Day 22 – Features of Perl 6.d

So there we are. Two years after the first official release of Rakudo Perl 6. Or 6.c to be more precise. Since Matt Oates already touched on the performance improvements since then, Santa thought to counterpoint this with a description of the new features for 6.d that have been implemented since then. Because there have been many, Santa had to make a selection.

Tweaking objects at creation

Any class that you create can now have a TWEAK method. This method will be called after all other initializations of a new instance of the class have been done, just before it is being returned by .new. A simple, bit contrived example in which a class A has one attribute, of which the default value is 42, but which should change the value if the default is specified at object creation:

class A {
    has $.value = 42;
    method TWEAK(:$value = 0) { # default prevents warning
        # change the attribute if the default value is specified
        $!value = 666 if $value == $!value;
    }
}
# no value specified, it gets the default attribute value
dd A.new;              # A.new(value => 42)

# value specified, but it is not the default
dd A.new(value => 77); # A.new(value => 77)

# value specified, and it is the default 
dd A.new(value => 42); # A.new(value => 666)

Concurrency Improvements

The concurrency features of Rakudo Perl 6 saw many improvements under the hood. Some of these were exposed as new features. Most prominent are Lock::Async (a non-blocking lock that returns a Promise) and atomic operators.

In most cases, you will not need to use these directly, but it is probably good that you know about atomic operators if you’re engaged in writing programs that use concurrency features. An often occurring logic error, especially if you’ve been using threads in Pumpking Perl 5, is that there is no implicit locking on shared variables in Rakudo Perl 6. For example:

   my int $a;
    await (^5).map: {
        start { ++$a for ^100000 }
    }
    say $a; # something like 419318

So why doesn’t that show 500000? The reason for this is that we had 5 threads that were incrementing the same variable at the same time. And since incrementing consists of a read step, an increment step and write step, it became very easy for one thread to do the read step at the same time as another thread. And thus losing an increment. Before we had atomic operators, the correct way of doing the above code would be:

   my int $a;
    my $l = Lock.new;
    await (^5).map: {
       start {
           for ^100000 {
               $l.protect( { ++$a } )
           }
       }
    }
    say $a; # 500000

This would give you the correct answer, but would be at least 20x as slow.

Now that we have atomic variables, the above code becomes:

   my atomicint $a;
    await (^5).map: {
        start { ++⚛$a for ^100000 }
    }
    say $a; # 500000

Which is very much like the original (incorrect) code. And this is at least 6x as fast as the correct code using Lock.protect.

Unicode goodies

So many, so many. For instance, you can now use , , as Unicode versions of <=, >= and != (complete list).

You can now also create a grapheme by specifying the Unicode name of the grapheme, e.g.:

say "BUTTERFLY".parse-names; # 🦋

or create the Unicode name string at runtime:

my $t = "THUMBS UP SIGN, EMOJI MODIFIER FITZPATRICK TYPE";
print "$t-$_".parse-names for 3..6; # 👍🏼👍🏽👍🏾👍🏿

Or collate instead of just sort:

# sort by codepoint value
say <ä a o ö>.sort; # (a o ä ö)
# sort using Unicode Collation Algorithm
say <ä a o ö>.collate; # (a ä o ö)

Or use unicmp instead of cmp:

say "a" cmp "Z"; # More
 say "a" unicmp "Z"; # Less

Or that you can now use any Unicode digits Match variables ( for $1), negative numbers ( for -1), and radix bases (:۳("22") for :3("22")).

It’s not for nothing that Santa considers Rakudo Perl 6 to have the best Unicode support of any programming language in the world!

Skipping values

You can now call .skip on Seq and Supply to skip a number of values that were being produced. Together with .head and .tail this gives you ample manipulexity with Iterables and Supplies.

By the way, .head now also takes a WhateverCode so you can indicate you want all values except the last N (e.g. .head(*-3) would give you all values except the last three). The same goes for .tail (e.g. .tail(*-3) would give you all values except the first three).

Some additions to the Iterator role make it possible for iterators to support the .skip functionality even better. If an iterator can be more efficient in skipping a value than to actually produce it, it should implement the skip-one method. Derived from this are the skip-at-least and skip-at-least-pull-one methods that can be provided by an iterator.

An example of the usage of .skip to find out the 1000th prime number:

say (^Inf).grep(*.is-prime)[999]; # 7919

Versus:

say (^Inf).grep(*.is-prime).skip(999).head; # 7919

The latter is slightly more CPU efficient, but more importantly much more memory efficient, as it doesn’t need to keep the first 999 prime numbers in memory.

Of Bufs and Blobs

Buf has become much more like an Array, as it now supports .push, .append, .pop, .unshift, .prepend, .shift and .splice. It also has become more like Str with the addition of a subbuf-rw (analogous with .substr-rw), e.g.:

my $b = Buf.new(100..105);
$b.subbuf-rw(2,3) = Blob.new(^5);
say $b.perl; # Buf.new(100,101,0,1,2,3,4,105)

You can now also .allocate a Buf or Blob with a given number of elements and a pattern. Or change the size of a Buf with .reallocate:

my $b = Buf.allocate(10,(1,2,3));
say $b.perl; # Buf.new(1,2,3,1,2,3,1,2,3,1)
$b.reallocate(5);
say $b.perl; # Buf.new(1,2,3,1,2)

Testing, Testing, Testing!

The plan subroutine of Test.pm now also takes an optional :skip-all parameter to indicate that all tests in the file should be skipped. Or you can call bail-out to abort the test run marking it as failed. Or set the PERL6_TEST_DIE_ON_FAIL environment variable to a true value to indicate you want the test to end as soon as the first test has failed.

What’s Going On

You can now introspect the number of CPU cores in your computer by calling Kernel.cpu-cores. The amount of CPU used since the start of the program is available in Kernel.cpu-usage, while you can easily check the name of the Operating System with VM.osname.

And as if that is not enough, there is a new Telemetry module which you need to load when needed, just like the Test module. The Telemetry module provides a number of primitives that you can use directly, such as:

use Telemetry;
say T<wallclock cpu max-rss>; # (138771 280670 82360)

This shows the number of microseconds since the start of the program, the number of microseconds of CPU used, and the number of Kilobytes of memory that were in use at the time of call.

If you want get to a report of what has been going on in your program, you can use snap and have a report appear when your program is done. For instance:

use Telemetry;
snap;
Nil for ^10000000;  # something that takes a bit of time

The result will appear on STDERR:

Telemetry Report of Process #60076
Number of Snapshots: 2
Initial/Final Size: 82596 / 83832 Kbytes
Total Time:           0.55 seconds
Total CPU Usage:      0.56 seconds
No supervisor thread has been running

wallclock  util%  max-rss
   549639  12.72     1236
--------- ------ --------
   549639  12.72     1236

Legend:
wallclock  Number of microseconds elapsed
    util%  Percentage of CPU utilization (0..100%)
  max-rss  Maximum resident set size (in Kbytes)

If you want a state of your program every .1 of a second, you can use the snapper:

use Telemetry;
snapper;
Nil for ^10000000;  # something that takes a bit of time

The result:

Telemetry Report of Process #60722
Number of Snapshots: 7
Initial/Final Size: 87324 / 87484 Kbytes
Total Time:           0.56 seconds
Total CPU Usage:      0.57 seconds
No supervisor thread has been running

wallclock  util%  max-rss
   103969  13.21      152
   101175  12.48
   101155  12.48
   104097  12.51
   105242  12.51
    44225  12.51        8
--------- ------ --------
   559863  12.63      160

Legend:
wallclock  Number of microseconds elapsed
    util%  Percentage of CPU utilization (0..100%)
  max-rss  Maximum resident set size (in Kbytes)

And many more options are available here, such as getting the output in .csv format.

The MAIN thing

You can now modify the way MAIN parameters are handled by setting options in %*SUB-MAIN-OPTS. The default USAGE message is now available inside the MAIN as the $*USAGE dynamic variable, so you can change it if you want to.

Embedding Perl 6

Two new features make embedding Rakudo Perl 6 easier to handle:
the &*EXIT dynamic variable now can be set to specify the action to be taken when exit() is called.

Setting the environment variable RAKUDO_EXCEPTIONS_HANDLER to "JSON" will throw Exceptions in JSON, rather than text, e.g.:

$ RAKUDO_EXCEPTIONS_HANDLER=JSON perl6 -e '42 = 666'
{
  "X::Assignment::RO" : {
    "value" : 42,
    "message" : "Cannot modify an immutable Int (42)"
  }
}

Bottom of the Gift Bag

While rummaging through the still quite full gift bag, Santa found the following smaller prezzies:

  • Native string arrays are now implemented (my str @a)
  • IO::CatHandle allows you to abstract multiple data sources into a single virtual IO::Handle
  • parse-base() performs the opposite action of base()

Time to catch a Sleigh

Santa would like to stay around to tell you more about what’s been added, but there simply is not enough time to do that. If you really want to keep up-to-date on new features, you should check out the Additions sections in the ChangeLog that is updated with each Rakudo compiler release.

So, catch you again next year!

Best wishes from

🎅🏾

Day 24 – One Year On

This time of year invites one to look back on things that have been, things that are and things that will be.

Have Been

I was reminded of things that have been when I got my new notebook a few weeks ago. Looking for a good first sticker to put on it, I came across an old ActiveState sticker:

If you don’t know Perl
you don’t know Dick

A sticker from 2000! It struck me that that sticker was as old as Perl 6. Only very few people now remember that a guy called Dick Hardt was actually the CEO of ActiveState at the time. So even though the pun may be lost on most due to the mists of time, the premise still rings true to me: that Perl is more about a state of mind, then about versions. There will always be another version of Perl. Those who don’t know Perl are doomed to re-implement it, poorly. Which, to me, is why so many ideas were borrowed from Perl. And still are!

Are

Where are we now? Is it the moment we know, We know, we know? I don’t think we are at twenty thousand people using Perl 6 just yet. But we’re keeping our fingers crossed. Just in case.

We are now 12 compiler releases after the initial Christmas release of Perl 6. In this year, many, many areas of Rakudo Perl 6 and MoarVM have dramatically improved in speed and stability. Our canary-in-the-coalmine test has dropped from around 14 seconds a year ago to around 5.5 seconds today. A complete spectest run is now about 3 minutes, whereas it was about 4.5 minutes at the beginning of the year, while about 4000 tests were added (from about 50K to 54K). And we now have 757 modules in the Perl 6 ecosystem (aka temporary CPAN for Perl 6 modules), with a few more added every week.

The #perl6 IRC channel has been too busy for me to follow consistently. But if you have a question related to Perl 6 and you want a quick answer, the #perl6 channel is the place to be. You don’t even have to install an IRC client: you can also use a browser to chat, or just follow “live” what is being said.

There are also quite a few useful bots on that channel: they e.g. take care of running a piece of Perl 6 code for you. Or find out at which commit the behaviour of a specific piece of code changed. These are very helpful for the developers of Perl 6, who usually also hang out on the #perl6-dev IRC channel. Which could be you! The past year, at least one contributor was added to the CREDITS every month!

Will Be

The coming year will see at least three Perl 6 books being published. First one will be Think Perl 6 – How To Think Like A Computer Scientist by Laurent Rosenfeld. It is an introduction to programming using Perl 6. But even for those of you with programming experience, it will be a good book to start learning Perl 6. And I can know. Because I’ve already read it :-)

Second one will be Learning Perl 6 by veteran Perl developer and writer brian d foy. It will have the advantage of being written by a seasoned writer going through the newbie experience that most people will have when coming from Perl 5.

The third one will be Perl 6 By Example by Moritz Lenz, which will, as the title already gives away, introduce Perl 6 topics by example.

There’ll be at least two (larger) Perl Conferences apart from many smaller Perl workshops: the The Perl Conference NA on 18-23 June, and the The Perl Conference in Amsterdam on 9-11 August. Where you will meet all sorts of nice people!

And for the rest? Expect a faster, leaner, Perl 6 and MoarVM compiler release on the 3rd Saturday every month. And an update of weekly events in the Perl 6 Weekly on every Monday evening/Tuesday morning (depending on where you live).

Day 16 – Quoting on Steroids

A few days ago, there was a blog post about String Interpolation. Most of the examples there used a simple double quoted string.

my $a = 42;
say "a = $a"; # a = 42

The possibilities of the double quoted string are quite powerful already. But in the end, they’re just a special case of a much more generic and malleable quoting construct, named Q.

Q’s Basic Features

In its most generic form, Q just copies the string without any changes or interpretation:

my $a = 42;
say Q/foo $a \n/; # foo $a \n

You can add adverbs to Q[…], to change the way the resulting string will be formatted. For instance, if you want to have interpolation of scalars, you can add :s. If you want interpretation of backslashes like \n, you can add :b. And you can combine them:

my $a = 42;
say Q:s/foo $a\n/;   # foo 42\n
say Q:b/foo $a\n/;   # foo $a␤
say Q:s:b/foo $a\n/; # foo 42␤

(If you wonder what a is, it is a U+2424 SYMBOL FOR NEWLINE [So] (␤) and should be show up in your browser as a character containing an N and an L as a visible representation of a new line character)

In fact, the list of adverbs of basic features is:

    short       long            what does it do
    =====       ====            ===============
    :q          :single         Interpolate \\, \q and \' (or whatever)
    :s          :scalar         Interpolate $ vars
    :a          :array          Interpolate @ vars
    :h          :hash           Interpolate % vars
    :f          :function       Interpolate & calls
    :c          :closure        Interpolate {...} expressions
    :b          :backslash      Interpolate \n, \t, etc. (implies :q at least)

The :q (:single) basically gives you single quoted string semantics. The other adverbs, give you typically the functionality that you would expect from double quoted strings. If you really want to be verbose on your double quoted strings, you can write them like this:

my $a = 42;
say Q :scalar :array :hash :function :closure :backslash /foo $a\n/; # foo 42␤

Of course, you can also specify the short versions of the adverbs, and not separate them by whitespace. So, if you want to be less verbose:

my $a = 42;
say Q:s:a:h:f:c:b/foo $a\n/; # foo 42␤

As with any adverbs (which are just named parameters, really), the order does not matter:

my $a = 42;
say Q:f:s:b:a:c:h/foo $a\n/; # foo 42␤

Actually, the story about the order of the named parameters is a little bit more complicated than this. But for this set of adverbs, it does not matter in which order they are specified.

.oO( is that a brother of Johann Sebastian? )

But seriously, that is still a mouthful. So an even shorter shortcut is provided: :qq

    short       long            what does it do
    =====       ====            ===============
    :qq         :double         Interpolate with :s, :a, :h, :f, :c, :b

So, you can:

my $a = 42;
say Q:double/foo $a\n/; # foo 42␤
say Q:qq/foo $a\n/; # foo 42␤

All that for simply doing a double quoted string with interpolation? Well, because people are using double quoted strings a lot, the simple remains the quickest way of interpolating values into a string. However, underneath that all, it’s really Q:qq, which in turn is really Q:s:a:h:f:c:b.

What about a double quoted string that has double quotes in it? For those cases, the Q:qq form is available, but that is still quite verbose. Synopsis 2 therefore specifies:

In fact, all quote-like forms derive from Q with adverbs:
    q//         Q:q//
    qq//        Q:qq//

Which means we can shorten the Q:qq in that last example to qq (and have double quotes in the double quoted string without any problems):

my $a = 42;
say qq/foo "$a"\n/; # foo "42"␤

Both q// and qq// also support (the same) adverbs. This initially seems the most useful with q//, for instance in combination with :s, which would (also) interpolate scalars:

my $a = 42;
say q:s/foo "$a"\n/; # foo "42"\n

However, adverbs (just as named parameters) are just a shortcut for a Pair: :s is really s => True. And :!s is really just s => False. Can we also apply this to quoting constructs? The answer is: yes, you can!

say qq:!s:!c/foo "$x{$y}"\n/; # foo "$x{$y}"␤

Even though we specified qq//, the scalar is not interpolated, because of the :!s adverb. And the scope is not interpolated, because of the :!c. This can for instance be handy when building strings to be EVALled. So, if you want all quoting features except one or more, you can easily de-activate that feature by negating those adverbs.

Some of Q’s Advanced Features

Quoting features do not stop here. This is a list of some of the other features that already work in Rakudo Perl 6:

    short       long            what does it do
    =====       ====            ===============
    :x          :exec           Execute as command and return results
    :w          :words          Split result on words (no quote protection)
    :ww         :quotewords     Split result on words (with quote protection)
    :to         :heredoc        Parse result as heredoc terminator

Here are some examples. Note that we don’t bother specifying these features as attributes, because they’re basically the first additional attribute, so there is a shortcut version for them. For example, qq:x// can be shortened to qqx//. Whether you could consider that more readable or not, I leave up to the reader. There is, after all, more than one way to do it.

Interpolate and execute as an external program:

my $w = 'World';
say qqx/echo Hello $w/; # Hello World

Interpolate as single quoted words (please look at what happens to the single quotes):

.say for qw/ foo bar 'first second' /;
==
foo
bar
'first
second'

Interpolate as single quoted words with quote protection. This will make sure that balanced quotes will be treated as one entity (and note again what happened to the single quotes).

.say for qww/ foo bar 'first second' /;
==
foo
bar
first second

Interpolate variables into a heredoc:

my $a = 'world';
say qqto/FOO/;
  Hello $a
  FOO
==
Hello world␤

The text is exdented automatically for the same number of indents as the target has.

Conclusion

Perl 6 has a very powerful basic quoting construct in Q[…], from which all other quoting constructs are derived. They allow mix and matching of features in various short and more verbose ways. There are still some adverbs unimplemented, but the ones that are mentioned here, should Just Work™.

Finally, the Synopsis also indicates that there will be a way out of this alphabet soup:

If you want to abbreviate further, just define a macro:

    macro qx { 'qq:x ' }          # equivalent to P5's qx//
    macro qTO { 'qq:x:w:to ' }    # qq:x:w:to//
    macro quote:<❰ ❱> ($text) { quasi { {{{$text}}}.quoteharder } }

We can only hope that someone will find enough quality tuits soon to implement this.

Day 13 – String Interpolation and the Zen Slice

So you think you know all about string interpolation in Perl 6?

Well, especially coming from Perl 5, you may find some things that do not work exactly as you’re used to. The simple examples all work, as it seems:

my $a = 42;
say 'value = $a'; # value = $a
say "value = $a"; # value = 42

my @a = ^10;
say 'value = @a'; # value = @a
say "value = @a"; # value = @a HUH??

In earlier versions of Perl 5 (or was it Perl 4?), this gave the same result. At one point, it was decided that arrays would be interpolated in double quoted strings as well. However, this caused quite a few problems with double quoted texts with email addresses in them: you would need to escape each @ (if you were using strict, which you of course should have). And if you forgot, and there was an array that just happened to have the same name as the user part of an email address, you would be in for a surprise. Or if you didn’t use strict, you would suddenly have the text without the email address. But then again, you got what you asked for.

So how can we make this work in Perl 6?

Introducing the Zen slice

The Zen slice on an object, returns the object. It’s like you specify nothing, and get everything. So what does that look like?

my @a = ^10;
say "value = @a[]"; # value = 0 1 2 3 4 5 6 7 8 9

You will have to make sure that you use the right indexers for the type of variable that you’re interpolating.

my %h = a => 42, b => 666;
say "value = %h{}"; # value = a 42 b 666

Note that the Zen slice on a hash returns both keys and values, whereas the Zen slice on an array only returns the values. This seems inconsistent, until you realize that you can think of a hash as a list of Pairs.

The Zen slice only really exists at compile time. So you will not get everything if your slice specification is an empty list at runtime:

my @a;
my %h = a => 42, b => 666;
# a slice, but not a Zen slice:
say "value = %h{@a}"; # value =

So the only way you can specify a Zen slice, is if there is nothing (but whitespace) between the slice delimiters.

The Whatever slice

The * ( Whatever ) slice is different. The Whatever will just fill in all keys that exist in the object, and thus only return the values of a hash.

my %h = a => 42, b => 666;
say "value = %h{*}"; # value = 42 666

For arrays, there isn’t really any difference at the moment (although that may change in the future when multi-dimensional arrays are fleshed out more).

Interpolating results from subs and methods

In double quoted strings, you can also interpolate subroutine calls, as long as they start with an ‘&‘ and have a set of parentheses (even if you don’t want to pass any arguments):

sub a { 42 }
say "value = &a()"; # value = 42

But it doesn’t stop at calling subs: you can also call a method on a variable as long as they have parentheses at the end:

my %h = a => 42, b => 666;
say "value = %h.keys()"; # value = b a

And it doesn’t stay limited to a single method call: you can have as many as you want, provided the last one has parentheses:

my %h = a => 42, b => 666;
say "value = %h.perl.EVAL.perl.EVAL.perl()"; # value = ("b" => 666, "a" => 42).hash

Interpolating expressions

If you want to interpolate an expression in a double quoted string, you can also do that by providing an executable block inside the string:

say "6 * 7 = { 6 * 7 }"; # 6 * 7 = 42

The result of the execution of the block, is what will be interpolated in the string. Well, what really is interpolated in a string, is the result of calling the .Str method on the value. This is different from just saying a value, in which case the .gist method is called. Suppose we have our own class with its own .gist and .Str methods:

class A {
    method Str { "foo" }
    method gist { "bar" }
}
say "value = { A }"; # value = foo
say "value = ", A;   # value = bar

Conclusion

String interpolation in Perl 6 is very powerful. As you can see, the Zen slice makes it easy to interpolate whole arrays and hashes in a string.

In this post I have only scratched the surface of string interpolation. Please check out Quoting on Steroids in a few days for more about quoting constructs.

Day 5 – Act with great Responsibility

The past year in Rakudo-land has been very exciting. It has brought us a new, full-fledged backend in the form of MoarVM. And with that came extended support for asynchronous execution of code. This means you can now more easily try out all these asynchronous features, without having to suffer from the long startup time of the JVM backend.

The “Supply” is still the main mechanism to use the Reactive Programming paradigm. Here is a simple (silly) example of using a Supply:

1: my $s = Supply.new;
2: $s.act: { print " Fizz" if $_ %% 3 }
3: $s.act: { print " Buzz" if $_ %% 5 }
4: $s.act: { print " $_" unless $_ %% 3 || $_ %% 5 }
5: $s.emit($_) for 1..20;
======================
1 2 Fizz 4 Buzz Fizz 7 8 Fizz Buzz 11 Fizz 13 14 Fizz Buzz 16 17 Fizz 19 Buzz

Let’s take this example apart. Line 1 just creates a new Supply. By itself, it doesn’t do anything at all, except for creating the object, of course. Lines 2, 3 and 4 set up code (closures) to be executed every time a new value arrives in the Supply $s.

Line 5 sends 20 consecutive values to the Supply. This in turn causes the closures to be executed (in turn for each value emitted). The result is a nice FizzBuzz.

But but…

Careful watchers of this code may wonder about two things. First of all: what’s emit ? Is that something new? No, it is not (really). It used to be called more. At the Austrian Perl Workshop last October it was decided to rename this method to emit (as nobody was really happy with any more). So what happens if you have code that uses more? Well, it will still work, but you will get a deprecation warning after your process finishes:

Saw 1 call to deprecated code during execution.
===============================================
Method more (from Supply) called at:
t/spec/integration/advent2014-day05.t, line 13
Deprecated since v2014.10, will be removed with release v2015.10!
Please use emit instead.
-----------------------------------------------

Note that even though the same method was called 20 times, it is only mentioned once. And that’s because all the calls happened at the same location in the code. If you have seen deprecation messages before, you might notice that it now also mentions when it was deprecated, and when the feature will most likely be removed.

The second thing that a careful watcher might wonder about is: why the use of act? Is that a also a rename (from tap)? No, it is not. The act method was added last April. It ensures that your closure will be the only one running on that Supply, even if the emits are coming from different threads. That is a much safer default than tap, where multiple threads can be executing the closure you provided at the same time.

Sharing everything comes with great power and great responsibility

So why is this important? This is important because all variables visible in a specific scope, are readable and changeable by all threads simultaneously. Reading a variable from different threads at the same time, is not an issue in Perl 6 (apart from some minor known issues to be resolved after the Great List Refactor). Changing a variable from multiple threads simultaneously, is an issue that may cause data loss, memory corruption and segfaults. To give an example:

1: my $times = 100000;
2: my $a = 0;
3: $*SCHEDULER.cue: { $a++ } for ^$times; # this may segfault because of unguarded changes
4: sleep 5; # poor man’s way to wait for all code to finish
5: say “Missed {$times - $a} updates”;
===================================
Missed 9 updates

Line 1 sets up the number of iterations we’re going to do. Line 2 initializes the variable that’s going to be incremented. Although ++ is magic, we initialize the variable explicitly, so that each increment will follow the same internal code path. Line 3 cues the increment to be executed that many times (in a very low level way using the underlying scheduler). Line 4 waits for the execution to be finished. And line 5 shows the result (if we didn’t segfault, which we might).

So please use act rather than tap, unless you are sure that closures will not be updating any shared variables simultaneously. Which could be hard to tell if you’re using external modules inside the closure.

FizzBuzzing some more

Coming back to the first example: should we have used act there, or could we have used tap? We could have used tap there, because print is threadsafe. It becomes a different story if we would have used a (shared) array instead of print:

1: my @seen;
2: my $s = Supply.new;
3: $s.act: { @seen.push: "Fizz" if $_ %% 3 }
4: $s.act: { @seen.push: "Buzz" if $_ %% 5 }
5: $s.act: { @seen.push: $_ unless $_%%3 || $_%%5 }
6: await do for 1..20 {
7:     start { rand.sleep; $s.emit($_) }
8: }
9: say @seen;
==========
11 14 4 Fizz 17 19 Fizz Buzz 16 13 Buzz Fizz Buzz 7 Fizz 8 2 Buzz 1 Fizz Fizz

Hmmm… that has the right number of Fizz’s and Buzz’s, but clearly the order is wrong. But because we have used act rather than tap, we are at least sure that no push was executed simultaneously on the @seen array. So we didn’t lose any values, nor did we segfault.

You might ask: what are that lines 6-8 about? Well, that’s making sure the emit’s are done in a random order, spread out in time (because the 0 to 1 second random sleep from rand.sleep, and the start making it run in a separate thread).

So, how could you make this work in a parallel way? Well generally, if you have some kind of identifier for each value emitted, then you can use that to store the result in a shared data-structure. In this case, $_ is exactly that. So with a slight modification:

1: my @seen;
2: my $s = Supply.new;
3: $s.act: { @seen[$_] = "Fizz" if $_ %% 3 }
4: $s.act: { @seen[$_] ~= "Buzz" if $_ %% 5 }
5: $s.act: { @seen[$_] //= $_ }
6: await do for 1..20 {start{rand.sleep;$s.emit($_)}}
7: say @seen[1..20];
=================
1 2 Fizz 4 Buzz Fizz 7 8 Fizz Buzz 11 Fizz 13 14 FizzBuzz 16 17 Fizz 19 Buzz

By assigning directly to an element in the array (lines 3, 4, 5), we automatically put the information in the right place. And because we are (implicitely) guaranteed that act’s on the same item are always executed in the order they were created, we can simplify the non-FizzBuzz case in line 5.

Conclusion

When using Supplies, please use act on the Supply if you’re not sure whether the closure is going to change shared variables. Only use tap if you’re really 100% sure.

Further reading

Last year saw 2 async execution related posts, the latter showing uses of tap. Please read up on them if the above was not telling you all you needed to know:

Tests

The above code can also be found in roast at t/spec/integration/advent2014-day05.t

Day 20 – How to mangle the initial state

The past year saw the (at least partial) implementation of several variable traits. But before we get into that, the past year also saw Nil getting implemented closer to spec in Rakudo. If you read the spec, you will know that Nil indicates the absence of a value. This is different from being undefined, as we saw in an earlier blogpost this year. So what does that mean?

$ perl6 -e 'my $a = 42; say $a; $a = Nil; say $a'
42
(Any)

So, assigning Nil will reset a scalar container to its default value, which (by default) is the type object of that container. Which is (Any) if nothing is specifically specified.

Enter the “is default” variable trait

If you want to specify the default value of a variable, you can now use the “is default” variable trait:

$ perl6 -e 'my $a is default(42); say $a; say $a.defined’
42
True

So oddly enough, even though we haven’t assigned anything to the variable, it now has a defined value. So let’s assign something else to it, and then assign Nil:

$ perl6 -e 'my $a is default(42) = 69; say $a; $a = Nil; say $a'
69
42

Of course, this is a contrived example.

“is default” on arrays and hashes

It gets more interesting with arrays and hashes. Suppose you want a Bool array where you can only switch “off” elements by assigning False? That is now possible:

$ perl6 -e 'my Bool @b is default(True); say @b[1000]; @b[1000]=False; say @b[1000]'
True
False

Of course, type checks should be in place when specifying default values.

$ perl6 -e 'my Bool $a is default(42)'
===SORRY!=== Error while compiling -e
Type check failed in assignment to '$a'; expected 'Bool' but got 'Int’

Note that this type check is occurring at compile time. Ideally, this should also happen with arrays and hashes, but that doesn’t happen just yet. Patches welcome!

Using is default on arrays and hashes interacts as expected with :exists (well, at least for some definition of expected :-):

$ perl6 -e 'my @a is default(42); say @a[0]; say @a[0].defined; say @a[0]:exists'
42
True
False

Note that even though each element in the array appears to have a defined value, it does not necessarily exist. Personally, I would be in favour of expanding that to scalar values as well, but for now :exists does not work on scalar values, just on slices.

It’s not the same as specifying the type

What’s wrong with using type objects as default values? Well, for one it doesn’t set the type of the variable. Underneath it is still an (Any) in this case:

$ perl6 -e 'my $a is default(Int) = "foo"; say $a'
foo

So you don’t get any type checking, which is probably not what you want (otherwise you wouldn’t take all the trouble of specifying the default). So don’t do that! Compare this with:

$ perl6 -e 'my Int $a = "foo"'
Type check failed in assignment to '$a'; expected 'Int' but got 'Str'

Which properly tells you that you’re doing something wrong.

Nil assigns the default value

Coming back to Nil: we already saw that assigning it will assign the default value. If we actually specify a default value, this becomes more clear:

$ perl6 -e 'my $a is default(42) = 69; say $a; $a = Nil; say $a'
69
42

In the context of arrays and hashes, assigning Nil has a subtly different effect from using the :delete adverb. After assigning Nil, the element still exists:

$ perl6 -e 'my @a is default(42) = 69; say @a[0]; @a[0] = Nil; say @a[0]:exists'
69
True

Compare this with using the :delete adverb:

$ perl6 -e 'my @a is default(42) = 69; @a[0]:delete; say @a[0]; say @a[0]:exists'
42
False

One could argue that assigning Nil (which assigns the default value, but which is also specced as indicating an absence of value) should be the same as deleting. I would be in favour of that.

Argh, I want to pass on Nil

The result of a failed match, now also returns Nil (one of the other changes in the spec that were implemented in Rakudo in the past year). However, saving the result of a failed match in a variable, will assign the default value, losing its Nilness, Nilility, Nililism (so much opportunity for creative wordsmithing :-).

$ perl6 -e 'my $a = Nil; say $a'
(Any)

It turns out you can specify Nil as the default for a variable, and then It Just Works™

$ perl6 -e 'my $a is default(Nil) = 42; say $a; $a = Nil; say $a'
42
Nil

Introspection

You can use the VAR macro to introspect variables to find out its default value, without having to do something as destructive as assigning Nil to it:

$ perl6 -e 'my @a is default(42); say @a.VAR.default'
42

This also works on system variables like $/:

$ perl6 -e 'say $/.VAR.default'
Nil

And can also be used to interrogate other properties:

$ perl6 -e 'say $/.VAR.dynamic’
True

But this gets us into material for a blog post for another day!

Conclusion

The “is default” variable trait allows you to manipulate the default value of variables and elements in arrays and hashes. Together with the new meaning of Nil, it is a powerful means of mangling initial values of variables to your satisfaction.

Day 12 – Slicing with adverbs, the only way!

My involvement with adverbs in Perl 6 began very innocently. I had the idea to creating a small, lightning talk size presentation about how the Perl 5 fat arrow corresponds to Perl 6’s fatarrow and adverbs. And how they relate to hash / array slices. And then had to find out you couldn’t combine them on hash / array slices. Nor could you pass values to them.

And so started my first bigger project on Rakudo Perl 6. Making adverbs work as specced on hashes and arrays, and on the way, expand the spec as well. So, do they now work? Well, all spectests pass. But while preparing this blog post, I happened to find a bug which is now waiting for my further attention. There’s always one more bug.

What are the adverbs you can use with hash and array slices?

name description
:exists whether element(s) exist(ed)
:delete remove element(s), return value (if any)
:kv return key(s) and value(s) as Parcel
:p return key(s) and value(s) as Parcel of Pairs
:k return key(s) only
:v return value(s) only

:exists

This adverb replaces the now deprecated .exists method. Adverbs provide a generic interface to hashes and arrays, regardless of number of elements requested. The .exists method only ever allowed checking for a single key.

Examples speak louder than words. To check whether a single key exists:

$ perl6 -e 'my %h = a=>1, b=>2; say %h<a>:exists’
True

If we expand this to a slice, we get a Parcel of boolean values:

$ perl6 -e 'my %h = a=>1, b=>2; say %h<a b c>:exists'
True True False

Note that if we ask for a single key, we get a boolean value back, not a Parcel with one Bool in it.

$ perl6 -e 'my %h = a=>1, b=>2; say (%h<a>:exists).WHAT’
(Bool)

If it is clear that we ask for multiple keys, or not clear at compile time that we are only checking for one key, we get back a Parcel:

$ perl6 -e 'my %h = a=>1, b=>2; say (%h<a b c>:exists).WHAT’
(Parcel)
$ perl6 -e 'my @a="a"; my %h = a=>1, b=>2; say (%h{@a}:exists).WHAT'
(Parcel)

Sometimes it is handier to know if something does not exist. You can easily do this by negating the adverb by prefixing it with !: they’re really just like named parameters anyway!

$ perl6 -e 'my %h = a=>1, b=>2; say %h<c>:!exists'
True

:delete

This is the only adverb that actually can make changes to the hash or array it is (indirectly) applied to. It replaces the now deprecated .delete method.

$ perl6 -e 'my %h = a=>1, b=>2; say %h<a>:delete; say %h.perl'
1
("b" => 2).hash

Of course, you can also delete slices:

$ perl6 -e 'my %h = a=>1, b=>2; say %h<a b c>:delete; say %h.perl'
1 2 (Any)
().hash

Note that the (Any) is the value returned for the non-existing key. If you happened to have given the hash a default value, it would have looked like this:

$ perl6 -e 'my %h is default(42) = a=>1, b=>2; say %h<a b c>:delete; say %h.perl'
1 2 42
().hash

But the behaviour of the is default maybe warrants a blog post of itself, so I won’t go into it now.

Like with :exists, you can negate the :delete adverb. But there wouldn’t be much point, as you might have well not specified it at all. However, since adverbs are basically just named parameters, you can make the :delete attribute conditional:

$ perl6 -e 'my $really = True; my %h = a=>1, b=>2; say %h<a b c>:delete($really); say %h.perl'
1 2 (Any)
().hash

Because the value passed to the adverb was true, the deletion actually took place. However, if we pass a false value:

$ perl6 -e ‘my $really; my %h = a=>1, b=>2; say %h<a b c>:delete($really); say %h.perl'
1 2 (Any)
("a" => 1, "b" => 2).hash

It doesn’t. Note that the return value did not change: the deletion was simply not performed. This can e.g. be very handy if you have a subroutine or method doing some kind of custom slice, and you want to have an optional parameter indicating whether the slice should be deleted as well: simply pass that parameter as the adverb’s value!

:kv, :p, :k, :v

These 4 attributes modify the returned values from any hash / array slice. The :kv attribute returns a Parcel with keys and values interspersed. The :p attribute returns a Parcel of Pairs. The :k and :v attributes return the key only, or the value only.

$ perl6
> my %h = a => 1, b => 2;
("a” => 1, "b” => 2).hash
> %h<a>:kv
a 1
> %h<a>:p
"a" => 1
> %h<a>:k
a
> %h<a>:v
1

Apart from modifying the return value, these attributes also act as a filter for existing keys only. Please note the difference in return values:

> %h<a b c>
1 2 (Any)
> %h<a b c>:v
1 2

Because the :v attribute acts as a filter, there is no (Any). But sometimes, you want to not have this behaviour. To achieve this, you can negate the attribute:

> %h<a b c>:k
a b
> %h<a b c>:!k
a b c

Combining adverbs

You can also combine adverbs on hash / array slices. The most useful combinations are with one or two of :exists and :delete, with zero or one of :kv, :p, :k, :v. Some examples, like putting a slice out of one hash into a new hash:

$ perl6 -e 'my %h = a=>1, b=>2; my %i = (%h<a c>:delete:p).list; say %h.perl; say %i.perl'
("b” => 2).hash
("a” => 1).hash

Or the keys that were actually deleted:

$ perl6 -e 'my %h = a=>1, b=>2; say %h<a b c>:delete:k’
a b

We actually have a spec that describes which combinations are valid, and what they should return.

Arrays are not Hashes

Apart from hashes using {} for slices, and arrays [] for slices, the adverbial syntax for hash and array slices are the same. But there are some subtle differences. First of all, the “key” of an element in an array, is its index. So, to show the indexes of elements in an array that have a defined value, one can use the :k attribute:

$ perl6 -e 'my @a; @a[3] = 1; say @a[]:k'
3

Or, to create a Parcel with all elements in an array:

$ perl6 -e 'my @a; @a[3] = 1; say @a[]:!k’
0 1 2 3

However, deleting an element from an array, is similar to assigning Nil to it, so it will return its default value (usually (Any)):

$ perl6 -e 'my @a = ^10; @a[3]:delete; say @a[2,3,4]; say @a[2,3,4]:exists'
2 (Any) 4
True False True

If we have specified a default value for the array, the result is slightly different:

$ perl6 -e 'my @a is default(42) = ^10; @a[3]:delete; say @a[2,3,4]; say @a[2,3,4]:exists'
2 42 4
True False True

So, even though the element “does not exist”, it can return a defined value! As said earlier, that may become a blog post for another day!

Conclusion

Slices with adverbs are a powerful way of handling your data structures, be they hashes or arrays. It will take a while to get used to all of the combinations of adverbs that can be specified. But once you’re used to them, they provide you with a concise way of dicing and slicing your data that would previously have involved more elaborate structures with loops and conditionals. Of course, if you want to, you can still do that: it’s not illegal to program Perl 5 in Perl 6 :-)