Author Archive

Day 16 – Quoting on Steroids

December 16, 2014

A few days ago, there was a blog post about String Interpolation. Most of the examples there used a simple double quoted string.

my $a = 42;
say "a = $a"; # a = 42

The possibilities of the double quoted string are quite powerful already. But in the end, they’re just a special case of a much more generic and malleable quoting construct, named Q.

Q’s Basic Features

In its most generic form, Q just copies the string without any changes or interpretation:

my $a = 42;
say Q/foo $a \n/; # foo $a \n

You can add adverbs to Q[…], to change the way the resulting string will be formatted. For instance, if you want to have interpolation of scalars, you can add :s. If you want interpretation of backslashes like \n, you can add :b. And you can combine them:

my $a = 42;
say Q:s/foo $a\n/;   # foo 42\n
say Q:b/foo $a\n/;   # foo $a␤
say Q:s:b/foo $a\n/; # foo 42␤

(If you wonder what a is, it is a U+2424 SYMBOL FOR NEWLINE [So] (␤) and should be show up in your browser as a character containing an N and an L as a visible representation of a new line character)

In fact, the list of adverbs of basic features is:

    short       long            what does it do
    =====       ====            ===============
    :q          :single         Interpolate \\, \q and \' (or whatever)
    :s          :scalar         Interpolate $ vars
    :a          :array          Interpolate @ vars
    :h          :hash           Interpolate % vars
    :f          :function       Interpolate & calls
    :c          :closure        Interpolate {...} expressions
    :b          :backslash      Interpolate \n, \t, etc. (implies :q at least)

The :q (:single) basically gives you single quoted string semantics. The other adverbs, give you typically the functionality that you would expect from double quoted strings. If you really want to be verbose on your double quoted strings, you can write them like this:

my $a = 42;
say Q :scalar :array :hash :function :closure :backslash /foo $a\n/; # foo 42␤

Of course, you can also specify the short versions of the adverbs, and not separate them by whitespace. So, if you want to be less verbose:

my $a = 42;
say Q:s:a:h:f:c:b/foo $a\n/; # foo 42␤

As with any adverbs (which are just named parameters, really), the order does not matter:

my $a = 42;
say Q:f:s:b:a:c:h/foo $a\n/; # foo 42␤

Actually, the story about the order of the named parameters is a little bit more complicated than this. But for this set of adverbs, it does not matter in which order they are specified.

.oO( is that a brother of Johann Sebastian? )

But seriously, that is still a mouthful. So an even shorter shortcut is provided: :qq

    short       long            what does it do
    =====       ====            ===============
    :qq         :double         Interpolate with :s, :a, :h, :f, :c, :b

So, you can:

my $a = 42;
say Q:double/foo $a\n/; # foo 42␤
say Q:qq/foo $a\n/; # foo 42␤

All that for simply doing a double quoted string with interpolation? Well, because people are using double quoted strings a lot, the simple remains the quickest way of interpolating values into a string. However, underneath that all, it’s really Q:qq, which in turn is really Q:s:a:h:f:c:b.

What about a double quoted string that has double quotes in it? For those cases, the Q:qq form is available, but that is still quite verbose. Synopsis 2 therefore specifies:

In fact, all quote-like forms derive from Q with adverbs:
    q//         Q:q//
    qq//        Q:qq//

Which means we can shorten the Q:qq in that last example to qq (and have double quotes in the double quoted string without any problems):

my $a = 42;
say qq/foo "$a"\n/; # foo "42"␤

Both q// and qq// also support (the same) adverbs. This initially seems the most useful with q//, for instance in combination with :s, which would (also) interpolate scalars:

my $a = 42;
say q:s/foo "$a"\n/; # foo "42"\n

However, adverbs (just as named parameters) are just a shortcut for a Pair: :s is really s => True. And :!s is really just s => False. Can we also apply this to quoting constructs? The answer is: yes, you can!

say qq:!s:!c/foo "$x{$y}"\n/; # foo "$x{$y}"␤

Even though we specified qq//, the scalar is not interpolated, because of the :!s adverb. And the scope is not interpolated, because of the :!c. This can for instance be handy when building strings to be EVALled. So, if you want all quoting features except one or more, you can easily de-activate that feature by negating those adverbs.

Some of Q’s Advanced Features

Quoting features do not stop here. This is a list of some of the other features that already work in Rakudo Perl 6:

    short       long            what does it do
    =====       ====            ===============
    :x          :exec           Execute as command and return results
    :w          :words          Split result on words (no quote protection)
    :ww         :quotewords     Split result on words (with quote protection)
    :to         :heredoc        Parse result as heredoc terminator

Here are some examples. Note that we don’t bother specifying these features as attributes, because they’re basically the first additional attribute, so there is a shortcut version for them. For example, qq:x// can be shortened to qqx//. Whether you could consider that more readable or not, I leave up to the reader. There is, after all, more than one way to do it.

Interpolate and execute as an external program:

my $w = 'World';
say qqx/echo Hello $w/; # Hello World

Interpolate as single quoted words (please look at what happens to the single quotes):

.say for qw/ foo bar 'first second' /;
==
foo
bar
'first
second'

Interpolate as single quoted words with quote protection. This will make sure that balanced quotes will be treated as one entity (and note again what happened to the single quotes).

.say for qww/ foo bar 'first second' /;
==
foo
bar
first second

Interpolate variables into a heredoc:

my $a = 'world';
say qqto/FOO/;
  Hello $a
  FOO
==
Hello world␤

The text is exdented automatically for the same number of indents as the target has.

Conclusion

Perl 6 has a very powerful basic quoting construct in Q[…], from which all other quoting constructs are derived. They allow mix and matching of features in various short and more verbose ways. There are still some adverbs unimplemented, but the ones that are mentioned here, should Just Work™.

Finally, the Synopsis also indicates that there will be a way out of this alphabet soup:

If you want to abbreviate further, just define a macro:

    macro qx { 'qq:x ' }          # equivalent to P5's qx//
    macro qTO { 'qq:x:w:to ' }    # qq:x:w:to//
    macro quote:<❰ ❱> ($text) { quasi { {{{$text}}}.quoteharder } }

We can only hope that someone will find enough quality tuits soon to implement this.

Day 13 – String Interpolation and the Zen Slice

December 13, 2014

So you think you know all about string interpolation in Perl 6?

Well, especially coming from Perl 5, you may find some things that do not work exactly as you’re used to. The simple examples all work, as it seems:

my $a = 42;
say 'value = $a'; # value = $a
say "value = $a"; # value = 42

my @a = ^10;
say 'value = @a'; # value = @a
say "value = @a"; # value = @a HUH??

In earlier versions of Perl 5 (or was it Perl 4?), this gave the same result. At one point, it was decided that arrays would be interpolated in double quoted strings as well. However, this caused quite a few problems with double quoted texts with email addresses in them: you would need to escape each @ (if you were using strict, which you of course should have). And if you forgot, and there was an array that just happened to have the same name as the user part of an email address, you would be in for a surprise. Or if you didn’t use strict, you would suddenly have the text without the email address. But then again, you got what you asked for.

So how can we make this work in Perl 6?

Introducing the Zen slice

The Zen slice on an object, returns the object. It’s like you specify nothing, and get everything. So what does that look like?

my @a = ^10;
say "value = @a[]"; # value = 0 1 2 3 4 5 6 7 8 9

You will have to make sure that you use the right indexers for the type of variable that you’re interpolating.

my %h = a => 42, b => 666;
say "value = %h{}"; # value = a 42 b 666

Note that the Zen slice on a hash returns both keys and values, whereas the Zen slice on an array only returns the values. This seems inconsistent, until you realize that you can think of a hash as a list of Pairs.

The Zen slice only really exists at compile time. So you will not get everything if your slice specification is an empty list at runtime:

my @a;
my %h = a => 42, b => 666;
# a slice, but not a Zen slice:
say "value = %h{@a}"; # value =

So the only way you can specify a Zen slice, is if there is nothing (but whitespace) between the slice delimiters.

The Whatever slice

The * ( Whatever ) slice is different. The Whatever will just fill in all keys that exist in the object, and thus only return the values of a hash.

my %h = a => 42, b => 666;
say "value = %h{*}"; # value = 42 666

For arrays, there isn’t really any difference at the moment (although that may change in the future when multi-dimensional arrays are fleshed out more).

Interpolating results from subs and methods

In double quoted strings, you can also interpolate subroutine calls, as long as they start with an ‘&‘ and have a set of parentheses (even if you don’t want to pass any arguments):

sub a { 42 }
say "value = &a()"; # value = 42

But it doesn’t stop at calling subs: you can also call a method on a variable as long as they have parentheses at the end:

my %h = a => 42, b => 666;
say "value = %h.keys()"; # value = b a

And it doesn’t stay limited to a single method call: you can have as many as you want, provided the last one has parentheses:

my %h = a => 42, b => 666;
say "value = %h.perl.EVAL.perl.EVAL.perl()"; # value = ("b" => 666, "a" => 42).hash

Interpolating expressions

If you want to interpolate an expression in a double quoted string, you can also do that by providing an executable block inside the string:

say "6 * 7 = { 6 * 7 }"; # 6 * 7 = 42

The result of the execution of the block, is what will be interpolated in the string. Well, what really is interpolated in a string, is the result of calling the .Str method on the value. This is different from just saying a value, in which case the .gist method is called. Suppose we have our own class with its own .gist and .Str methods:

class A {
    method Str { "foo" }
    method gist { "bar" }
}
say "value = { A }"; # value = foo
say "value = ", A;   # value = bar

Conclusion

String interpolation in Perl 6 is very powerful. As you can see, the Zen slice makes it easy to interpolate whole arrays and hashes in a string.

In this post I have only scratched the surface of string interpolation. Please check out Quoting on Steroids in a few days for more about quoting constructs.

Day 5 – Act with great Responsibility

December 5, 2014

The past year in Rakudo-land has been very exciting. It has brought us a new, full-fledged backend in the form of MoarVM. And with that came extended support for asynchronous execution of code. This means you can now more easily try out all these asynchronous features, without having to suffer from the long startup time of the JVM backend.

The “Supply” is still the main mechanism to use the Reactive Programming paradigm. Here is a simple (silly) example of using a Supply:

1: my $s = Supply.new;
2: $s.act: { print " Fizz" if $_ %% 3 }
3: $s.act: { print " Buzz" if $_ %% 5 }
4: $s.act: { print " $_" unless $_ %% 3 || $_ %% 5 }
5: $s.emit($_) for 1..20;
======================
1 2 Fizz 4 Buzz Fizz 7 8 Fizz Buzz 11 Fizz 13 14 Fizz Buzz 16 17 Fizz 19 Buzz

Let’s take this example apart. Line 1 just creates a new Supply. By itself, it doesn’t do anything at all, except for creating the object, of course. Lines 2, 3 and 4 set up code (closures) to be executed every time a new value arrives in the Supply $s.

Line 5 sends 20 consecutive values to the Supply. This in turn causes the closures to be executed (in turn for each value emitted). The result is a nice FizzBuzz.

But but…

Careful watchers of this code may wonder about two things. First of all: what’s emit ? Is that something new? No, it is not (really). It used to be called more. At the Austrian Perl Workshop last October it was decided to rename this method to emit (as nobody was really happy with any more). So what happens if you have code that uses more? Well, it will still work, but you will get a deprecation warning after your process finishes:

Saw 1 call to deprecated code during execution.
===============================================
Method more (from Supply) called at:
t/spec/integration/advent2014-day05.t, line 13
Deprecated since v2014.10, will be removed with release v2015.10!
Please use emit instead.
-----------------------------------------------

Note that even though the same method was called 20 times, it is only mentioned once. And that’s because all the calls happened at the same location in the code. If you have seen deprecation messages before, you might notice that it now also mentions when it was deprecated, and when the feature will most likely be removed.

The second thing that a careful watcher might wonder about is: why the use of act? Is that a also a rename (from tap)? No, it is not. The act method was added last April. It ensures that your closure will be the only one running on that Supply, even if the emits are coming from different threads. That is a much safer default than tap, where multiple threads can be executing the closure you provided at the same time.

Sharing everything comes with great power and great responsibility

So why is this important? This is important because all variables visible in a specific scope, are readable and changeable by all threads simultaneously. Reading a variable from different threads at the same time, is not an issue in Perl 6 (apart from some minor known issues to be resolved after the Great List Refactor). Changing a variable from multiple threads simultaneously, is an issue that may cause data loss, memory corruption and segfaults. To give an example:

1: my $times = 100000;
2: my $a = 0;
3: $*SCHEDULER.cue: { $a++ } for ^$times; # this may segfault because of unguarded changes
4: sleep 5; # poor man’s way to wait for all code to finish
5: say “Missed {$times - $a} updates”;
===================================
Missed 9 updates

Line 1 sets up the number of iterations we’re going to do. Line 2 initializes the variable that’s going to be incremented. Although ++ is magic, we initialize the variable explicitly, so that each increment will follow the same internal code path. Line 3 cues the increment to be executed that many times (in a very low level way using the underlying scheduler). Line 4 waits for the execution to be finished. And line 5 shows the result (if we didn’t segfault, which we might).

So please use act rather than tap, unless you are sure that closures will not be updating any shared variables simultaneously. Which could be hard to tell if you’re using external modules inside the closure.

FizzBuzzing some more

Coming back to the first example: should we have used act there, or could we have used tap? We could have used tap there, because print is threadsafe. It becomes a different story if we would have used a (shared) array instead of print:

1: my @seen;
2: my $s = Supply.new;
3: $s.act: { @seen.push: "Fizz" if $_ %% 3 }
4: $s.act: { @seen.push: "Buzz" if $_ %% 5 }
5: $s.act: { @seen.push: $_ unless $_%%3 || $_%%5 }
6: await do for 1..20 {
7:     start { rand.sleep; $s.emit($_) }
8: }
9: say @seen;
==========
11 14 4 Fizz 17 19 Fizz Buzz 16 13 Buzz Fizz Buzz 7 Fizz 8 2 Buzz 1 Fizz Fizz

Hmmm… that has the right number of Fizz’s and Buzz’s, but clearly the order is wrong. But because we have used act rather than tap, we are at least sure that no push was executed simultaneously on the @seen array. So we didn’t lose any values, nor did we segfault.

You might ask: what are that lines 6-8 about? Well, that’s making sure the emit’s are done in a random order, spread out in time (because the 0 to 1 second random sleep from rand.sleep, and the start making it run in a separate thread).

So, how could you make this work in a parallel way? Well generally, if you have some kind of identifier for each value emitted, then you can use that to store the result in a shared data-structure. In this case, $_ is exactly that. So with a slight modification:

1: my @seen;
2: my $s = Supply.new;
3: $s.act: { @seen[$_] = "Fizz" if $_ %% 3 }
4: $s.act: { @seen[$_] ~= "Buzz" if $_ %% 5 }
5: $s.act: { @seen[$_] //= $_ }
6: await do for 1..20 {start{rand.sleep;$s.emit($_)}}
7: say @seen[1..20];
=================
1 2 Fizz 4 Buzz Fizz 7 8 Fizz Buzz 11 Fizz 13 14 FizzBuzz 16 17 Fizz 19 Buzz

By assigning directly to an element in the array (lines 3, 4, 5), we automatically put the information in the right place. And because we are (implicitely) guaranteed that act’s on the same item are always executed in the order they were created, we can simplify the non-FizzBuzz case in line 5.

Conclusion

When using Supplies, please use act on the Supply if you’re not sure whether the closure is going to change shared variables. Only use tap if you’re really 100% sure.

Further reading

Last year saw 2 async execution related posts, the latter showing uses of tap. Please read up on them if the above was not telling you all you needed to know:

Tests

The above code can also be found in roast at t/spec/integration/advent2014-day05.t

Day 20 – How to mangle the initial state

December 20, 2013

The past year saw the (at least partial) implementation of several variable traits. But before we get into that, the past year also saw Nil getting implemented closer to spec in Rakudo. If you read the spec, you will know that Nil indicates the absence of a value. This is different from being undefined, as we saw in an earlier blogpost this year. So what does that mean?

$ perl6 -e 'my $a = 42; say $a; $a = Nil; say $a'
42
(Any)

So, assigning Nil will reset a scalar container to its default value, which (by default) is the type object of that container. Which is (Any) if nothing is specifically specified.

Enter the “is default” variable trait

If you want to specify the default value of a variable, you can now use the “is default” variable trait:

$ perl6 -e 'my $a is default(42); say $a; say $a.defined’
42
True

So oddly enough, even though we haven’t assigned anything to the variable, it now has a defined value. So let’s assign something else to it, and then assign Nil:

$ perl6 -e 'my $a is default(42) = 69; say $a; $a = Nil; say $a'
69
42

Of course, this is a contrived example.

“is default” on arrays and hashes

It gets more interesting with arrays and hashes. Suppose you want a Bool array where you can only switch “off” elements by assigning False? That is now possible:

$ perl6 -e 'my Bool @b is default(True); say @b[1000]; @b[1000]=False; say @b[1000]'
True
False

Of course, type checks should be in place when specifying default values.

$ perl6 -e 'my Bool $a is default(42)'
===SORRY!=== Error while compiling -e
Type check failed in assignment to '$a'; expected 'Bool' but got 'Int’

Note that this type check is occurring at compile time. Ideally, this should also happen with arrays and hashes, but that doesn’t happen just yet. Patches welcome!

Using is default on arrays and hashes interacts as expected with :exists (well, at least for some definition of expected :-):

$ perl6 -e 'my @a is default(42); say @a[0]; say @a[0].defined; say @a[0]:exists'
42
True
False

Note that even though each element in the array appears to have a defined value, it does not necessarily exist. Personally, I would be in favour of expanding that to scalar values as well, but for now :exists does not work on scalar values, just on slices.

It’s not the same as specifying the type

What’s wrong with using type objects as default values? Well, for one it doesn’t set the type of the variable. Underneath it is still an (Any) in this case:

$ perl6 -e 'my $a is default(Int) = "foo"; say $a'
foo

So you don’t get any type checking, which is probably not what you want (otherwise you wouldn’t take all the trouble of specifying the default). So don’t do that! Compare this with:

$ perl6 -e 'my Int $a = "foo"'
Type check failed in assignment to '$a'; expected 'Int' but got 'Str'

Which properly tells you that you’re doing something wrong.

Nil assigns the default value

Coming back to Nil: we already saw that assigning it will assign the default value. If we actually specify a default value, this becomes more clear:

$ perl6 -e 'my $a is default(42) = 69; say $a; $a = Nil; say $a'
69
42

In the context of arrays and hashes, assigning Nil has a subtly different effect from using the :delete adverb. After assigning Nil, the element still exists:

$ perl6 -e 'my @a is default(42) = 69; say @a[0]; @a[0] = Nil; say @a[0]:exists'
69
True

Compare this with using the :delete adverb:

$ perl6 -e 'my @a is default(42) = 69; @a[0]:delete; say @a[0]; say @a[0]:exists'
42
False

One could argue that assigning Nil (which assigns the default value, but which is also specced as indicating an absence of value) should be the same as deleting. I would be in favour of that.

Argh, I want to pass on Nil

The result of a failed match, now also returns Nil (one of the other changes in the spec that were implemented in Rakudo in the past year). However, saving the result of a failed match in a variable, will assign the default value, losing its Nilness, Nilility, Nililism (so much opportunity for creative wordsmithing :-).

$ perl6 -e 'my $a = Nil; say $a'
(Any)

It turns out you can specify Nil as the default for a variable, and then It Just Works™

$ perl6 -e 'my $a is default(Nil) = 42; say $a; $a = Nil; say $a'
42
Nil

Introspection

You can use the VAR macro to introspect variables to find out its default value, without having to do something as destructive as assigning Nil to it:

$ perl6 -e 'my @a is default(42); say @a.VAR.default'
42

This also works on system variables like $/:

$ perl6 -e 'say $/.VAR.default'
Nil

And can also be used to interrogate other properties:

$ perl6 -e 'say $/.VAR.dynamic’
True

But this gets us into material for a blog post for another day!

Conclusion

The “is default” variable trait allows you to manipulate the default value of variables and elements in arrays and hashes. Together with the new meaning of Nil, it is a powerful means of mangling initial values of variables to your satisfaction.

Day 12 – Slicing with adverbs, the only way!

December 12, 2013

My involvement with adverbs in Perl 6 began very innocently. I had the idea to creating a small, lightning talk size presentation about how the Perl 5 fat arrow corresponds to Perl 6’s fatarrow and adverbs. And how they relate to hash / array slices. And then had to find out you couldn’t combine them on hash / array slices. Nor could you pass values to them.

And so started my first bigger project on Rakudo Perl 6. Making adverbs work as specced on hashes and arrays, and on the way, expand the spec as well. So, do they now work? Well, all spectests pass. But while preparing this blog post, I happened to find a bug which is now waiting for my further attention. There’s always one more bug.

What are the adverbs you can use with hash and array slices?

name description
:exists whether element(s) exist(ed)
:delete remove element(s), return value (if any)
:kv return key(s) and value(s) as Parcel
:p return key(s) and value(s) as Parcel of Pairs
:k return key(s) only
:v return value(s) only

:exists

This adverb replaces the now deprecated .exists method. Adverbs provide a generic interface to hashes and arrays, regardless of number of elements requested. The .exists method only ever allowed checking for a single key.

Examples speak louder than words. To check whether a single key exists:

$ perl6 -e 'my %h = a=>1, b=>2; say %h<a>:exists’
True

If we expand this to a slice, we get a Parcel of boolean values:

$ perl6 -e 'my %h = a=>1, b=>2; say %h<a b c>:exists'
True True False

Note that if we ask for a single key, we get a boolean value back, not a Parcel with one Bool in it.

$ perl6 -e 'my %h = a=>1, b=>2; say (%h<a>:exists).WHAT’
(Bool)

If it is clear that we ask for multiple keys, or not clear at compile time that we are only checking for one key, we get back a Parcel:

$ perl6 -e 'my %h = a=>1, b=>2; say (%h<a b c>:exists).WHAT’
(Parcel)
$ perl6 -e 'my @a="a"; my %h = a=>1, b=>2; say (%h{@a}:exists).WHAT'
(Parcel)

Sometimes it is handier to know if something does not exist. You can easily do this by negating the adverb by prefixing it with !: they’re really just like named parameters anyway!

$ perl6 -e 'my %h = a=>1, b=>2; say %h<c>:!exists'
True

:delete

This is the only adverb that actually can make changes to the hash or array it is (indirectly) applied to. It replaces the now deprecated .delete method.

$ perl6 -e 'my %h = a=>1, b=>2; say %h<a>:delete; say %h.perl'
1
("b" => 2).hash

Of course, you can also delete slices:

$ perl6 -e 'my %h = a=>1, b=>2; say %h<a b c>:delete; say %h.perl'
1 2 (Any)
().hash

Note that the (Any) is the value returned for the non-existing key. If you happened to have given the hash a default value, it would have looked like this:

$ perl6 -e 'my %h is default(42) = a=>1, b=>2; say %h<a b c>:delete; say %h.perl'
1 2 42
().hash

But the behaviour of the is default maybe warrants a blog post of itself, so I won’t go into it now.

Like with :exists, you can negate the :delete adverb. But there wouldn’t be much point, as you might have well not specified it at all. However, since adverbs are basically just named parameters, you can make the :delete attribute conditional:

$ perl6 -e 'my $really = True; my %h = a=>1, b=>2; say %h<a b c>:delete($really); say %h.perl'
1 2 (Any)
().hash

Because the value passed to the adverb was true, the deletion actually took place. However, if we pass a false value:

$ perl6 -e ‘my $really; my %h = a=>1, b=>2; say %h<a b c>:delete($really); say %h.perl'
1 2 (Any)
("a" => 1, "b" => 2).hash

It doesn’t. Note that the return value did not change: the deletion was simply not performed. This can e.g. be very handy if you have a subroutine or method doing some kind of custom slice, and you want to have an optional parameter indicating whether the slice should be deleted as well: simply pass that parameter as the adverb’s value!

:kv, :p, :k, :v

These 4 attributes modify the returned values from any hash / array slice. The :kv attribute returns a Parcel with keys and values interspersed. The :p attribute returns a Parcel of Pairs. The :k and :v attributes return the key only, or the value only.

$ perl6
> my %h = a => 1, b => 2;
("a” => 1, "b” => 2).hash
> %h<a>:kv
a 1
> %h<a>:p
"a" => 1
> %h<a>:k
a
> %h<a>:v
1

Apart from modifying the return value, these attributes also act as a filter for existing keys only. Please note the difference in return values:

> %h<a b c>
1 2 (Any)
> %h<a b c>:v
1 2

Because the :v attribute acts as a filter, there is no (Any). But sometimes, you want to not have this behaviour. To achieve this, you can negate the attribute:

> %h<a b c>:k
a b
> %h<a b c>:!k
a b c

Combining adverbs

You can also combine adverbs on hash / array slices. The most useful combinations are with one or two of :exists and :delete, with zero or one of :kv, :p, :k, :v. Some examples, like putting a slice out of one hash into a new hash:

$ perl6 -e 'my %h = a=>1, b=>2; my %i = (%h<a c>:delete:p).list; say %h.perl; say %i.perl'
("b” => 2).hash
("a” => 1).hash

Or the keys that were actually deleted:

$ perl6 -e 'my %h = a=>1, b=>2; say %h<a b c>:delete:k’
a b

We actually have a spec that describes which combinations are valid, and what they should return.

Arrays are not Hashes

Apart from hashes using {} for slices, and arrays [] for slices, the adverbial syntax for hash and array slices are the same. But there are some subtle differences. First of all, the “key” of an element in an array, is its index. So, to show the indexes of elements in an array that have a defined value, one can use the :k attribute:

$ perl6 -e 'my @a; @a[3] = 1; say @a[]:k'
3

Or, to create a Parcel with all elements in an array:

$ perl6 -e 'my @a; @a[3] = 1; say @a[]:!k’
0 1 2 3

However, deleting an element from an array, is similar to assigning Nil to it, so it will return its default value (usually (Any)):

$ perl6 -e 'my @a = ^10; @a[3]:delete; say @a[2,3,4]; say @a[2,3,4]:exists'
2 (Any) 4
True False True

If we have specified a default value for the array, the result is slightly different:

$ perl6 -e 'my @a is default(42) = ^10; @a[3]:delete; say @a[2,3,4]; say @a[2,3,4]:exists'
2 42 4
True False True

So, even though the element “does not exist”, it can return a defined value! As said earlier, that may become a blog post for another day!

Conclusion

Slices with adverbs are a powerful way of handling your data structures, be they hashes or arrays. It will take a while to get used to all of the combinations of adverbs that can be specified. But once you’re used to them, they provide you with a concise way of dicing and slicing your data that would previously have involved more elaborate structures with loops and conditionals. Of course, if you want to, you can still do that: it’s not illegal to program Perl 5 in Perl 6 :-)

Day 07 – Bagging the changes in the Set specification

December 7, 2013

In 2012, colomon++ implemented most of the then Set / Bag specification in Niecza and Rakudo, and blogged about it last year.

Then in May this year, it became clear that there was a flaw in the implementation that prohibited creating Sets of Sets easily. In June, colomon++ re-implemented Sets and Bags in Niecza using the new views on the spec. And I took it upon myself to port these changes to Rakudo. And I was in for a treat (in the Bertie Bott’s Every Flavour Beans kind of way).

Texan versus (non-ASCII) Unicode

Although the Set/Bag modules were written in Perl 6, there were some barriers to conquer: it was not as simple as a copy-paste operation. First of all, all Set operators were implemented in their Unicode form in Niecza foremost, with the non-Unicode (ASCII) versions (aka Texan versions) implemented as a dispatcher to the Unicode version. At the time, I was mostly developing in rakudo on Parrot. And Parrot has this performance issues with parsing code that contains non-ASCII characters (at any place in the code, even in comments). Therefore, the old implementation of Sets and Bags in Rakudo, only had the Texan versions of the operators. So I had to carefully figure out which Unicode operator in the Niecza code (e.g. ) matched which Texan operator (namely (<=)) in the Rakudo code, and make the necessary changes.

Then I decided: well, why don’t we have Unicode operators for Sets and Bags in Rakudo either? I mentioned this on the #perl6 channel, and I think it was jnthn++ who pointed out that there should be a way to define Unicode operators without actually having to use Unicode characters. After trying this, and having jnthn++ fix some issues in that area, it was indeed possible.

So how does that look?

  # U+2286 SUBSET OF OR EQUAL TO
  only sub infix:<<"\x2286">>($a, $b --> Bool) {
      $a (<=) $b;
  }

One can only say: Yuck! But it gets the job done. So now one can write:

  $ perl6 -e 'say set( <a b c> ) ⊆ set( <a b c d> )'
  True

Note that Parcels (such as <a b c>) are automatically upgraded to Sets by the set operators. So one can shorten this similar statement to:

  $ perl6 -e 'say <a b c> ⊆ <a b d>'  # note missing c
  False

Of course, using the Unicode operator in Rakudo comes at the expense of an additional subroutine call. But that’s for some future optimizer to take care of.

Still no bliss

But alas, the job was still not done. The implementation using Hash in Rakudo, would not allow Sets within Sets yet still. It would look like it worked, but that was only because the stringification of a Set was used as a key in another set. So, when you asked for the elements of such a Set of Sets, you would get strings back, rather than Sets.

Rakudo allows objects to be used as keys (and still remain objects), by mixing in the TypedHash role into a Hash, so that .keys returns objects, rather than strings. Unfortunately, using the TypedHash role is only completely functional for user programs, not when building the Perl 6 internals using Perl 6, as is done in the core settings. Bummer.

However, the way TypedHash is implemented, could also be used for implementing Sets and Bags. For Sets, there is simply an underlying Hash, in which the key is the .WHICH value of the thing in the set, and the value is the thing. For Bags, that became a little more involved, but not a lot: again, the key is the .WHICH of the thing, and the value is a Pair with the thing as the key, and the count as the value. So, after refactoring the code to take this into account as well, it seemed I could finally consider this job finished (at least for now).

It’s all about values

Then, the nitty gritty of the Set spec struck again.

  "they are subject to === equality rather than eqv equality"

What does that mean?

  $ perl6 -e 'say <a b c> === <a b c>'
  False
  $ perl6 -e 'say <a b c> eqv <a b c>'
  True

In other words, because any Set consisting of <a b c> is identical to any other Set consisting of <a b c>, you would expect:

  $ perl6 -e 'say set(<a b c>) === set(<a b c>)'
  False

to return True (rather than False).

Fortunately, there is some specification that comes in aid. It’s just a matter of creating a .WHICH method for Sets and Bags, and we’re set. So now:

  $ perl6 -e 'say set(<a b c>) === set(<a b c>)’
  True

just works, because:

  $ perl6 -e 'say set(<a b c>).WHICH; say set(<a b c>).WHICH'
  Set|Str|a Str|b Str|c
  Set|Str|a Str|b Str|c

shows that both sets, although defined separately, are really the same.

Oh, and some other spec changes

In October, Larry invoked rule #2 again, but those changes were mostly just names. There’s a new immutable Mix and mutable MixHash, but you could consider those just as Bags with floating point weights, rather than unsigned integer weights. Creating Mix was mostly a Cat license job.

Conclusion

Sets, Bags, Mixes, and their mutable counterparts SetHash, BagHash and MixHash, are now first class citizens in Perl 6 Rakudo. So, have fun with them!

Day 05 – Changes in specification and operational fallout

December 4, 2013

Perl 6 has become much more stable in the past year. There have however been some potentially disrupting changes to the Perl 6 specification, followed by implementation changes to adhere to that new spec.

bless() changes

One of the most visible changes is the removal of an object candidate in bless(). If you wanted to call bless() yourself in your code, rather than supplying your own BUILD() method, you needed to provide an object candidate as the first parameter. Over the years, this turned out to basically always be * (as in Whatever). Which is pretty useless and an obstacle for future optimisations. So TimToady invoked rule #2 to remove that first parameter.

The changes to calls to bless() in the core setting were implemented by moritz++. For those Perl 6 modules in the wild, a warning was added:

 Passing an object candidate to Mu.bless is deprecated

The first parameter would then be removed and execution would continue.

This has the disadvantage of generating a warning every time you create an object of that class with the deprecated call to bless(). So there must be a better way to do this!

Enter “is DEPRECATED”

It turns out there is a better way. Already in June 2012, pmichaud++ added an “is DEPRECATED” routine trait that did nothing until earlier this year when I decided to add some functionality to it. Initially it was just a warn, but that just had the same annoying quality as the warning with bless().

Since the idea behind the “is DEPRECATED” trait was not specced yet, I figured I could turn it any way I wanted, unless I would not be forgiven by the #perl6 crowd. So I re-used an idea I had had at former $work, already years ago. Instead of warning at the moment a transgression is spotted, it feels better, especially for these types of deprecations, to just remember where these transgressions take place. Only when the program is finished, report the transgressions that were spotted (on STDERR).

One of the other standard methods that has been deprecated in Perl 6, is ucfirst(). One should use the tc() (for “title case”) method instead. So what happens if you do call ucfirst()? That is easily demonstrated with a one-liner:

$ perl6 -e 'say "foo".ucfirst; say "done"'
Foo
done
Saw 1 call to deprecated code during execution.
================================================================================
Method ucfirst (from Cool) called at:
  -e, line 1
Please use 'tc' instead.
--------------------------------------------------------------------------------
Please contact the author to have these calls to deprecated code adapted,
so that this message will disappear!

After this has been live in the repo for a while, and spectested, and since nobody on #perl6 complained, I decided to spec this behavior not only for routines, but also for attributes and classes. Unfortunately, the latter ones have not been implemented yet (although you can already specify the traits). But there is a patch -p1 coming up, which should give me some quality time to look at this.

So why is bless(*) not properly DEPRECATED

Indeed. Why? Simply because I missed it. So I just fixed this: nothing like blog-driven development! So this one-liner now says:

$ perl6-p -e 'class A { method new { self.bless(*) } }; say A.new'
A.new()
Saw 1 call to deprecated code during execution.
================================================================================
Method bless (from Mu) called at:
  -e, line 1
Please use a call to bless without initial * parameter instead.
--------------------------------------------------------------------------------
Please contact the author to have these calls to deprecated code adapted,
so that this message will disappear!

So how do you specify the text?

The “is DEPRECATED” trait currently takes one parameter: the string to be shown between Please use and instead. If you don’t specify anything, the text something else will be assumed. Since that is not entirely useful, it is advised to always specify a text that makes sense in that context. Additional parameters may be added in the future to allow for more customisation, but so far they have not been needed.

Conclusion

Perl 6 will continue to evolve. Changes will still be made. To not break early adopter’s code, any non-compatible changes in the implementation of Perl 6 can be marked as deprecated without interfering with the execution of existing programs much. Perl 6 module authors can do the same should they feel the need to change the API of their modules.


Follow

Get every new post delivered to your Inbox.

Join 45 other followers