Here at the end of megayears of human adventure,
we schedule a silent night to stop time,
once again awaiting the next advent
of the answer to our questionable venture.
Are we there yet?
After a gigayear or so of slogging from slime to silicon
mercifully forgetting (most of) the unmerciful past,
and the scars left by unnatural nature upon our pedigree,
we now remember to remember the future once more.
Are we there yet?
So in this month, 26 year-moments after the advent of Perl,
(including 13 year-eternities of precocious brat sisterhood)
our little family celebrates, 24 tales at a time,
its victories in the struggle to find our way home.
Are we there yet, Daddy, are we there yet?
We follow after all those who wander but are not lost:
We follow Abraham, looking for a city because it isn't there yet;
We follow Strider, guarding the hobbits who will redistribute the future;
We follow Magi and wizard, scholar and explorer, saint and scientist.
I wonder as I wander out under the sky...whether I'm lost yet...
But wise man or hobbit, we must all take that journey in the dark,
groping ahead for the path to better air and a little hope,
following the encapsulated starlight past monsters and chasms
out to a land where the weary can rest, and be healed of grief.
"Wait, what do you mean, I can't go there?" —Gandalf
We must all wander in this desert for forty years,
burying the bones of naysayers and yaysayers alike,
so that their children can someday cross the Jordan
into a land flowing with milk and honey and fancy new phones!
"Wait, what do you mean, I can't go there?" —Moses
We carry these old stories to the future,
cadences to chant over the confusion of the road,
backpacks full of epics, pockets stuffed with tales,
leaving our own litter of anecdotal evidence behind us.
You haven't heard some of the good ones yet.
So kids, along with the old stories, pack a few new tools,
light but powerful tools that will help you and help you help us.
The lightest tools, the most powerful tools, are ideas,
so pack lots and lots of 'em. I'll wait here while you do.
I'm here yet. Which means I'm not there yet. Hurry up!
Pick some good friends, and let some good friends pick you.
Take turns waiting patiently, running impatiently,
or walking hopefully, crawling hopelessly,
standing up yet again defiantly. Or woozily, that works too.
Be the protagonist some of the time, yet not all of the time.
Trust your journey to provide you with new companions;
trust your new companions to provide you with your journey.
Be prepared to say your eternal hellos and temporary goodbyes.
(No one's ever ready for the temporary hellos and eternal goodbyes.)
And I'm not sure I want to be there quite yet.
Enjoy the companions your journey gives today, for life is bittersweet.
Enjoy the bittersweet songs and the bittersweet beer.
Enjoy the bitter fights and sweet hugs.
And, yes, enjoy the resulting bruises, but not too much.
Enjoy knowing that you're not there yet.
Welcome, my friends, to the here, and to the not-there-yet.
Welcome to the clan's quantum superposition of joy and grief and longing.
Welcome to our ongoing effort to steal more of that Promethean fire
that burns too fast yet never fast enough to fit the firepits of our lives.
Are we getting warmer yet?
As they say, "Give a man a fire..." Hold that thought, some breaking news...
This just in: Fire from heaven is now free and open-sourced?! Well, huh...
Seems a blogger heard some angels singing popup advent adverts in the cloud?
Hmm...better do some fact checking...hang in there...tum tiddly tum...
Darn flakey connection...almost there...
Well, hey, whaddya know?! The physicists figured it out.
The whole universe has just finished compiling without error...
Now they're looking for someone to debug the silly thing;
Hey, I know, I'll just use the Perl 6 test suite.
[...now you have thousands of problems...]
Did you say something?
(louder) Does sanity test #1 pass yet? What's the output?
The road goes ever on and on,
Over the river and through the woods,
You take the high road, and I'll take the low road,
We're all bound for the Promised Land.
[TimToady gets blessed and starts directing the choir of Perl Pilgrims.]
We're marching to Zion,
Beautiful, beautiful Zion,
We're marching upward to Zion, that beautif—
You can't go there.
Wait, what do you mean, I can't go there?
Bugfix #1: kill all the bad poets.
Aw...crap...
Category 2013
Day 23 – Unary Sort
Most languages or libraries that provide a generic sort routine allow you to specify a comparator, that is a callback that tells the sort routine how two given elements compare. Perl is no exception.
For example in Perl 5, which defaults to lexicographic ordering, you can request numeric sorting like this:
use v5;
my @sorted = sort { $a <=> $b } @values;Perl 6 offers a similar option:
use v6;
my @sorted = sort { $^a <=> $^b }, @values;The main difference is that the arguments are not passed through the global variables $a and $b, but rather as arguments to the comparator. The comparator can be anything callable, that is a named or anonymous sub or a block. The { $^a <=> $^b} syntax is not special to sort, I have just used placeholder variables to show the similarity with Perl 5. Other ways to write the same thing are:
my @sorted = sort -> $a, $b { $a <=> $b }, @values;
my @sorted = sort * <=> *, @values;
my @sorted = sort &infix:«<=>», @values;The first one is just another syntax for writing blocks, * <=> * use * to automatically curry an argument, and the final one directly refers to the routine that implements the <=> "space ship" operator (which does numeric comparison).
But Perl strives not only to make hard things possible, but also to make simple things easy. Which is why Perl 6 offers more convenience. Looking at sorting code, one can often find that the comparator duplicates code. Here are two common examples:
# sort words by a sort order defined in a hash:
my %rank = a => 5, b => 2, c => 10, d => 3;
say sort { %rank{$^a} <=> %rank{$^b} }, 'a'..'d';
# ^^^^^^^^^^ ^^^^^^^^^^ code duplication
# sort case-insensitively
say sort { $^a.lc cmp $^b.lc }, @words;
# ^^^^^^ ^^^^^^ code duplicationSince we love convenience and hate code duplication, Perl 6 offers a shorter solution:
# sort words by a sort order defined in a hash:
say sort { %rank{$_} }, 'a'..'d';
# sort case-insensitively
say sort { .lc }, @words;sort is smart enough to recognize that the code object code now only takes a single argument, and now uses it to map each element of the input list to new values, which it then sorts with normal cmp sort semantics. But it returns the original list in the new order, not the transformed elements. This is similar to the Schwartzian Transform, but very convenient since it's built in.
So the code block now acts as a transformer, not a comparator.
Note that in Perl 6, cmp is smart enough to compare strings with string semantics and numbers with number semantics, so producing numbers in the transformation code generally does what you want. This implies that if you want to sort numerically, you can do that by forcing the elements into numeric context:
my @sorted-numerically = sort +*, @list;And if you want to sort in reverse numeric order, simply use -* instead.
The unary sort is very convenient, so you might wonder why the Perl 5 folks haven't adopted it yet. The answer is that since the sort routine needs to find out whether the callback takes one or two arguments, it relies on subroutine (or block) signatures, something not (yet?) present in Perl 5. Moreover the "smart" cmp operator, which compares number numerically and strings lexicographically, requires a type system which Perl 5 doesn't have.
I strongly encourage you to try it out. But be warned: Once you get used to it, you'll miss it whenever you work in a language or with a library that lacks this feature.
Day 22 – A catalogue of operator types
Perl 6 has a very healthy relationship to operators. "An operator is just a funnily-named subroutine", goes the slogan in the community.
In practice, it means you can do this:
sub postfix:<!>($N) {
[*] 2..$N;
}
say 6!; # 720
Yeah, that’s the prototypical factorial example. I promise I won’t do it any more in this post. I swear the only reason we don’t have factorial as a standard operator in the language, is so that we can impress people by defining it.
Anyway, so a postfix operator ! is really just a "funnily-named" subroutine postfix:<!>. Similarly, we have prefix and infix operators, like prefix:<-> and infix:<*>. They’re all just subroutines. I wrote about that before, so I’m not going to hammer on that point.
Operators have different precedence (like infix:<*> binds tighter than infix:<+>)…
$x + $y * $z # compiler sees $x + ($y * $z) $x * $y + $z # compiler sees ($x * $y) + $z
…and different associativity (like infix:</> associates to the left but infix:<=> associates to the right).
$x / $y / $z # compiler sees ($x / $y) / $z $x = $y = $z # compiler sees $x = ($y = $z)
But I wrote about that before too, at quite some length, so I’m not going to revisit that topic.
No, today I just want to talk about the operator categories in general. I think Perl 6 does a nice job of describing the operator types themselves. I don’t see that in many other languages.
Here are all the different types:
type position syntax ====== ========== ======== prefix before a term !X infix between two terms X ! Y postfix after a term X! circumfix around [X] postcircumfix after & around X[Y]
A lot of other languages give you the ability to define your own operators. Many of them provide approaches which are hackish afterthoughts, and only allow you to override existing operators. Some other languages do approach the problem head-on, but end up simplifying the language to decrease the complexity of defining new operators.
Perl 6 approaches, and solves, the problem, head-on. You get all of the above operators, you can refine or override old ones, and you can talk within the language about things like precedence and associativity.
All that is rather nice. But, as usual, Perl 6 goes one step further and starts classifying metaoperators, too.
What’s a metaoperator? That’s our name for when you can extend an operator in a certain way. For example, many languages allow some kind of not-equal operator like infix:<!=>. Perl 6 has another one for string non-equality: infix:<ne>. And then maybe you want to do smart non-matching: infix:<!~~>. And so on — pretty soon you catch on to the pattern: you may, at one point or another, want to invert the result of any boolean matcher. So that’s what Perl 6 provides.
Here are a few examples:
normal op negated op ========= ========== eq !eq ( synonym of ne ) ~~ !~~ < !< ( synonym of >= ) before !before
Because this particular metaoperator attaches itself before an infix operator, it gets the name infix_prefix_meta_operator:<!>. I was going to say "and yes, you can add your own user-defined metaoperators too", but currently no implementation supports that. Maybe sometime in the future.
There are many other categories of metaoperators. For example the "multiplication reducer" [*] that we used in the factorial example at the top (which takes a list of numbers and multiplies them all together) is really an infix:<*> surrounded by the metaop prefix_circumfix_meta_operator:sym<[ ]>. (It’s "prefix" because it goes before the list of numbers.)
Luckily, we don’t have meta-meta-ops. Already thinking about the metaops is quite a challenge sometimes. But what I like about Perl 6 and the approach it takes to grammar and operators is this: someone did think about it, long and hard. The resulting system has a pleasing simplicity to it.
Perl 6 is very well-suited for building parsers for languages. As one of its neatest tricks, it takes this expressive power and directs it towards the task of defining itself. As a Perl 6 user, you’re given not just an excellent toolbox, but a well-organized workshop to adapt and extend to fit your needs.
Day 21 – Signatures
In today’s post, we’ll go through the basics of Perl 6’s subroutine signatures, which allow us to declare our parameters instead of coding them as we do in Perl 5.
In Perl 5, arguments to a sub are accessible via the @_ array, so if you want to access an argument to a sub, you can either access the array directly (which is typically only done where speed is a concern)…
use v5;
sub greet {
print "Hello, " . @_[0] . "\n";
}
Or, more commonly, pull off any arguments as lexicals:
use v5;
sub greet {
my $name = shift @_;
print "Hello, $name\n";
}
Positionals
Perl 6 lets you declare required positional arguments for your subs as part of the signature:
use v6;
sub greet($name) {
print "Hello, $name\n";
}
Inside the sub, $name is available as a readonly alias to the passed in variable.
By default, parameters are required. If you try to call this function with no parameters as greet, you’ll get a compile time error:
===SORRY!===
CHECK FAILED:
Calling 'greet' requires arguments (line 1)
Expected: :($name)
You can make the parameter optional by adding a ? to the signature. Then you’ll have to examine the parameter to see if a defined value was passed in. Or, you can declare a default value to be used if none is passed in:
use v6;
sub guess($who, $what?) {
# manually check to see if $what is defined
}
sub dance($who, $dance = "Salsa") {
...
}
dance("Rachael")
dance("Rachael", "Watusi")
Another way to handle optional arguments is to use multis. Multis are subs that can have multiple implementations that differ by their signature. When invoking a multi, the argument list used in the invocation is used to determine which version of the multi to dispatch to.
use v6;
multi sub dance($who, $dance) {
say "$who is doing the $dance";
}
multi sub dance($who) {
dance($who, "Salsa");
}
Types
The parameters in the previous section allow any type of value to be passed in. You can declare that only certain types are valid:
sub greet(Str $name) {
say "hello $name";
}
Now hello("joe") will work as it did before. But hello(3) will generate a compile time error:
===SORRY!===
CHECK FAILED:
Calling 'greet' will never work with argument types (int) (line 1)
Expected: :(Str $name)
In addition to any of Perl 6 builtin types or user-built classes, you can also add programmatic checks inline in the signature with where clauses.
use v6;
multi odd-or-even(Int $i where * %% 2) { say "even" };
multi odd-or-even(Int $i) { say "odd"};
Note that we didn’t have to add a where clause to the second sub. The multi-dispatch algorithm will prefer the more detailed signature when it matches, but fallback to the less descriptive version when it doesn’t.
You can even use literal values as arguments. This style allows you to move some of your logic to the multi dispatcher.
use v6;
multi fib(1) { 1 }
multi fib(2) { 1 }
multi fib(Int $i) { fib($i-1) + fib($i-2) }
say fib(10);
Note that this isn’t very efficient… yet. Once the is cached trait for subs is implemented, we can use that to provide a builtin analog to Perl 5’s Memoize module.
Named
Perl 6 provides for named parameters; if the positionals are analogous to an array of parameters, the named arguments are like a hash. You use a preceding colon in the argument declaration, and then pass in a pair for each parameter when invoking the sub.
use v6;
sub doctor(:$number, :$prop) {
say "Doctor # $number liked to play with his $prop";
}
doctor(:prop("cricket bat"), :number<5>);
doctor(:number<4>, :prop<scarf>);
Note that the order the parameters are passed in doesn’t matter for named parameters.
If you have a variable of the same name in the calling scope, you can simplify the calling syntax to avoid creating an explicit pair.
use v6;
my $prop = "fez";
my $number = 11;
doctor(:$prop, :$number)
You can also use different names internally (using the expanded pair syntax) for the named arguments. This allows you to use different (simplified or sanitized) versions for the caller.
use v6;
sub doctor(:number($incarnation), :prop($accoutrement)) {
say "Doctor # $incarnation liked to play with his $accoutrement";
}
my $number = 2;
my $prop = "recorder";
doctor(:$number, :$prop)
Slurpy
To support functions like sprintf, we need to be able to take a variable number of arguments. We call these args “slurpy”, since they “slurp” up the arguments.
# from rakudo's src/core/Cool.pm
sub sprintf(Cool $format, *@args) {
...
}
When invoking this sub, the first argument must be of type Cool (kind of a utility supertype), and all the remaining positional arguments are slurped up into the @args variable. The preceding * indicates the slurp.
Symmetrically, we can also slurp up named arguments into a hash.
# from rakudo's src/core/control.pm
my &callwith := -> *@pos, *%named {
...
}
This snippet introduces to the pointy block syntax for anonymous subs. The -> begins the declaration, followed by the signature (without parentheses), and finally the sub definition in the block. The signature here shows all the positionals ending up in *@pos, and all the named arguments in %named.
This also shows that positionals and named arguments can be combined in the same sub.
Methods
Method and sub declarations are virtually identical. All the parameters mentioned so far are usable in methods.
The main difference is that methods can be passed an invocant (the object associated with the method call), and when you define the sub, you have the opportunity to name it. It must be the first parameter if present, and is marked with a trailing :.
use v6;
method explode ($self: $method) {...}
Note that there is no comma separating these the invocant and the first positional. In this context, the colon functions as a comma.
Parameter Traits
Each parameter can additionally specify a trait that changes the behavior of that parameter:
is readonly– this is the default behavior for a parameter; the sub cannot modify the passed in value.is rw– the argument can be modified. Forces the argument to be required.is copy– the sub gets a modifiable copy of the original value.
More Information
Check out Synopsis #6 for more information, and the roast test suite for more examples – any of the directories starting with S06-.
Day 20 – How to mangle the initial state
The past year saw the (at least partial) implementation of several variable traits. But before we get into that, the past year also saw Nil getting implemented closer to spec in Rakudo. If you read the spec, you will know that Nil indicates the absence of a value. This is different from being undefined, as we saw in an earlier blogpost this year. So what does that mean?
$ perl6 -e 'my $a = 42; say $a; $a = Nil; say $a' 42 (Any)
So, assigning Nil will reset a scalar container to its default value, which (by default) is the type object of that container. Which is (Any) if nothing is specifically specified.
Enter the “is default” variable trait
If you want to specify the default value of a variable, you can now use the “is default” variable trait:
$ perl6 -e 'my $a is default(42); say $a; say $a.defined’ 42 True
So oddly enough, even though we haven’t assigned anything to the variable, it now has a defined value. So let’s assign something else to it, and then assign Nil:
$ perl6 -e 'my $a is default(42) = 69; say $a; $a = Nil; say $a' 69 42
Of course, this is a contrived example.
“is default” on arrays and hashes
It gets more interesting with arrays and hashes. Suppose you want a Bool array where you can only switch “off” elements by assigning False? That is now possible:
$ perl6 -e 'my Bool @b is default(True); say @b[1000]; @b[1000]=False; say @b[1000]' True False
Of course, type checks should be in place when specifying default values.
$ perl6 -e 'my Bool $a is default(42)' ===SORRY!=== Error while compiling -e Type check failed in assignment to '$a'; expected 'Bool' but got 'Int’
Note that this type check is occurring at compile time. Ideally, this should also happen with arrays and hashes, but that doesn’t happen just yet. Patches welcome!
Using is default on arrays and hashes interacts as expected with :exists (well, at least for some definition of expected :-):
$ perl6 -e 'my @a is default(42); say @a[0]; say @a[0].defined; say @a[0]:exists' 42 True False
Note that even though each element in the array appears to have a defined value, it does not necessarily exist. Personally, I would be in favour of expanding that to scalar values as well, but for now :exists does not work on scalar values, just on slices.
It’s not the same as specifying the type
What’s wrong with using type objects as default values? Well, for one it doesn’t set the type of the variable. Underneath it is still an (Any) in this case:
$ perl6 -e 'my $a is default(Int) = "foo"; say $a' foo
So you don’t get any type checking, which is probably not what you want (otherwise you wouldn’t take all the trouble of specifying the default). So don’t do that! Compare this with:
$ perl6 -e 'my Int $a = "foo"' Type check failed in assignment to '$a'; expected 'Int' but got 'Str'
Which properly tells you that you’re doing something wrong.
Nil assigns the default value
Coming back to Nil: we already saw that assigning it will assign the default value. If we actually specify a default value, this becomes more clear:
$ perl6 -e 'my $a is default(42) = 69; say $a; $a = Nil; say $a' 69 42
In the context of arrays and hashes, assigning Nil has a subtly different effect from using the :delete adverb. After assigning Nil, the element still exists:
$ perl6 -e 'my @a is default(42) = 69; say @a[0]; @a[0] = Nil; say @a[0]:exists' 69 True
Compare this with using the :delete adverb:
$ perl6 -e 'my @a is default(42) = 69; @a[0]:delete; say @a[0]; say @a[0]:exists' 42 False
One could argue that assigning Nil (which assigns the default value, but which is also specced as indicating an absence of value) should be the same as deleting. I would be in favour of that.
Argh, I want to pass on Nil
The result of a failed match, now also returns Nil (one of the other changes in the spec that were implemented in Rakudo in the past year). However, saving the result of a failed match in a variable, will assign the default value, losing its Nilness, Nilility, Nililism (so much opportunity for creative wordsmithing :-).
$ perl6 -e 'my $a = Nil; say $a' (Any)
It turns out you can specify Nil as the default for a variable, and then It Just Works™
$ perl6 -e 'my $a is default(Nil) = 42; say $a; $a = Nil; say $a' 42 Nil
Introspection
You can use the VAR macro to introspect variables to find out its default value, without having to do something as destructive as assigning Nil to it:
$ perl6 -e 'my @a is default(42); say @a.VAR.default' 42
This also works on system variables like $/:
$ perl6 -e 'say $/.VAR.default' Nil
And can also be used to interrogate other properties:
$ perl6 -e 'say $/.VAR.dynamic’ True
But this gets us into material for a blog post for another day!
Conclusion
The “is default” variable trait allows you to manipulate the default value of variables and elements in arrays and hashes. Together with the new meaning of Nil, it is a powerful means of mangling initial values of variables to your satisfaction.
Day 19 – Perl 6 Supplies Reactive Programming
Several days back, we took a look at promises and channels. Promises provided a synchronization mechanisms for asynchronous things that produced a single result, while channels were ideal for setting up producer/consumer style workflows, with producers and consumers able to work in parallel. Today we’ll take a look at a third mechanism aimed at introducing asynchrony and coping with concurrency: supplies!
Synchronous = Pull, Asynchronous = Push
One of the most important features of promises is the then method. It enables one or more things to be run whenever the asynchronous work the promise represents is completed. In a sense, it’s like a publish/subscribe mechanism: you call then to subscribe, and when the work is done then the notification is published. This happens asynchronously. You don’t sit around waiting for the promise to complete, but instead just say what to do when this takes place. Thinking about the way data flows, values are pushed along to the next step of the process.
This is rather different to things like sub and method calls. There, we make the call, then we block until it has completed. Iterating over a lazy list is the same: you can’t progress with the iteration until the next value is available. Thus, iteration is really about pulling stuff from a source. So, if a promise can be thought of as something that can push a single values out as it becomes available, do we have something that can push a whole stream of values outwards, as they are produced over time?
Supplies! We do!
A while back, it was realized that the observer pattern is the mathematical dual of the iterator pattern. Why is this exciting? Quite simply, because it means that all the things you can sensibly do with something you can iterate (map, grep, zip, etc.), you can also sensibly do with something you can observe. Out of this was born reactive programming, and the Rx (Reactive Extensions) library, which has now been ported to many platforms. In Perl 6, we’re providing support for this in core.
The basics
Let’s start out simple. First, we create a new Supply:
my $measurements = Supply.new;
We can then tap the supply, passing a closure that should be called whenever a value is made available:
$measurements.tap(-> $value {
say "Measured: $value";
});
Finally, we produce some values:
$measurements.more(1.5); $measurements.more(2.3); $measurements.more(4.6);
On each of these calls, the closure we tapped the supply with is invoked. Just as we can call then many times, so we can tap many times too:
$measurements.tap(-> $value {
say "Also measured: $value";
});
Now, when we produce a value:
$measurements.more(2.8);
Both of the closures tapping the supply will be called. Note that tap returns an object which can be used to express you’re no longer interested in the supply, essentially turning that tap off.
Note that we didn’t introduce any asynchrony so far. However, supplies are built for it. You can safely have multiple threads supplying values. By default, the supplying thread is used to execute the taps.
Enter the combinators!
Since supplies are essentially a thread-safe observer implementation, we can define many of the same things on them as we’re used to having on lists. For example, imagine we just wanted to tap high measurements. Well, we just re-use knowledge from how we’d filter a list: using grep!
$measurements.grep(* > 4).tap(-> $value {
say "HIGH: $value";
});
Calling grep on a supply produces another supply, just as calling grep on a list gives another list. We could, if we wished, store it in a variable and tap this derived supply many times, grep it again, map it, etc.
Supply factories
There are ways to get supplies besides simply creating them directly. The Supply class has various factory methods that create various interesting kinds of supply, while introducing asynchrony. For example, interval gives a supply that, when tapped, will produce an ascending integer once per time interval.
my $secs = Supply.interval(1);
$secs.tap(-> $s { say "Started $s seconds ago" });
sleep 10;
Factories can also help map between paradigms. The Supply.for method produces a supply that, when tapped, will iterate the specified (potentially lazy) list and push the values out to the tapper. It does the iteration asynchronously. While it’s not implemented yet, we’ll be able to define a similar mechanism for taking a Channel and tapping each value that is received.
Crossing the streams
Some of the most powerful – and tricky to implement – combinators are those that involve multiple supplies. For example, merge gives a single supply whose values are those of the two other supplies it tapped, and zip pairs together values from two different supplies. These are tricky to implement because it’s entirely possible that two different threads will be supplying values. Thankfully, though, we just need to manage this once inside of the supplies implementation, and save those using them from worrying about the problem! In a sense, combinators on lists factor out flow control, while combinators on supplies factor out both flow control and synchronization. Both let us program in a more declarative style, getting the imperative clutter out of our code.
Let’s bring all of this together with an example from one of my recent presentations. We simulate a situation where we have two sets of readings coming in: first, measurements from a belt, arriving in batches of 100, which we need to calculate the mean of, and second another simpler value arriving once every 5 seconds. We want to label them, and get a single supply merging these two streams of readings together. Here’s how it can be done:
my $belt_raw = Supply.interval(1).map({ rand xx 100 });
my $belt_avg = $belt_raw.map(sub (@values) {
([+] @values) / @values
});
my $belt_labeled = $belt_avg.map({ "Belt: $_" });
my $samples = Supply.interval(5).map({ rand });
my $samples_labeled = $samples.map({ "Sample: $_" });
my $merged = $belt_labeled.merge($samples_labeled);
$merged.tap(&say);
sleep 20;
Notice how it’s not actually that much harder than code that maps and greps lists we already had – and yet we’re managing to deal with both time and concurrently arriving data.
The future
Supplies are one of the most recently implemented things in Rakudo, and what’s done so far works on Rakudo on the JVM. With time, we’ll flesh out the range of combinators and factories, keep growing our test coverage, and deliver this functionality on Rakudo on MoarVM too, for those who don’t want to use the JVM.
Day 18 – A Grammar with duplicate checking
Today’s example constructs a grammar for tracking playing cards in a single deal. We’ll say it’s poker with one or more players and that each player is being dealt a hand that contains exactly five cards.
There is, however, the need to detect duplicate cards. We’ll need some way of tracking cards both within each card-hand and between hands.
A simple Card Game Grammar
To start with, here’s the basic grammar (no duplicate checks yet):
grammar CardGame {
rule TOP { ^ <deal> $ }
rule deal {
<hand>+ % ';'
}
rule hand { [ <card> ]**5 }
token card {<face><suit>}
proto token suit {*}
token suit:sym<♥> {<sym>}
token suit:sym<♦> {<sym>}
token suit:sym<♣> {<sym>}
token suit:sym<♠> {<sym>}
token face {:i <[2..9]> | 10 | j | q | k | a }
}
say CardGame.parse("2♥ 5♥ 7♦ 8♣ 9♠");
say CardGame.parse("2♥ a♥ 7♦ 8♣ j♥");
The top-level rule consists of a deal. The deal consists of one or more hands separated by ';'. Each hand consists of 5 playing cards.
Each card is represented by a face, one of: a (ace), j (jack), q (queen) or k (king), or 2 – 10. This is followed by a suite: ♥ (hearts) ♦ (diamonds) ♣ (clubs) or ♠ (spades).
[We could have used the playing cards characters, newly introduced in Unicode 6.0, but these aren’t widely supported yet].
As expected, the first cut of the grammar cheerily parses any hand:
say CardGame.parse("a♥ a♥ 7♦ 8♣ j♥");
# one hand, duplicate a♥
say CardGame.parse("a♥ 7♥ 7♦ 8♣ j♥; 10♥ j♥ q♥ k♥ a♥");
# two hands, duplicate j♥
Detecting Duplicates
We start by adding a Perl 6 variable declaration to the grammar. This will be used to track cards:
rule deal {
:my %*PLAYED = ();
<hand>+ % ';'
}This declares %*PLAYED [1]. The '%*' twigil indicates that it’s a hash '%' and that’s dynamically scoped '*'.
Dynamic scoping is not only for subroutine and method calls [1]. It also works seamlessly with grammar rules, tokens and actions.
Being dynamically scoped, %*PLAYED is available to callees of the deal rule; the hand token, and its callee, the card token.
It’s also available to any actions, that then get called. So we can track and report on duplicates by creating an action class with a method for the card token:
class CardGame::Actions {
method card($/) {
my $card = $/.lc;
say "Hey, there's an extra $card"
if %*PLAYED{$card}++;
}
}
my $a = CardGame::Actions.new;
say CardGame.parse("a♥ a♥ 7♦ 8♣ j♥", :actions($a));
# "Hey there's an extra a♥"
say CardGame.parse("a♥ 7♥ 7♦ 8♣ j♥; 10♥ j♥ q♥ k♥ a♦",
:actions($a));
# "Hey there's an extra j♥"
And that might be all that’s needed for tracking and reporting on duplicates. There’s a pretty good separation between the declarative grammar and procedural actions, with just one dynamically scoped hash variable.
Disallowing Duplicates
But I had a situation where I wanted duplicate checking to be a parsing constraint. Parsing needed to fail when duplicates were encountered.
I achieved this by moving the duplicate check grammar side:
token card {<face><suit>
<?{
# only allow each card to appear once
my $card = $/.lc;
say "Hey, there's an extra $card"
if %*PLAYED{$card};
! %*PLAYED{$card}++;
}>
}
This has introduced a code assertion between the <?{ and }> [2]. The rule succeeds when the code evaluates to a True value. The card token thus fails when the same card is detected more than once in a single deal.
say CardGame.parse("2♥ 7♥ 2♦ 3♣ 3♦");
# legitimate, parses
say CardGame.parse("a♥ a♥ 7♦ 8♣ j♥");
# fails with message: Hey, there's an extra a♥
say CardGame.parse("a♥ 7♥ 7♦ 8♣ j♥; 10♥ j♥ q♥ k♥ a♦");
# fails with message: Hey, there's an extra j♥
Discussion/Conclusion
One thing to be careful of with this type of technique is back-tracking (trying of alternatives). If, for instance, the grammar was modified in such a way that the card token could be called more than once for single input card, then we might erroneously report a duplicate. It’s still possible to track, but becomes a bit more involved. The simplest answer is to keep the grammars as simple as possible and minimize back-tracking.
If in any doubt, please consider using one or more of the Grammar::Debugger, Grammar::Tracer [3] or the debugger [4] modules [5] to track what’s going on. You can also insert debugging code into tokens or rules as closures: { say "here" } [6].
That the exercise for today; a simple Perl 6 Grammar to parse playing-cards in a card-game, but with duplicate checks using either actions or code assertions.
Day 17 – Of a new contributor
If you’re anything like me, you’ve read last year’s advent calendar posts with delight, squeeing a little bit at the neat things you could express so easily with Perl 6. You might have wondered – like I have – if all those sundry features are just shiny bells and whistles that were added on top of an okay language. And you might be wondering how to find out more, dig deeper, and maybe even contribute.
As you can tell from the fact that I’m writing this article, I’ve decided to become a contributor to the Perl 6 effort. I’m going to tell you how I got interested and then involved (and inloved) with Perl 6.
Before my involvement with Perl 6 I mostly used Python. However beautiful and flexible Python is, I worked on a project where it was a poor fit and writing proper test code was exceptionally uncomfortable. Thus, I often made mistakes – misused my data structures, passed values of incorrect types to methods – that I felt the language should be able to detect early without sacrificing flexibility. The “gradual typing” approach of Perl 6 sounded like a very good fit to me.
Having a friend show me bits and pieces of Perl 6 quickly led to looking at the Advent Calendar last year. I also joined the IRC channel and asked a whole bunch of questions. Not having done any Perl 5 programming before made figuring out the nooks and crannies of Perl 6 syntax a bit harder than I would have liked, especially when using the Perl 6 book. Fortunately, the IRC channel was always active and eager to help.
After having learnt a bit more about Perl 6, I quickly started helping out here and there. In part because I’ve already enjoyed doing that for PyPy – which is implemented in a subset of Python, much like Rakudo is implemented in NQP – but also because I kept hitting little problems and bugs.
So, how do you actually get to contributing?
Well, starting small is always good. Come to the IRC channel and ask for “low hanging fruit”. Check out bugs blocking on test coverage or easy tickets from the Perl 6 bug tracker. Try porting or creating one of the most wanted modules. Experiment with the language and look what bugs you hit. One thing that’s always good and often not very hard is fixing “LTA error messages”; those are error messages that are “less than awesome”.
At some point you’ll find something you’d like to fix. For me, the first thing was making spectests for bugs that were already fixed, but didn’t have a test yet. After that, giving a more helpful error message when a programmer tries to concatenate strings with . instead of ~. Then came fixing the output of nested Pair objects to have proper parenthesis. And then I dug deep into the Grammar, Actions and World classes to implement a suggestion mechanism for typos in variables, classes and subs.
The fact that most of your potential contributions will likely be done either in Perl 6 code or at least in NQP code makes it rather easy to get started, since even NQP is fairly high-level. And if you work through the materials from the Rakudo and NQP Internals Workshop 2013, you’ll get a decent head start in understanding how Rakudo and NQP work.
Whenever you get stuck, the people in #perl6 – including me, of course – will be happy to help you out and give advice along the way.
Let’s try to tackle a simple “testneeded” bug. this bug about parsing :$<a> pair shorthand syntax looks simple enough to write a test case for.
Here’s the executive summary:
The :$foo syntax has already been introduced a few days ago by the post on adverbly adverbs. The syntax here is a combination of the shorthand notation $<foo> to mean $/<foo>, which refers to the named match group “foo” from the regex we just matched.
So :$<foo> is supposed to give the pair "foo" => $<foo>. It didn’t in the past, but someone came along and fixed it. Now all we need to do is create a simple test case to make sure Rakudo doesn’t regress and other implementations don’t make the same mistake.
In the last comment of the discussion, jnthn already wrote a much shortened test case for this bug, hidden behind a “show quoted text” link:
'abc' ~~ /a $<a>=[\w+]/; say :$<a>.perl
Since sometimes it happens that such a test case causes a compilation error on one of the implementations, it’s useful to be able to “fudge” the whole thing in one go. That means writing something like #?niecza skip in front of a block. That’s why we wrap our whole little test in curly braces, like so:
# RT #76998
{
my $res = do { 'abc' ~~ /a $<a>=[\w+]/; :$<a> };
ok $res ~~ Pair, ':$<a> returns a pair';
ok $res.key eq 'a', 'its key is "a"';
ok $res.value ~~ Match:D, 'the pair's value is a defined match object';
}
So what have we done here? We executed the code in a do block, so that we can grab the value of the last piece of the block. The next three lines inspect the return value of our code block: Is it a Pair object? Is its key “a”, like expected? Is its value a defined Match object instance? (See day 2’s post about the :D smiley).
Also, we wrapped the whole test in a big block and put a comment on top to mention the ticket ID from the bug tracker. That way, it’s easier to see why this test exists and we can reference the commit to the perl6/roast repository in the ticket discussion when we close it (or have someone close it for us).
Next up, we have to “bump up the plan”, which means we go to the beginning of the test file and increase the line that calls “plan” with a number by 3 (which is exactly how many more ok/not ok outputs we expect our test case to generate).
Then we can either fork perl6/roast on github and make a pull-request from our change or submit a git format-patch created patch via IRC, the bug tracker, or the mailing lists.
Note that I didn’t commit anything I wrote here yet. If you’re faster than the other readers of this blog, you can feel free to go ahead and submit this. If you’re not, feel free to select a different ticket to improve upon.
Finally, if you apply for a commit bit – meaning the right to directly commit to the repositories on github – make sure to mention I sent you to get 50% off of your first 10 commits; what an incredible deal!
Day 16 – Slangs
use v6;
my $thing = "123abc";
say try $thing + 1; # this will fail
{
use v5;
say $thing + 1 # will print 124
}
Slangs are pretty interesting things in natural languages, so naturally they will be pretty awesome in computer languages as well. Without it the cross-language communication is like talking through a thin pipe, like it is when calling C functions. It does work, but calling functions is not the only nor the most comfortable thing out there.
The example above shows that we create a variable in Perl 6 land and use it in a nested block, which derives from another language. (This does only work if the nested language is capable of handling the syntax, a dollar-sigilled variable in this case.)
We use this other language on purpose: it provides a feature that we need to solve our task.
I hope that slangs will pop up not just to provide functionality to solve a given problem, but also help in writing the code in a way that fits the nature of that said problem.
How does that even work?
The key is that the module that lets you switch to the slang provides a grammar and appropriate action methods. That is not different from how Perl 6 is implemented itself, or how JSON::Tiny works internally.
The grammar will parse all statements in our nested block, and the actions are there to translate the parsed source code (text) into something a compiler can handle better: usually abstracted operations in form of a tree, called AST.
The v5 slang compiles to QAST, which is the name of the AST that Rakudo uses. The benefit of that approach is that this data structure is already known by the guts of the Rakudo compiler. So our slang would just need to care about translating the foreign source code text into something known. The compiler takes this AST then and maps it to something the underlying backend understands.
So it does not matter if we’re running on Parrot, on the JVM or something else, the slang’s job is done when it produced the AST.
A slang was born.
In March this year at the GPW2013, I felt the need for something that glues both Perl 6 and Perl 5 together. There were many nice people that shared this urge, so I investigated how to do so.
Then I found a Perl 5 parser in the std repository. Larry Wall took the Perl 6 parser years ago and modified it to be Perl 5 conform. The Perl 6 parser it is based on is the very same that Rakudo is built upon. So the first step was to take this Perl 5 grammar, take the action methods from Rakudo, and try to build something that compiles.
(In theory this is all we needed: grammar + action = slang.)
I can’t quite remember whether it took one week or two, but then there was a hacked Rakudo that could say “Hallo World”. And it already insisted on putting parens around conditions for example. Which might be the first eye catcher for everyone when looking at both languages.
Since then there was a lot of progress in merging in Perl 5’s test suite, and implementing and fixing things, and making it a module rather than a hacked standalone Rakudo-ish thing.
Today I can proudly say that it passes more than 4600 of roughly 45000 tests. These 4600 passing tests are enough so you can play with it and feed it simple Perl 5 code. But the main work for the next weeks and months is to provide the core modules so that you can actually use a module from CPAN. Which, after all, was the main reason to create v5.
What is supported at the moment?
- all control structures like loops and conditions
- functions like shift, pop, chop, ord, sleep, require, …
- mathematical operations
- subroutine signatures that affect parsing
- pragmas like vars, warnings, strict
- core modules like Config, Cwd and English
The main missing pieces that hurt are:
- labels and goto
- barewords and pseudo filehandles
- many many core modules
Loop labels for next LABEL, redo LABEL and last LABEL will land soon in rakudo and v5. The other missing parts will take their time but will happen :o).
The set goals of v5:
- write Perl 5 code directly in Perl 6 code, usually as a closure
- allow Perl 6 lexical blocks inside Perl 5 ones
- make it easy to use variables declared in an outer block (outer means the other language here)
- provide the behaviour of Perl 5 operators and built-ins for v5 blocks only, nested Perl 6 blocks should not be affected
- and of course: make subs, packages, regexes, etc available to the other language
All of the statements above are already true today. If you do a numeric operation it will behave differently in a v5 block than a Perl 6 block like the example at the top shows. That is simply because in Perl 6 the + operator will dispatch to a subroutine called &infix:<+>, but in a v5 block it translates to &infix:<P5+>.
Oversimplified it looks a bit like this:
Perl 6/5 code:
1 + 2;
{
use v5;
3 + 4
}
Produced AST:
- QAST::CompUnit
- QAST::Block 1 + 2; { use v5; 3 + 4 }
- QAST::Stmts 1 + 2; { use v5; 3 + 4 }
- QAST::Stmt
- QAST::Op(call &infix:<+>) +
- QAST::IVal(1)
- QAST::IVal(2)
- QAST::Block { use v5; 3 + 4 }
- QAST::Stmts use v5; 3 + 4
- QAST::Stmt
- QAST::Op(call &infix:<P5+>)
- QAST::IVal(3)
- QAST::IVal(4)
The nice thing about this is that you can use foreign operators (of the used slang) in your Perl 6 code. Like &prefix:<P5+>("123hurz") would be valid Perl 6 code that turn a string into a number even when there are trailing word characters.
To get v5 you should follow its README, but be warned, at the moment this involves recompiling Rakudo.
Conclusion: When was the last time you’ve seen a language you could extend that easily? Right. I was merely astonished how easy it is to get started. Next on your TODO list: the COBOL slang. :o)
Day 15 – Numbers and ways of writing them
Consider the humble integer.
my $number-of-dwarfs = 7; say $number-of-dwarfs.WHAT; # (Int)
Integers are great. (Well, many of them are.) Computers basically run on integers. Also, God made the integers, and everything else is basically cheap counterfeit knock-offs, smuggled into the Platonic realm when no-one’s looking. If they’re so important, we’d expect to have lots of good ways of writing them in any respectable programming language.
Do we have lots of ways of writing integers in Perl 6? We do. (Does that make Perl 6 respectable? No, because it’s a necessary but not sufficient condition. Duh.)
One thing we might want to do, for example, is write our numbers in other number bases. The need for this crops up now and then, for example if you’re stranded on Binarium where everyone has a total of two fingers:
my $this-is-what-we-humans-call-ten = 0b1010; # 'b' for 'binary', baby my $and-your-ten-is-our-two = 0b10;
Or you may find yourself on the multi-faceted world of Hexa X, whose inhabitants (due to an evolutionary arms race involving the act of tickling) sport 16 fingers each:
my $open-your-mouth-and-say-ten = 0xA; # 'x' is for, um, 'heXadecimal' my $really-sixteen = 0x10;
If you’re really unlucky, you will find yourself stuck in a file permissions factory, with no other means to signal the outer world than by using base 8:
my $halp-i'm-stuck-in-this-weird-unix-factory = 0o644;
(Fans of other C-based languages may notice the deviation from tradition here: for your own sanity, we no longer write the above number as 0644 — doing so rewards you with a stern (turn-offable) warning. Maybe octal numbers were once important enough to merit a prefix of just 0, but they ain’t no more.)
Of course, just these bases are not enough. Sometimes you will need to count with a number of fingers that’s not any of 2, 8, or 16. For those special occasions, we have another nice syntax for you:
say :3<120>; # 15 (== 1 * 3**2 + 2 * 3**1 + 0 * 3**0) say :28<aaaargh>; # 4997394433 say :5($n); # let me parse that in base 5 for you
Yes, that’s the dear old pair syntax, here hijacked for base conversions from a certain base. You will recognize this special syntax by the fact that it uses colonpairs (:3<120>), not the "fat arrow" (3 => 120), and that the key is an integer. For the curious, the integer goes up to 36 (at which point we’ve reached ‘z’ in the alphabet, and there’s no natural way to make up more symbols for "digits").
I once used :4($_) to translate DNA into proteins. Perl 6 — fulfilling your bioinformatics dreams!
Oh, and sometimes you want to convert into a certain base, essentially taking a good-old-integer and making it understandable for a being with a certain amount of fingers. We’ve got you covered there, too!
say 0xCAFEBABE; # 3405691582 say 3405691582.base(16); # CAFEBABE
Moving on. Perl 6 also has rationals.
my $tenth = 1/10; say $tenth.WHAT; # (Rat)
Usually computers (and programming languages) are pretty bad at representing numbers such as a tenth, because most of them are stranded on the planet Binarium. Not so Perl 6; it stores your rational numbers exactly, most of the time.
say (0, 1/10 ... 1); # 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 say (1/3 + 1/6) * 2; # 1 say 1/10 + 1/10 + 1/10 == 0.3; # True (\o/)
The rule here is, if you give some number as a decimal (0.5) or as a ratio (1/2), it will be represented as a Rat. After that, Perl 6 does its darnedest to strike a balance between representing numbers without losing precision, and not losing too much performance in tight loops.
But sometimes you do reach for those lossy, precision-dropping, efficient numbers:
my $pi = 3.14e0; my $earth-mass = 5.97e24; # kg say $earth-mass.WHAT; # (Num)
You get floating-point numbers by writing things in the scientific notation (3.14e0, where the e0 means "times ten to the power of 0"). You can also get these numbers by doing some lossy calculation, such as exponentiation.
Do keep in mind that these are not exact, and will get you into trouble if you treat them as exact:
say (0, 1e-1 ... 1); # misses the 1, goes on forever say (0, 1e-1 ... * >= 1); # 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1 (oops) say 1e0/10 + 1/10 + 1/10 == 0.3; # False (awww!)
So, don’t do that. Don’t treat them as exact, that is.
You’re supposed to be able to write lists of numbers really neatly with the good old Perl 6 quote-words syntax:
my @numbers = <1 2.0 3e0>; say .WHAT for @numbers; # (IntStr) (RatStr) (NumStr)
But this, unfortunately, is not implemented in Rakudo yet. (It is implemented in Niecza.)
If you’re into Mandelbrot fractals or Electrical Engineering, Perl 6 has complex numbers for you, too:
say i * i == -1; # True say e ** (i * pi) + 1; # 0+1.22464679914735e-16i
That last one is really close to 0, but the exponentiation throws us into the imprecise realm of floating-point numbers, and we lose a tiny bit of precision. (But what’s a tenth of a quadrillionth between friends?)
my $googol = EVAL( "1" ~ "0" x 100 ); say $googol; # 1000…0 (exactly 100 zeros) say $googol.WHAT; # (Int)
Finally, if you have a taste for the infinite, Perl 6 allows handing, passing, and storage of Inf as a constant, and it compares predictably against numbers:
say Inf; # Inf say Inf.WHAT; # (Num) say Inf > $googol; # True my $max = -Inf; $max max= 5; # (max= means "store only if greater") say $max; # 5
Perl 6 meets your future requirements by giving you alien number bases, integers and rationals with excellent precision, floating point and complex numbers, and infinities. This is what has been added — now go forth and multiply.
Raku Advent Calendar (2009-2019) 