Archive for the ‘2013’ Category

Day 14 – Asynchronous Programming: Promises and Channels

December 14, 2013

Some of the most exciting progress in Perl 6 over the last year has been in the area of asynchronous, concurrent and parallel programming. In this post, we’ll take a look at two of the language features that relate to this: promises and channels. But first…

A Little Design Philosophy

Threads and locks are the assembly language of parallel programming. In the spirit of “make the hard things possible”, Perl 6 does let you spawn a thread and provide you with a Lock primitive. But these are absolutely aimed at those doing the hard things. I’ve written, code-reviewed and taught parallel programming in languages where these were the primary primitives for a while. Doing code reviews was often a fairly depressing affair. It’s not just that there were bugs, it’s that often it felt like the approach taken by the code’s author was, “just throw locks all over the place and all will be well”.

In this post, I’ll focus on the things we have in Perl 6 to help make the easy things easy. They are designed around a number of principles:

  • The paradigms we provide should have a strong focus on being composable, to make it easy to extend, re-use and refactor code
  • Furthermore, it should be easy to compose the various paradigms together, as well as having ways to move between the synchronous and asynchronous worlds where needed
  • Both asynchrony and synchronization should be explicit, happen at clearly defined boundaries, and be done at a fairly high level

In general, the Perl 6 approach is that you achieve concurrency by decomposing a problem into many pieces, communicating through the provided synchronization mechanisms (those in the language, and no doubt a bunch of extra ones that will be provided by the module ecosystem over time). The approach is not about mutating shared memory. That’s decidedly in the “hard things possible” category. The fact that it’s really hard to get right is the main problem, but from a performance perspective, lots of threads competing to write to the same bit of memory is the worst case for CPU caches – which really matter these days.

Promises

A promise is a synchronization primitive for a piece of asynchronous work that will produce a single result at some point in the future, or fail to do so because something went wrong. Different languages have evolved different terms for this idea, or use the terms with different nuances. Both “future” and “task” are often used.

The easiest way to create a promise is:

my $p10000 = start {
    (1..Inf).grep(*.is-prime)[9999]
}

This schedules the work in the block to be done. By default, this means it will be scheduled to run on a pool of threads. Thus, start introduces asynchrony into a program. We continue by executing the next line of code, and the work we specified will be done on another thread. If it runs to completion and produces a result, we say that the promise was kept. If, by contrast, it dies by throwing an exception, then we say the promise was broken.

So, what can you do with a promise? Well, you can ask it for the result:

say $p10000.result;

This blocks until the promise is kept or broken. If it is kept, the value it produced is returned. If it’s broken, the exception is thrown. There’s a neater way to write this:

say await $p10000;

This may take many promises, and so you can do things like:

my @quotes = await @currency_exchanges.map(-> $ex { start { $ex.get_quote($val) } });

Although this will throw an exception if any of them fail. Thus, we may wish to wait on all of them, then just extract those that produced a result:

my @getting = @currency_exchanges.map(-> $ex { start { $ex.get_quote($val) } });
await Promise.allof(@getting);
my @quotes = @getting.grep(*.status == Kept).map(*.result);

There’s something a little interesting in there: allof. This is an example of a promise combinator: something that takes one or more promises as its arguments and creates some kind of composite promise that relates to them. And this brings us to the next interesting and important thing: a promise need not be backed by a piece of asynchronously executing code! For example, we can create a promise that will be kept after a certain amount of time has elapsed:

my $kept_in_10 = Promise.in(10);

Thus, we might provide a basic timeout mechanism, making sure any exchange that doesn’t give us a result in 5 seconds doesn’t get blocked on:

my @getting = @currency_exchanges.map(-> $ex { start { $ex.get_quote($val) } });
await Promise.anyof(Promise.allof(@getting), Promise.in(5));
my @quotes = @getting.grep(*.status == Kept).map(*.result);

Of course, sitting around and waiting for results is just one thing we can do with a promise. We can also provide things that should be done upon the promise being completed. These will also be scheduled and run asynchronously. For example:

my $p10000 = start {
    (1..Inf).grep(*.is-prime)[9999]
}
my $base16 = $p10000.then(sub ($res) {
    $res.result.base(16)
});
my $pwrite = $base16.then(sub ($res) {
    spurt 'p10000.txt', $res.result;
    return 'p10000.txt';
});

Here, we use then in order to specify something that should be done after the promise is kept or broken. This also returns a promise, meaning you can chain another operation into the process. And you can call then multiple times on one promise too, giving a kind of one-off publish/subscribe mechanism (see a future article on supplies for a much richer way to do this kind of thing, however). Note that promise takes care internally to make sure races work out OK (for example, the work being done in the promise is already completed by the time we call then).

You can also create your own promises, keeping or breaking them as you desire. This is as simple as:

# Create the promise.
my $p = Promise.new;

# Take the "vow" object, used to keep/break it.
my $v = $p.vow;

# Later, one of...
$v.keep($result);
$v.break($exception_or_message);

Thus, you can write your own promise factories and combinators too.

Channels

A promise is OK for conveying a single result, but what about producer/consumer scenarios where the producer will produce many values over time, and the consumer will process them as they are available? This is where a channel can come in useful.

Let’s say we want to read in a bunch of INI configuration files, parse each one using a grammar, and then flatten the configuration results into a single hash. There are three distinct steps here, in a producer/consumer relationship, which we can do in parallel. While the final result is a single value, and so a promise feels suitable, there are many files to read and parse. This is where channels come in. Let’s explore them using this example.

First, here is the top level of the program:

sub MAIN() {
    loop {
        my @files = prompt('Files: ').words;
        read_all(@files);
    }
}

This prompts the user for a bunch of filenames, then calls read_all. This is a little more interesting:

sub read_all(@files) {
    my $read = Channel.new;
    my $parsed = Channel.new;
    read_worker(@files, $read);
    parse_worker($read, $parsed) for 1..2;
    my %all_config = await config_combiner($parsed);
    say %all_config.perl;
}

This creates two channels, $read and $parsed. The $read channel will be used by read_worker in order to send the contents of each of the files it reads in along to the parse_worker. Here is read_worker:

sub read_worker(@files, $dest) {
    start {
        for @files -> $file {
            $dest.send(slurp($file));
        }
        $dest.close();
        CATCH { $dest.fail($_) }
    }
}

It uses the send method in order to send along the contents of each file it slurps. After slurping them all, it calls last on the channel to indicate there will be no more. The CATCH block calls fail on the channel to indicate that the producer failed. This will, when reached, throw an exception in the consumer. A channel that has had last or fail called on it can no longer be used to send values. Finally, the whole thing is wrapped in a start block so it is done on a thread in the thread pool.

The parse_worker is a little more interesting:

sub parse_worker($source, $dest) {
    my grammar INIFile {
        token TOP {
            ^
            <entries>
            <section>+
            $
        }

        token section {
            '[' ~ ']' <key> \n
            <entries>
        }

        token entries {
            [
            | <entry> \n
            | \n
            ]*
        }

        rule entry { <key> '=' <value> }

        token key   { \w+ }
        token value { \N+ }

        token ws { \h* }
    }

    my class INIFileActions {
        method TOP($/) {
            my %result;
            %result<_> = $<entries>.ast;
            for @<section> -> $sec {
                %result{$sec<key>} = $sec<entries>.ast;
            }
            make %result;
        }

        method entries($/) {
            my %entries;
            for @<entry> -> $e {
                %entries{$e<key>} = ~$e<value>;
            }
            make %entries;
        }
    }

    start {
        loop {
            winner $source {
                more $source {
                    if INIFile.parse($_, :actions(INIFileActions)) -> $parsed {
                        $dest.send($parsed.ast);
                    }
                    else {
                        $dest.fail("Could not parse INI file");
                        last;
                    }
                }
                done $source { last }
            }
        }
        $dest.close();
        CATCH { $dest.fail($_) }
    }
}

It starts off with a grammar and actions class for INI files. We then sit in a loop, watching the $source channel, which is the one that read_worker is placing results in. If a channel has one more value available, then the more block will be called. Inside it, $_ will contain the slurped contents of an INI file. We then parse it, and provided this worked out send along the hash of hashes representing the INI file’s content (sections at the top level, then key/value pairs). Again, we take care to call fail and last appropriately.

Finally, config_combiner takes each of those hash of hashes, and does the work to combine them into a single hash. It uses a promise to convey the final, single, result.

sub config_combiner($source) {
    my $p = Promise.new;
    my $v = $p.vow;
    start {
        my %result;
        loop {
            winner $source {
                more $source {
                    for %^content.kv -> $sec, %kvs {
                        for %kvs.kv -> $k, $v {
                            %result{$sec}{$k} = $v;
                        }
                    }
                }
                done $source { last }
            }
        }
        $v.keep(%result);
        CATCH { $v.break($_) }
    }
    return $p;
}

And there we have it: a program using promises and channels happily together, in a producer/consumer, map/reduce style.

Day 13 – Roasting Rakudo Star

December 13, 2013

Roasting Rakudo Star

When is Perl 6 going to be Ready? We get this question a lot in the Perl 6 community, and the answer is never
as simple as we or the inquirers would like.

One part of the answer involves the specification; When we have an implementation that passes all of the tests marked “perl 6.0”, that will be a Perl 6.

Many people think of the specs as the Synopses, but Patrick Michaud makes a good point that the specification is really more about the tests.

Thus it was recognized early on (in Synopsis 1) that acceptance tests provide a far more objective measure of specification conformance than an English description. There are likely things that need to be “spec” that cannot be fully captured by testing… but I still believe that the test suite should be paramount.

Every language feature must have corresponding spec tests. Trying to find a test? Tests are broken up first by Synopsis (which themselves follow the numbering scheme of the Camel chapters), with multiple directories broken out by a group of features, then individual tests. For example, Synopsis 4 (S04) is about Blocks and Statements, including phasers. So to find the tests for the BEGIN phaser, you’ll want S04-phasers/begin.t in the roast suite.

We call the specification tests roast to follow in the tradition of “smoke test”, and also because TimToady can’t resist a punny backronym: “Repository Of All Spec Tests’.

Each of these files tries to thoroughly test something from the Synopsis including a lot of edge cases that aren’t necessarily mentioned in the prose. This goes to Patrick’s point about the tests being the more canonical answer about what the spec is.

Are we there yet?

Just a little further… 

So, how can we use these tests to determine if we’re done?

Each time a developer adds a feature or a language designer documents something in the Synopses, the team must add corresponding tests in roast – each change in the Synopses text may potentially increase the gap between prose and tests, and we have to regularly verify that both sets of documents are in agreement.

The tests even turn out to inform the prose, because they are concrete code – if you cannot write a coherent test, not just because it isn’t implemented in one of the compilers yet, but because it’s inefficient or breaks other tests, this will in turn require changes to the Synopses. Over the course of Perl 6′s development, compiler authors have pushed back on the specification in this way.

Before the compiler author checks in any code, ideally they should run the full spectest suite (roast) to insure not only their new tests work, but also that nothing else broke. This can be time consuming, so it’s possible they might just run a few test files for that particular feature or Synopsis.

So, despite best efforts, failing tests might be introduced. Even if you run all of roast, something might work on your compiler, but might have problems on another. Or another VM, or another OS, or hardware, or… So, there’s a need for regular testing that’s outside of the normal code/test/commit (or test/code/commit, if you like) workflow.

Speaking of other compilers and VMs, the current landscape of Perl 6 compilers is dominated by Rakudo. It has the most passing tests, two functioning backends (parrot and the JVM), a third on the way (MoarVM), and a possible fourth (JavaScript) landing in 2014. All of the virtual machines here support a wide variety of actual hardware and OSes. Niecza is another compiler, implemented in C#. It passes a substantial number of tests. We also test Pugs (targeting Haskell), but only for historical reasons.

Roasting

It can take a while to build and run the full roast suite for even a single compiler, and we are trying to keep track of between four and six at the moment (And that’s just for one architecture!) So, with limited infrastructure, rather than have a continual integration, we setup a single daily run that builds the latest version of every compiler from a fresh checkout using a shared copy of roast (so we can compare like to like), and then saves out the information into a github repository so we can see the current state. Github provides a nice interface for viewing the CSV data.

So, every day, we get a list of which test files are failing for each compiler/backend, which of the compilers is in the lead. When Rakudo/JVM started passing more spectests than Rakudo/Parrot, we were able see that immediately on the daily run.  Given the historical data available in the github repository, one could easily chart out things like (click to embiggen)

sixchart

number of passing tests per compiler/backend on the first of each month during 2013

Ecosystem 

But that’s just the compiler. The Rakudo team bundles a distribution called Rakudo Star (also spelled Rakudo *) that includes many modules from the Perl 6 ecosystem – kind of a mini-CPAN. This source distribution includes everything you need to build Rakudo from scratch and get a bunch of usable modules. Right now it’s Rakudo/Parrot, but work is being done for the distribution to support Rakudo/JVM and other backends.

We’ve had issues in the past where modules didn’t keep up with spec changes, and the person cutting the Star release would find issues with modules just before a release, causing delays.

Now we have a daily test that builds Rakudo Star, using the latest version of every module, Rakudo, NQP, and runs all the module tests, allowing us to catch any deprecation warnings or test errors as soon as they are made, rather than when we’re trying to cut a release. It’s plain text at the moment, but functions well as a warning indicator.

Test to the Future

Two other projects are in place for testing:

  • Colomon has an ad hoc process that tests all the modules in the ecosystem, not just star.
  • japhb has created a benchmark suite to help us prevent performance regressions across the various compilers. Here’s a video presentation.

Going forward, we need to setup and encourage the use of a smoke server so that we can take the daily runs on the testing platform, and combine them with the results from other platforms, compilers, etc.

Drop by the #perl6 channel on freenode or leave a comment here if you want to chat more about testing!

[Coke]

 

Day 12 – Slicing with adverbs, the only way!

December 12, 2013

My involvement with adverbs in Perl 6 began very innocently. I had the idea to creating a small, lightning talk size presentation about how the Perl 5 fat arrow corresponds to Perl 6’s fatarrow and adverbs. And how they relate to hash / array slices. And then had to find out you couldn’t combine them on hash / array slices. Nor could you pass values to them.

And so started my first bigger project on Rakudo Perl 6. Making adverbs work as specced on hashes and arrays, and on the way, expand the spec as well. So, do they now work? Well, all spectests pass. But while preparing this blog post, I happened to find a bug which is now waiting for my further attention. There’s always one more bug.

What are the adverbs you can use with hash and array slices?

name description
:exists whether element(s) exist(ed)
:delete remove element(s), return value (if any)
:kv return key(s) and value(s) as Parcel
:p return key(s) and value(s) as Parcel of Pairs
:k return key(s) only
:v return value(s) only

:exists

This adverb replaces the now deprecated .exists method. Adverbs provide a generic interface to hashes and arrays, regardless of number of elements requested. The .exists method only ever allowed checking for a single key.

Examples speak louder than words. To check whether a single key exists:

$ perl6 -e 'my %h = a=>1, b=>2; say %h<a>:exists’
True

If we expand this to a slice, we get a Parcel of boolean values:

$ perl6 -e 'my %h = a=>1, b=>2; say %h<a b c>:exists'
True True False

Note that if we ask for a single key, we get a boolean value back, not a Parcel with one Bool in it.

$ perl6 -e 'my %h = a=>1, b=>2; say (%h<a>:exists).WHAT’
(Bool)

If it is clear that we ask for multiple keys, or not clear at compile time that we are only checking for one key, we get back a Parcel:

$ perl6 -e 'my %h = a=>1, b=>2; say (%h<a b c>:exists).WHAT’
(Parcel)
$ perl6 -e 'my @a="a"; my %h = a=>1, b=>2; say (%h{@a}:exists).WHAT'
(Parcel)

Sometimes it is handier to know if something does not exist. You can easily do this by negating the adverb by prefixing it with !: they’re really just like named parameters anyway!

$ perl6 -e 'my %h = a=>1, b=>2; say %h<c>:!exists'
True

:delete

This is the only adverb that actually can make changes to the hash or array it is (indirectly) applied to. It replaces the now deprecated .delete method.

$ perl6 -e 'my %h = a=>1, b=>2; say %h<a>:delete; say %h.perl'
1
("b" => 2).hash

Of course, you can also delete slices:

$ perl6 -e 'my %h = a=>1, b=>2; say %h<a b c>:delete; say %h.perl'
1 2 (Any)
().hash

Note that the (Any) is the value returned for the non-existing key. If you happened to have given the hash a default value, it would have looked like this:

$ perl6 -e 'my %h is default(42) = a=>1, b=>2; say %h<a b c>:delete; say %h.perl'
1 2 42
().hash

But the behaviour of the is default maybe warrants a blog post of itself, so I won’t go into it now.

Like with :exists, you can negate the :delete adverb. But there wouldn’t be much point, as you might have well not specified it at all. However, since adverbs are basically just named parameters, you can make the :delete attribute conditional:

$ perl6 -e 'my $really = True; my %h = a=>1, b=>2; say %h<a b c>:delete($really); say %h.perl'
1 2 (Any)
().hash

Because the value passed to the adverb was true, the deletion actually took place. However, if we pass a false value:

$ perl6 -e ‘my $really; my %h = a=>1, b=>2; say %h<a b c>:delete($really); say %h.perl'
1 2 (Any)
("a" => 1, "b" => 2).hash

It doesn’t. Note that the return value did not change: the deletion was simply not performed. This can e.g. be very handy if you have a subroutine or method doing some kind of custom slice, and you want to have an optional parameter indicating whether the slice should be deleted as well: simply pass that parameter as the adverb’s value!

:kv, :p, :k, :v

These 4 attributes modify the returned values from any hash / array slice. The :kv attribute returns a Parcel with keys and values interspersed. The :p attribute returns a Parcel of Pairs. The :k and :v attributes return the key only, or the value only.

$ perl6
> my %h = a => 1, b => 2;
("a” => 1, "b” => 2).hash
> %h<a>:kv
a 1
> %h<a>:p
"a" => 1
> %h<a>:k
a
> %h<a>:v
1

Apart from modifying the return value, these attributes also act as a filter for existing keys only. Please note the difference in return values:

> %h<a b c>
1 2 (Any)
> %h<a b c>:v
1 2

Because the :v attribute acts as a filter, there is no (Any). But sometimes, you want to not have this behaviour. To achieve this, you can negate the attribute:

> %h<a b c>:k
a b
> %h<a b c>:!k
a b c

Combining adverbs

You can also combine adverbs on hash / array slices. The most useful combinations are with one or two of :exists and :delete, with zero or one of :kv, :p, :k, :v. Some examples, like putting a slice out of one hash into a new hash:

$ perl6 -e 'my %h = a=>1, b=>2; my %i = (%h<a c>:delete:p).list; say %h.perl; say %i.perl'
("b” => 2).hash
("a” => 1).hash

Or the keys that were actually deleted:

$ perl6 -e 'my %h = a=>1, b=>2; say %h<a b c>:delete:k’
a b

We actually have a spec that describes which combinations are valid, and what they should return.

Arrays are not Hashes

Apart from hashes using {} for slices, and arrays [] for slices, the adverbial syntax for hash and array slices are the same. But there are some subtle differences. First of all, the “key” of an element in an array, is its index. So, to show the indexes of elements in an array that have a defined value, one can use the :k attribute:

$ perl6 -e 'my @a; @a[3] = 1; say @a[]:k'
3

Or, to create a Parcel with all elements in an array:

$ perl6 -e 'my @a; @a[3] = 1; say @a[]:!k’
0 1 2 3

However, deleting an element from an array, is similar to assigning Nil to it, so it will return its default value (usually (Any)):

$ perl6 -e 'my @a = ^10; @a[3]:delete; say @a[2,3,4]; say @a[2,3,4]:exists'
2 (Any) 4
True False True

If we have specified a default value for the array, the result is slightly different:

$ perl6 -e 'my @a is default(42) = ^10; @a[3]:delete; say @a[2,3,4]; say @a[2,3,4]:exists'
2 42 4
True False True

So, even though the element “does not exist”, it can return a defined value! As said earlier, that may become a blog post for another day!

Conclusion

Slices with adverbs are a powerful way of handling your data structures, be they hashes or arrays. It will take a while to get used to all of the combinations of adverbs that can be specified. But once you’re used to them, they provide you with a concise way of dicing and slicing your data that would previously have involved more elaborate structures with loops and conditionals. Of course, if you want to, you can still do that: it’s not illegal to program Perl 5 in Perl 6 :-)

Day 11 – Installing Modules

December 11, 2013

“Honey, I can’t find my keys!”
– “Hmmm, have you already looked at home or site?”

Preface: This post is about a new feature which currently resides in the branches rakudo/eleven and panda/eleven.

So this post is about installing “modules” and finding them again later. I quoted the word “modules” here because we are not really talking about modules. Even when we say that we meant classes, roles, grammars and every other packagy type by that term, we’re in fact talking about distributions.

That is what we see when we look at modules.perl6.org. These things that have a name, an author or authority and hopefully a version, are the things that provide compilation units which then can be loaded later using statements like use Foo, need Bar or require Baz.

But these distributions can ship other information as well: executable scripts or music, graphics or fonts that are used by another application.
And this bunch of information that is put in a paper bag called distribution, labeled with name/auth/ver is meant to be downloaded by an installer (panda), placed safely on your harddisk, your stick or a webspace, and should be easily locatable later when we need it.

But we are devs, right? We want to use our in-developement-modules without the need to install them. So, there should be a way of telling the compiler that we have a directory structure where our github clones are. These directories should be considered when searching for candidates of a use statement. And, to the fact that we are lacking the paper bag in such a situation, these should be preferred, whatever name/auth/version-trait a use statement may have attached.

This could be one of our rules of thumb: Not installed modules in a known path are preferred over installed ones.

Our first crux, or: Use it.

use Foo:ver<1.2.3> does not mean you are loading a module Foo with version v1.2.3. You are in case loading a package Foo that is part of a distribution that has the required version and that provides such a namespace.

Al right, we are all good hackers, we can handle that. We would just need a (sort of) database were we can put all installed distributions that we would query later, say, when useing a module.

After a few days and the first prototype we would come at a point where we play with panda, our installer toolchain.
We would be ready in so far that panda would install dists into our database. Our tests would show that we could load these installed modules by name, auth and version even when several distributions would supply modules that only differ by version number.
Wasn’t that hard… All fine now?

The second crux, or: The installer installs the installer.

Even panda itself must be installed in our new environment. And that will become insteresting in two ways. We take the pathy way first:
What panda does when we execute its bootstrap.pl script is that it loads the not-yet-installed File::Find for example, compiles it, and installes it to the destination path, just to pick it up to compile Shell::Command. That breaks the our rule of thumb badly. Now a installed module should preferred.
It seems like we would need some sort of ordering there.¹

The third crux, or: I thought it is all about modules?

Panda (or perhaps pandora) offers another box for us: It is our first distribution that has executable files.
Okay, we have a problem here. Our task is to install several versions of the same distribution, but all of them are going to provide executables with the same name, but likely with different functionality?
Clearly we need a way of invoking the correct executable. Our shell would just pick the executable that is found in PATH first. We need something better.
What if we would only create one `bin` folder per installation repository? We could have a script that delegates to the correct version of the wanted executable. Querying our wrapper would then look like this:

panda --ver=1.2 install Foo::Bar

Our wrapper would only need to know about parameters named `–auth`, `–name` and `–ver`, and would just pass everything else to the original executable panda in this case.
Luckily this helps us in another aspect. We could install wrappers like panda-p and panda-j also, which would explicitely invoke the backends parrot and jvm.

The final chapter.

Let us forget about the subjunctive for a moment, what can we do *now*?

There are two interesting branches: rakudo/eleven and panda/eleven. Called after today’s date and the fact that the corresponding spec is the S11.
With these two branches you are able to:

  1. configure your directories for vendor, perl, site and home and also your developement paths using the libraries.cfg.
  2. bootstrap panda which gives you panda, panda-p and panda-j executeables
  3. install modules the “new” way, and also locate them in the following way:
    use Foo:ver(*);
    use Foo:ver(1.*);
    use Foo:ver(/alpha$/);
    use Foo:auth<FROGGS>
    use Foo:auth({ .substr(0,3) eq 'Bar' });
    ...
  4. you can invoke executables like:
    myscript --auth=Peter rec0001.wav
    yourscript --ver="2.*" index.html
    ...
    

I hope this will land in the master/nom branch soon, but I think there are a few glitches that need to be discovered and fixed before doing so. (One glitch might be just less Windows® testing from my side.)

Another glitch, now that I think about it: When you load a specific version of a module or execute a script, the magic must make sure that it prefers its own distribution when it loads modules without the need to specify this in the use statements. Otherwise you would execute the panda script version v1 while this loads modules of version v2.
This will require additional thought in the S11 specification.

A note for module authors:

You probably know about the META.info, in most cases you need to add a “provides” section as shown here.
Without that the packages can’t be used. This “provides” section will not break current code, so please add that.

¹) You can set the ordering of the repositories in your libraries.cfg and in -I compiler switches like:

perl6 -ICompUnitRepo::Local::File:prio[10]=/home/peter/project-a:/home/peter/project-b

Day 10 — Adverbly Adverby Adverbs

December 10, 2013

So yesterday masak introduced and covered quite well Hash objects, and the things that go inside them, Pairs. Here quickly are the two ways you can create Pair objects: there’s fat arrow notation,

    my %h = debug => True;

and there’s colonpair notation:

    my %h = :debug(True);

Today I’ll show you how the colonpair notation in particular is so useful, Perl 6 uses them as a major language feature.

What are Adverbs?

Adverbs in natural languages change in slight ways the meaning of verbs and adjectives. For example,

The dog fetched the stick.

is a simple statement of something the dog did. By adding an adverb, such as:

The dog quickly fetched the stick.

clarifies that dog was able to do this within a short amount of time. Adverbs can make more drastic changes, as seen here:

This batch of cookies was chewy.
This batch of cookies was oddly chewy.

The second sentence, with the “oddly” adverb, lets you know that chewy cookies were not the goal of the baker, while the first sentence alternately does nothing to discredit the efforts of the cook.

Adverbs in Perl 6 perform much of the same tasks, telling functions and other language features to do what they’re about to do differently.

The Basics of Adverbs

Adverbs are expressed with colonpair syntax. Most often, you’ll use them as on/off switches, though it’s perfectly fine for them to take on non-binary information.

The way you turn on an adverb is like this:

    :adverb

which is the same as

    :adverb(True)

To turn off an adverb, you write

    :!adverb

which is just like

    :adverb(False)

If you’re passing a literal string, such as

    :greet('Hello')
    :person("$user")

you can instead do

    :greet<Hello>
    :person«$user» or :person<<$user>>

as long as there’s no whitespace in the string (the angle bracket forms actually create a list of terms, separating on whitespace, which could potentially break whatever’s given the adverb).

You can also abbreviate variable values if the variable’s name is equal to the key’s name:

    :foo($foo)
    :$foo

And if you’re supplying a decimal number, there’s two ways to do that:

    :th(4)
    :4th

(The :4th form only works on quoting construct adverbs, like m// and q[], in Rakudo at the moment.)

Note that the negation form of adverb (:!adv) and the sigil forms (:$foo, :@baz) can’t be given a value, because you already gave it one.

Adverbs in Function Calls

Adverbs used within function calls act more like the named arguments they are than adverbs, but they still count as adverbs.

How do you use adverbs in a function call? Here’s a couple of ways:

    foo($z, :adverbly);
    foo($z, :bar, :baz);
    foo($z, :bar :baz);

Each adverb is a named parameter, so multiple commas separate each adverb, like with any other parameter. Of note is that you’re allowed to “stack” adverbs like you see in the last example (though Rakudo as of yet doesn’t handle this within function calls). You can do this anywhere one adverb is allowed, by the way.

Adverbs on Operators

Adverbs can be supplied to operators just as they can be to functions. They function at a precedence level tighter than item assignment and looser than conditional. (See this part of the Synopses for details on precedence levels.)

Here are a couple of simple uses of adverbs on operators:

    foo($z) :bar :baz  # equivalent to foo($z, :bar, :baz)
    1 / 3 :round       # applies to /
    $z & $y :adverb    # applies to &

When it comes to more complex cases, it’s helpful to remember that adverbs work similar to how an infix operator at that precedence level would (if it helps, think of the colon as a double bond in chemistry, binding both “sides” of the infix to the left-hand side). It operates on the loosest precedence operator no looser than adverbs.

    1 || 2 && 3 :adv   # applies to ||
    1 || (2 && 3 :adv) # applies to &&
    !$foo.bar() :adv   # applies to !
    !($foo.bar() :adv) # applies to .bar()
    @a[0..2] :kv       # applies to []
    1 + 2 - 3 :adv     # applies to -
    1 ** 2 ** 3 :adv   # applies to the leftmost **

Notice that the behavior of adverbs on operators looser than adverbs is currently undefined.

    1 || 2 and 3 :adv  # error ('and' too loose, applies to 3)
    1 and 2 || 3 :adv  # applies to ||

Adverbs on Quoting Constructs

Various quote-like constructs change behavior through adverbs as well.

(Note: this post will refrain from providing an exhaustive list of potential adverbs. S02 and S05 are good places to see them in more detail.)

For example, to have a quoting construct that functions like single quotes but also interpolates closures, then you would do something like:

    q:c 'Hello, $name. You have { +@msgs } messages.' # yes, a space between c and ' is required

Which comes out as

Hello, $name. You have 12 messages.

(This implies your @msgs array has 12 elements.)

If you instead just wanted a double-quote-like construct that didn’t interpolate scalars, you’d do

    qq:!s ' ... etc ...'

Regexes allow you to use adverbs within the regex in addition to outside. This allows you to access features brought by those adverbs in situations where you’d otherwise be unable to use them.

    $a ~~ m:i/HELLO/; # matches HELLO, hello, Hello ...
    $a ~~ /:i HELLO/; # same
    regex Greeting {
        :i HELLO
    }                 # same

One thing to keep in mind is that adverbs on a quoting construct must use parentheses to pass values. This is because normally any occurrence of brackets after an adverb is considered to be passing a value to that adverb, which conflicts with you being able to choose your own quoting brackets.

    m:nth(5)// # OK
    m:nth[5]// # Not OK
    q:to(EOF)  # passing a value to :to, no delimiters found
    q:to (EOF) # string delimited by ()

Your Very Own Adverbs

So you’ve decided you want to make your own adverbs for your function. If you’ll remember, adverbs and named arguments are almost the same thing. So to create an adverb for your function, you just have to declare named parameters:

    sub root3($number, :$adverb1, :$adverb2) {
        # ... snip ...
    }

Giving adverbs a default value is the same as positional parameters, and making an adverb required just needs a ! after the name:

    sub root4($num, :$adv1 = 42, :$adv2, :$adv3!) {
        # default value of $adv1 is 42,
        # $adv2 is undefined (boolifies to False)
        # $adv3 must be supplied by the user
    }

If you want to catch all the adverbs somebody throws at you, you can use a slurpy hash:

    sub root3($num, *%advs) {
        # %advs contains all the :adverbs
        # that were passed to the function.
    }

And if you define named parameters for the MAIN sub, they become commandline options! This is the one time where you should use Bool on boolean named parameters, even if you don’t normally, just to keep the option from accepting a value on the commandline.

It’s the same for operators, as operators are just functions with funny syntax.

Now that you learned how to apply the humble Pair to much more than just Hashes, I hope you’ll quickly start using them in your code, and joyously read the rest of the advent!

Day 09 – Hashes and pairs

December 9, 2013

Hashes are nice. They can work as a kind of “poor man’s objects” when creating a class seems like just too much ceremony for the occasion.

my $employee = {
    name => 'Fred',
    age => 51,
    skills => <sweeping accounting barking>,
};

Perl (both 5 and 6) takes hashes seriously. So seriously, in fact, that there’s a dedicated sigil for them, and by using it you can omit the braces in the above code:

my %employee =
    name => 'Fred',
    age => 51,
    skills => <sweeping accounting barking>,
;

Note that Perl 6, just as Perl 5, allows you to use barewords for hash keys. People coming from Perl 5 seem to expect that we’ve dropped this feature — I don’t know why, but I suspect that as much as they like the ability, they also feel that it’s secretly “dirty” or “wrong” somehow, and thus they just assume that hash keys need to be quotes in Perl 6. Don’t worry, fivers, you can omit the quotes there without any feelings of guilt!

Another nice thing is that final comma. Yes, Perl allows a comma even after the last hash entry. This makes rearranging lines later a lot less of a hair-pulling experience, because all lines have a final comma. So a big win for maintainability, and not a lot of extra bookkeeping for the Perl 6 parser.

Hashes make great “configuration objects”, too. You want to pass some options into a routine somewhere, but the options (for reasons of future compatibility, perhaps) need to be an open set.

my %options =
    rpm => 440,
    duration => 60,
;
$centrifuge.start(%options);

Actually, we have two options with that last line. Either we pass in the whole hash like that, and the method in the centrifuge class will need to look like this:

method start(%options) {
    # probably need to start by unpacking options here
    # ...
}

Or we decide to “gut” the hash as we pass it in, effectively turning it into a bunch of named arguments:

$centrifuge.start( |%options );  # means :rpm(440), :duration(60)

In which case the method declaration will have to look like this instead:

method start(:$rpm!, :$duration!) {
    # ...
}

(In this case, we probably want to put in those exclamation marks, to make those named parameters obligatory. Unless we’re fine with providing some of them with a default, such as :$duration = 120.)

The “gut” operator prefix:<|> is really called “flattening” or “interpolation”. I really like how, in Perl 6, arrays flatten into positional parameters, and hashes flatten into named parameters. Decades after the fact, it gives some extra rationale to making arrays and hashes special enough to have their own sigils.

my @args = "Would you like fries with that?", 15, 5;
say substr(|@args);    # fries

my %details = :year(1969), :month(7), :day(16),
              :hour(20), :minute(17);
my $moonlanding = DateTime.new( |%details );

This brings us neatly into my next point: hash entries in Perl 6 really try to look like named parameters. They aren’t, of course, they’re just keys and values in a hash. But they try really hard to look the same. So much so that we even have *two* syntaxes for writing a hash entry. We have the fat-arrow syntax:

my %opts = blackberries => 42;

And we have the named argument syntax:

my %opts = :blackberries(42);

They each have their pros and cons. One of the nice things about the latter syntax is that it mixes nicely with variables, and (in case your variables are fortunately named) eliminates a bit of redundancy:

my $blackberries = 42;
my %opts = :$blackberries;   # means the same as :blackberries($blackberries)

We can’t do that in the fat-arrow syntax, not without repeating the word blackberries. And no-one likes to do that.

So hash entries (a key plus a value) really become more of a thing in Perl 6 than they ever were in Perl 5. In Perl 5 they’re a bit of a visual hack, since the fat-arrow is just a synonym for the comma, and hashes are initialized through lists.

In Perl 6, hash entries are syntactically pulled into visual pills through the :blackberries(42) syntax, and even more so through the :$blackberries syntax. Not only that, but we’re passing hashes into routines entry by entry, making the entries stand out a bit more.

In the end, we give up and realize that we care a bunch about those hash entries as units, so we give them a name: Pair. A hash is an (unordered) bunch of Pair objects.

So when you’re saying this:

say %employee.elems;

And the answer comes back “3″… that’s the number of Pair objects in the hash that were counted.

But in the end, Pair objects even turn out to have a sort of independent existence, graduating from their role as hash constituents. For, example, you can treat them as cons pairs and simulate Lisp lists with them:

my $lisp-list = 1 => 2 => 3 => Nil;  # it's nice that infix:<< => >> is right-associative

And then, as a final trick, let’s dynamically extend the Pair class to recognize arbitrary cadr-like method calls. (Note that .^add_fallback is not in the spec and currently Rakudo-only.)

Pair.^add_fallback(
    -> $, $name { $name ~~ /^c<[ad]>+r$/ },  # should we handle this? yes, if /^c<[ad]>+r$/
    -> $, $name {                            # if it turned out to be our job, this is what we do
        -> $p {
            $name ~~ /^c(<[ad]>*)(<[ad]>)r$/;        # split out last 'a' or 'd'
            my $r = $1 eq 'a' ?? $p.key !! $p.value; # choose key or value
            $0 ne '' ?? $r."c{$0}r"() !! $r;               # maybe recurse
        } 
    }
);

$lisp-list.caddr.say;    # 3

Whee!

Day 08 – Array-based objects

December 8, 2013

With the advent of a fully-realized object system in Perl 6, complete with formal attributes and accessors, hash-based objects are no longer the de-facto standard. Indeed, the underlying representation for the vast majority of classes (including Arrays and Hashes) is something called “P6Opaque”. This is because such complexities have been moved into the guts of the implementation, which provides a consistent interface to various internal data types transparently. Today’s post isn’t about underlying representations, though. The whole point is for us to be able to take advantage of the new arrangement by spending our time worrying about practical intentions instead of data type semantics.

So, what are we talking about, and if nearly everything is just “opaque”, what makes an array look and act like an array? Arrays (and hashes too) are in many ways just another class. Specifically, the array class is named “Array” with a capital A. We can subclass it, add our own methods and properties, and it will still behave as any array, because it still is one. Many language elements which used to be a largely static part of Perl syntax have been reimplemented as first-class citizens such as grammars and roles. Creating an array-based class truly is as simple as:

class Vector is Array {}

Binding and assignment

It doesn’t look like much yet, but we get a lot for that short bit of code. As might be expected from a Perl array, Vector is an auto-sizing sequence of writable scalars which are indexed by sequential integers. Our Vector class inherits Array’s constructor, which takes positional arguments. It does the Positional role, too, so it is followed by square brackets for subscripting and can be bound to containers declared with the “@” sigil, like so:

my @vec := Vector.new(1, 2, 3);

Note the “:=” binding instead of “=” assignment. Without the colon, a positional container will be created as an empty Array whose contained values are then set to the list after the “=”. In other words:

my @vec = Vector.new(1, 2, 3); # WRONG: assignment instead of binding
say @vec.WHAT; # (Array) instead of (Vector)

because it is the same as:

my @vec := Array.new;
@vec[] = Vector.new(1, 2, 3);

which isn’t what we want, of course. Since “$” means “any type” in Perl 6, as opposed to “a single value” as in Perl 5, we can also use scalar containers for our array-based objects, like any other object. No consideration of binding versus assignment is necessary in this case:

my $vec = Vector.new(1, 2, 3);
say $vec.WHAT; # (Vector)

Methods

Now we know how to create and store our array-based objects in a couple of ways. But our class doesn’t actually do anything useful, so we add some methods:

class Vector is Array {
    method magnitude () {
        sqrt [+] self »**» 2 
    }

    method subtract (@vec) {
        self.new( self »-« @vec )
    }
}

We have a couple simple vector operations. The subtract method takes any positional object as its argument, including Arrays and Vectors, and returns another Vector to allow method chaining:

my @position := Vector.new(1, 2);
my @destination = 4, 6;
my $distance = @position.subtract(@destination).magnitude;
    # $distance = 5

In practical reality we would of course do various sanity checks such as type and dimension comparisons. Needlessly complicating the example serves no purpose, though. This is working pretty well so far. For us, at least.

TIMTOWTDI

Many programmers, however, dislike the idea of treating an object as an array or vice-versa, for various reasons. They would rather have their objects all look like opaque scalars, and access their values via ordinary-looking method calls instead of subscripts. We might not share this view (we are, after all, writing an Array-based class), but imposing that singular vision on all of our users would be decidedly un-Perlish. And we’re half way there. Our Vector can already be stored in a scalar, as discussed before:

my $vec = Vector.new(0, 1);
say $vec.magnitude; # 1

It’s not a bad facade, for an array. But let’s see if we can do better, so we don’t alienate our more traditional users. What about the values? A normal Perl 6 object would expose those via lvalue accessors, so let’s add a few of our own:

class Vector is Array {
    method x () is rw { self[0] }
    method y () is rw { self[1] }
    method z () is rw { self[2] }

    method magnitude () {
        sqrt [+] self »**» 2 
    }

    method subtract (@vec) {
        self.new( self »-« @vec )
    }
}

Here we’re simply defining lvalue accessors with “is rw”, which means their return value can be assigned to with “=” like normal variables instead of through arguments to the accessor:

my $vec = Vector.new(1, 2);
$vec.z = 3;
say $vec; # 1 2 3

Customizing the Constructor

Only one step remains to complete our somewhat seamless illusion: honoring those names if they are passed to the constructor. To accomplish this, we’ll write our own constructor to turn named arguments into positional ones, and pass them on to Array’s constructor implementation positionally:

class Vector is Array {
    method new (*@values is copy, *%named) {
        @values[0] = %named<x> if %named<x> :exists;
        @values[1] = %named<y> if %named<y> :exists;
        @values[2] = %named<z> if %named<z> :exists;

        nextwith(|@values);
    }

    method x () is rw { self[0] }
    method y () is rw { self[1] }
    method z () is rw { self[2] }

    method magnitude () {
        sqrt [+] self »**» 2 
    }

    method subtract (@vec) {
        self.new( self »-« @vec )
    }
}

Don’t forget to declare any parameters which you make changes to as “is copy”, as in the constructor above. Otherwise you run the risk of causing unintended changes to your users’ variables or having your routine just plain die when it tries to perform the assignment.

Seeing is Believing

With the final piece in place, we can now treat our Vector class as a “normal” class with named properties:

my $vec = Vector.new(:x(1), :y(2));
$vec.z = 3;

as an array:

my @vec := Vector.new(1, 2);
@vec.push(3);

or even a mixture of the two, if you just plain want to do it how you feel like at any given moment, and don’t care what anyone else thinks:

my $vec = Vector.new(:y(3), 0);
$vec.x++;
$vec[1]--;
$vec.z = 3;
.say for @$vec;
    # 1
    # 2
    # 3

Closing Remarks

That concludes our demonstration for today, but our Vector class is still far from complete. If you’re looking for something to sharpen your Perl 6 teeth on, you could add methods for more of the basic vector operations, and generally experiment with your working model. Optimize methods with internal caching in private attributes. Export a convenience sub for construction (something like vector(1, 2, 3) instead of Vector.new(1, 2, 3)). Or maybe you want x, y, and z to be actual attributes with all of the extra semantics and meta goodness thereof, in which case you would probably do something in your constructor along the lines of binding the attributes to the array slots, or vice-versa. Or try something different based on a hash. Go wild. -Ofun, as the mantra goes.

Here’s hoping that on day 8 (a number thought by some to variously represent building, power, balance, and new beginnings), you’ve received the gift of inspiration, if nothing else, to look long and hard at all the new ways Perl 6 allows you to bend and blend traditionally rigid design patterns along with more specialized approaches. We have illustrated in our explorations that this power, as with any, can create ugliness and danger if not exercised with due care. It has also been made apparent, however, that wise application of this level of flexibility can sometimes better suit the practical goals, personal thinking style, and human nature of yourself and/or users of your code, and can be accomplished without introducing unreasonable complexity.

For a complete implementation of vectors in Perl 6, including overloaded operators for extra-sugary, very math-looking expressions, see Math::Vector, one of a growing number of modules listed on modules.perl6.org and easily installed via panda.

Day 07 – Bagging the changes in the Set specification

December 7, 2013

In 2012, colomon++ implemented most of the then Set / Bag specification in Niecza and Rakudo, and blogged about it last year.

Then in May this year, it became clear that there was a flaw in the implementation that prohibited creating Sets of Sets easily. In June, colomon++ re-implemented Sets and Bags in Niecza using the new views on the spec. And I took it upon myself to port these changes to Rakudo. And I was in for a treat (in the Bertie Bott’s Every Flavour Beans kind of way).

Texan versus (non-ASCII) Unicode

Although the Set/Bag modules were written in Perl 6, there were some barriers to conquer: it was not as simple as a copy-paste operation. First of all, all Set operators were implemented in their Unicode form in Niecza foremost, with the non-Unicode (ASCII) versions (aka Texan versions) implemented as a dispatcher to the Unicode version. At the time, I was mostly developing in rakudo on Parrot. And Parrot has this performance issues with parsing code that contains non-ASCII characters (at any place in the code, even in comments). Therefore, the old implementation of Sets and Bags in Rakudo, only had the Texan versions of the operators. So I had to carefully figure out which Unicode operator in the Niecza code (e.g. ) matched which Texan operator (namely (<=)) in the Rakudo code, and make the necessary changes.

Then I decided: well, why don’t we have Unicode operators for Sets and Bags in Rakudo either? I mentioned this on the #perl6 channel, and I think it was jnthn++ who pointed out that there should be a way to define Unicode operators without actually having to use Unicode characters. After trying this, and having jnthn++ fix some issues in that area, it was indeed possible.

So how does that look?

  # U+2286 SUBSET OF OR EQUAL TO
  only sub infix:<<"\x2286">>($a, $b --> Bool) {
      $a (<=) $b;
  }

One can only say: Yuck! But it gets the job done. So now one can write:

  $ perl6 -e 'say set( <a b c> ) ⊆ set( <a b c d> )'
  True

Note that Parcels (such as <a b c>) are automatically upgraded to Sets by the set operators. So one can shorten this similar statement to:

  $ perl6 -e 'say <a b c> ⊆ <a b d>'  # note missing c
  False

Of course, using the Unicode operator in Rakudo comes at the expense of an additional subroutine call. But that’s for some future optimizer to take care of.

Still no bliss

But alas, the job was still not done. The implementation using Hash in Rakudo, would not allow Sets within Sets yet still. It would look like it worked, but that was only because the stringification of a Set was used as a key in another set. So, when you asked for the elements of such a Set of Sets, you would get strings back, rather than Sets.

Rakudo allows objects to be used as keys (and still remain objects), by mixing in the TypedHash role into a Hash, so that .keys returns objects, rather than strings. Unfortunately, using the TypedHash role is only completely functional for user programs, not when building the Perl 6 internals using Perl 6, as is done in the core settings. Bummer.

However, the way TypedHash is implemented, could also be used for implementing Sets and Bags. For Sets, there is simply an underlying Hash, in which the key is the .WHICH value of the thing in the set, and the value is the thing. For Bags, that became a little more involved, but not a lot: again, the key is the .WHICH of the thing, and the value is a Pair with the thing as the key, and the count as the value. So, after refactoring the code to take this into account as well, it seemed I could finally consider this job finished (at least for now).

It’s all about values

Then, the nitty gritty of the Set spec struck again.

  "they are subject to === equality rather than eqv equality"

What does that mean?

  $ perl6 -e 'say <a b c> === <a b c>'
  False
  $ perl6 -e 'say <a b c> eqv <a b c>'
  True

In other words, because any Set consisting of <a b c> is identical to any other Set consisting of <a b c>, you would expect:

  $ perl6 -e 'say set(<a b c>) === set(<a b c>)'
  False

to return True (rather than False).

Fortunately, there is some specification that comes in aid. It’s just a matter of creating a .WHICH method for Sets and Bags, and we’re set. So now:

  $ perl6 -e 'say set(<a b c>) === set(<a b c>)’
  True

just works, because:

  $ perl6 -e 'say set(<a b c>).WHICH; say set(<a b c>).WHICH'
  Set|Str|a Str|b Str|c
  Set|Str|a Str|b Str|c

shows that both sets, although defined separately, are really the same.

Oh, and some other spec changes

In October, Larry invoked rule #2 again, but those changes were mostly just names. There’s a new immutable Mix and mutable MixHash, but you could consider those just as Bags with floating point weights, rather than unsigned integer weights. Creating Mix was mostly a Cat license job.

Conclusion

Sets, Bags, Mixes, and their mutable counterparts SetHash, BagHash and MixHash, are now first class citizens in Perl 6 Rakudo. So, have fun with them!

Day 06 – Parsing and generating recurring dates

December 6, 2013

There are a lot of events that are scheduled on particular days of the week each month, for example the regular Windows Patch Day on the second Tuesday of each month, or in Perl 6 land that Rakudo Perl 6 compiler release, which is scheduled for two days after the Parrot release day, which again is scheduled for the third Tuesday of the month.

So let's write something that calculates those dates.

The specification format I have chosen looks like 3rd tue + 2 for the Rakudo release date, that is, two days after the 3rd Tuesday of each month (note that this isn't always the same as the 3rd Thursday).

Parsing it isn't hard with a simple grammar:

grammar DateSpec::Grammar {
    rule TOP {
        [<count><.quant>?]?
        <day-of-week>
        [<sign>? <offset=count>]?
    }
    token count { \d+ }
    token quant { st | nd | rd | th }
    token day-of-week { :i
        [ mon | tue | wed | thu | fri | sat | sun ]
    }
    token sign { '+' | '-' }
}

As you can see, everything except the day of the week is optional, so sun would simply be the first Sunday of the month, and 2 sun - 1 the Saturday before the second Sunday of the month.

Now it's time to actually turn this specification into a data structure that does something useful. And for that, a class wouldn't be a bad choice:

my %dow = (mon => 1, tue => 2, wed => 3, thu => 4,
        fri => 5, sat => 6, sun => 7);

class DateSpec {
    has $.day-of-week;
    has $.count;
    has $.offset;

    multi method new(Str $s) {
        my $m = DateSpec::Grammar.parse($s);
        die "Invalid date specification '$s'\n" unless $m;
        self.bless(
            :day-of-week(%dow{lc $m<day-of-week>}),
            :count($m<count> ?? +$m<count>[0] !! 1),
            :offset( ($m<sign> eq '-' ?? -1 !! 1)
                    * ($m<offset> ?? +$m<offset> !! 0)),
        );
    }

We only need three pieces of data from those date specification strings: the day of the week, whether the 1st, 2nd, 3rd. etc is wanted (here named $.count), and the offset. Extracting them is a wee bit fiddly, mostly because so many pieces of the grammar are optional, and because the grammar allows a space between the sign and the offset, which means we can't use the Perl 6 string-to-number conversion directly.

There is a cleaner but longer method of extracting the relevant data using an actions class.

The closing } is missing, because the class doesn't do anything useful yet, and that should be added. The most basic operation is to find the specified date in a given month. Since Perl 6 has no built-in type for months, we use a Date object where the .day is one, that is, a Date object for the first day of the month.

    method based-on(Date $d is copy where { .day == 1}) {
        ++$d until $d.day-of-week == $.day-of-week;
        $d += 7 * ($.count - 1) + $.offset;
        return $d;
    }

The algorithm is quite simple: Proceed to the next date (++$d) until the day of week matches, then advance as many weeks as needed, plus as many days as needed for the offset. Date objects support addition and subtraction of integers, and the integers are interpreted as number of days to add or subtract. Handy, and exactly what we need here. (The API is blatantly copied from the Date::Simple Perl 5 module).

Another handy convenience method to implement is next, which returns the next date matching the specification, on or after a reference date.

    method next(Date $d = Date.today) {
        my $month-start = $d.truncated-to(month);
        my $candidate   = $.based-on($month-start);
        if $candidate ge $d {
            return $candidate;
        }
        else {
            return $.based-on($month-start + $month-start.days-in-month);
        }
    }
}

Again there's no rocket science involved: try the date based on the month of $d, and if that's before $d, try again, but with the next month as base.

Time to close the class :-).

So, when is the next Rakudo release? And the next Rakudo release after Christmas?

my $spec = DateSpec.new('3rd Tue + 2');
say $spec.next;
say $spec.next(Date.new(2013, 12, 25));

Output:

2013-12-19
2014-01-23

The code works fine on Rakudo with both the Parrot and the JVM backend.

Happy recurring hollidates!

Day 05 – Changes in specification and operational fallout

December 4, 2013

Perl 6 has become much more stable in the past year. There have however been some potentially disrupting changes to the Perl 6 specification, followed by implementation changes to adhere to that new spec.

bless() changes

One of the most visible changes is the removal of an object candidate in bless(). If you wanted to call bless() yourself in your code, rather than supplying your own BUILD() method, you needed to provide an object candidate as the first parameter. Over the years, this turned out to basically always be * (as in Whatever). Which is pretty useless and an obstacle for future optimisations. So TimToady invoked rule #2 to remove that first parameter.

The changes to calls to bless() in the core setting were implemented by moritz++. For those Perl 6 modules in the wild, a warning was added:

 Passing an object candidate to Mu.bless is deprecated

The first parameter would then be removed and execution would continue.

This has the disadvantage of generating a warning every time you create an object of that class with the deprecated call to bless(). So there must be a better way to do this!

Enter “is DEPRECATED”

It turns out there is a better way. Already in June 2012, pmichaud++ added an “is DEPRECATED” routine trait that did nothing until earlier this year when I decided to add some functionality to it. Initially it was just a warn, but that just had the same annoying quality as the warning with bless().

Since the idea behind the “is DEPRECATED” trait was not specced yet, I figured I could turn it any way I wanted, unless I would not be forgiven by the #perl6 crowd. So I re-used an idea I had had at former $work, already years ago. Instead of warning at the moment a transgression is spotted, it feels better, especially for these types of deprecations, to just remember where these transgressions take place. Only when the program is finished, report the transgressions that were spotted (on STDERR).

One of the other standard methods that has been deprecated in Perl 6, is ucfirst(). One should use the tc() (for “title case”) method instead. So what happens if you do call ucfirst()? That is easily demonstrated with a one-liner:

$ perl6 -e 'say "foo".ucfirst; say "done"'
Foo
done
Saw 1 call to deprecated code during execution.
================================================================================
Method ucfirst (from Cool) called at:
  -e, line 1
Please use 'tc' instead.
--------------------------------------------------------------------------------
Please contact the author to have these calls to deprecated code adapted,
so that this message will disappear!

After this has been live in the repo for a while, and spectested, and since nobody on #perl6 complained, I decided to spec this behavior not only for routines, but also for attributes and classes. Unfortunately, the latter ones have not been implemented yet (although you can already specify the traits). But there is a patch -p1 coming up, which should give me some quality time to look at this.

So why is bless(*) not properly DEPRECATED

Indeed. Why? Simply because I missed it. So I just fixed this: nothing like blog-driven development! So this one-liner now says:

$ perl6-p -e 'class A { method new { self.bless(*) } }; say A.new'
A.new()
Saw 1 call to deprecated code during execution.
================================================================================
Method bless (from Mu) called at:
  -e, line 1
Please use a call to bless without initial * parameter instead.
--------------------------------------------------------------------------------
Please contact the author to have these calls to deprecated code adapted,
so that this message will disappear!

So how do you specify the text?

The “is DEPRECATED” trait currently takes one parameter: the string to be shown between Please use and instead. If you don’t specify anything, the text something else will be assumed. Since that is not entirely useful, it is advised to always specify a text that makes sense in that context. Additional parameters may be added in the future to allow for more customisation, but so far they have not been needed.

Conclusion

Perl 6 will continue to evolve. Changes will still be made. To not break early adopter’s code, any non-compatible changes in the implementation of Perl 6 can be marked as deprecated without interfering with the execution of existing programs much. Perl 6 module authors can do the same should they feel the need to change the API of their modules.


Follow

Get every new post delivered to your Inbox.

Join 37 other followers