Archive for the ‘2010’ Category

Day 15 – Calling native libraries from Perl 6

December 15, 2010

If you’ve used Perl 5 for any length of time, you’ve probably encountered a package with “::XS” in the name. Maybe you’ve even written one, in which case you can skip this paragraph. XS is Perl 5’s way of calling into native code (C libraries), and it works great for what it does. Writing such modules takes effort though; there’s C code and compilers involved, and it’s not much fun if you just want to get something working with a minimum of effort.

Perl 6 puts a lot of effort into letting you be lazy. That goes for using foreign code too. Let’s tell it we’re going to do that:

#!/usr/bin/env perl6
use v6;
use NativeCall;

That’s it. NativeCall does the hard work of talking to C libraries; you don’t need to manually compile anything. For Rakudo, you just need to install zavolaj.

Let’s have some fun with it. For demonstration, I’ll go and port part 1 of XMMS2’s C client tutorial to Perl 6.

First off, NativeCall needs to know how to talk to the C library. At present it knows how to translate to/from strings, numbers, and a thing called OpaquePointer. That last one is for things like database connection handles that you should only poke with a 10-foot pole.

We’re dealing with weird structs and a socket connection in this code, so let’s pretend those are subclasses of OpaquePointer. These don’t have any effect on the code, but it’ll be easier to follow:

class xmmsc_connection_t is OpaquePointer;
class xmmsc_result_t is OpaquePointer;
class xmmsv_t is OpaquePointer;

Now the C functions we’re using, in alphabetical order. One of these is slightly different from the rest – the original C code expects a **char to be passed in, or in other words a place in memory to write a string. We give it a Str is rw, and it just works.

sub xmmsc_connect(xmmsc_connection_t, Str $path) returns Int is native('libxmmsclient') { ... }
sub xmmsc_get_last_error(xmmsc_connection_t) returns Str is native('libxmmsclient') { ... }
sub xmmsc_init(Str $clientname) returns xmmsc_connection_t is native('libxmmsclient') { ... }
sub xmmsc_playback_start(xmmsc_connection_t) returns xmmsc_result_t is native('libxmmsclient') { ... }
sub xmmsc_result_get_value(xmmsc_result_t) returns xmmsv_t is native('libxmmsclient') { ... }
sub xmmsc_result_unref(xmmsc_result_t) is native('libxmmsclient') { ... }
sub xmmsc_result_wait(xmmsc_result_t) is native('libxmmsclient') { ... }
sub xmmsc_unref(xmmsc_connection_t) is native('libxmmsclient') { ... }
sub xmmsv_get_error(xmmsv_t, Str $error is rw) returns Int is native('libxmmsclient') { ... }
sub xmmsv_is_error(xmmsv_t) returns Int is native('libxmmsclient') { ... }

And because the point of this is to *avoid* writing C code, let’s make a nice wrapper object:

class XMMS2::Client {
    has xmmsc_connection_t $!connection;
    method new($client_name = 'perl6', $path = %*ENV) {
        self.bless(*, :$client_name, :$path);
    }
    method play returns Bool {
        my $result = xmmsc_playback_start($!connection);
        xmmsc_result_wait($result);
        return True if self.check-result($result);
        warn "Playback start failed!"; return False;
    }
    method !check-result(xmmsc_result_t $result) returns Bool {
        my $return_value = xmmsc_result_get_value($result);
        my Bool $failed = xmmsv_is_error($return_value);
        if $failed {
            xmmsv_get_error($return_value, my $error-str) and warn $error-str;
        }
        xmmsc_result_unref($result);
        return not $failed;
    }
    submethod BUILD($client_name, $path) {
        $!connection = xmmsc_init($client_name);
        xmmsc_connect($!connection, $path) or die "Connection failed with error: {xmmsc_get_last_error($!connection)}";
    }
    submethod DESTROY {
        xmmsc_unref($!connection);
    }
};

And here’s the magic part:

XMMS2::Client.new.play;

It connects! Then it spits out a Null PMC error. Looks like some work still needs to be done in NativeCall for passing null pointers around, but it’s still a work in progress.

NativeCall has been put to good use already though: check out
MiniDBI, which gives you a (working!) MySQL driver, among others. :)

(Late edit: I forgot the xmmsc_result_wait(). The above code should actually start playback now, though you’ll still get the Null PMC access error.)

Day 14 – nextsame and its cousins

December 14, 2010

Maybe you’re familiar with the way the keyword super in Java allows you to delegate to the method (or constructor) of a base class. Perl 6 has something similar… but in a world with multiple inheritance and mixins it makes less sense to call it super. So it’s called nextsame. Here’s an example of its use:

class A {
    method sing {
        say "life is but a dream.";
    }
}

class B is A {
    method sing {
        say ("merrily," xx 4).join(" ");
        nextsame;
    }
}

class C is B {
    method sing {
        say "row, row, row your boat,";
        say "gently down the stream.";
        nextsame;
    }
}

Now, when we call C.new.sing, our class hierarchy will output this:

row, row, row your boat,
gently down the stream.
merrily, merrily, merrily, merrily,
life is but a dream.

You’ll note how the call finds its way from C.sing via B.sing over to A.sing. Those transitions are (of course) mediated by the nextsame calls. You’ll note the similarity to, for example, Java’s super.

But calling along the inheritance chain is not the only place where nextsame proves useful. Here’s an example not involving object orientation:

sub bray {
    say "EE-I-EE-I-OO.";
}

# Oh right, forgot to add the first line of the song...
&bray.wrap( {
    say "Old MacDonald had a farm,";
    nextsame;
} );

bray(); # Old MacDonald had a farm,
        # EE-I-EE-I-OO.

So that’s another reason nextsame is not called super: it’s not necessarily related to the base class, because there might not be a base class. Instead, there’s some more general phenomenon involved. What might that be?

Every time we do a call to something, there’s a part of the language runtime making sure that the call ends up in the right routine. Such a part is called a dispatcher. A dispatcher makes sure that the following multi call ends up in the appropriate routine:

multi foo(    $x) { say "Any argument" }
multi foo(Int $x) { say "Int argument" }

foo(42) # Int argument

(And a nextsame in the second multi foo would re-dispatch to the first. But that doesn’t work in Rakudo yet.)

Dispatchers are everywhere in Perl 6. They’re involved in method calls, so that a method can defer along the inheritance chain, as we did in the beginning of the post. They’re in wrapped routines, so that the code doing the wrapping can call into the code being wrapped. And they participate in multi dispatch, so that multi variants can defer to each other. It’s all the same principle, but in different guises.

And nextsame is just a way to talk directly to your friendly neighborhood dispatcher. By the way, the keyword is called nextsame because it instructs the dispatcher to defer to the next candidate with the same signature. There are variants, as you’ll see below.

You can use nextsame in mixins, too:

class A {
    method foo { "OH HAI" }
}

role LogFoo {
    method foo {
        note ".foo was called";
        nextsame;
    }
}

my $logged_A = A.new but LogFoo;

say $logged_A.foo; # .foo was called
                   # OH HAI

I like this way to use mixins to inject behavior. I once wrote a post about it, and jnthn has written a Perl 6 module that exploits it.

Though pretty cool, this use of nextsame isn’t really anything new; in fact it’s just another example of the defer-along-the-OO-callchain dispatcher. That’s because mixing in the role LogFoo with but causes an anonymous subclass to be created, one that also does LogFoo. So role mixin nextsame boils down to just inheritance nextsame. (But we don’t need to actually grok that to use it, and it still feels slightly magical and very nice to use.)

In summary, nextsame works in a lot of places you’d expect it to work, and it works the way you expect it to. It defers to the next thing.

Oh, and nextsame has three closely related cousin keywords:

nextsame             Defer with the same arguments, don't return
callsame             Defer with the same arguments, then  return

nextwith($p1, $p2, ...) Defer with these arguments, don't return
callwith($p1, $p2, ...) Defer with these arguments, then  return

Naturally, the other three can be used in the same situations nextsame can.

Day 13 – The Perl6 Community

December 13, 2010

Today is the halfway point of the Perl 6 Advent Calendar (and indeed any advent calendar). For 12 days you have learned various nuggets of information, and most likely experimented with those various bits of knowledge. You feel great, wonderful, stuff works!

And then it didn’t work.

You may have found your way to some documentation (available at http://perl6.org/documentation by the way), and potentially even some talk of Real People™ out there ready to help you. Whatever you’ve done, today’s post is given to you to display the two main methods of helping in the Perl 6 Universe.

The Mailing Lists

The first of two major communication channels is the mailing lists. If you head on over to http://perl6.org/community, you can see on the right hand side a list of the several lists you can subscribe to/peruse the archives of. If you have question s about the language, such as “How do I add up all the numers in an array without a loop?”, go for perl6-users. The mailing list is great for those who can’t use IRC or want more permanent discussions. Have an idea for the language? Try perl6-language, where people discuss the language itself. Things like “I think there should be a beat-object-with-catfish operator in the language” go here.

Now, the mailing lists aren’t the only option. Your question could be simple-when-you-know-it, or maybe that feature you really really want is already there! For these reasons, you may want to run your comments by the IRC community.

The #perl6 IRC channel

If you want to use something more instantaneous instead, or you hate the idea of signing up to a mailing list for one question, head over to the channel #perl6 on irc.freenode.net. Here, the community is reputed as being one of the nicest on the Internet (so if you’ve been burned by IRC channels before and vowed “Never again shall I use IRC!”, try #perl6.)

#perl6 is like an eccentric incarnation of The Doctor, multifaceted in its nature. We have punfests, technotalk, lolz, above-your-head language design, and more (thanks sorear for reminding me of that)! So come and ask away!

Final Words

These are the two ways of communication in the Perl 6 universe. There are also blogs, screencasts, and more. Go to http://perl6.org to check it all out!

Day 12 – Smart matching

December 12, 2010

Remember the http://perl6advent.wordpress.com/2010/12/04/the-sequence-operator/ sequence operator? As the last argument it takes a limit, which makes the sequence generation stop. For example:

    1, 2, 4 ... 32;         # 1 2 4 8 16 32
    1, 2, 4 ... * > 10;     # 1 2 4 8 16

You can see that in the first case, numerical equality is used. The second is a bit more interesting: * > 10 is internally rewritten into a closure like -> $x { $x > 10 } (through currying).

The sequence operator then does some magic comparison, depending on the type of the matcher. This comparison is called "smartmatching", and is a concept that reappears in many places in Perl 6. Examples:

    # after the 'when' keyword:
    given $age {
        when 100    { say "congratulations!"      }
        when * < 18 { say "You're not of age yet" }
    }
    # after 'where':
    subset Even of Int where * %% 2;
    # with an explicit smartmatch operator:
    if $input ~~ m/^\d+$/ {
        say "$input is an integer";
    }
    # arguments to grep(), first() etc.:
    my @even = @numbers.grep: Even;

On the right-hand side of the ~~ operator, and after when and where, the value to be matched against is set to the topic variable $_. This allows us to use constructs that operate on $_, like regexes created with m/.../ and .method_call.

Here is what the smart match operator does with some matchers on the right-hand side:

    # is it of type Str?
    $foo ~~ Str
    # is it equal to 6?
    $foo ~~ 6
    # or is it "bar"?
    $foo ~~ "bar"
    # does it match some pattern?
    $foo ~~ / \w+ '-' \d+ /
    # Is it between 15 and 25?
    $foo ~~ (15..25)
    # call a closure
    $foo ~~ -> $x { say 'ok' if 5 < $x < 25 }
    # is it an array of 6 elems in which every odd element is 1?
    $foo ~~ [1, *, 1, *, 1, *]

The full table for smart matching behavior can be found at
http://perlcabal.org/syn/S03.html#Smart_matching.

Notice how, unlike in Perl 5, in Perl 6 smartmatching is the only syntactic way to match a regex, it has no special operator. Also, while most smart matching cases return a Bool, matching against a regex returns a Match object – which behaves appropriately in boolean context too.

You have probably started wondering: a’right, that for built-in types, how do I use it in my own classes? You need to write a special ACCEPTS method for it. Say we have our good, old class Point:


    class Point {
        has $.x;
        has $.y;
        method ACCEPTS(Positional $p2) {
            return $.x == $p2[0] and $.y == $p2[1]
        }
    }

Everything clear? Let’s see how it works:

    my $a = Point.new(x => 7, y => 9);
    say [3, 5] ~~ $a; # Bool::False
    say (7, 9) ~~ $a; # Bool::True

Perl 6 can now do exactly what you mean, even with your own classes.

Day 11 – Markov Sequence

December 11, 2010

On Day 4, I teased with mention of non-numeric sequences. Today I’d like to explore one such sequence, based on the common idea of using Markov chains on text. We will determine the next letter in the sequence randomly based on the previous two letters. The distribution follows the patterns contained in a model text, so that the result approximates the language of the model.

use v6;
use List::Utils;

my $model-text = $*IN.slurp.lc;
$model-text .=subst(/<[_']>/, "", :global);
$model-text .=subst(/<-alpha>+/, " ", :global);

my %next-step;
for sliding-window($model-text.comb, 3) -> $a, $b, $c {
    %next-step{$a ~ $b}{$c}++;
}

my $first = $model-text.substr(0, 1);
my $second = $model-text.substr(1, 1);
my @chain := $first, $second, -> $a, $b { %next-step{$a ~ $b}.roll.key } ... *;
say @chain.munch(80);

After the initial use statements, the code divides neatly into three sections. The first section inputs the model text and gets rid of the non-alphabetic characters. Line 4 uses slurp to read standard input ($*IN) into a single string, and lc makes it all lowercase. The first subst (line 5) removes all underscores and apostrophes from the text. The second (line 6) replaces each string of non-alphabetic characters with a single space.

The second section combines the sliding-window sub from List::Utils with a bit of the good old Perl hash magic. (You can get List::Utils using neutro.)

$model-text.comb splits the text into individual characters.

sliding-windows iterates through a list, giving you the next N (3, in this case) elements in the list starting with each element in the list. (That is, you get the 1st, 2nd, and 3rd elements, then the 2nd, 3rd, and 4th elements, then the 3rd, 4th, and 5th elements, etc.) In this case we use it to get every set of three consecutive characters in the text.

In that loop, we construct a hash table of hash tables. The keys to the outer are the first two of those three consecutive characters. The keys to the inner are the third consecutive character, and its value is how many times that character follows the first two. So, for instance, if I feed the lyrics to the Aqualung album into this program, then %next-step{"qu"} looks like this:

{"a" => 5, "e" => 2}

That is to say, if you have “q” and “u”, they are followed by “a” five times (presumably all in the name Aqualung) and “e” twice (“requests” and “question”).

The third section of the code, then uses the knowledge we’ve just accumulated to build the sequence. First we get the first and second characters of the model space, just so we can start with a pair of letters we know for sure will have a letter coming after them. Then we construct a sequence which starts with those two, and uses -> $a, $b { %next-step{$a ~ $b}.roll } as a generator. The generator uses the previous two characters in the sequence to look up the appropriate frequency hash for the next character. The roll method randomly returns one of the keys of that hash, weighted by the value. (In the “qu” example above, you could think of this as rolling a seven-sided die, five of whose faces say “a” and two “e”.) If there is no known character which follows the previous two (for instance, if the last two characters in the model text are a pair unique in the text and you reach them both in order), then an undefined value is returned, which stops the sequence. We get the first 80 characters from the sequence using the munch method, chosen because it is well-behaved if the sequence terminates early.

Running the script on the lyrics to Aqualung produces sequences like
“t carealven thead you he sing i withe and upon a put saves pinsest to laboonfeet” and “t steall gets sill a creat ren ther he crokin whymn the gook sh an arlieves grac”. (Why does it start with “t “? My Aqualung lyrics file starts with some ASCII-art apparently attempting to imitate the original liner notes, and when we removed the non-alphabetic characters it boils down to “t “.)

Note that nowhere here does this script make assumptions about the character set it is working with. Anything that Perl 6 recognizes as alphabetic can be processed this way. If you feed it the standard “Land der Berge” file that p6eval uses as stdin, you’ll get strings like “laß in ber bist brüften las schören zeites öst froher land der äckerzeichöne lan”. (Fingers crossed that WordPress and your browser handle the non-ASCII characters correctly!)

One word of warning: As I was finishing this, the #perl6 channel raised the question of what Hash.roll should actually do. Right now in Rakudo it has the functionality of the (not yet implemented) KeyBag.roll method. Once it is implemented, KeyBag could be substituted if Hash.roll ends up spec’d differently.

There is a simple alternative which works today, however. If you change to %next-step{$a ~ $b}{$c}++ to %next-step.push($a ~ $b, $c), %next-step will be constructed as a Hash of Arrays. Each array will list all the characters $c which appear after $a and $b, with each distinct character repeated the number of times it appears in the file. This naturally acts as a weighting for .roll to use in the sequence generator.

I need to give a big thank you to Moritz Lenz, who was a huge help cleaning up and simplifying this script.

Day 10 – Feed operators

December 10, 2010

Anyone who has programmed in Perl 5 for a while has probably run across or written code similar to this:

    my @new = sort { ... } map { ... } grep { ... } @original;

In this construction, the data flows from the @original array which feeds into the grep, and that, in turn, feeds into the map, and then into the sort, and then finally, the result is assigned to the @new array. Because they each take a list as their final parameter, simply by juxtposition, the data feeds leftward from one operation to the next.

Perl 6, on the other hand, makes this idea of data flowing from one operation to another explicit by introducing the feed operator. The above Perl 5 code could be written like this in Perl 6:

    my @new <== sort { ... } <== map { ... } <== grep { ... } <== @original;

Note that TMTOWTDI is alive and well in Perl 6. You could have also written much the same as in Perl 5:

    my @new = sort { ... }, map { ... }, grep { ... }, @original;

The only difference would be the addition of commas.

So, what do we gain from these feed operators? Normally, when reading code, you read from left to right. In the original Perl 5 code you would read from left to right until you realize that you’re dealing with constructions where the direction of flow is right to left, then you jump to the end and follow the processing in a right-to-left manner. In Perl 6 there is now a prominent syntactic marker that clues you in to the leftward flowing nature of the data.

Still, the right-to-left nature of this code is somewhat troublesome. It may not seem so onerous if all of the code fits on one line as above. But imagine if the blocks associated with grep, map, and sort were little longer. Finding the end of the statement could be a bit annoying.

Luckily, Perl 6 has another feed operator that allows you to write the same code in a left-to-right fashion:

    @original ==> grep { ... } ==> map { ... } ==> sort { ... }  ==> my @new;

This works exactly the same as before only the direction of flow has been changed.

Here are a couple of examples of real, working Perl 6 code using the feed operators:

    my @random-nums = (1..100).pick(*);
    my @odds-squared <== sort() <== map { $_ ** 2 } <== grep { $_ % 2 } <== @random-nums;
    say ~@odds-squared;
    my @rakudo-people = <scott patrick carl moritz jonathan jerry stephen>;
    @rakudo-people ==> grep { /at/ } ==> map { .ucfirst } ==> my @who-it's-at;
    say ~@who-it's-at;

Day 9 – The module ecosystem

December 9, 2010

The Perl 6 module database on http://modules.perl6.org is certainly not CPAN yet, but there are still a number of things worth using, or at least knowing about. There’s no "standard" module installer for Perl 6, like there’s a cpan shell for Perl 5, but the most commonly used and most often working is neutro. It’s a simple script fetching, building, and installing modules from the ecosystem, resolving dependencies and checking if the tests are passing: not much more we need. Let’s see how to install something interesting with it, say JSON::Tiny, a JSON parser.

First, we need to get neutro. We will assume you use git to obtain it; notice that git is also obligatory to download modules (all of them live on github currently).

    git clone git://github.com/tadzik/neutro.git
    cd neutro
    PERL6LIB=tmplib bin/neutro .

That will download neutro and bootstrap it using the supplied libs. What we end up is the module installer itself, and the File::Tools and Module::Tools distributions. Make sure ~/.perl6/bin is in your PATH enviroment variable, so you will be able to run neutro without specifying its exact location. You are now able to install modules as simply as with cpanminus:

    neutro json
    neutro perl6-Term-ANSIColor
    neutro perl6-lwp-simple

You may notice module names are not similar to what you may be used to from Perl 5. They’re not standarized, they’re just the names of a git repos they live in. To make sure what you are looking for, consult the
list:

    neutro update # fetch the fresh list of modules
    neutro list

Modules will be installed to ~/.perl6/lib, which is in the default search path of Rakudo, so you don’t really need to set PERL6LIB yourself:

    perl6 -e 'use Term::ANSIColor; say colored("Hello blue world!", "blue")'

You probably just can’t wait to write your first module and make it available for the whole world. There’s no CPAN where you can send your packages; the usual workflow is creating a repository on Github and adding it to the projects.list file in the ecosystem. You don’t need to have a direct access to the repo to get your module published. You can either send a pull request for your forked ecosystem repo, send a patch, or just ask some of the commiters or people on the #perl6 channel of Freenode.

How to write a module, what tools to use? If you’re interested, look into the guide.

Happy hacking!

Day 8 – Different Names of Different Things

December 8, 2010

(By masak and moritz)

Newcomers to the Perl programming language, version 5, often complain that
they can’t reverse strings. There’s a built-in reverse function, but it doesn’t seem to work at first glance:

    $ perl -E "say reverse 'hello'"
    hello

When such programmers ask more experienced Perl programmers, the solution is quickly found: reverse has actually two different operation modes. In list context it reverses lists, in scalar context it reverses strings.

    $ perl -E "say scalar reverse 'hello'"
    olleh

Sadly this an exception from Perl’s usual context model. For most operators or functions the operator determines the context, and the data is interpreted in that context. For example + and * work on numbers, and . works on strings (concatenation). So a symbol (or function name, in the case of uc) stands for an operation, and provides context. Not so reverse.

In Perl 6, we try to learn from previous mistakes, and get rid of historical inconsistencies. This is why list reversal, string reversal and hash inversion have been split up into separate built-ins:

    # string reversal, sorry, flipping:
    $ perl6 -e 'say flip "hello"'
    olleh
    # reversing lists
    # perl6 -e 'say join ", ", reverse <ab cd ef>'
    ef, cd, ab
    # hash inversion
    perl6 -e 'my %capitals = France => "Paris", UK => "London";
              say %capitals.invert.perl'
    ("Paris" => "France", "London" => "UK")

Hash inversion differs from the other two operations in that the result is of a different type than the input. Since hash values need not be unique, inverting a hash and returning the result as a hash would either change the structure (by grouping the keys of non-unique values together somehow), or lose information (by having one value override the other ones).

Instead hash inversion returns a list of pairs, and the user can decide which mode of operation they want for turning the result back into a hash, if that’s necessary at all.

Here’s how you do it if you want the hash inversion operation to be non-destructive:

    my %inverse;
    %inverse.push( %original.invert );

As pairs with already-seen keys get pushed onto the %inverse hash, the original value isn’t overriden but instead "promoted" to an array. Here’s a small demonstration:

    my %h;
    %h.push('foo' => 1);    # foo => 1
    %h.push('foo' => 2);    # foo => [1, 2]
    %h.push('foo' => 3);    # foo => [1, 2, 3]

All three (flip/reverse/invert) coerce their arguments to the expected type (if possible). For example if you pass a list to flip, it is coerced into a string and then the string is rever^W flipped.

Day 7 – Lexical variables

December 7, 2010

Programming is tough going. Well, stringing together lines of code isn’t that difficult, and prototyping an idea can be pleasant and easy. But as the size of the program scales up, and the maintenance time lengthens, things tend to get tricky. Eventually, if we’re unlucky, we’re overcome by the complexity — not necessarily the complexity of the problem we started out solving, but the complexity of the program itself. We get gray hairs from debugging, or we’re simply at a loss for how to extend the program to do what we want.

So we turn to the history of programming, seeking advice as to how to combat complexity. And the answer sits there, clear as day: limit extent. If you’re architecting programs with hundreds or thousands of types of components, you’ll want those components to interact through only a very small set of surfaces… otherwise, you’ll simply lose control. Otherwise, combinatorics will defeat you.

We see this principle at every single level of programming, simply because it’s such a primary thing: Separation of Concerns, Do One Thing And Do It Well, BCNF, monads, routines, classes, roles, modules, packages. All of them urge us or guide us to limit the extent of things, so we don’t lose to combinatorics. Perhaps the simplest example of this is the lexical variable.

{
    my $var;
    # $var is visible in here
}
# $var is not visible out here

Yeah — that is today’s “cool feature”. :-) Here’s what makes it interesting:

Perl got this one wrong from version 1 and onwards. The default variable scope in Perl 5 is the “package variable”, a kind of global variable. Define something inside a block; still see it outside.

$ perl -v
This is perl 5, version 12, subversion 1 (v5.12.1)

$ perl -E '{ $var = 42 }; say $var'
42

$ perl -wE '{ my $var= 42 }; say $var'
Name "main::var" used only once: possible typo at -e line 1.
Use of uninitialized value $var in say at -e line 1.

In Perl 6, the lexical variable is the default. You won’t get past compilation if you try to pull off the above trick in Rakudo:

$ perl6 -e '{ $var = 42 }; say $var'    # gotta initialize with 'my'
===SORRY!===
Symbol '$var' not predeclared in <anonymous>

$ perl6 -e '{ my $var = 42 }; say $var' # still won't work! not visible outside
===SORRY!===
Symbol '$var' not predeclared in <anonymous>

You might say “okay, this is great for catching a typo now and then”. Yes, sure, but the big advantage is that this keeps you honest about variable scoping. And that helps you manage complexity.

Now let me just rush to the defense of Perl 5 by saying a variety of things at the same time. Perl 5 does try to steer you in the right way by having you use strict and use warnings by reflex; Perl 5 is bound by its promise of backwards compatibility, which is very good and noble; Perl 1 certainly was not about writing large applications and managing the resulting complexity; and global variables do make a lot of sense in a one-line script.

Perl 6 has an inherent focus to help you start small, and then help you put in more strictures and architectural underpinnings as your application scales up. In the case of variables, this means that in scripts and modules, lexical variables (à la strict) are the default, but in those -e one-liners the default is package variables. (Rakudo doesn’t implement this distinction yet, and one has to use lexical variables even at the command line. After it’s been implemented in Rakudo, I’d expect the perl6 invocations above to get past compilation, and produce outputs similar to the perl invocations.)

Moving along. At this point you might consider all that’s worth saying about lexical variables to have been said — not so. You see, the result of designing things right is that surprising and awesome bonuses keep falling out. Consider this subroutine:

sub counter($start_value) {
    my $count = $start_value;
    return { $count++ };
}

What’s returned at return { $count++ } is a block of code. So each time we call counter, what we get back is a little disconnected piece of code that can be called, as many times as we want.

Now look what happens when we create two such pieces of code and play around with them:

my $c1 = counter(5);
say $c1();           # 5
say $c1();           # 6

my $c2 = counter(42);
say $c2();           # 42
say $c1();           # 7
say $c2();           # 43

See that? The vital observation here is that $c1 and $c2 are acting entirely independently of each other. Both keep their own state, in the form of the $count variable, and although this might look like the same variable to us, to the two invocations of counter it looks like two different storage locations — because each time we enter a block of code, we start out afresh. The little block of code returned from some run of counter retains a relation to that particular storage location (it “closes over” the storage location, protecting it from the grasp of the Grim Garbage Collector; thus this kind of block is called a closure.)

If closures look a lot like lightweight objects to you, congratulations; they are. The principle behind closures, regulating the way values are accessed, is the same as the principle behind encapsulation and information hiding in OO. It’s all about limiting the extent of things, so that they can wreak as little havoc as possible when things turn ugly.

You can do nifty things like closures with lexical variables. You can’t with package variables. Lexical variables are cooler. QED.

Day 6 – The X and Z metaoperators

December 6, 2010

One of the new ideas in Perl 6 is the metaoperator, an operator which combines with an ordinary operator in order to change its behaviour. There are several of them, but in this post we will concentrate on just two of them: X and Z.

The X operator is one you might already have seen in its ordinary role as the infix cross operator. It combines lists together, one element from each, in every combination:

> say ((1, 2) X ('a', 'b')).perl
((1, "a"), (1, "b"), (2, "a"), (2, "b"))

However, this infix:<X> operator is actually just a shorthand for the X metaoperator applied to the list concatenation operator infix:<,>. Indeed, you’re perfectly at liberty to write

> say ((1, 2) X, (10, 11)).perl
((1, 10), (1, 11), (2, 10), (2, 11))

If you like. So what happens if you apply X to a different infix operator? How about infix:<+>

> say ((1, 2) X+ (10, 11)).perl
(11, 12, 12, 13)

What’s it done? Instead of creating a list of all the elements picked for each combination, the operator applies the addition infix to them, and the result is not a list but a single number, the sum of all the elements in that combination.

This works for any infix operator you care to use. How about string concatenation, infix:<~>?

> say ((1, 2) X~ (10, 11)).perl
("110", "111", "210", "211")

Or perhaps the numeric equality operator infix:<==>?

> say ((1, 2) X== (1, 1)).perl
(Bool::True, Bool::True, Bool::False, Bool::False)

But this post is also meant to be about the Z metaoperator. We expect you may have already figured out what it does, if you’ve encountered the infix:<Z> operator before, which is of course just a shortcut for Z,. If a Haskell programmer understands infix:<Z> as being like the zip function, then the Z metaoperator is like zipWith.

> say ((1, 2) Z, (3, 4)).perl
((1, 3), (2, 4))
> say ((1, 2) Z+ (3, 4)).perl
(4, 6)
> say ((1, 2) Z== (1, 1)).perl
(Bool::True, Bool::False)

Z, then, operates on each element of each list in turn, working on the first elements together, then the second, then the third for however many there are. It stops when it reaches the end of a list regardless of which side that list is on.

Z is also lazy, so you can apply it to two infinite lists and it will only generate as many results as you need. X can only handle an infinite list on the left, otherwise it would never manage to get anywhere at all.

At the time of writing, Rakudo appears to suffer from a bug where infix:<Z> and infix:<Z,> are not identical: the former produces a flattened result list. S03 shows that the behaviour of the latter is correct.

These metaoperators, then, become powerful tools for performing operations encompassing the individual elements of multiple lists, whether those elements are associated in some way based on their indexes as with Z, or whether you just want to examine all possible combinations with X.

Got a list of keys and values and you want to make a hash? Easy!

my %hash = @keys Z=> @values;

Or perhaps you want to iterate over two lists in parallel?

for @a Z @b -> $a, $b { ... }

Or three?

for @a Z @b Z @c -> $a, $b, $c { ... }

Or maybe you want to find out all the possible totals you could get from rolling three ten-sided dice:

my @d10 = 1 ... 10;
my @scores = (@d10 X+ @d10) X+ @d10;

If you want to see some real-world use of these metaoperators, Sudoku.pm in Moritz Lenz’s Sudoku solver.


Follow

Get every new post delivered to your Inbox.

Join 37 other followers