Day 4 – Having Fun with Rakudo and Project Euler

December 4, 2012 by

Rakudo, the leading Perl6 implementation, is not perfect, and performance is a particularly sore subject. However, the pioneer does not ask ‘Is it fast?’, but rather ‘Is it fast enough?’, or perhaps even ‘How can I help to make it faster?’.

To convince you that Rakudo can indeed be fast enough, we’ll take a shot at a bunch of Project Euler problems. Many of those involve brute-force numerics, and that’s something Rakudo isn’t particularly good at right now. However, that’s not necessarily a show stopper: The less performant the language, the more ingenious the programmer needs to be, and that’s where the fun comes in.

All code has been tested with Rakudo 2012.11.

We’ll start with something simple:

Problem 2

By considering the terms in the Fibonacci sequence whose values do not exceed four million, find the sum of the even-valued terms.

The solution is beautifully straight-forward:

    say [+] grep * %% 2, (1, 2, *+* ...^ * > 4_000_000);

Runtime: 0.4s

Note how using operators can lead to code that’s both compact and readable (opinions may vary, of course). We used

  • whatever stars * to create lambda functions
  • the sequence operator (in its variant that excludes the right endpoint) ...^ to build up the Fibonacci sequence
  • the divisible-by operator %% to grep the even terms
  • reduction by plus [+] to sum them.

However, no one forces you to go crazy with operators – there’s nothing wrong with vanilla imperative code:

Problem 3

What is the largest prime factor of the number 600,851,475,143?

An imperative solution looks like this:

    sub largest-prime-factor($n is copy) {
        for 2, 3, *+2 ... * {
            while $n %% $_ {
                $n div= $_;
                return $_ if $_ > $n;
            }
        }
    }

    say largest-prime-factor(600_851_475_143);

Runtime: 2.6s

Note the is copy trait, which is necessary as Perl6 binds arguments read-only by default, and that integer division div is used instead of numeric division /.

Nothing fancy going on here, so we’ll move along to

Problem 53

How many, not necessarily distinct, values of nCr, for 1 ≤ n ≤ 100, are greater than one-million?

We’ll use the feed operator ==> to factor the algorithm into separate steps:

    [1], -> @p { [0, @p Z+ @p, 0] } ... * # generate Pascal's triangle
    ==> (*[0..100])()                     # get rows up to n = 100
    ==> map *.list                        # flatten rows into a single list
    ==> grep * > 1_000_000                # filter elements exceeding 1e6
    ==> elems()                           # count elements
    ==> say;                              # output result

Runtime: 5.2s

Note the use of the Z meta-operator to zip the lists 0, @p and @p, 0 with +.

The one-liner generating Pascal’s triangle has been stolen from Rosetta Code, another great resource for anyone interested in Perl6 snippets and exercises.

Let’s do something clever now:

Problem 9

There exists exactly one Pythagorean triplet for which a + b + c = 1000. Find the product abc.

Using brute force will work (solution courtesy of Polettix), but it won’t be fast (~11s on my machine). Therefore, we’ll use a bit of algebra to make the problem more managable:

Let (a, b, c) be a Pythagorean triplet

    a < b < c
    a² + b² = c²

For N = a + b + c it follows

    b = N·(N - 2a) / 2·(N - a)
    c = N·(N - 2a) / 2·(N - a) + a²/(N - a)

which automatically meets b < c.

The condition a < b gives the constraint

    a < (1 - 1/√2)·N

We arrive at

    sub triplets(\N) {
        for 1..Int((1 - sqrt(0.5)) * N) -> \a {
            my \u = N * (N - 2 * a);
            my \v = 2 * (N - a);

            # check if b = u/v is an integer
            # if so, we've found a triplet
            if u %% v {
                my \b = u div v;
                my \c = N - a - b;
                take $(a, b, c);
            }
        }
    }

    say [*] .list for gather triplets(1000);

Runtime: 0.5s

Note the declaration of sigilless variables \N, \a, …, how $(…) is used to return the triplet as a single item and .list – a shorthand for $_.list – to restore listy-ness.

The sub &triplets acts as a generator and uses &take to yield the results. The corresponding &gather is used to delimit the (dynamic) scope of the generator, and it could as well be put into &triplets, which would end up returning a lazy list.

We can also rewrite the algorithm into dataflow-driven style using feed operators:

    constant N = 1000;

    1..Int((1 - sqrt(0.5)) * N)
    ==> map -> \a { [ a, N * (N - 2 * a), 2 * (N - a) ] }
    ==> grep -> [ \a, \u, \v ] { u %% v }
    ==> map -> [ \a, \u, \v ] {
        my \b = u div v;
        my \c = N - a - b;
        a * b * c
    }
    ==> say;

Runtime: 0.5s

Note how we use destructuring signature binding -> […] to unpack the arrays that get passed around.

There’s no practical benefit to use this particular style right now: In fact, it can easily hurt performance, and we’ll see an example for that later.

It is a great way to write down purely functional algorithms, though, which in principle would allow a sufficiently advanced optimizer to go wild (think of auto-vectorization and -threading). However, Rakudo has not yet reached that level of sophistication.

But what to do if we’re not smart enough to find a clever solution?

Problem 47

Find the first four consecutive integers to have four distinct prime factors. What is the first of these numbers?

This is a problem where I failed to come up with anything better than brute force:

    constant $N = 4;

    my $i = 0;
    for 2..* {
        $i = factors($_) == $N ?? $i + 1 !! 0;
        if $i == $N {
            say $_ - $N + 1;
            last;
        }
    }

Here, &factors returns the number of prime factors. A naive implementations looks like this:

    sub factors($n is copy) {
        my $i = 0;
        for 2, 3, *+2 ...^ * > $n {
            if $n %% $_ {
                ++$i;
                repeat while $n %% $_ {
                    $n div= $_
                }
            }
        }
        return $i;
    }

Runtime: unknown (33s for N=3)

Note the use of repeat while … {…}, the new way to spell do {…} while(…);.

We can improve this by adding a bit of caching:

    BEGIN my %cache = 1 => 0;

    multi factors($n where %cache) { %cache{$n} }
    multi factors($n) {
        for 2, 3, *+2 ...^ * > sqrt($n) {
            if $n %% $_ {
                my $r = $n;
                $r div= $_ while $r %% $_;
                return %cache{$n} = 1 + factors($r);
            }
        }
        return %cache{$n} = 1;
    }

Runtime: unknown (3.5s for N=3)

Note the use of BEGIN to initialize the cache first, regardless of the placement of the statement within the source file, and multi to enable multiple dispatch for &factors. The where clause allows dynamic dispatch based on argument value.

Even with caching, we’re still unable to answer the original question in a reasonable amount of time. So what do we do now? We cheat and use Zavolaj – Rakudo’s version of NativeCall – to implement the factorization in C.

It turns out that’s still not good enough, so we refactor the remaining Perl code and add some native type annotations:

    use NativeCall;

    sub factors(int $n) returns int is native('./prob047-gerdr') { * }

    my int $N = 4;

    my int $n = 2;
    my int $i = 0;

    while $i != $N {
        $i = factors($n) == $N ?? $i + 1 !! 0;
        $n = $n + 1;
    }

    say $n - $N;

Runtime: 1m2s (0.8s for N=3)

For comparison, when implementing the algorithm completely in C, the runtime drops to under 0.1s, so Rakudo won’t win any speed contests just yet.

As an encore, three ways to do one thing:

Problem 29

How many distinct terms are in the sequence generated by ab for 2 ≤ a ≤ 100 and 2 ≤ b ≤ 100?

A beautiful but slow solution to the problem can be used to verify that the other solutions work correctly:

    say +(2..100 X=> 2..100).classify({ .key ** .value });

Runtime: 11s

Note the use of X=> to construct the cartesian product with the pair constructor => to prevent flattening.

Because Rakudo supports big integer semantics, there’s no loss of precision when computing large numbers like 100100.

However, we do not actually care about the power’s value, but can use base and exponent to uniquely identify the power. We need to take care as bases can themselves be powers of already seen values:

    constant A = 100;
    constant B = 100;

    my (%powers, %count);

    # find bases which are powers of a preceeding root base
    # store decomposition into base and exponent relative to root
    for 2..Int(sqrt A) -> \a {
        next if a ~~ %powers;
        %powers{a, a**2, a**3 ...^ * > A} = a X=> 1..*;
    }

    # count duplicates
    for %powers.values -> \p {
        for 2..B -> \e {
            # raise to power \e
            # classify by root and relative exponent
            ++%count{p.key => p.value * e}
        }
    }

    # add +%count as one of the duplicates needs to be kept
    say (A - 1) * (B - 1) + %count - [+] %count.values;

Runtime: 0.9s

Note that the sequence operator ...^ infers geometric sequences if at least three elements are provided and that list assignment %powers{…} = … works with an infinite right-hand side.

Again, we can do the same thing in a dataflow-driven, purely-functional fashion:

    sub cross(@a, @b) { @a X @b }
    sub dups(@a) { @a - @a.uniq }

    constant A = 100;
    constant B = 100;

    2..Int(sqrt A)
    ==> map -> \a { (a, a**2, a**3 ...^ * > A) Z=> (a X 1..*).tree }
    ==> reverse()
    ==> hash()
    ==> values()
    ==> cross(2..B)
    ==> map -> \n, [\r, \e] { (r) => e * n }
    ==> dups()
    ==> ((A - 1) * (B - 1) - *)()
    ==> say();

Runtime: 1.5s

Note how we use &tree to prevent flattening. We could have gone with X=> instead of X as before, but it would make destructuring via -> \n, [\r, \e] more complicated.

As expected, this solution doesn’t perform as well as the imperative one. I’ll leave it as an exercise to the reader to figure out how it works exactly ;)

That’s it

Feel free to add your own solutions to the Perl6 examples repository under euler/.

If you’re interested in bioinformatics, you should take a look at Rosalind as well, which also has its own (currently only sparsely populated) examples directory rosalind/.

Last but not least, some solutions for the Computer Language Benchmarks Game – also known as the Debian language shootout – can be found under shootout/.

You can contribute by sending pull requests, or better yet, join #perl6 on the Freenode IRC network and ask for a commit bit.

Have the appropriate amount of fun!

Day 3 – Whatever the layout manager is

December 3, 2012 by

Introduction

This article aims to demonstrate how Whatever — one of the many interesting Perl 6 curiosities — could be useful to easily implement and use complex things like a layout manager. In a couple of words, a layout manager is the part of a graphical interface in charge of the spatial arrangement of objects like windows or widgets. For the sake of simplicity, the layout manager implemented in this article will comply with the following three rules:

  • there are only two kinds of widgets: terminal or container, the latter can contain other widgets of either kind;
  • a widget cannot be overlapped, except for containers which fully contain their sub-widgets; and
  • only the height can be adjusted, this size can be either static, dynamic, or intentionally left unspecified.

Usage

From the user's point-of-view, this layout manager aims to be as easy to use as possible. For example, it shouldn't be hard to specify such typical interface below, inspired from a text-based program. In this example, the interface and body widgets are containers, all the others are terminals:

interface (X lines)
+----> +------------------------------------------+
|      | menu bar (1 line)                        |  body (remaining space)
|      +------------------------------------------+ <----+
|      | subpart 1 (1/3 of the remaining space)   |      |
|      |                                          |      |
|      |                                          |      |
|      +------------------------------------------+      |
|      | subpart 2 (remaining space)              |      |
|      |                                          |      |
|      |                                          |      |
|      |                                          |      |
|      |                                          |      |
|      |                                          |      |
|      +------------------------------------------+ <----+
|      | status bar (1 line)                      |
+----> +------------------------------------------+

The user don't know what the remaining space is in advance because such an interface is arbitrary resizable. As a consequence it should be specified as a non-predefined value; this is where * — the Whatever object — comes in handy. This object is interesting for two reasons:

  • from the user's point-of-view, the definition of non-static sizes is as simple as: * / 3 for the subpart 1 (dynamic) and just * for the subpart 2 (unspecified); and
  • from the developer's point-of-view, Perl 6 transforms automatically things like $size = * / 3 into a closure: x → x / 3. Then, it could be called like a regular function: $size($x).

That way, the previous GUI can be transliterated into the following lines of code:

my $interface =
    Widget.new(name => 'interface', size => $x, sub-widgets => (
        Widget.new(name => 'menu bar', size => 1),
        Widget.new(name => 'main part', size => *, sub-widgets => (
            Widget.new(name => 'subpart 1', size => * / 3),
            Widget.new(name => 'subpart 2', size => *))),
        Widget.new(name => 'status bar', size => 1)));

Implementation

The drawing of terminal widgets is straightforward since most of the work is done by containers. Those are in charge to compute the remaining space as well as to uniformly distribute widgets that have an unspecified size:

class Widget {
    has $.name;
    has $.size is rw;
    has Widget @.sub-widgets;

    method compute-layout($remaining-space? is copy, $unspecified-size? is copy) {
        $remaining-space //= $!size;

        if @!sub-widgets == 0 {  # Terminal
            my $computed-size = do given $!size {
                when Real     { $_                  };
                when Callable { .($remaining-space) };
                when Whatever { $unspecified-size   };
            }

            self.draw($computed-size);
        }
        else {  # Container
            my @static-sizes   =  grep Real,     @!sub-widgets».size;
            my @dynamic-sizes  =  grep Callable, @!sub-widgets».size;
            my $nb-unspecified = +grep Whatever, @!sub-widgets».size;

            $remaining-space -= [+] @static-sizes;

            $unspecified-size = ([-] $remaining-space, @dynamic-sizes».($remaining-space))
                                 / $nb-unspecified;

            .compute-layout($remaining-space, $unspecified-size) for @!sub-widgets;
        }
    }

    method draw(Real $size is copy) {
        "+{'-' x 25}+".say;
        "$!name ($size lines)".fmt("| %-23s |").say;
        "|{' ' x 25}|".say while --$size > 0;
    }
}

Here, any Callable object can be used to specify a dynamic size, as far as it takes the computed remaining space as argument. That means it is possible to specify more sophisticated dynamic size by passing a code Block. For example, { max(5, $^x / 3) } ensures the widget has a proportional size that can't decrease below 5.

Conclusion

It's time to check if this trivial layout manager works correctly both in Rakudo and Niecza, the two most advanced implementations of Perl 6. The following test is rather simple, it creates and draws an interface, then resize it and draws it again:

my $interface =
    Widget.new(name => 'interface', size => 11, sub-widgets => (
        Widget.new(name => 'menu bar', size => 1),
        Widget.new(name => 'main part', size => *, sub-widgets => (
            Widget.new(name => 'subpart 1', size => * / 3),
            Widget.new(name => 'subpart 2', size => *))),
        Widget.new(name => 'status bar', size => 1)));

$interface.compute-layout;  # Draw
$interface.size += 3;       # Resize
$interface.compute-layout;  # Redraw

The results before and after resizing are respectively displayed below. They are close enough from the initial mockup, n'est-ce pas?

+-------------------------+            +-------------------------+
| menu bar (1 lines)      |            | menu bar (1 lines)      |
+-------------------------+            +-------------------------+
| subpart 1 (3 lines)     |            | subpart 1 (4 lines)     |
|                         |            |                         |
|                         |            |                         |
+-------------------------+            |                         |
| subpart 2 (6 lines)     |            +-------------------------+
|                         |            | subpart 2 (8 lines)     |
|                         |            |                         |
|                         |            |                         |
|                         |            |                         |
|                         |            |                         |
+-------------------------+            |                         |
| status bar (1 lines)    |            |                         |
                                       |                         |
                                       +-------------------------+
                                       | status bar (1 lines)    |

Finally, the implementation of such a flexible program is really simple in Perl 6: everything is already there, in the core language. Obviously, this trivial layout manager isn't ready for prime-time since a lot of things are missing: sanity checks, multiple dimensions, … but those are left as exercises to you, the reader ;) For any questions or comments, feel free to meet Perl 6 fellows on IRC (#perl6 on freenode).

Bonus

As seen previously, $!size can be Whatever, but it can't be whatever you want. For example, a negative Real or a string are not correct values. Once again Perl 6 provides a simple yet powerful feature: constrained types. In a couple of words this permits to define new types from a set of constraints:

subset PosReal of Real where * >= 0;
subset Size where {   .does(PosReal)
                   or .does(Callable) and .signature ~~ :(PosReal --> PosReal)
                   or .does(Whatever) };

has Size $.size is rw;

Day 2 – Anonymous functions for great good

December 2, 2012 by

Perl 6 has great support for functions. It packs function signatures full with awesome, and lets you have your cake and eat it a couple of times over with all the ways you can specify a function. You can specify parameter types, optional parameters, named parameters, and even those cool where clauses. If I didn’t know better, I’d suspect Perl 6 was compensating for some predecessor’s rather rudimentary handling of parameters. (*cough* @_ *cough*)

Among all these other things, Perl 6 also allows you to define functions without naming them.

sub { say "lol, I'm so anonymous!" }

How is this useful? If you can’t name the function, you can’t call it, right? Wrong.

You can store the function in a variable. Or return it from another function. Or pass it to another function. In fact, when you don’t name your function, the focus becomes much more what code you’re going to run later. Like an executable “to do”-list.

Of course, Perl 5 has anonymous functions, too. With exactly the same syntax, even. In fact, all the big languages do anonymous functions, according to this list of languages on Wikipedia. Except, it seems, the historically significant languages C and Pascal. And the more modern but lumbering Java. “Planned for Java 8″. Haha, Java, catch up! Even C++ has them now.

How important are anonymous functions? Very. In the 1930s, Alan Turing showed how all computer processes could be simulated using just a pre-programmed machine that looks like a tape recorder, reading and writing values on a really long tape. (The Turing Machine.) Meanwhile, across the Atlantic, Alonzo Church showed how all computer processes could be simulated using just anonymous functions, no tape recorder required. (Lambda calculus.) It’s all quite elegant.

Later languages like Lisp and Scheme lean heavily on anonymous functions as a key component in the language. And lately a scrappy language called JavaScript, which also leans heavily on anonymous functions, has taken over the world while we were all busy surfing the web.

But let’s talk possibilities here. What can anonymous functions do for us? And how would it look in Perl 6?

Well, take sorting as a famous example. You could imagine Perl 6 having a sort_lexicographically function and a sort_numerically function. But it doesn’t. It has a sort function. When you want it to sort in a certain way, you just pass an anonymous function to it.

my @sorted_words = @words.sort({ ~$_ });
my @sorted_numbers = @numbers.sort({ +$_ });

(Technically, those are blocks, not functions. But the difference isn’t significant if you’re not planning to return anywhere inside.)

And of course it goes further than just those two sort orders. You could sort by shoe size, or maximum ground speed, or decreasing likelihood of spontaneous combustion. All because you can pass in any logic as an argument. Object-oriented people are very proud of this pattern, and call it “dependency injection”.

Come to think of it, map and grep and reduce all depend on this kind of function-passing. We sometimes refer to passing functions to functions as “higher order programming”, as if it was only something people with special privileges should be doing. But in fact it’s a very useful and broadly applicable technique.

The above examples all run the anonymous functions as part of their own execution. But there’s no need to restrict ourselves to this. We can create functions, return them, and then run them later:

sub make_surprise_for($name) {
    return sub { say "Sur-priiise, $name!" };
}

my $reveal_surprise = make_surprise_for("Finn");    # nothing happens, yet
# ...wait for it...
# ...wait...
# ...waaaaaaait...
$reveal_surprise();        # "Sur-priiise, Finn!"

The function in $reveal_surprise remembers the value of $name even though the original function passing it in has exited long ago. That’s pretty nice. This effect is referred to as the anonymous function closing over the variable $name. But there’s no need to get technical — the long and short of it is “it’s awesome”.

And in fact, it feels quite natural if we just look at anonymous functions alongside other staple storage mechanisms such as arrays and hashes. All of these can be stored in variables, passed as arguments or returned from functions. An anonymous array allows you to store a sequence of things for later. An anonymous hash allows you to store mappings/translations of things for later. An anonymous function allows you to store calculations or behavior for later.

Later this month, I’ll go through how to exploit dynamic scoping in Perl 6 to create nice DSL-y interfaces. We’ll see how anonymous functions come into play there as well.

Perl 6 Advent Calendar 2012: Table of Contents

December 1, 2012 by

This post serves as a table of contents for the 2012 Perl 6 advent calendar. Links to new posts will appear here during the course of this month.

See also: table of contents for 2011, 2010, 2009.

Day 1 – State of Perl 6 in 2012

December 1, 2012 by

Welcome to another edition of your annual Perl 6 advent calendar.

As is tradition on the first of December, you can read a short overview over what has changed in the past year, and where we are standing now.

The list of major changes to the specification is pretty short. The IO subsystem has undergone a rewrite, and now much better reflects the realities in implementations, and actually has a measure of common sense applied. S32::Exceptions has gone through lots of changes (mostly extensions), and now there is a decent core of exception classes in Perl 6.

Both Rakudo and Niecza, the two major Perl 6 compilers, have matured a great deal. Contrary to last year, chances are pretty good that if your program works on one of the compilers, it also works on the other. Niecza also temporarily overtook Rakudo on the count of passing tests.

Niecza had a revamp of the roles implementation, has gained constant folding, awesome Unicode support in regexes, list comprehensions and a no strict; mode. To name just a few of the major changes.

Rakudo now supports heredocs, all phasers (special blocks like BEGIN, END, FIRST, …), longest-token matching in regexes, typed exceptions, much nicer backtraces and operator adverbs. And it now has a debugger, which is shipped with the Rakudo Star distribution.

The module ecosystem has grown a lot, and there is much more documentation for Perl 6 than a year ago.

So, after all these changes, where are we now?

Reports from production uses of Perl 6 are slowly starting to trickle in, and these days if your Perl 6 code has bugs, the chances are much higher that your code is to blame than the compilers. Perl 6 has never been this much fun to use. It surely has been a good and productive year for Perl 6, and we’re sure that this last month will continue the tradition. Have fun!

Day 25 – Merry Christmas!

December 25, 2011 by

The kind elves who spend the rest of the year working in Santa’s shop to bring you more of Perl 6 each year would like to wish you a very warm and fuzzy Christmas vacation. December is always a special time for us, because we get to interact with you all through the interface of the advent calendar. We think that’s wonderful.

Be sure to check out this year’s Perl 6 coding contest, where you can win €100 worth of books!

Merry Christmas!

Day 24 — Subs are Always Better in multi-ples

December 24, 2011 by

Hey look, it’s Christmas Eve! (Also, the palindrome of 42!) And today, we’re going to learn about multi subs, which are essentially synonyms (like any natural language would have). Let’s get started!

An Informative Introduction

multi subs are simply subroutines (or anything related to it, such as methods, macros, etc.) that start with the multi keyword and are then allowed to have the same name as another sub before, provided that sub starts with a multi (or proto — that’s later) keyword. What has to be different between these subs is their signature, or list of formal parameters.

Sound complicated? It isn’t, just take a look at the example below:

multi sub steve(Str $name) {
    return "Hello, $name";
}

multi sub steve(Int $number) {
    return "You are person number $number to use this sub!";
}

Every sub was started with the multi keyword, and has the same name of “steve”, but its parameters are different. That’s how Perl 6 knows which steve to use. If I were to later type steve("John"), then the first steve gets called. If, however, I were to type steve(35), then I’d get the second steve sub.

Equal Footing with built-ins

When you write a multi sub, and it happens to have the same name as some other built-in, your sub is on equal footing with the compiler’s. There’s no preferring Perl 6′s multi sub over yours, so if you write a multi sub with the same name as a built-in and with the same signature, say

multi sub slurp($filename) {
    say "Yum! $filename was tasty. Got another one?";
}

And then try calling it with something like slurp("/etc/passwd"), I get this:

Ambiguous dispatch to multi 'slurp'. Ambiguous candidates had signatures:
:(Any $filename)
:(Any $filename)

Why? Because Perl 6 found two equally valid choices for slurp("/etc/passwd"), my sub and its own, and was unable to decide. That’s the easiest way I know to demonstrate equal footing.

A Fun Conclusion

Now, since it’s Christmas, let’s try writing another open sub, but unlike the built-in open sub, which opens files, this one open presents! Here’s our Present class for this example:

class Present {
    has $.item;
    has $.iswrapped = True;

    method look() {
        if $.iswrapped {
            say "It's wrapped.";
        }
        else {
            say $.item;
        }
    }

    method unwrap() {
        $!iswrapped = False;
    }
}

Now, our open multi sub looks like this:

multi sub open(Present $present) {
    $present.unwrap;
    say "You unwrap the present and find...!";
}

The signature is vastly different from Perl 6′s own open sub, which is a good thing. And here’s the rest of the code, which makes a Present and uses our new multi sub:

my $gift = Present.new(:item("sock"));
$gift.look;
open($gift);
$gift.look;

But wait!

Running this gets us an error in the latest pull of Rakudo:

$ ./perl6 present.p6
It's wrapped.
This type cannot unbox to a native string
⋮

This means that Perl 6′s original open sub is being used, so perhaps it’s being interpreted as an only sub (only subs are the default — only sub unique() {...} and sub unique() {...} mean the same thing). No matter, let’s try adding a proto sub line before our multi sub:

proto sub open(|$) {*}

A proto sub allows you to specify the commonalities between multi subs of the same name. In this case, |$ means “every possible argument”, and {*} means “any kind of code”. It also turns any sub with that name into a multi sub (unless explicitly defined as something other than multi). This is useful if you’re, say, importing a &color sub from a module that isn’t defined as multi (or explicitly as only) and you want to have your own &color sub as well.

After adding this before our multi sub open, we get this result:

$ ./perl6 present.p6
It's wrapped.
You unwrap the present and find...!
sock

It works! Well, that’s it for multi subs. For all the nitty-gritty details, see the most current S06. Enjoy your multi subs and your Christmas Eve!

Day 23 – Idiomatic Perl 6

December 23, 2011 by

Perl 6 Idioms, and Idiomatic Perl 6

(Butterflies of the world, alight!)

Perl is a richly expressive language, with a warm and playful community. When someone crafts a succinct way to solve a common problem, the Perl community often adopts that solution's phrasing as a idiom. (Other-times, the community recoils in horror and proposes a less obtuse phrasing.)

Perl 6 adds many elements to the language, and not just keywords; there are metaoperators, clean exceptions, new contexts, better interpolation, frugal OO, and much more. Those additions were shaped by patterns of Perl 5 use (with an eye to future uses), so at least one should scratch some itch you have long ignored. The new bits aren't required, but we like the shiny, so be prepared for a slew of new idioms soon after Christmas.

Most Perl 6 idioms will not just be translated versions of familiar Perl 5 idioms; they will use new features where appropriate, and may seem unfamiliar at first. If you regularly use any of the Perl 5 idioms below, you can expect to grok its new form soon after you embrace Perl 6 itself.

Idiomatic Perl 6 should feel just as Perlish as Perl 5, once you get used to it. After all, it is all Perl.

Conventions

(Details, details)

Definitions (corrupted^W adapted for computer languages):

Idiom

A phrase expected to be used to express an idea.

Idiomatic

Expressed as the language's native users would state it.

In most examples below, the code is shown in four versions:

1. Non-idiomatic Perl 5,
2. then made idiomatic.
#
3. Perl 5 idiom, naively translated into Perl 6,
4. then made idiomatic.

Any versions past #4 are to show TMTOWTDI. Notice how the code gets clearer or more concise as it goes from 1 to 4.

Idiom ==> Word

(Movin' on up)

Some Perl 5 idioms were so useful and conceptually concise, that they became Perl 6 "words" in their own right. These words take the form of new built-in operators, functions, and methods.

 # Pick a random array element
 $z = $array[ int(rand scalar(@array)) ];
 $z = $array[ rand @array ];
 #
 $z = @array[ rand*@array ];
 $z = @array.pick;            


 # Loop over the keys (indexes) of an array
 for ( my $i=0; $i<@array; $i++ ) {...}
 for my $i ( 0 .. $#array ) {...}
 #
 for 0 .. @array.end -> $i {...}
 for @array.keys -> $i {...}
 
 
 # Whole number division
 ( ($x - ($x % 3) ) / 3 )
 int( $x / 3 )
 #
 Int( $x / 3 )
 $x div 3                  # Integer division op


 # Print the count of the elements of an array.
 say scalar @array;
 say 0+@array;
 #
 say 0+@array;      # Identical in Perl 6
 say +@array;       # + forces the new "numeric" context
 say @array.elems;  # .elems method is more explicit.


 # Do something every 5th time
 if ( ($x/5) == int($x/5) ) {...}
 if ( !($x % 5) ) {...}
 #
 if !($x % 5) {...}
 if $x %% 5 {...}           # %% means "is evenly divisible by"


 # Do something $n times, counting up to $n-1
 for ( $_=0; $_ < $n; $_++ ) {...}
 for ( 0 .. ($n-1) ) {...}
 #
 for 0 ..^ $n {...}
 for ^$n {...}                      # ^9 means 0 ..^ 9, or 0..8

Bare method calls are *always* methods on $_, eliminating Perl 5's confusion on which functions default to $_.

Other defaults have been tweaked to move from something-else-I-must-remember to obvious.

 # Split on whitespace
 @words = split /\s+/, $_;
 @words = split;          # Default is useful, but not intuitive
 #
 @words = .split(/\s+/);  # split() now has no default pattern
 @words = .words;         # split's old default behavior is now a separate method: .words

 # Split a string into individual characters.
 @chars = map { substr $word, $_, 1 } 0..length($word);
 @chars = split '', $word; # Split on nothingness
 #
 @chars = $word.split('');
 @chars = $word.comb;      # Default is to "keep everything"


 # Infinite loop
 for (;;) {...}    # Spoken with a 'C' accent
 while (1) {...}
 #
 while 1 {...}
 loop {...}        # No limit given, so endless by default

Some idioms that became words were already words in Perl 5, if you used the appropriate module.

 # Return the unique elements from a list, in original order
 my %s, @r; for @a { push @r, $_ if !$s{$_}; $s{$_}++; } return @r;
 my %s; return grep { !$s{$_}++ } @a;    # or List::MoreUtils::uniq
 #
 my %s; return grep { !%s{$_}++ }, @a;
 return @a.uniq;


 # Add up all list elements
 my $sum = 0; for my $num (@a) { $sum += $num }
 my $sum; $sum += $_ for @a;    # or List::Util::sum
 #
 my $sum = @a.reduce(*+*);
 my $sum = [+] @a;              # [op] applies op to entire list

Idiom ==> Idiom

(The song remains the same)

Some idioms remain the same, modulo required syntax changes.

 @alpha = 'A' .. 'Z';

 @a = qw{ able baker charlie };
 
 %meta = ( foo => 'bar', baz => 'quz' );
 
 @squares = map { $_ * $_ }, @a;
 
 @starts_with_number = grep { /^\d/ }, @a;

Didn't those all look familiar? Sure, parenthesis are no longer needed on list assignment, and qw{} now has a an angle-bracket shortcut, but those are optional, and don't really change the gist of the idioms. Add that comma after the map|grep block, and these Perl 5 idioms still stand strong in Perl 6.

Perl 5's "magic diamond" idiom still exists, just spelled more sanely (and less diamond-ly).

 # Process each line from STDIN or from command-line files.
 for my $file (@ARGV) { open FH, $file; while (<FH>) {...} }
 while (<>) {...}               # Null filehandle is magical
 #
 for $*ARGFILES.lines {...}
 for lines() {...}              # lines() defaults to $fh = $*ARGFILES

Idiom 5 ==> Idiom 6

(The more things change, the more they remain the same)

Some idioms have taken new form, when the idea behind the original idiom found better expression through new language elements.

 # Hash initialization to constant
 my %h;   for (@a) { $h{$_} = 1 }
 my %h = map { $_ => 1 } @a;
 #
 my %h = map { $_ => 1 }, @a;
 my %h = @a X=> 1;


 # Hash initialization for enumeration
 my %h;   for (0..$#a) { $h{ $a[$_] } = $_ }
 my $c;   my %h = map { $_ => ++$c } @a;
 #
 my $c;   my %h = map { $_ => ++$c }, @a;
 my %h = @a Z=> 1..*;
 my %h = @a.pairs».invert;  # if zero based

 # Hash initialization from parallel arrays
 my %h;   for (@a) { $h{$_} = shift @b }
 my %h;   @h{@a} = @b;
 #
 my %h;   %h{@a} = @b;
 my %h = @a Z=> @b;

 # Swap two variables
 my $temp = $x; $x = $y; $y = $temp;
 ( $x, $y ) = ( $y, $x );
 #
 ( $x, $y ) =   $y, $x;
 ( $x, $y ) .= reverse;   # .= makes reverse into a "mutating" method
 # Tastes great on array swaps, too!   @a[ $j, $k ] .= reverse;

 # Rotate array left by 1 element
 my $temp = shift @a; push @a, $temp;
 push @a, shift @a;
 #
 @a.push: @a.shift;
 @a .= rotate;

 # Create an object
 my $pet = new Dog;
 my $pet = Dog->new;
 #
 my $pet = Dog.new;
 my Dog $pet .= new;    # $pet *always* isa Dog; Compiler can optimize!

Combining transformation with selection was an advanced idiom in Perl 5. The new return values for if provide a bite-sized idiom.

 # Three copies of elements > 5
 @z = map { ($_) x 3 } grep { $_ > 5 } @y;    # map,grep
 @z = map { $_ > 5 ? ($_) x 3 : () } @y;      # map as grep
 #
 @z = map { $_ > 5 ?? ($_) xx 3 !! Nil }, @y;
 @z = @y.map: { $_ xx 3 if $_ > 5 };          # !if == Empty list
 @z = ($_ xx 3 if $_ > 5 for @y);             # List comprehension

That fifth form is a list comprehension, popular in functional languages, and in our closer cousin.

Sentence|Paragraph ==> Idiom

(The territory becomes the map)

Some larger code blocks can now be expressed so concisely that they become idioms in their own right.

 # Random integer between 3 and 7 inclusive
 do { $z = int rand 8 } until $z >= 3;
 $z = 3 + int rand 5;
 #
 $z = 3 + Int(5.rand);
 $z = (3..7).pick;

 # Count by 3 in an infinite loop
 for ( my $i = 1; ; $i++ ) { my $n = 3 * $i; ... }
 for ( my $n = 3; ; $n += 3 ) {...}
 #
 loop ( my $n = 3; ; $n += 3 ) {...}
 for 3, * + 3 ... * -> $n {...}      # `...` is the "sequence" operator
 for 3, 6, 9 ... * -> $n {...}       # `...` can infer from example list

 # Loop over a range, excluding the start and end points
 for my $i ( $start .. $limit ) { next if $i == $start or $i == $limit; ... }
 for my $i ( ($start+1) .. ($limit-1) ) {...}
 #
 for ($start+1) .. ($limit-1) -> $i {...}
 for $start ^..^ $limit -> $i {...}

Predictions

(Gaze into my crystal ball)

The idioms above are not written in stone. After Christmas, they may be given the boot, eighty-sixed, or traded-in for a newer model. Furthermore, other forces are at work to season this stew. Please indulge these prognostications:

  • We will see more use of exceptions. They are now cleaner and cheaper and don't need eval(), and users will be introduced to them much sooner due to open()'s new exception-based error handling.
  • We will see more hash|array slicing. Beginners had to grok another sigil rule to use slices, now they don't! In fact, single-element access now just looks like a degenerate case of multiple-element access.
  • Lazy lists are cool and efficient. Coding practices will be shaped to allow lists to stay lazy longer.
  • The new hyper-operators are implicitly parallel. We will see less explicit threading, and more set-at-a-time thinking.
  • In Perl 5, OO (before Moose) was overly wordy to the point of being grating. Functional programming worked best using the `&` prototype, but we are all told to never use Perl 5's prototypes!

    In Perl 6, OO and functional styles are so low-friction that you will hoist code across paradigm boundaries with the same ease that you refactor same-paradigm code today.

  • `say` will rule the world, with legions of inlined ».methods as its army.

OK, maybe not that last one :)

The crystal ball may be hazy, but the future looks bright!

Day 22 – Operator overloading, revisited

December 22, 2011 by

Today’s post is a follow-up. Exactly two years ago, Matthew Walton wrote on this blog about overloading operators:

You can exercise further control over the operator’s parsing by adding traits to the definition, such as tighterequiv and looser, which let you specify the operator’s precedence in relationship to operators which have already been defined. Unfortunately, at the time of writing this is not supported in Rakudo so we will not consider it further today.

Rakudo is still lagging in precedence support (though at this point there are no blockers that I know about to simply going ahead and implementing it). But there’s a new implementation on the block, one that didn’t exist two years ago: Niecza.

Let’s try out operator precedence in Niecza.

$ niecza -e 'sub infix:<mean>($a, $b) { ($a + $b) / 2 }; say 10 mean 4 * 5'
15

Per default, an operator gets the same precedence as infix<+>. This is per spec. (How do we know it got the same precedence as infix<+> above? Well, we know it’s not tighter than multiplication, otherwise we’d have gotten the result 35.)

That’s all well and good, but what if we want to make our mean little operator evaluate tighter than multiplication? Nothing could be simpler:

$ niecza -e 'sub infix:<mean>($a, $b) is tighter(&infix:<*>) { ($a + $b) / 2 }; say 10 mean 4 * 5'
35

See what we did there? is tighter is a trait that we apply to the operator definition. The trait accepts an argument, in this case the language-provided multiplication operator. It all reads quite well, too: “infix mean is tighter [than] infix multiplication”.

Note the explicit use of intuitive naming for the precedence levels. Rather than the inherently confusing terms “higher/lower”, Perl 6 talks about “tighter/looser”, as in “multiplication binds tighter than addition”. Easier to think about precedence that way.

Internally, the precedence levels are stored not as numbers but as strings. Each original precedence level gets a letter of the alphabet and an equals sign (=). Subsequent added precendence levels append either a less-than sign (<) or a greater-than sign (>) to an existing precedence level representation. Using this system, we never “run out” of levels between existing ones (as we could if we were using integers, for example), and tighter levels always come lexigographically before looser ones. Language designers, take heed.

A few last passing notes about operators in Perl 6, while we’re on the subject:

  • In Perl 6, operators are subroutines. They just happen to have funny names, like prefix:<-> or postfix:<++> or infix:<?? !!>. This actually takes a lot of the hand-wavey magic out of defining them. The traits that we’ve seen applied to operators are really subroutine traits… these just happen to be relevant to operator definitions.
  • As a consequence, just like subroutines, operators are lexically scoped by default. Lexical scoping is something we like in Perl 6; it keeps popping up in unexpected places as a solid, sound design principle in the language. In practice, this means that if you declare an operator within a given scope, the operator will be visible and usable within that scope. You’re modifying the parser, but you’re doing it locally, within some block or other. (Or within the whole file, of course.)
  • Likewise, if you want to export your operators, you just use the same exporting mechanism used with subroutines. See how this unification between operators and subroutines keeps making sense? (In Perl 6-land, we say “operators are just funny-looking subroutines”.)
  • Multiple dispatch in operators works just as with ordinary subroutines. Great if you want to dispatch your operators on different types. As with all other routines in the core library in Perl 6, all operators are declared multi to be able to co-exist peacefully with module extensions to the language.
  • Operators can be macros, too. This is not an exceptions to the rule that operators are subroutines, because in Perl 6, macros are subroutines. In other words, if you want some syntactic sugar to execute at parse time (which is what a macro does), you can dress it up either as a normal-looking sub, or as an operator.

That’s it for today. Now, go forth and multiply, or even define your own operator that’s either tighter or looser than multiplication.

Native libraries, native objects

December 21, 2011 by

Last year flussence++ wrote a nice post about writing XMMS bindings for Perl 6 using the Native Call Interface. It has improved a bit since then, (at least NCI, I don’t know about XMMS), so let’s show it off a bit.

To run the examples below you need a NativeCall module installed. Then add use NativeCall; at the top of the file.

Previously, we were carefully writing all the C subs we needed to use and then usually writing some Perl 6 class which wrapped it in a nice, familiar interface. That doesn’t change much, except that now a class is not really an interface for some C-level data structure. Thanks to the new metamodel we can now make our class to actually be a C-level data structure, at least under the hood. Consider a class representing a connection to Music Player Daemon:

    class Connection is repr('CPointer') {
        sub mpd_connection_new(Str $host, Int $port)
            returns Connection
            is native('libmpdclient.so') {}
        sub mpd_connection_free()
            is native('libmpdclient.so') {}
        method new(Str $host, Int $port) {
            self.bless(mpd_connection_new($host, $port))
        }
        method DESTROY {
            mpd_connection_free(self)
        }
    }

The first line does not necesarilly look familiar. The is repr trait tells the compiler that the internal representation of the class Connection is a C pointer. It still is a fully functional Perl 6 type, which we can use in method signatures or wherever (as seen in the lines below).

We then declare some native fuctions we’re going to use. It’s quite convenient to put them inside the class body, so they don’t pollute the namespace and don’t confuse the user. What we are really exposing here is the new method, which uses bless to set the object’s internal representation to what mpd_connection_new has returned. From now on our object is a Perl 6 level object, while under the hood being a mere C pointer. In method DESTROY we just pass self to another native function, mpd_connection_free, without the need to unbox it or whatever. The NativeCall module will just extract its internal representation and pass it around. Ain’t that neat?

Let’s see some bigger example. We’ll use taglib library to extract the metadata about some music files lying around. Let’s see the Tag class first:

    class Tag is repr('CPointer') {
        sub taglib_tag_title(Tag)  returns Str is native('libtag_c.so') {}
        sub taglib_tag_artist(Tag) returns Str is native('libtag_c.so') {}
        sub taglib_tag_album(Tag)  returns Str is native('libtag_c.so') {}
        sub taglib_tag_genre(Tag)  returns Str is native('libtag_c.so') {}
        sub taglib_tag_year(Tag)   returns Int is native('libtag_c.so') {}
        sub taglib_tag_track(Tag)  returns Int is native('libtag_c.so') {}
        sub taglib_tag_free_strings(Tag)       is native('libtag_c.so') {}

        method title  { taglib_tag_title(self)  }
        method artist { taglib_tag_artist(self) }
        method album  { taglib_tag_album(self)  }
        method genre  { taglib_tag_genre(self)  }
        method year   { taglib_tag_year(self)   }
        method track  { taglib_tag_track(self)  }

        method free   { taglib_tag_free_strings(self) }
    }

That one is pretty boring: plenty of native functions, and plenty of methods being exactly the same things. You may have noticed the lack of new: how are we going to get an object and read our precious tags? In taglib, the actual Tag object is obtained from a Tag_File object first. Why didn’t we implement it first? Well, it’s going to have a method returning the Tag object shown above, so it was convenient to declare it first.

    class TagFile is repr('CPointer') {
        sub taglib_file_new(Str) returns TagFile is native('libtag_c.so') {}
        sub taglib_file_free(TagFile)            is native('libtag_c.so') {}
        sub taglib_file_tag(TagFile) returns Tag is native('libtag_c.so') {}
        sub taglib_file_is_valid(TagFile) returns Int
            is native('libtag_c.so') {}

        method new(Str $filename) {
            unless $filename.IO.e {
                die "File '$filename' not found"
            }
            my $self = self.bless(taglib_file_new($filename));
            unless taglib_file_is_valid($self) {
                taglib_file_free(self);
                die "'$filename' is invalid"
            }
            return $self;
        }

        method tag  { taglib_file_tag(self)  }

        method free { taglib_file_free(self) }
    }

Note how we use native functions in new to check for exceptional situations and react in an appropriately Perl 6 way. Now we only have to write a simple MAIN before we can test it on our favourite music files.

    sub MAIN($filename) {
        my $file = TagFile.new($filename);
        my $tag  = $file.tag;
        say 'Artist: ', $tag.artist;
        say 'Title:  ', $tag.title;
        say 'Album:  ', $tag.album;
        say 'Year:   ', $tag.year;

        $tag.free;
        $file.free;
    }

Live demo! Everyone loves live demos.

    $ perl6 taginfo.pl some-track.mp3
    Artist: Diablo Swing Orchestra
    Title:  Balrog Boogie
    Album:  The Butcher's Ballroom
    Year:   2009

Works like a charm. I promise I’ll wrap it up in some nice Audio::Tag module and release it on Github shortly.

Of course there’s more to do with NativeCall than just passing raw pointers around. You could, for example, declare it as a repr('CStruct') and access the struct field directly, as you would in good, old C. This is only partly implemented as for now though, but that shouldn’t stop you from experimenting and seeing what you can do before Christmas. Happy hacking!


Follow

Get every new post delivered to your Inbox.

Join 25 other followers