Day 9 – Data munging in Perl 6 vs Perl 5

December 9, 2014 by

One thing that Perl has traditionally been used for a lot, is small scripts which read some data (usually from a file), put it into a data structure, transform said data structure, and print some output based on it.

I believe that Perl 6, too, has the potential to appeal to people with such data wrangling needs (who may not necessarily be professional programmers, e.g. powerusers/sysadmins or students/scientists). Perl 6 does not share Perl 5’s selling-point of coming pre-installed on most *nix systems, but it entices with convenience features that allow script writers to focus more on what they’re trying to accomplish, and less on low-level technical details.

Let me showcase and compare an idiomatic Perl 5 solution and an idiomatic Perl 6 solution for the same simple data munging use-case (which might be a little contrived, but practical enough to be transferable to more complex problems), so you can form your own opinion:

Read the rest of this entry »

Day 8 – A HTTP User Agent with SSL/TLS

December 8, 2014 by

Introduction

This year’s Google Summer of Code brings us some new cool stuff.
One of them is web crawling which comes with HTTP::UserAgent and IO::Socket::SSL modules!

You all know what web crawling is, so I describe it with just a few words:
Web crawling (limited for the purposes of this article) is everything we do with the websites using scripts, sending and receiving requests.

HTTP::UserAgent

We can write a simple Web Crawler with just a few lines of code.

 use HTTP::UserAgent;
 my $ua = HTTP::UserAgent.new(useragent => 'firefox_linux');
 my $response = $ua.get: "http://perl6advent.wordpress.com/";

Simple as that.

What could draw attention is the second line where we use a magic phrase ‘firefox_linux’.
It automatically generates an User-Agent header’s field for us. The rule is simple, just write ‘browser_system’ and the correct user agent will be used.
The list of predefined user agents can be found here, feel free to add more.

We want to decode the content.

 my $content = $response.decoded_content;

As we have the content decoded, we can do everything we want with it. It’s simple, isn’t it?

TLS support

The most fascinating thing which comes with this project is SSL/TLS support built on OpenSSL library.
To use SSL/TLS we must install IO::Socket::SSL.

 panda install IO::Socket::SSL

And again, that’s it, simple.

All of that gives us some opportunities like WWW::Mechanize, Net::GitHub and so on.

I really want these modules to be available in Perl 6 — do you want to help us writing? Join us. :)

Merry Christmas!

Day 7 – that’s how we .roll these days

December 7, 2014 by

This is a public service announcement.

Since Wolfman++ wrote his blog post about .pick back in the year two thousand nine (nine years after the distant future), we’ve dropped the :replace flag, and we’ve gained a .roll method. (Actually, the change happened in September 2010, but no-one has advent blogged about it.)

So this is the new way to think about the two methods:

.pick    like picking pebbles out of an urn, without putting them back
.roll    like rolling a die

I don't remember how I felt about it back in 2010, but nowadays I'm all for keeping these methods separate. Also, it turns out, I use .roll($N) a whole lot more. In that light, I'm glad it isn't spelled .pick($N, :replace).

Just to give a concrete example:

> say <Rock Paper Scissor>.roll
Rock                    # good ol' Rock

.roll is equivalent to .roll(1). When we're only interested in one element, .roll and .pick work exactly the same since replacement doesn't enter into the picture. It's a matter of taste which one to use.

I just want to show two more tricks that .roll knows about. One is that you can do .roll on a Bag:

> my $bag = (white-ball => 9, black-ball => 1).Bag;
bag(white-ball(9), black-ball)
> say $bag.roll         # 90% chance I'll get white-ball...
white-ball              # ...told ya

Perfect for all your weighted-randomness needs.

The other trick is that you can .roll on an enumeration:

> enum Dwarfs <Dipsy Grumpy Happy Sleepy Snoopy Sneezy Dopey>
> Dwarfs.roll
Happy                   # me too!
> enum Die <⚀ ⚁ ⚂ ⚃ ⚄ ⚅>
> Die.roll
⚄
> Die.roll(2)
⚀ ⚀                     # snake eyes!
> Die.roll(5).sort
⚀ ⚀ ⚁ ⚂ ⚃

The most frequent use I make of this is the idiom Bool.roll for randomly tossing a coin in my code.

if Bool.roll {
    # heads, I win
}
else {
    # tails, you lose
}

Bool is an enum by spec, even though it isn't one in Rakudo (yet) for circularity saw reasons.

> say Die.HOW
Perl6::Metamodel::EnumHOW.new()
> say Bool.HOW
Perl6::Metamodel::ClassHOW.new()    # audience goes "awwww!"

This has been a public service announcement. Enjoy your holidays, and have a merry Perl 6.

Day 6 – Running External Programs from Perl 6

December 6, 2014 by

There are several ways to run an external program in Perl 6, let’s take a look at each way in turn.

shell

Shell is used to pass an expression to the system shell. The shell function invokes the system shell and passes a string argument to it. The return value of `shell` is the exit code of the program:

    my $exit_code = shell 'echo "Hello, World!"';
    say $exit_code; # 0 == success

If you want redirects, pipes, globs and other shell features to work in your command lines, this is the way to go.

run

Run is used to execute a program. The run function accepts a stringified program name and optional list of arguments to pass to the external program. Like shell, the return value is the exit code of the program:

    my $exit_code = run 'echo', 'Hello, World!';
    say $exit_code; # 0 == success

Capturing Output

What if you wanted to capture the output of a command, rather than just know if the command was executed successfully or not? Just use a quoting operator “q” and “qq” with the “:x” adverb to execute an expression:

    my $message = q:x/echo "Hello, World!"/;
    say $message; # Hello, World!

If you want to interpolate the expression, use qq:

    my $audience = 'Everyone';
    my $message = qq:x/echo "Hello, $audience!"/;
    say $message; # Hello, Everyone!

With quoting mechanisms in Perl 6, the colon is optional for the first adverb so they can be slightly shortened as “qx” and “qqx”.

You’re not limited to slurping results into a single scalar either. Split strings into a list:

    # get a list of paths (also see %*ENV)
    my @env_path = qx/echo $PATH/.split(':'); # Unix-based systems 
    my @env_path = qx/echo %PATH%/.split(';'); # Windows version

Or read a program’s output into an array of lines:

    my @commit_results = qqx/git commit -am "$message"/.lines;

Or extract a particular value from tabular data:

    # requires "free" program
    my $free_memory_mb = qx/free -m/.lines[1].words[3]

These examples just scratch the surface of what’s possible, but they should give you enough to get started any time you need to run an external program. Give them a shot!

More Info

Day 5 – Act with great Responsibility

December 5, 2014 by

The past year in Rakudo-land has been very exciting. It has brought us a new, full-fledged backend in the form of MoarVM. And with that came extended support for asynchronous execution of code. This means you can now more easily try out all these asynchronous features, without having to suffer from the long startup time of the JVM backend.

The “Supply” is still the main mechanism to use the Reactive Programming paradigm. Here is a simple (silly) example of using a Supply:

1: my $s = Supply.new;
2: $s.act: { print " Fizz" if $_ %% 3 }
3: $s.act: { print " Buzz" if $_ %% 5 }
4: $s.act: { print " $_" unless $_ %% 3 || $_ %% 5 }
5: $s.emit($_) for 1..20;
======================
1 2 Fizz 4 Buzz Fizz 7 8 Fizz Buzz 11 Fizz 13 14 Fizz Buzz 16 17 Fizz 19 Buzz

Let’s take this example apart. Line 1 just creates a new Supply. By itself, it doesn’t do anything at all, except for creating the object, of course. Lines 2, 3 and 4 set up code (closures) to be executed every time a new value arrives in the Supply $s.

Line 5 sends 20 consecutive values to the Supply. This in turn causes the closures to be executed (in turn for each value emitted). The result is a nice FizzBuzz.

But but…

Careful watchers of this code may wonder about two things. First of all: what’s emit ? Is that something new? No, it is not (really). It used to be called more. At the Austrian Perl Workshop last October it was decided to rename this method to emit (as nobody was really happy with any more). So what happens if you have code that uses more? Well, it will still work, but you will get a deprecation warning after your process finishes:

Saw 1 call to deprecated code during execution.
===============================================
Method more (from Supply) called at:
t/spec/integration/advent2014-day05.t, line 13
Deprecated since v2014.10, will be removed with release v2015.10!
Please use emit instead.
-----------------------------------------------

Note that even though the same method was called 20 times, it is only mentioned once. And that’s because all the calls happened at the same location in the code. If you have seen deprecation messages before, you might notice that it now also mentions when it was deprecated, and when the feature will most likely be removed.

The second thing that a careful watcher might wonder about is: why the use of act? Is that a also a rename (from tap)? No, it is not. The act method was added last April. It ensures that your closure will be the only one running on that Supply, even if the emits are coming from different threads. That is a much safer default than tap, where multiple threads can be executing the closure you provided at the same time.

Sharing everything comes with great power and great responsibility

So why is this important? This is important because all variables visible in a specific scope, are readable and changeable by all threads simultaneously. Reading a variable from different threads at the same time, is not an issue in Perl 6 (apart from some minor known issues to be resolved after the Great List Refactor). Changing a variable from multiple threads simultaneously, is an issue that may cause data loss, memory corruption and segfaults. To give an example:

1: my $times = 100000;
2: my $a = 0;
3: $*SCHEDULER.cue: { $a++ } for ^$times; # this may segfault because of unguarded changes
4: sleep 5; # poor man’s way to wait for all code to finish
5: say “Missed {$times - $a} updates”;
===================================
Missed 9 updates

Line 1 sets up the number of iterations we’re going to do. Line 2 initializes the variable that’s going to be incremented. Although ++ is magic, we initialize the variable explicitly, so that each increment will follow the same internal code path. Line 3 cues the increment to be executed that many times (in a very low level way using the underlying scheduler). Line 4 waits for the execution to be finished. And line 5 shows the result (if we didn’t segfault, which we might).

So please use act rather than tap, unless you are sure that closures will not be updating any shared variables simultaneously. Which could be hard to tell if you’re using external modules inside the closure.

FizzBuzzing some more

Coming back to the first example: should we have used act there, or could we have used tap? We could have used tap there, because print is threadsafe. It becomes a different story if we would have used a (shared) array instead of print:

1: my @seen;
2: my $s = Supply.new;
3: $s.act: { @seen.push: "Fizz" if $_ %% 3 }
4: $s.act: { @seen.push: "Buzz" if $_ %% 5 }
5: $s.act: { @seen.push: $_ unless $_%%3 || $_%%5 }
6: await do for 1..20 {
7:     start { rand.sleep; $s.emit($_) }
8: }
9: say @seen;
==========
11 14 4 Fizz 17 19 Fizz Buzz 16 13 Buzz Fizz Buzz 7 Fizz 8 2 Buzz 1 Fizz Fizz

Hmmm… that has the right number of Fizz’s and Buzz’s, but clearly the order is wrong. But because we have used act rather than tap, we are at least sure that no push was executed simultaneously on the @seen array. So we didn’t lose any values, nor did we segfault.

You might ask: what are that lines 6-8 about? Well, that’s making sure the emit’s are done in a random order, spread out in time (because the 0 to 1 second random sleep from rand.sleep, and the start making it run in a separate thread).

So, how could you make this work in a parallel way? Well generally, if you have some kind of identifier for each value emitted, then you can use that to store the result in a shared data-structure. In this case, $_ is exactly that. So with a slight modification:

1: my @seen;
2: my $s = Supply.new;
3: $s.act: { @seen[$_] = "Fizz" if $_ %% 3 }
4: $s.act: { @seen[$_] ~= "Buzz" if $_ %% 5 }
5: $s.act: { @seen[$_] //= $_ }
6: await do for 1..20 {start{rand.sleep;$s.emit($_)}}
7: say @seen[1..20];
=================
1 2 Fizz 4 Buzz Fizz 7 8 Fizz Buzz 11 Fizz 13 14 FizzBuzz 16 17 Fizz 19 Buzz

By assigning directly to an element in the array (lines 3, 4, 5), we automatically put the information in the right place. And because we are (implicitely) guaranteed that act’s on the same item are always executed in the order they were created, we can simplify the non-FizzBuzz case in line 5.

Conclusion

When using Supplies, please use act on the Supply if you’re not sure whether the closure is going to change shared variables. Only use tap if you’re really 100% sure.

Further reading

Last year saw 2 async execution related posts, the latter showing uses of tap. Please read up on them if the above was not telling you all you needed to know:

Tests

The above code can also be found in roast at t/spec/integration/advent2014-day05.t

Day 4 – Composers, Coercers, and the Case of a Camel’s Curious Corner

December 4, 2014 by

“Every programming language has its curious corners, it’s just that some languages’ corners are curiouser than others’.”

When viewed from space, flame wars over programming languages are all about flinging various code or other examples around in order to prompt Yet Another Potentially Awkward Explanation (YAPAE) from the other side.

Even God has to give a YAPAE every once in a while

Even God has to give a YAPAE every once in a while… [image courtesy of xkcd++]

Everyone seems to know that Perl has corners. What is perhaps less famous than these corners themselves is the fact that their YAPAEs by and large make total sense. It may be a very Perlish form of sense, quite particular to Perl 5’s associativity, context, or precedence rules, but the reasoning always works out right. The language itself has a pretty simple set of rules, it’s the interactions of those rules that any good YAPAE code snippet will actually be highlighting.

Yet if the explanation has the potential to make sense, then it is only ever potentially awkward. Behind every YAPAE is the story of a decision in language design, a place where some of the implicit rules of operation suddenly become explicit1. As such they form the intersecting seams of design choices that were made at the core language level. The explanations are never all that awkward to people who understand those choices and agree with them. I like to think of these seams in the language as “curious corners,” which refers both to the surface-level surprise that comes from a YAPAE as well as my own curiosity in the ways different languages handle their seams that show.

Perl 5 does some pretty incredible things with what syntactically amount to a few rules about precedence, associativity, and context. There’s a lot more to the language, to be sure, but when something blows up in your face it is almost one of these categories that you’ve botched. As such, almost any of syntactical surprise in Perl 5 can be solved with liberal application of parentheses. (If Perl 5 is the duct tape of the internet, then the parenthesis is the duct tape of Perl 5). These usually work together so well that you only notice them where they collide in strange ways, and once you get bitten you start to know better. That said, it was these curious corners which led to the development of a new Perl, which would have different rules and different interactions of those rules. Some of these rules exist precisely to avoid YAPAEs from previous versions.

The original text that I’m using as a source for this blog post began as something of a screed: I thought I had found a curious corner to which I was firmly on the “awkward” side of. However my opinion flipped completely about halfway through the analysis that was to form the basis of my argument. I hope by sharing the process of that reasoning with you that I can convince you of just how low the potential awkwardness of this YAPAE is.

So, join me as we explore a ‘curious corner’ of Perl 6, a journey that will take us straight to the source.

set() which is also Set.new() and ().Set which is also Set()


    > Set("wise men" => 3, "star" => 1, "camels" => 9)
    set(wise men, star, camels)

    > ( "wise men" => 3, "star" => 1, "camels" => 9 ).Set
    set(wise men, star, camels)

    > set("wise men" => 3, "star" => 1, "camels" => 2)
    set("wise men" => 3, "star" => 1, "camels" => 2)

    > Set.new("wise men" => 3, "star" => 1, "camels" => 2)
    set("wise men" => 3, "star" => 1, "camels" => 2)

Well, there’s gotta be more than one way to do it, right? This is Perl, after all…

When I first ran into this, it actually upset me a fair bit. I hack in Perl at my $day_job2 and YAPAEs are a part of life when you are bringing new devs into the world of Perl. So I saw that set() and Set() both exist while also both doing different things and bit my fingernails / ranted on #perl6.

If you read that log, you can maybe tell that I’m a bit worked up. Until TimToady stated it simply: composers are not coercers.

Even though I understood the distinction more clearly, I was still a bit unconvinced and went off to investigate.

Composers like Set.new() and set()

Luckily in Rakudo, investigating the internals of the language mainly implies reading how one would implement Perl 6, using Perl 6! All the examples here are taken from their respective files in src/core without modification or truncation.

First we will take a look at some behavior, and then at the code which implements it.


    > set "we wish" => "you", "a" => "merry", "camel" => "ride"
    set("we wish" => "you", "a" => "merry", "camel" => "ride")

That looks exactly like a Perl 6 subroutine invocation: calling set with an argument which is a single List of Pairs. A quick git grep 'sub set' in rakudo.git reveals this declaration:


   #~~( set_operators.pm ) 
   sub set(*@args --> Set) { Set.new(@args) }

It appears to be some syntactic sugar, a sub which takes all of its arguments as a single list (the * prefix is ‘slurpy’) and then passes it to a constructor. I don’t know about you, but that sugar tastes guilt-free to me. Yet our investigation leads us elsewhere.


    #~~( Set.new() ... is in `submethod Build`? )
    submethod BUILD (:%elems) {
        nqp::bindattr(self, Set, '%!elems', %elems);
    }

There’s no .new definition to be found, but what is this submethod business? It sure sounds constructor related. This is a good candidate for looking up in the Perl 6 specification, which tells us that submethods are “infrastructural methods that shouldn’t be inherited by subclasses” (S12). Useful to note is that BUILD is not a phaser, but it shares Huffman coding3 with them: the all-caps is a hint that the code will run automatically.

blessed be the BUILDers

“The bless method automatically calls all appropriate BUILD routines for the current class, which initializes the object in least-derived to most-derived order,” from Synopsis 12: Objects. This is illustrated by the following snippet:


   class Elf {
        submethod BUILD { 
            say "This Elf is busy BUILDing"; 
        }
        method new {
            say "A shiny new Elf";
        }
    }

    role Build { 
        method new { 
            say "Did someone want to BUILD something?";
            self.bless;
        } 
    }

    class Santa is Elf does Build {
       submethod BUILD { 
          say "Even Santa BUILDs";
       }
    }
    Santa.new;
    
    Did someone want to BUILD something?
    This Elf is busy BUILDing
    Even Santa BUILDs

It is the self.bless invocation in Build.new that triggers BUILDALL, which descends the inheritance chain, calling a non-inherited BUILD if it is present. Elf, which does not call self.bless in it’s new method, would only produce the following when constructed:


    > Elf.new
    A shiny new Elf

BUILD is never called here, but it will be if any descendant or role were to mix in a self.bless in a constructor. For this reason, it is generally a good idea to call self.bless when you create a new method, especially if it is in a subclass. Unless, of course, you’ve got other plans for introducing the .bless call.

Back to the investigation

So, the presence of a BUILD in our Set class means that it wants to do something special during initialization. Even though I don’t know NQP, I can guess at what this code might do by asking myself, “what does a set want to do that is so special?”

The presence of the BUILD method is interesting, but as we’ve just seen, it doesn’t do anything without a .new. Just another file over from Set.pm in src/core is Setty.pm. That looks promising, especially if we noticed class Set does Setty in Set.pm and recognized the role declaration syntax.


    #~~( Setty.pm )
    method new(*@args --> Setty) {
        my %e;
        %e{$_.WHICH} = $_ for @args;
        self.bless(:elems(%e));
    }

Well, the %e{$_.WHICH} = $_ for @args line in the new method from is already taking care of one key component of a Set: making a unique list by internally storing each element of the (again slurpy) @args in a hash. That to me implies that the NQP code is taking the other important bit: immutability. Once again we can reach into src/core to see if we can confirm that suspicion. Perl 6 has a mutable form of Set, called SetHash. Let’s check that out.


    #~~( SetHash.pm )
    # No submethod BUILD to be found...

… so it appears that our assumption is correct! Now we know how to implement ‘hard immutability’ for an attribute in a pinch, even if we don’t have a spare second in the heat of that pinch to learn NQP.

Coercers like ().Set and Set()

Coercers are different. They don’t just flatten your lists into a big list, they reduce any Pairs in your list to just the key, or left-hand value, of any given Pair.


    > %( somehow => "special", snow => "flake" ).Set
    set(somehow, snow)

And because Any defines the .Set method, this allows a Set() call with an argument which descends from Any to coerce. If you are curious you can investigate in Synopsis 13: Overloading.


    > Set( (special => "is", as => "does") )
    set(special, as)

Note I didn’t have to turn it into a hash there. The code below shows that it will take all the thingies in the list and extract keys if it can. Otherwise it will “be on its merry,” so to speak.


    #~~( Any.pm )
    method Set()     {     Set.new-from-pairs(self.list) }

    #~~( Set.pm )
    method new-from-pairs(*@pairs --> Setty) {
        my %e;
        for @pairs {
            when Pair {
                %e{.key.WHICH} //= $_.key if .value;
            }
            default {
                %e{.WHICH} //= $_;
            }
        }
        self.bless(:elems(%e));
    }

Et voila! Coercion of a list of Pairs (or a Hash, as in the first example) no longer produces a set of Pairs as it does in the composer, but rather a set of the keys of those Pairs.


    > (special => "is", as => "does", 'a', bag 'of', 
      'snowflakes', 'snowflakes', 'snowflakes').Set
    set(special, as, a, bag(of, snowflakes(3)))

In contrast to earlier versions of Perl, 'a' does not become a key with the value of bag(of, snowflakes(3)). The difference with Perl 6 is that a Pair, denoted by the two terms separated by =>, is it’s very own type — not just a ‘fat comma’ as we have in Perl 5 (just one example of how P5 YAPAEs influenced the design of P6). The behavior is implemented by the when/default behavior: if you are a Pair, we put your key into the set. Otherwise we stuff your object as a whole in, just as we do in the composer.


    > Set( (special => "is", as => "does", 'a' => bag 'of',
      'snowflakes', 'snowflakes', 'snowflakes') )
    set(special, as, a)

Note the extra pair of parentheses when using the Set() form of the coercer. This is because Pairs are used to pass named parameters to subroutines. We can resolve this either by adding another pair of parentheses, as above, or by quoting our keys (which you can see by comparing this last snippet against the very first one in the post).

Conclusion

I hope that this brief, if twisty, tour of composers and coercers in the context of Set has introduced you to a few new dynamics in Perl 6. If nothing else, if it encourages an impulse to pop open src/core to perform your own investigation then I will consider it a success. A merry Christmas, and a jolly curious corner to you all!

Footnotes:

  • 1 Why else would they result in so much cursing?
  • 2 As opposed to $day-job ;)
  • 3 We need a shorthand which represents Huffman Coding. Perhaps we can borrow a character from a logographic language with a similar associated meaning to stand in?

Day 3 – cap your junctions

December 3, 2014 by

“With great power comes great responsibility.” These days I muse on how this applies not just to movie superheroes, but to programming language features as well.

It happens all over the place. Database connection? Awesome. Well, until you become the victim of an SQL injection or a SELECT N+1 problem. Regular expressions? Also awesome… until someone uses it to parse HTML and gets a visitation from elder non-beings from beyond the fabric of reality. Ouch.

Of course, we still like our powerful features. But we need to learn to contain them, to harness them. The awesome should work for us and the purpose we have set it to. It should not work for attackers, elder non-beings, or Murphy. We need to make sure the awesome doesn’t leak beyond its designated boundaries.

In 2009, Matthew Walton blogged about junctions. His explanation of what makes junctions exciting is great, and I agree with it. Today I would simply like to add the following point: junctions are so awesome that they should be contained.

Let’s demonstrate the problem real quickly.

> my @list = 3..7
3 4 5 6 7
> my $answer = all(@list) > 0;

Quick, what value ends up in $answer? Well, all the values in @list are
indeed greater than 0, so…

all(True, True, True, True, True)

Aw, dang. Not again.

Just to be clear, this is by spec, and correctly implemented and everything. But unless you ask specifically for Perl 6 to collapse the junction, it just keeps on going. Keeps on going, like a juggernaut, through walls and oceans, propagating through your program whether you want it to or not.

What you have to do then is to collapse the junction, putting a cap on all that awesome. The junction is useful as long as you’re doing the junctive computation itself (in this case all(@list) > 0), but immediately after that, we’ll want to collapse back into the normal, sane, non-junctive world:

> ?$answer      # symbolic prefix
True
> so $answer    # listop prefix
True
> $answer.Bool  # coerce method
True

Moritz points out that the whole “collapsing” metaphor comes from quantum physics, where something collapses as it’s measured. The analogy holds up; after we’re done junctioning around with our “quantum” superposition, we want to get a single boolean value back out. We do that by “measuring” — boolifying — the junction.

I will hasten to add that the most common use of junctions doesn’t suffer from this problem, namely junctions in if statements:

if all(@list) > 0 {
...
}

The if statement itself provides a natural cap to the junction. It does its own boolification of it under the hood, but more importantly, the junction value doesn’t stick around. We only see the effects of the boolean value of the junction.

No, the real risk comes when you’re a library writer, and you provide routines that happen to compute things using junctions:

sub all-positive(@list) {
    all(@list) > 0;     # DANGER
}

That’s where you want to remember the old tired adage. With great power… comes… right, that’s right. So you cap your junction before you let it run out into the wild and do untold structural damage to people, buildings, cattle, and those cute fluffy dogs that would otherwise suffer Puppy Death By Junction.

sub all-positive(@list) {
    so all(@list) > 0;
}

Yes. That’s better.

If you declare your return type, you can get a runtime error for leaving out the `so`.

sub all-positive(@list --> Bool) {
    all(@list) > 0;
}

say all-positive([1, 2, 3]); # Type check failed for return value;
                             # expected 'Bool' but got 'Junction'

Maybe that’s an argument for declaring return types for your routines. And maybe some day we’ll catch that one at compile-time, too.

Cap your junctions as early as possible.
— masak’s Rule of Responsible Junction Use

The longest I can remember legitimately keeping junctions around in an uncollapsed state was when storing junctions of regexes in a hash as part of this password-analyzing script. Even that’s not an exception to the above rule, though — note that the first thing that happens to the junctions in the subsequent code is that they become the rhs in a smartmatch statement, and then get capped by an if statement. So even here, they don’t escape out into the wild.

Code responsibly. Cap your junctions.

Day 2 – Rakudobrew

December 2, 2014 by

Traditionally in Perl, and probably in any other scripting language (I’m going to use the term “scripting” to describe “one that requires a runtime”, bear with me) there’s quite a big discrepancy between the latest released version of the toolchain and the one that your OS distribution ships (if it ships one at all!). My Ubuntu for instance, the freshly updated 14.10 not only offers me to install a nine-month-old Rakudo 2014.03, but also one running on top of Parrot, which I don’t remember voluntarily running anytime during 2014, frankly. Even if you look at languages with a longer release cycle, like Perl 5 which releases every year, system distributions shipping old versions is still a big deal: modern Perl modules still often maintain backwards compatibility with Perl 5.8 released in 2002, because that’s what some versions of CentOS comes with.

A popular idea is to say that system Perl (or system Python, system Ruby or whatever floats your boat) is for the system itself, while we programmers can freely uses whatever version we wish as long as we install it and maintain it in our home directory and not mess with system stuff. Perl 5 people usually recommend either Perlbrew or plenv for that; in Perl 6 rakudobrew is often said to be the easiest way to stay up-to-date with recent developments.

Rakudobrew is a commandline tool that makes it easy to install and update different flavours of Rakudo, as well as switch between them any time you want to. It also updates Panda, the module installer, after every update, along with all modules you had installed before (precompiled modules need to be updated after every update of Rakudo).

I made an effort to keep rakudobrew reasonably free of dependencies: it only requires Perl 5 and git, which are roughly what’s required to build Rakudo anyway. It’s known to work well on Linuxes and OSX, but I’ve also recently got some pull requests to improve Windows support (which apparently worked for a while by itself, without me knowing about it. Cool!) so it may happen to be useful to you, our Windows-running readers just as well.

Rakudobrew requires almost zero setup to get running; the only thing you need to do is to get it from Github (either by cloning it or downloading a tarball from the website), and get your system to recognize its bin/ subdirectory as part of your PATH. What I’m usually going for on my machines is the following:

$ git clone https://github.com/tadzik/rakudobrew ~/.rakudobrew
$ export PATH=~/.rakudobrew/bin:$PATH
$ echo 'export PATH=~/.rakudobrew/bin:$PATH' >> ~/.zshrc
$ # or ~/.bashrc or whatever

After that it requires nothing like ‘rakudobrew init’ or anything; it’s ready to get useful. The most basic thing you need from it is of course installing Rakudo:

$ rakudobrew build moar

The above command will download and build Rakudo on MoarVM, the latest version available from Github. For extra stability you may want to install the latest available release and just stick to that:

$ rakudobrew build moar 2014.11

This will take a while, but after a while it should finish saying “Done, moar-2014.11 built.” From that point on you are ready to go; if you adjusted your PATH correctly, the “perl6″ command will be available for you to use, and you’re ready to hack!

That said, you may want something more from life than just the bare compiler. The “useful and usable” Perl 6 distribution called Rakudo Star doesn’t have its own Starbrew, but that doesn’t mean you can’t eat the cake and still have it. Let’s also install Panda, the module installer, and whatever Rakudo Star usually ships.

$ rakudobrew build panda
$ panda install Task::Star

This gets you not only the module installer (which is instantly ready to go, just as shown above), but also all the widely appreciated goodies like: NativeCall, Term::ANSIColor, DBIish or Grammar::Debugger. Life’s good!

But what if you want more from life than just MoarVM? Perhaps you want to try out the famous JVM interop? Couldn’t be easier!

$ rakudobrew build jvm
$ rakudobrew switch jvm

Contrary to what it may suggest the above commands don’t build the JVM itself (you’re expected to have that installed), but the version of Rakudo running on the Java Virtual Machine. Note that you’ll still have to install panda and all the modules you want; those are installed per every instance of Rakudo you built. In case you got lost in all the versions you installed, the list command is there to help you again:

$ rakudobrew list
  jvm-HEAD
  moar-2014.11
* moar-HEAD

The asterisk is marking the one currently active (‘rakudobrew current’ can also tell you that). You don’t need to type the full name either; both ‘rakudobrew switch jvm’ and ‘rakudobrew switch moar-H’ will work just fine.

In case you want to get rid of something you installed (perhaps you wanted to test your code on some older version of Rakudo, or see how slow things were back in the days (the further you reach the more amazing it gets)), it’s as easy as removing a subdirectory from rakudobrew’s directory:

$ rm -rf ~/.rakudobrew/jvm-HEAD

Or in case you want to get rid of the entire rakudobrew for any reason, you don’t have to be afraid of any globally installed leftovers: there aren’t any. It’s all self-contained, so as simple as

$ rm -rf ~/.rakudobrew

And poof! Gone. Before you get rid of it though, you may want to keep your still warm Rakudo around to try out more interesting stuff that our Advent Calendar will have to show you in the coming weeks. Stay tuned!

Perl 6 Advent Calendar 2014: Table of Contents

December 1, 2014 by

This post serves as a table of contents for the 2014 Perl 6 advent calendar. Links to new posts will appear here during the course of this month.

Day 01 – The State of Perl 6 in 2014

December 1, 2014 by

Welcome to the 6th annual edition of the Perl 6 advent calendar, code name 2014.

In 2014, MoarVM has become the de facto standard backend for the Rakudo Perl 6 compiler. Parrot and JVM are still supported as backends, but MoarVM provides much lower memory usage, faster startup time, and is significantly faster than parrot at runtime.

Rakudo on MoarVM also has concurrency support, including reactive programming.

Much work has been done to improve performance, some in MoarVM, some in Rakudo and NQP. Maybe most notably is the JIT compiler for MoarVM that came out of the Google Summer of Code project by Bart Wiegmans.

Another Google Summer of Code project by Filip Sergot brought us HTTP::UserAgent, a versatile HTTP client library with SSL/TLS support.

During the Austrian Perl Workshop in fall 2014, many of the Perl 6 and Rakudo core contributors met, and identified three roadblocks for a "final" Perl 6.0 release: GLR, NFG and NSA.

Here GLR stands for the Grand List Refactoring, a plan to make the list-y types more transparent, list iteration faster, and more obvious when a list will flatten.

NFG is the Normal Form Grapheme, a plan for implementing grapheme-based string indexing.

Finally, our NSA has nothing to do with surveillance. Natively Shaped Arrays are a flexible feature for declaring typed, potentially multi-dimensional arrays, potentially with pre-defined dimensions. It will make memory-efficient matrix storage and operations possible, for example.

With these three blockers defined, Larry Wall submitted a talk to FOSDEM called Get ready to party!, predicting that 2015 will be the year that Perl 6 will get a production release.

Sorry, this was meant to become a summary of the current state of Perl 6, and it derailed into an outlook. To allow me to keep the "state" in the title, let me just tell you that Rakudo on the MoarVM is quite fun to work with. It's fast enough for small (and sometimes even mid-sized) tasks, module installation works fairly reliably, and the range of available modules has also increased.

Also I feel that any attempt to summarize the progress of this awesome community is bound to be very incomplete; I hope that my fellow Perl 6 hackers will fill in some details in the upcoming 23 posts.

Have a nice pre-Christmas time, and enjoy the show!


Follow

Get every new post delivered to your Inbox.

Join 43 other followers