Archive for December, 2013

Day 04 — Heredocs, Theredocs, Everywheredocs docs

December 4, 2013

So let’s say you’ve got a bit of documentation to print out, a help statement perhaps. You could use an ordinary string, but it always looks like something you really shouldn’t be doing.

    sub USAGE {
        say "foobar Usage:
    ./foobar <args> <file>



Perl 6 has a much better idea for you, fortunately: heredocs! They work a bit differently from Perl 5, and are now invoked using the adverb :heredoc on quoting constructs:

    say q:heredoc/END/;
    Hello world!

When you use :heredoc, the contents of the string are no longer the final contents; they become the string that signifies the end of a heredoc. q"END" results in the string "END", q:heredoc"END" results in everything before the next END to appear on its own line.

You will have also noticed that heredocs only start on the next possible line for them to start, not immediately after the construct closes. That semicolon after the construct never gets picked up as part of a heredoc, don’t worry :) .

The :heredoc adverb is nice, but it seems a bit long, doesn’t it? Luckily it has a short form, :to, which is much more commonly used. So that’s what we’ll be using through the rest of the post.

    say q:to"FIN";
    Hello again.

You can use any sort of string for the delimiter, so long as there’s no leading whitespace in it. A null delimiter (q:to//) is fine too, it just means you end the heredoc with two newlines, effectively a blank line.

And yes, delimiters need to be on their own line. This heredoc never ends:

    say q:to"noend";
    HELLO WORLD noend

A note about indentation: look at this heredoc

    say q:to[finished];
      Hello there

Which of those three heredoc lines decides how much whitespace is removed from the beginning of each line (and thus sets the base level of indentation)? It’s the line with the end delimiter, “finished” in the last example. Lines with more indentation than the delimiter will appear indented by however much extra space they use, and lines with less indentation will be as indented as the delimiter, with a warning about the issue.

(Tabs are considered to be 8 spaces long, unless you change $?TABSTOP. This usually doesn’t matter unless you mix spaces and tabs for indentation anyway though.)

It doesn’t matter how much the delimiter indentation is, all that matters is indentation relative to the delimiter. So these are all the same:

    say q:to/END/;
    say q:to/END/;
    say q:to/END/;

One other thing to note is that what quoting construct you use will affect how the heredoc contents are parsed, so

    say q:to/EOF/;
    $dlrs dollars and {$cnts} cents.

Interpolates nothing,

    say q:to:c/EOF/;
    $dlrs dollars and {$cnts} cents.

Interpolates just {$cnts} (the :c adverb allows for interpolation of just closures), and

    say qq:to/EOF/;
    $dlrs dollars and {$cnts} cents.

Interpolates both $dlrs and {$cnts}.

Here’s the coolest part of heredocs: using more than one at once! It’s easy too, just use more than one heredoc quoting construct on the line!

    say q:to/end1/, qq:to/end2/, Q:to/end3/;
    This is q.\\Only some backslashes work though\t.
    $sigils don't interpolate either.
    This is qq. I can $interpolate-sigils as well as \\ and \t.
    Neat, yes?
    This is Q. I can do \\ no \t such $things.

Which, assuming you’ve defined $interpolate-sigils to hold the string "INTERPOLATE SIGILS", prints out

    This is q.\Only some backslashes work though\t.
    $sigils don't interpolate either.
    This is qq. I can INTERPOLATE SIGILS as well as \ and   .
    Neat, yes?
    This is Q. I can do \\ no \t such $things.

After every end delimiter, the next heredoc to look for its contents starts.

Of course, indentation of different heredocs will help whenever you have to stack a bunch of them like this.

    say qq:to/ONE/, qq:to/TWO/, qq:to/THREE/, qq:to/ONE/;
    The first one.
        The second one.
    The third one.
        The fourth one.

Which outputs:

    The first one.
    The second one.
    The third one.
    The fourth one.

(And yes, you don’t have to come up with a unique end delimiter every time. That could have been four q:to/EOF/ statements and it’d still work.)

One final note you should be aware of when it comes to heredocs. Like the rest of Perl 6 (barring a couple of small exceptions), heredocs are read using one-pass parsing (this means your Perl 6 interpreter won’t re-read or skip ahead to better understand the code you wrote). For heredocs this means Perl 6 will just wait for a newline to start reading heredoc data, instead of looking ahead to try and find the heredoc.

As long as the heredoc contents and the statement that introduces the heredoc are part of the same compilation unit, everything’s fine. In addition to what you’ve seen so far, you can even do stuff like this:

    sub all-info { return q:to/END/ }
    This is a lot of important information,
    and it is carefully formatted.

(If you didn’t put the brace on the same line, it would be part of the heredoc, and then you’d need another brace on a line after END.)

However, things like BEGIN blocks start compiling before normal code, so trying that last one with BEGIN block fails:

    BEGIN { say q:to/END/ }
    This is only the BEGINning.

You have to put the heredoc inside the BEGIN block, with the quoting construct, in order to place them in the same compilation unit.

    BEGIN {
        say q:to/END/;
        This is only the BEGINning.

That’s it for heredocs! When should you use them? I would say whenever you need to type a literal newline (by hitting Enter) into the string. Help output from the USAGE sub is probably the most common case. The one at the beginning could easily (and more readably) be written as

    sub USAGE {
        say q:to"EOHELP";
            foobar Usage:
            ./foobar <args> <file>



Day 03 – Rakudo Perl 6 on the JVM

December 3, 2013

There have been a number of exciting developments for Perl 6 during 2013. In this post, we’ll take a look at one of them in some detail: running Perl 6 on the JVM (Java Virtual Machine).

Why the JVM?

There are many reasons for a language to have an implementation that targets the JVM. Here are some that drove us to bring Perl 6 to this platform.

  • The JVM is a stable, widely deployed,  trusted-in-the-enterprise platform. There are places where they don’t mind which language you write it, but they do care that it can run on the JVM.
  • The JVM has been very well optimized over the years. It most certainly isn’t fast to get started – but for long running things it typically performs well.
  • These days, the JVM is most certainly not just for Java. In fact, the commitment to run other languages – including those very different to Java – is serious. For example, there’s now a yearly JVM Language Summit, and the invokedynamic instruction and infrastructure was added in JDK7, and being improved in JDK8. Since Perl 6 is a gradually typed language, a VM that can play host to both static and dynamic languages is a good fit. Furthermore, every other major dynamic language is on the JVM. So, why not Perl too?
  • The JVM has widely used, well exercised support for concurrent, parallel and asynchronous programming. A wide range of primitives are available. Given that before this year, the Perl 6 story in these areas was also rather weak so far as implementation went, being on the JVM would provide an opportunity for fast prototyping and exploration, to help drive things forward.

But…another Perl 6 implementation?!

Implementing Perl 6 is a large undertaking – as those of us who got sucked into the process along the way have discovered. Many languages have got to the JVM by having a JVM-specific implementation of the language: JRuby, Jython, Nashorn, etc. For Perl 6, we’ve taken a different path.

The Rakudo Perl 6 compiler may only have targeted Parrot for much of its life, but those designing it have had VM portability in mind for a good while. Furthermore, the basic architecture has always been to have strongly isolated compilation stages, communicating by well-defined data structures. This put Rakudo in a good place to gain a JVM backend – at least, in theory. Over the course of the last year, what we hoped would work out well in theory has played out very nicely in practice.

The vast majority of the Rakudo codebase is not in any way VM-specific. Better still, the bits that need to change most often and that undergo most active development are almost always VM-independent. Many developers working on Rakudo test their changes against a single backend, and it’s relatively uncommon to find breakage on the other backend as a result. That said, we have automated daily spectest runs to catch any regressions.


First, let’s consider the specification test suite. You might think at this point I’d mention how close Rakudo on JVM is to passing the number of specification tests that Rakudo on Parrot does. In fact, we need to do it the other way around these days: Rakudo on Parrot passes 99.64% of the spectests that Rakudo on the JVM does. “Huh,” you might think. “How’d the JVM backend come out ahead?” The answer is relatively simple: the JVM backend runs a bunch of concurrency tests that we don’t run on the Parrot backend. There actually are a small number of tests (tens rather than hundreds) that only pass on Rakudo on Parrot, largely due to “interesting” edge-case behaviors that have yet to be hunted down. However, these days the vast majority of programs run unmodified on both.

In the wider ecosystem, things are not quite so polished yet. Panda, the module installer, runs on Rakudo on the JVM. However, a number of modules depend on the NativeCall library, for calling into native code. The NativeCall porting effort is very much underway; last time I looked at it I could do basic things, like calling simple Win32 APIs. But it’s not all the way there yet. This is, however, really the last major missing piece. To say this time last year, we couldn’t run Perl 6 on the JVM at all, we’ve come a very long way.

Is it faster?

Well, it depends. For quick one liners and short-running scripts? No, startup will kill you. For something long running? Yes, usually it’s faster, and sometimes it’s significantly faster (perhaps five times or even forty times). And that’s before we’ve really done a great deal of optimization work on the JVM backend; the focus thus far has largely been “make it work”.

Can I call Java libraries?

Yes, but… :-) We do have some basic interop support in place already. Here’s an example:

use java::util::zip::CRC32:from<java>;

my $crc =;
for 'Hello, Java'.encode('utf-8') {
say $crc.getValue();

It doesn’t look all that bad until you hit the method call in the loop. What’s that funny method/update/(B)V thing about? In Java you can statically overload methods. When there’s no overloading, we quite happily give you a short name. When there’s multiple, for now you need to use the JVM’s method descriptors to indicate the desired one. We’ll improve that, and many other aspects of interop, over the coming months. In summary, it’s often quite possible to call code from Java libraries today, it’s just not pleasant yet.

The future

Much has been done, yet of course there’s still plenty to do. Once NativeCall support is in shape, we’ll be able to add the JVM as an option to the Rakudo Star distribution release (for now, it’s only available in compiler releases – or fresh from Git, of course). Beyond that, the main areas of focus will be convergence, Java interop and performance. Given that this year took us from zero JVM support to Rakudo on JVM being the implementation passing the most spectests, it’s exciting to think where we’ll be in another year from now.

Day 02 – The humble type object

December 2, 2013

For some inscrutable reason, we have defined a Dog class in today’s post.

class Dog {
    has $.name;

Don’t ask me why — maybe we’re writing software for a kennel? Maybe we’re writing software for dogs? “Teach your dog how to type!” Clever dogs can do up to 10 words a minute, with surprisingly few typos.

Anyway. Having a Dog class gives us the dubious pleasure of being able to create dogs out of thin air and passing them to functions. No surprise there.

sub check(Dog $d) {
    say "Yup, that's a dog for sure.";

my Dog $dog .= new(:name);
check($dog);     # Yup, that's a dog for sure.

But where there might be some surprise — if you haven’t gotten used to the idea of Perl 6’s type objects yet — is that you can also do this:

check(Dog);      # Yup, that's a dog for sure.

What did we just do there?

We didn’t pass a dog to the function, we passed Dog to the function. And the function accepts it and says it’s a dog, too. So, Dog is a dog. Huh.

What’s going on here is that in Perl 6 when we declare a class Dog like we just did, the word Dog ends up having two meanings:

  • The class Dog that we declared.
  • The type object Dog, kind of a patron saint for all Dog objects ever instantiated.

Cramming two concepts into one word like this seems like a recipe for failure. But it turns out it works surprisingly well.

Before we look at the reasons for using type objects, let’s find out a bit more about what they are.

say Dog;          # (Dog)
say;     # ERROR: Cannot look up attributes in a type object
say ?Dog;         # False
say defined Dog;  # False

So, in summary, the Dog type object identifies itself as (Dog), it refuses to have its attribute inspected, it boolifies to False, and it’s not defined. Contrast this with an instance of Dog:

say $dog;         # => "Fido")
say $;    # Fido
say ?$dog;        # True
say defined $dog; # True

An instance is everthing the type object isn’t: it knows how to output itself, it will happily tell you its name, and it’s both True and defined. Nice.

(Being undefined is only almost a surefire way to identify the type object. Someone could have gone through the trouble of making their instance object undefined. As they say in the industry, you don’t have to be a type object to be undefined… but it helps!)

And now, as promised, the Top Five Reasons Type Objects Work Surprisingly Well:

  1. Classes actually make sense as objects in the program. There’s this idea that classes have to haughtily refuse to play among the rest of the values in a program, that they have to somehow be like gods looking down on the instances from a Parthenon of class-hood. Perl 5 kind of has it like that. But both Ruby and Python show that classes can behave as more or less normal objects. In Perl 6, Dog is also what you get if you do $dog.WHAT.
  2. It fits quite well with the whole smartmatching thing. So what $dog ~~ Dog actually means is something like “hey, Dog type object, does this $fido look anything like you?”. The type object doesn’t just sit there, it does useful things like smartmatching.
  3. Another thing: the whole reason that line, my Dog $dog .= new(:name);, works as it does is because we end up calling .new on the type object. So here’s what that line does in slow motion. It desugars to a declaration and an assignment. The declaration is my Dog $dog; and so, because the $dog variable needs to start out with some undefined value, it starts out with Dog, the type object. The assignment then is simply $dog.=new, which is short for $dog = $ Conveniently, because the type object Dog is an object of the type Dog, it has a .new method (inherited from Mu in this case) that knows how to construct dogs.
  4. A little detail from that last point, which actually turns out to be a rather big deal: Perl 6 doesn’t really have undef like Perl 5 does. It turned out that undef wasn’t a really good fit with a type system; undef gets in everywhere, and doesn’t really have a type at all. (A bit like Java’s null which is known to have caused people no end of suffering.) So what Perl 6 has instead are these typed undefined values, namely — you guessed it — the type objects. If you declare a variable my Int $i, then $i will start out as undefined, that is, containing the type object Int.
  5. Not only do you sometimes want to call .new on the type object, sometimes you have other methods which don’t require an instance. (These kinds of methods are sometimes known as static methods in some languages, and class methods in other languages. Some languages have both of these, and they’re different, just to be confusing.) Again, the type object comes to the rescue here, sort of acts like an instance so that you can call your method on it, and then once again fades into the background. For example, if the class Dog had a method bark { say "woof" } then the Dog type object would be able to bark just as well as actual dog instances. (But the type object still refuses to tell you its .name, or any of its attributes.)

So that’s type objects for you. They’re sitting in a convenient semantic spot halfway between the class and its instances, sometimes representing one end of the spectrum, sometimes the other.

One thing before we part ways today. It doesn’t happen often, but sometimes you do want to be able to tell type objects and real instances apart, for example when accepting parameters in a function:

multi sniff(Dog:U $dog) {
    say "a type object, of course"
multi sniff(Dog:D $dog) {
    say "definitely a real dog instance"

sniff Dog;    # a type object, of course
sniff $dog;   # definitely a real dog instance

Here, :U stands for “undefined” and :D for “defined”. (And that, dear friends, is how we got a smiley into the design of Perl 6. Program language designers, take heed.) As I mentioned parenthetically before, it’s actually possible to be an undefined object without being a type object. For these special occasions, we have :T, but this modifier isn’t implemented in Rakudo as of this writing. (Though moritz++ informs me that, in Rakudo, :U currently has the semantics that :T should have.)

Let’s just end this post with maybe the corniest one-liner ever to see the light of day in #perl6:

$ perl6 -e 'say (my @a = "Hip " x 2), @a.^name, "!"'
Hip Hip Array!

Day 01 – The State of Perl 6 in 2013

December 1, 2013

Welcome to the 2013 Perl 6 advent calendar!

In Perl 6 land, 2013 will be remembered as the year that brought proper concurrency support.

But I'm getting ahead of myself.

There is also sad news. Niecza, the Perl 6 compiler on the CLR (.NET/Mono) platform, and the Perl 6 compiler with the best runtime characteristics, had its last release in March. Since then there were a few maintenance patches and new built-in types and routines, but little in terms of actual compiler features.

A little later, Rakudo gained support to run on the Java Virtual machine. There are still some bits missing, mostly notably support for the native call interface, but all in all it works quite well, passes more than 99.9% of the tests that Rakudo on Parrot passes, and has two key advantages: it is much faster at run time, and has proper concurrency/parallelism support.

Jonathan Worthington prototyped and implemented it, and later specified it in S17, which again led to lots of improvements. Stay tuned for more advent calendar posts on the JVM and concurrency/parallelism topics.

Another big news this year was the revelation of MoarVM, a virtual machine designed to run Perl 6. With the JVM's high startup time and Parrot being mostly unmaintained and having lots of unsolved problems, there is a niche to be filled. NQP, the "Not Quite Perl" Perl 6 compiler used to bootstrap Rakudo already runs on MoarVM; Rakudo support for MoarVM is on its way, and progressing well so far.

There was also lots of progress in terms of built-in types likes Set and Bag, and IO::Path for handling path and directory objects.

As a developer and early adopter, I find Perl 6 to be pleasant to work with. In 2013 it has gotten easier to use, due to better error reporting and improved IO.


Get every new post delivered to your Inbox.

Join 50 other followers