Day 17: Gtk Mandelbrot

Day 17: Gtk Mandelbrot

Two years ago today, the Advent post was on making Mandelbrot sets in Perl 6. At the time, they were in black and white, slow to produce, Rakudo was prone to crashing, and the only user interface thing you could control was how big the resulting PPM file was.

As they say, that was then. This is now.

Full Mandelbrot set

The new script is 423 lines of Perl 6 code — targeted at Niecza, threaded, and using the GtkSharp library. It allows you to move and resize the windows, zoom in (left mouse button, drag across image to define zoom boundaries), create Julia set images (right click on a Mandelbrot set image), increase the number of iterations (press ‘m’), and output a PPM file for a window (press ‘s’).

The threading doesn’t actually improve performance on my MacBook Pro (still looking into why) but it does make the script much more responsive.

It would be far too long to go through all the code, but lets hit on the highlights. The core is almost unchanged:

        sub julia(Complex $c, Complex $z0) {
            my $z = $z0;
            my $i;
            loop ($i = 0; $i < $max_iterations; $i++) {
                if $z.abs > 2 {
                    return $i + 1;
                $z = $z * $z + $c;
            return 0;

It’s named julia instead of mandel now, because it is more general. If you call it with $z0 set to 0, it calculates the same thing the old mandel did. Allowing $z0 to vary allows you to calculate Julia sets as well.

The code around it is very different, though! Stefan O’Rear wrote the threading code, using Niecza’s Threads library, which is a thin wrapper on C#’s threading libraries, and probably not very close to what Perl 6’s built-in threading will look like when it is ready to go. He establishes a WorkQueue with a list of the work that needs to be done, and then starts N running threads, where N comes from the environment variable THREADS if it is present, and the reported processor count otherwise:

for ^(%*ENV<THREADS> // CLR::System::Environment.ProcessorCount) {{ });
} is pretty simple:

    method run() {
        loop {
            my $item = self.shift;
            next if $item.cancelled;

This is an infinite loop that starts by getting the next WorkItem off the queue, checks to see if it has been cancelled, and if it hasn’t, calls the .run Callable attribute and then the mark-done method.

The WorkItems on the queue look like this:

class WorkItem {
    has Bool $!done = False;
    has Bool $!cancelled = False;

    has Callable &.run;
    has Callable &.done-cb;

    method is-done() { WorkQueue.monitor.lock({ $!done }) }
    method mark-done() {
        &.done-cb.() unless WorkQueue.monitor.lock({ $!done++ })

    method cancelled() { WorkQueue.monitor.lock({ $!cancelled }) }
    method cancel() { WorkQueue.monitor.lock({ $!cancelled = True }) }

Each WorkItem has two flags, $!done and $!cancelled, and two Callable attributes, &.run, already mentioned as what is called by, and &.done-cb, which is the callback function to be called when the &.run method finishes.

The two methods (for now) we use in our WorkItem are relatively simple:

            sub row() {
                my $row = @rows[$y];
                my $counter = 0;
                my $counter_end = $counter + 3 * $width;
                my $c = $ur - $y * $delta * i;

                while $counter < $counter_end {
                    my $value = $is-julia ?? julia($julia-z0, $c) !! julia($c, 0i);
                    $row.Set($counter++, @red[$value % 72]);
                    $row.Set($counter++, @green[$value % 72]);
                    $row.Set($counter++, @blue[$value % 72]);
                    $c += $delta;

            sub done() {
                Application.Invoke(-> $ , $ {
                    $.draw-area.QueueDrawArea(0, $y, $width, 1);

            my $wi = => &row, done-cb => &done);
            push @.line-work-items, $wi;

As you might expect, row calculates one line of the set we are working on. It may look like it is using global variables, but these subs are actually local to the FractalSet.start-work method and the variables are local to it from there. The done invokes a Gtk function noting that a portion of the window needs to be redrawn (namely the portion we just calculated).

The above block of code is called once for each row of the fractal window being generated, which has the effect of queuing up all of the fractal to be handled as there are available threads.

Moving upward in the code’s organization, each fractal window we generate is managed by an instance of the FractalSet class.

class FractalSet {
    has Bool $.is-julia;
    has Complex $.upper-right;
    has Real $.delta;
    has Int $.width;
    has Int $.height;
    has Int $.max_iterations;
    has Complex $.c;
    has @.rows;
    has @.line-work-items;
    has $.new-upper-right;

    has $.draw-area;

$.is-julia and $.max_iterations are self-explanatory. $.upper-right is the fixed complex number anchoring the image. $.delta is the amount of change in the previous number per-pixel; we assume the pixels are square. $.width and $.height are the size of the window in pixels. $.c only has meaning for Julia sets, where it is the value $c in the equation $new-z = $z * $z + $c. @.rows the pixel information generated by the row sub above; @.line-work-items saves a reference to all of the WorkItems generating those rows. $.new-upper-right is temporary used during the zoom mouse operation. $.draw-area is the Gtk.DrawingArea for the related window.

Once all that is set up, the rest of the code is pretty straightforward. The Gtk windowing code is set up in

    method build-window()
        my $index = +@windows;

        my $window =$.is-julia ?? "julia $index" !! "mandelbrot $index");
        $window.Resize($.width, $.height);  # TODO: resize at runtime NYI

        my $event-box =;

        my $drawingarea = $.draw-area =;


We store a global array @windows tracking all the FractalSets in play. Each of the different objects here gets the "Id" data set to this set’s index into the @windows array so we can easily look up the FractalSet from callback functions. The rest of the method is just plugging the right callback into each component — simple conceptually but it took this Gtk novice a lot of work figuring it all out.

As an example, consider the KeyReleaseEvent callback, which responds to presses on the keyboard.

sub KeyReleaseEvent($obj, $args) {
    my $index = $obj.GetData("Id").ToInt32();
    my $set = @windows[$index];
    given $args.Event.Key {
        when 'm' | 'M' {
        when 's' | 'S' {

First we lookup the index into @windows, then we get the $set we’re looking at. Then we just call the appropriate FractalSet method, for instance

    method increase-max-iterations() {
        $.max_iterations += 32;

.stop-work cancels all the pending operations for this FractalSet, then we bump up the number of iterations, and then we .start-work again to queue up a new set of rows with the new values.

The full source code is here. As of this writing it agrees with the code here, but this is an active project, and probably will change again in the not-too-distant future. Right now my biggest goals are figuring out how to get the threading to actually improve performance on my MacBook Pro and cleaning up the code. Both suggestions and questions are welcome.

Where Have All The References Gone?

Where Have All The References Gone?

Perl 5 programmers that start to learn Perl 6 often ask me how to take a reference to something, and my answers usually aren’t really helpful. In Perl 6, everything that can be held in a variable is an object, and objects are passed by reference everywhere (though you don’t always notice that, because objects like strings and numbers are immutable, so there’s no difference to passing by value). So, everything is already treated as a reference in some sense, and there’s no point in explicitly taking references.

But people aren’t happy with that answer, because it doesn’t explain how to get stuff done that involved references in Perl 5. So here are a few typical use cases of references, and how Perl 6 handles them.

Creating Objects

In Perl 5, an object is really just a reference to a blessed value (but people usually say "blessed reference", because you virtually never use the blessed value without going through a reference).

So, in Perl 5 you’d write

 package My::Class;
 # constructor
 sub new { bless {}, shift };
 # an accessor
 sub foo {
     my $self = shift; 
     # the ->{} dereferences $self as a hash
     $self->{foo} // 5;
 # use the object:
 say My::Class->new->foo;

In Perl 6, you just don’t think about references; classes are much more declarative, and there’s no need for dereferencing anything anywhere:

 class My::Class {
     # attribute with accessor (indicated by the dot)
     # and default value
     has $.foo = 5;
 # use it:

If you don’t like the default constructor, you can still use bless explicitly, but even then you don’t have to think about references:

 method new() {
     # the * specifies the storage, and means "default storage"

So, no explicit reference handling when dealing with OO. Great.

Nested Data Structures

In both Perl 5 and Perl 6, lists flatten automatically by default. So if you write

 my @a = (1, 2);
 my @b = (3, 4);
 push @a, @b

then @a ends up with the four elements 1, 2, 3, 4, not with three elements of which the third is an array.

In Perl 5, nesting the data structure happens by taking a reference to @b:

 push @a, \@b;

In Perl 6, item context replaces this use of references. It is best illustrated by a rather clumsy method to achieve the same thing:

 my $temp = @b;
 push @a, $temp;  # does not flatten the two items in $temp,
                 # because $temp is a scalar

Of course there are shortcuts; the following lines work too:

 push @a, $(@b);
 push @a, item @b;

(As a side note, push @a, $@b is currently not allowed, it tries to catch a p5ism; I will also try to persuade Larry and the other language designers to allow it, and have it mean the same thing as the other two).

On the flip side you need explicit dereferencing to get the values out of item context:

 my @a = 1, 2;
 my $scalar = @a;
 for @a { 
     # two iterations
 for $scalar {
     # one iteration only
 for @$scalar {
     # two iterations again

This explicit use of scalar and list context is the closest analogy to Perl 5 references, because it requires explicit context annotations in the same places where referencing and dereferencing is used in Perl 5.

But it’s not really the same, because there are cases where Perl 5 needs references, but Perl 6 can deduce the item context all on its own:

 @a[3] = @b; # automatically puts @b in item context

Mutating Arguments

Another use references in Perl 5 is for passing data to routines that should be modified inside the routine:

 sub set_five; {
     my $x = shift;
     # explicit dereferencing with another $:
     $$x = 5;
 my $var;
 # explicit taking of a reference
 set_five \$var;

In Perl 6, there is a separate mechanism for this use case:

 sub set_five($x is rw) {
     # no dereferencing
     $x = 5;
 my $var;
 # no explicit reference taking
 set_five $var;

So again a use case of Perl 5 references is realized by another mechanism in Perl 6 (signature binding, or binding in general).


Nearly everything is a reference in Perl 6, but you still don’t see them, unless you take a very close look. The control of list flattening with item and list context is the one area where Perl 5’s referencing and dereferencing shines through the most.

Day 15 – Something Exceptional

Day 15 – Something Exceptional

The Perl 6 exception system is currently in development; here is a small example demonstrating a part of the current state:

 use v6;
 sub might_die(Real $x) {
     die "negative" if $x < 0;
 for 5, 0, -3, 1+2i -> $n {
     say "The square root of $n is ", might_die($n);
     CATCH {
         # CATCH sets $_ to the error object,
         # and then checks the various cases:
         when 'negative' {
             # note that $n is still in scope,
             # since the CATCH block is *inside* the
             # to-be-handled block
             say "Cannot take square root of $n: negative"
         default {
             say "Other error: $_";

This produces the following output under rakudo:

 The square root of 5 is 2.23606797749979
 The square root of 0 is 0
 Cannot take square root of -3: negative
 Other error: Nominal type check failed for parameter '$x'; expected Real but got Complex instead

A few interesting points: the presence of a CATCH block automatically makes the surrounding block catch exceptions. Inside the CATCH block, all lexical variables from the outside are normally accessible, so all the interesting information is available for error processing.

Inside the CATCH block, the error object is available in the $_ variable, on the outside it is available in $!. If an exception is thrown inside a CATCH block, it is not caught — unless there is a second, inner CATCH that handles it.

The insides of a CATCH block typically consists of when clauses, and sometimes a default clause. If any of those matches the error object, the error is considered to be handled. If no clause matches (and no default block is present), the exception is rethrown.

Comparing the output from rakudo to the one that niecza produces for the same code, one can see that the last line differs:

 Other error: Nominal type check failed in binding Real $x in might_die; got Complex, needed Real

This higlights a problem in the current state: The wording of error messages is not yet specified, and thus differs among implementations.

I am working on rectifying that situation, and also throwing interesting types of error objects. In the past week, I have managed to start throwing specific error objects from within the Rakudo compiler. Here is an example:

 $ ./perl6 -e 'try EVAL q[ class A; method { $!x } ]; say "error: $!"; say $!.perl'
 error: Attribute $!x not declared in class A
         name => "\$!x",
         package-type => "class",
         package-name => "A", filename => "",
         line => 1,
         column => Any,
         message => "Attribute \$!x not declared in class A"
 # output reformatted for clarity

The string that is passed to EVAL is not a valid Perl 6 program, because it accesses an attribute that wasn’t declared in class A. The exception thrown is of type X::Attribute::Undeclared, and it contains several details: the name of the attribute, the type of package it was missing in (could be class, module, grammar and maybe others), the name of the package, the actual error message and information about the source of the error (line, cfile name (empty because EVAL() operates on a string, not on a file), and column, though column isn’t set to a useful value yet).

X::Attribute::Undeclared inherits from type X::Comp, which is the common superclass for all compile time errors. Once all compile time errors in Rakudo are switched to X::Comp objects, one will be able to check if errors were produced at run time or at compile with code like

 EVAL $some-string;
     when X::Comp { say 'compile time' }
     default      { say 'run time'     }

The when block smart-matches the error object against the X::Comp type object, which succeeds whenever the error object conforms to that type (so, is of that type or a subclas of X::Comp).

Writing and using new error classes is quite easy:

 class X::PermissionDenied is X::Base {
     has $.reason;
     method message() { "Permission denied: $.reason" };
 # and using it somewhere:
 die reason => "I don't like your nose");

So Perl 6 has a rather flexible error handling mechanism, and libraries and applications can choose to throw error objects with rich information. The plan is to have the Perl 6 compilers throw such easily introspectable error objects too, and at the same time unify their error messages.

Many thanks go to Ian Hague and The Perl Foundation for funding my work on exceptions.

Meta-programming: what, why and how

Meta-programming: what, why and how

Sometimes, it’s good to take ones understanding of a topic, throw it away and try to build a new mental model of it from scratch. I did that in the last couple of years with object orientation. Some things feel ever so slightly strange to let go of and re-evaluate. For many people, an object really is “an instance of a class” and inheritance really is a core building block of OOP. I suspect many people who read this post will at this point be thinking, “huh, of course they really are” – and if so, that’s totally fair enough. Most people’s view of OOP will, naturally, be based around the languages they’ve applied object orientation in, and most of the mainstream languages really do have objects that are instances of classes and really do have inheritance as a core principle.

Step back and look around, however, and things get a bit more blurry. JavaScript doesn’t have any notion of classes. CLOS (the Common Lisp Object System) does have classes, but they don’t have methods. And even if we do just stick with the languages that have classes with methods, there’s a dizzying array of “extras” playing their part in the language’s OO world view; amongst them are interfaces, mixins and roles.

Roles – more often known as traits in the literature – are a relatively recent arrival on the OO scene, and they serve as an important reminder than object orientation is not finished yet. It’s a living, breathing paradigm, undergoing its own evolution just as our programming languages in general are.

And that brings me nicely on to Perl 6 – a language that from the start has set out to be able to evolve. At a syntax level, that’s done by opening up the grammar to mutation – in a very carefully controlled way, such that you always know what language any given lexical scope is in. Meta-programming plays that same role, but in the object orientation and type system space.

So what is a meta-object? A meta-object is simply an object that describes how a piece of our language works. What sorts of things in Perl 6 have meta-objects? Here’s a partial list.

  • Classes
  • Roles
  • Subsets
  • Enumerations
  • Attributes
  • Subroutines
  • Methods
  • Signatures
  • Parameters

So that’s meta-objects, but what about the protocol? You can read protocol as “API” or “interface”. It’s an agreed set of methods that a meta-object will provide if it wants to expose certain features. Let’s consider the API for anything that can have methods, such as classes and roles. At a minimum, it will provide:

  • add_method – adds a method to the object
  • methods – enables introspection of the methods that the object has
  • method_table – provides a hash of the methods in this type, excluding any that may be inherited

What about something that you can call a method on? It just has to provide one thing:

  • find_method – takes an object and a name, and returns the method if one exists

By now you may be thinking, “wait a moment, is there something that you can call a method on, but that does not have methods”? And the answer is – yes. For example, an enum has values that you can call a method on – the methods that the underlying type of the enumeration provides. You can’t actually add a method to an enum itself, however.

What’s striking about this is that we are now doing object oriented programming…to implement our object oriented language features. And this in turn means that we can tweak and extend our language – perhaps by subclassing an existing meta-object, or even by writing a new one from scratch. To demonstrate this, we’ll do a simple example, then a trickier one.

Suppose we wanted to forbid multiple inheritance. Here’s the code that we need to write.

my class SingleInheritanceClassHOW
    is Metamodel::ClassHOW
    method add_parent(Mu $obj, Mu $parent) {
        if +self.parents($obj, :local) > 0 {
            die "Multiple inheritance is forbidden!";
my module EXPORTHOW { }
EXPORTHOW.WHO.<class> = SingleInheritanceClassHOW;

What are we doing here? First, we inherit from the standard Perl 6 implementation of classes, which is defined by the class Metamodel::ClassHOW. (For now, we also inherit from Mu, since meta-objects currently consider themselves outside of the standard type hierarchy. This may change.) We then override the add_parent method, which is called whenever we want to add a parent to a class. We check the current number of (local) parents that a class has; if it already has one, then we die. Otherwise, we use callsame in order to just call the normal add_parent method, which actually adds the parent.

You may wonder what the $obj parameter that we’re taking is, and why it is needed. It is there because if we were implementing a prototype model of OOP, then adding a method to an object would operate on the individual object, rather than stashing the method away in the meta-object.

Finally, we need to export our new meta-object to anything that uses our module, so that it will be used in place of the “class” package declarator. Do do this, we stick it in the EXPORTHOW module, under the name “class”. The importer pays special attention to this module, if it exists. So, here it is in action, assuming we put our code in a module This program works as usual:

use si;
class A { }
class B is A { }

While this one:

class A { }
class B { }
class C is A is B { }

Will die with:

Multiple inheritance is forbidden!

At compile time.

Now for the trickier one. Let’s do a really, really simple implementation of aspect oriented programming. We’ll write an aspects module. First, we declare a class that we’ll use to mark aspects.

my class MethodBoundaryAspect is export {

Next, when a class is declared with “is SomeAspect”, where SomeAspect inherits from MethodBoundaryAspect, we don’t want to treat it as inheritance. Instead, we’d like to add it to a list of aspects. Here’s an extra trait modifier to do that.

multi trait_mod:<is>(Mu:U $type, MethodBoundaryAspect:U $aspect) is export {
    $aspect === MethodBoundaryAspect ??
        $type.HOW.add_parent($type, $aspect) !!
        $type.HOW.add_aspect($type, $aspect);

We take care to make sure that the declaration of aspects themselves – which will directly derive from this class – still works out by continuing to call add_parent for those. Otherwise, we call a method add_aspect, which we’ll define in a moment.

Supposing that our aspects work by optionally implementing entry and exit methods, which get passed the details of the call, here’s our custom meta-class, and the code to export it, just as before.

my class ClassWithAspectsHOW
    is Metamodel::ClassHOW
    has @!aspects;
    method add_aspect(Mu $obj, MethodBoundaryAspect:U $aspect) {
    method compose(Mu $obj) {
        for @!aspects -> $a {
        for self.methods($obj, :local) -> $m {
            $m.wrap(-> $obj, |args {
                $a.?entry($, $obj, args);
                my $result := callsame;
                $a.?exit($, $obj, args, $result);
my module EXPORTHOW { }
EXPORTHOW.WHO.<class> = ClassWithAspectsHOW;

Here, we see how add_aspect is implemented – it just pushes the aspect onto a list. The magic all happens at class composition time. The compose method is called after we’ve parsed the closing curly of a class declaration, and is the point at which we finalize things relating to the class declaration. Ahead of that, we loop over any aspects we have, and the wrap each method declared in the class body up so that it will make the call to the entry and exit methods.

Here’s an example of the module in use.

use aspects;
class LoggingAspect is MethodBoundaryAspect {
    method entry($method, $obj, $args) {
        say "Called $method with $args";
    method exit($method, $obj, $args, $result) {
        say "$method returned with $result.perl()";
class Example is LoggingAspect {
    method double($x) { $x * 2 }
    method square($x) { $x ** 2 }
say Example.double(3);
say Example.square(3);

And the output is:

Called double with 3
double returned with 6
Called square with 3
square returned with 9

So, a module providing basic aspect orientation support in 30 or so lines. Not so bad.

As you can imagine, we can go a long way with meta-programming, whether we want to create policies, development tools (like Grammar::Debugger) or try to add entirely new concepts to our language. Happy meta-hacking.

Bailador — A small Dancer clone

Bailador — A small Dancer clone

Today we’ll write a simple Dancer clone in Perl 6. Simple also means Very Minimal — it will only recognize basic GET requests. Let’s look at how the simplest Dancer app possible looks like:

    get '/' => sub {
        "Hello World!"

So we need something to add routes to our app, and something to run it. Let’s take care of adding routes first. We’ll create an array to store all those, and thus get() will just add them to it.

    my @routes;
    sub get(Pair $x) is export {
        @routes.push: $x;

In case you’re not familiar with the Pair thing, in Perl 6 a fat comma (=>) creates an actual data structure containing a key and a value. In this case, the key is just a string ‘/’, and the value is a subroutine.

Having @routes being a simple array of keys and values we can now write a simple dispatcher:

    sub dispatch($env) {
        for @routes -> $r {
            if $env<REQUEST_URI> ~~ $r.key {
                return $r.value.();
        return "404";

dispatch() takes a hash representing our environment, which contains the REQUEST_URI string, basing on which we’ll try to find an appropriate candidate to dispatch to.

The above example is also cheating a bit: it just returns a ‘404’ string instead of creating a proper HTTP response. Making it respond properly is left as an exercise for the reader (not the last one in this short article, I assure you :)).

Since we got that far already, writing our own dance() is a piece of cake. We’re going to call it baile() though. Why do we write all this in Spanish? If you can guess on which classes I was bored and wrote this thing on a piece of paper, then on the next YAPC I’ll show you the fastest possible way to tie a shoe. No kidding! But let’s finish this thing first.

Luckily we don’t need to write our own web server now. We have HTTP::Server::Simple::PSGI in Perl 6, which will do most of the hard work for us. The only thing we have to do is to create a PSGI app. In case you’ve never heard of it, a PSGI app is a mere subroutine, taking the environment as an argument, and returning an array of three things: an HTTP response code, an array of HTTP headers and a response body. Once we have our PSGI app ready, we just feed HTTP::Server::Simple::PSGI with it, and we’re good to go.

    sub baile is export {
        my $app = sub($env) {
            my $res = dispatch($env);
            return ['200', [ 'Content-Type' => 'text/plain' ], $res];

        given {
            .host = 'localhost';

Yep, we’re cheating again and returning 200 no matter what. Remember the part about “an exercise for the reader?” You can think about it while celebrating a working Dancer clone.

But wait, there’s more!

Let’s look at our dispatch() once again:

    sub dispatch($env) {
        for @routes -> $r {
            if $env<REQUEST_URI> ~~ $r.key {
                return $r.value.();
        return "404";

You probably noticed that we’ve used ~~ — a smartmatching operator. Thanks to that, we can match REQUEST_URI against strings, but not only. Junctions will work fine too:

    get any('/h', '/help', '/halp') => sub {
        "A helpful help message"

And regexes:

    get /greet\/(.+)/ => sub ($x) {
        "Welcome $x"

The last one will need a bit of tweaking in dispatch(). Yes, ~~ does the regex matching for us, but we have to take care of passing the match results to the callback function. Let’s modify the if body then:

    sub dispatch($env) {
        for @routes -> $r {
            if $env<REQUEST_URI> ~~ $r.key {
                if $/ {
                    return $r.value.(|$/.list);
                } else {
                    return $r.value.();
        return "404";

The if $/ part checks whether the match resulted in setting the Match object in the $/ variable. If it did, we flatten the Match to a list, and pass it to the callback function. We need a | prefix, so it becomes expanded to a parameter list instead of being passed as a mere array. From now on, the above example with greet will Just Work. Yay!

The Bailador code presented here is available in the Github repository. If you feel challenged by the “exercises for the reader”, or just want to make it a bit more proper Dancer port, you’re welcome to hack on it a bit and contribute. I hope I showed you how simple it is to write a simple, yet useful thing, and going with those simple steps we can hopefully get to something close to a full-blown Dancer port. Happy hacking, and have an appropriate amount of fun!

Exploratory parsing with Perl 6

Exploratory parsing with Perl 6

There have already been some delectable little grammar goodies this Advent Calendar. We hope to add to that today, with a discussion of the concept of “Exploratory Parsing” and Perl 6.

There’s no question that many modern programming languages having embraced regular expressions as a central part of the language or via a standard library. Some languages have introduced full-blown-parsing facilities as core features of their design. For example, many functional languages give us powerful parser combinators. Perl 6, as the astute advent reader knows, gives us “Regexes and Rules”. It turns out that Perl 6 Regexes are an implementation of parsing expression grammars, or PEGs for short, originally formulated by Bryan Ford.

I was inspired by a recent Ward Cunnigham post where he uses PEGs to explore seemingly unstructured text. Ward used an implementation of a PEG parser generator written in C by Ian Piumarta.

We however have a powerful PEG-like parser built right into the core of the Perl 6 language, so what better way to play with exploratory parsing than getting cozy with your favourite Perl 6 compiler, pouring yourself another glass of eggnog, and divining meaning from random text out on the interwebs!

Our first example is taken directly from Ward’s “Exploratory Parsing” post. If you haven’t at least perused it yet, may I encourage you to do so.

The first thing we do is retrieve some data to explore:


We then translate Ward’s first example into Perl 6:

use v6;

# Inspired by
# but using Perl 6 regexes and tokens

grammar ExploratoryParsing {
    token TOP {

    token fact { <key> <value> | <other_char> }

    token key { <whitespace> <word>+ ':' }

    token value { [<!key> .]+ }

    token word { <alpha>+ ' '* }

    token whitespace { '\n' ' '* }

    token other_char { . }


I don’t know about you, but I love how declarative this is. I encourage you to compare the two implementations. The translation is almost trivial, no?

We introduce a MAIN method which slurps up a data file, uses our grammar definition to parse the text, and tells us how many “facts” we’ve found.

sub MAIN() {
    my $text = slurp('world192.txt');
    say "Read world factbook. Parsing...";

    my $match = ExploratoryParsing.parse($text);
    say "Found ", +$match, " facts.";

Running this with the Rakudo Perl 6 compiler we get:

$ perl6
Read world factbook. Parsing...
Found 16814 facts.

If we use the awesomely awesome Grammar::Tracer or Grammar::Debugger already unwrapped for us earlier this month, we can even step further into this and explore matches.

The remaining embellishments from Ward’s original post are left as an exercise for the reader.

You can see how powerful this idea is. We start with some semi-structured text and use the power of Perl 6 Regexes and Rules to start pulling things apart, stirring the precipitate meaning and exploring pattern and trends we see in the data. This kind of work is trivial in a language like Perl 6 with powerful parsing support. You can even imagine jumping into a Perl 6 REPL and doing this interactively.

Hopefully this has whet your appetite for playing with Perl 6 regexes. Happy parsing.

Privacy and OOP

Privacy and OOP

There are a number of ways in which Perl 6 encourages you to restrict the scope of elements of your program. By doing so, you can better understand how they are used and will be able to refactor them more easily later, potentially aiding agility. Lexical scoping is one such mechanism, and subroutines are by default lexically scoped.

Let’s take a look at a class that demonstrates some of the object oriented related privacy mechanisms.

    class Order {
        my class Item {
            has $.name;
            has $.price;
        has Item @!items;
        method add_item($name, $price) {
            @!items.push($name, :$price))
        method discount() {
        method total() {
            self!compute_subtotal() - self!compute_discount();
        method !compute_subtotal() {
            [+] @!items>>.price
        method !compute_discount() {
            my $sum = self!compute_subtotal();
            if $sum >= 1000 {
                $sum * 0.15
            elsif $sum >= 100 {
                $sum * 0.1
            else {

Taking a look at this, the first thing we notice is that Item is a lexical class. A class declared with “my” scope can never be referenced outside of the scope it is declared within. In our case, we never leak instances of it outside of our Order class either. This makes our class an example of the aggregate pattern: it prevents outside code from holding direct references to the things inside of it. Should we ever decide to change the way that our class represents its items on the inside, we have complete freedom to do so.

The other example of a privacy mechanism at work in this class is the use of private methods. A private method is declared just like an ordinary method, but with an exclamation mark appearing before its name. This gives it the same visibility as an attribute (which, you’ll note, are also declared with an exclamation mark – a nice bit of consistency). It also means you need to call it differently, using the exclamation mark instead of the dot.

Private methods are non-virtual. This may seem a little odd at first, but is consistent: attributes are also not visible to subclasses. By being non-virtual, we also get some other benefits. The latest Rakudo, with its optimizer cranked up to its highest level, optimizes calls to private methods and complains about missing ones at compile time. Thus a typo:

    self!compite_subtotal() - self!compute_discount();

Will get us a compile time error:

    Undefined private method 'compite_subtotal' called (line 18)

You may worry a little over the fact that we now can’t subclass the discount computation, but that’s likely not a good design anyway; for one, we’d need to also expose the list of items, breaking our aggregate boundary. If we do want pluggable discount mechanisms we’d probably be better implementing the strategy pattern.

Private methods can, of course, not be called from outside of the class, which is also a compile time error. First, if you try:

    say $order!compute_discount;

You’ll be informed:

    Private method call to 'compute_discount' must be fully qualified
    with the package containing the method

Which isn’t so surprising, given they are non-virtual. But even if we do:

    say $o!Order::compute_discount;

Our encapsulation-busting efforts just get us:

    Cannot call private method 'compute_discount' on package Order
    because it does not trust GLOBAL

This does, however, hint at the get-out clause for private methods: a class may choose to trust another one (or, indeed, any other package) to be able to call its private methods. Critically, this is the decision of the class itself; if the class declaration didn’t decide to trust you, you’re out of luck. Generally, you won’t need “trusts”, but occasionally you may be in a situation where you have two very closely coupled classes. That’s usually undesirable in itself, though. Don’t trust too readily. :-)

So, lexical classes, private methods and some nice compiler support to help catch mistakes. Have an agile advent. :-)