Author Archive

Traits — Meta Data With Character

December 4, 2011

Traits are a nice, extensible way to attach meta data to all sorts of objects in Perl 6.

An example is the is cached trait that automatically caches the functions return value, based on the argument(s) passed to it.

Here is a simple implementation of that trait:

 # this gets called when 'is cached' is added
 # to a routine
 multi sub trait_mod:<is>(Routine $r, :$cached!) {
     my %cache;
     #wrap the routine in a block that..
     $r.wrap(-> $arg {
         # looks up the argument in the cache
         %cache.exists($arg)
             ?? %cache{$arg}
             # ... and calls the original, if it
             # is not found in the cache
             !! (%cache{$arg} = callwith($arg))
         }
     );
 }
 
 # example aplication:
 sub fib($x) is cached {
     say("fib($x)");
     $x <= 1 ?? 1 !! fib($x - 1) + fib($x - 2);
 }
 # only one call for each value from 0 to 10
 say fib(10);

A trait is applied with a verb, here is. That verb appears in the routine name that handles the trait, here trait_mod:<is>. The arguments to that handler are the object on which the trait is applied, and the name of the trait (here cached) as a named argument.

Note that a production grade is cached would need to handle multiple arguments, and maybe things like limiting the cache size.

In this example, the .wrap method is called on the routine, but of course you can do whatever you want. Common applications are mixing roles into the routine or adding them to a dispatch table.

Traits can not only be applied to routines, but also to parameters, attributes and variables. For example writable accessors are realized with the is rw trait:

 class Book {
     has @.pages is rw;
     ...
 }

Traits are also used to attach documentation to classes and attributes (stay tuned for an addvent calendar post on Pod6), marking routine parameters as writable and declaring class inheritance and role application.

This flexibility makes them ideal for writing libraries that make the user code look like a domain-specific language, and supplying meta data in a safe way.

Buffers and Binary IO

December 3, 2011

Perl 5 is known to have very good Unicode support (starting from version 5.8, the later the better), but people still complain that it is hard to use. The most important reason for that is that the programmer needs to keep track of which strings have been decoded, and which are meant to be treated as binary strings. And there is no way to reliably introspect variables to find out if they are binary or text strings.

In Perl 6, this problem has been addressed by introducing separate types. Str holds text strings. String literals in Perl 6 are of type Str. Binary data is stored in Buf objects. There is no way to confuse the two. Converting back and forth is done with the encode and decode methods.

    my $buf = Buf.new(0x6d, 0xc3, 0xb8, 0xc3, 0xbe, 0x0a);
    $*OUT.write($buf);

    my $str = $buf.decode('UTF-8');
    print $str;

Both of those output operations have the same effect, and print møþ to the standard output stream, followed by a newline. Buf.new(...) takes a list of integers between 0 and 255, which are the byte values from which the new byte buffer is constructed. $*OUT.write($buf) writes the $buf buffer to standard output.

$buf.decode('UTF-8') decodes the buffer, and returns a Str object (or dies if the buffer doesn’t consistute valid UTF-8). The reverse operation is $Buf.encode($encoding). A Str can simply be printed with print.

Of course print also needs to convert the string to a binary representation somewhere in the process. There is a default encoding for this and other operations, and it is UTF-8. The Perl 6 specification allows the user to change the default, but no compiler implements that yet.

For reading, you can use the .read($no-of-bytes) methods to read a Buf, and .get for reading a line as a Str.

The read and write methods are also present on sockets, not just on the ordinary file and stream handles.

One of the particularly nasty things you can accidentally do in Perl 5 is
concatenating text and binary strings, or combine them in another way (like with join or string interpolation). The result of such an operation is a string that happens to be broken, but only if the binary string contains any bytes above 127 — which can be a nightmare to debug.

In Perl 6, you get Cannot use a Buf as a string when you try that, avoiding that trap.

The existing Perl 6 compilers do not yet provide the same level of Unicode support as Perl 5 does, but the bits that are there are much harder to misuse.

Day 22 – The Meta-Object Protocol

December 22, 2010

Have you ever wondered how to create a class in your favorite programming language, not by writing a class definition, but by running some code? Some languages allow that by simple API calls. The API behind it is called the Meta-Object Protocol, short MOP.

Perl 6 has a MOP, and it allows you to create classes, roles and grammars, add methods and attributes, and to introspect classes. For example we can use calls to the MOP in Rakudo to find out how the Rat type (rational numbers) is implemented. Calls to methods of the MOP generally start with .^ instead of just .:

 $ perl6
 > say join ', ', Rat.^attributes
 $!numerator, $!denominator
 > # the list of all methods is a bit long,
 > # so here is a random selection
 > say join ', ', Rat.^methods(:local).pick(5)
 unpolar, ceiling, reals, Str, round
 > say Rat.^methods(:local).grep('log').[0].signature.perl
 :(Numeric $x: Numeric $base = { ... };; *%_)

Most of these lines should be fairly self-explanatory: objects of class Rat has two attributes, $!numerator and $!denominator, as well as many methods. The log method takes a Numeric value as invocant (marked by the colon after the parameter name), and an optional second parameter called $base, which has a default value (but which Rakudo can’t show you. It’s Euler’s number).

A nice use case comes from the Perl 6 database interface. It has the option to log calls on an object, and to restrict this to only log methods from a certain role (for example only a role related to connection management, or related to data retrieval). Here is the example, and a possible way to call it:

 sub log-calls($obj, Role $r) {
     my $wrapper = RoleHOW.new;
     for $r.^methods -> $m {
         $wrapper.^add_method($m.name, method (|$c) {
             # print logging information
             # note() writes to standard error
             note ">> $m";
             # call the next method of the same name,
             # with the same arguments
             nextsame;
         });
     }
     $wrapper.^compose();
     # the 'does' operator works just like 'but', but
     # only modifies a copy of the object
     $obj does $wrapper;
 }
 role Greet {
     method greet($x) {
         say "hello, $x";
     }
 }
 class SomeGreeter does Greet {
     method LOLGREET($x) {
         say "OH HAI "~ uc $x;
     }
 }
 my $o = log-calls(SomeGreeter.new, Greet);
 # logged, since provided by role Greet
 $o.greet('you');
 # not logged, because not provided by the role
 $o.LOLGREET('u');

Output:

 >> greet
 hello, you
 OH HAI U

So with a Meta-Object Protocol, classes, roles and grammars are not just accessible by special syntax, but can be accessed as a normal API. This gives new flexibility to object oriented code, and allows easy introspection and modification of objects.

Day 8 – Different Names of Different Things

December 8, 2010

(By masak and moritz)

Newcomers to the Perl programming language, version 5, often complain that
they can’t reverse strings. There’s a built-in reverse function, but it doesn’t seem to work at first glance:

    $ perl -E "say reverse 'hello'"
    hello

When such programmers ask more experienced Perl programmers, the solution is quickly found: reverse has actually two different operation modes. In list context it reverses lists, in scalar context it reverses strings.

    $ perl -E "say scalar reverse 'hello'"
    olleh

Sadly this an exception from Perl’s usual context model. For most operators or functions the operator determines the context, and the data is interpreted in that context. For example + and * work on numbers, and . works on strings (concatenation). So a symbol (or function name, in the case of uc) stands for an operation, and provides context. Not so reverse.

In Perl 6, we try to learn from previous mistakes, and get rid of historical inconsistencies. This is why list reversal, string reversal and hash inversion have been split up into separate built-ins:

    # string reversal, sorry, flipping:
    $ perl6 -e 'say flip "hello"'
    olleh
    # reversing lists
    # perl6 -e 'say join ", ", reverse <ab cd ef>'
    ef, cd, ab
    # hash inversion
    perl6 -e 'my %capitals = France => "Paris", UK => "London";
              say %capitals.invert.perl'
    ("Paris" => "France", "London" => "UK")

Hash inversion differs from the other two operations in that the result is of a different type than the input. Since hash values need not be unique, inverting a hash and returning the result as a hash would either change the structure (by grouping the keys of non-unique values together somehow), or lose information (by having one value override the other ones).

Instead hash inversion returns a list of pairs, and the user can decide which mode of operation they want for turning the result back into a hash, if that’s necessary at all.

Here’s how you do it if you want the hash inversion operation to be non-destructive:

    my %inverse;
    %inverse.push( %original.invert );

As pairs with already-seen keys get pushed onto the %inverse hash, the original value isn’t overriden but instead "promoted" to an array. Here’s a small demonstration:

    my %h;
    %h.push('foo' => 1);    # foo => 1
    %h.push('foo' => 2);    # foo => [1, 2]
    %h.push('foo' => 3);    # foo => [1, 2, 3]

All three (flip/reverse/invert) coerce their arguments to the expected type (if possible). For example if you pass a list to flip, it is coerced into a string and then the string is rever^W flipped.

Day 5 – Why Perl syntax does what you want

December 5, 2010

Opening the fifth door of our advent calendar, we don’t find a recipe of how to do something cool with Perl 6 – rather an explanation of how some of the intuitiveness of the language works.

As an example, consider these two lines of code:

    say 6 / 3;
    say 'Price: 15 Euro' ~~ /\d+/;

They print out 2 and 15, respectively. For a Perl programmer this is not surprising. But look closer: the forward slash / serves two very different purposes, the numerical division in the first line, and delimits a regex in the second line.

How can Perl know when a / means what? It certainly doesn’t look at the text after the slash to decide, because a regex can look just like normal code.

The answer is that Perl keeps track of what it expects. Most important are two things it expects: terms and operators.

A term can be literal like 23 or "a string". After parser finds such a literal, there can either be the end of a statement (indicated by a semicolon), or an operator like +, * or /. After an operator, the parser expects a term again.

And that’s already the answer: When the parser expects a term, a slash is recognized as the start of a regex. When it expects an operator, it counts as a numerical division operator.

This has far reaching consequences. Subroutines can be called without parenthesis, and after a subroutine name an argument list is expected, which starts with a term. On the other hand type names are followed by operators, so at parse time all type names must be known.

On the upside, many characters can be reused for two different syntaxes in a very convenient way.

Day 3 – File operations

December 3, 2010

Directories

Instead of opendir and friends, in Perl 6 there is a single dir subroutine, returning a list of the files in a specified directory, defaulting to the current directory. A piece of code speaks a thousand words (some result lines are line-wrapped for better readability):

    # in the Rakudo source directory
    > dir
    build parrot_install Makefile VERSION parrot docs Configure.pl 
    README dynext t src tools CREDITS LICENSE Test.pm
    > dir 't'
    00-parrot 02-embed spec harness 01-sanity pmc spectest.data

dir has also an optional named parameter test, used to grep the results


    > dir 'src/core', test => any(/^C/, /^P/)
    Parcel.pm Cool.pm Parameter.pm Code.pm Complex.pm
    CallFrame.pm Positional.pm Capture.pm Pair.pm Cool-num.pm Callable.pm Cool-str.pm

Directories are created with mkdir, as in mkdir('foo')

Files

The easiest way to read a file in Perl 6 is using slurp. slurp returns the contents of a file, as a String,


    > slurp 'VERSION'
    2010.11

The good, old way of using filehandles is of course still available

    > my $fh = open 'CREDITS'
    IO()<0x1105a068>
    > $fh.getc # reads a single character
    =
    > $fh.get # reads a single line
    pod
    > $fh.close; $fh = open 'new', :w # open for writing
    IO()<0x10f3e704>
    > $fh.print('foo')
    Bool::True
    > $fh.say('bar')
    Bool::True
    > $fh.close; say slurp('new')
    foobar

File tests

Testing the existence and types of files is done with smartmatching (~~). Again, the code:

    > 'LICENSE'.IO ~~ :e # does the file exist?
    Bool::True
    > 'LICENSE'.IO ~~ :d # is it a directory?
    Bool::False
    > 'LICENSE'.IO ~~ :f # a file then?
    Bool::True

Easy peasy.

File::Find

When the standard features are not enough, modules come in handy. File::Find (available in the File::Tools package) traverses the directory tree looking for the files you need, and generates a lazy lists of the found ones. File::Find comes shipped with Rakudo Star, and can be easily installed with neutro if you have just a bare Rakudo.

Example usage? Sure. find(:dir<t/dir1>, :type<file>, :name(/foo/)) will generate a lazy list of files (and files only) in a directory named t/dir1 and with a name matching the regex /foo/. Notice how the elements of a list are not just plain strings: they’re objects which strinigify to the full path, but also provide accessors for the directory they’re in (dir) and the filename itself (name). For more info please refer to the documentation.

Useful idioms

Creating a new file
    open('new', :w).close
"Anonymous" filehandle
    given open('foo', :w) {
        .say('Hello, world!');
        .close
    }

Day 2 – Interacting with the command line with MAIN subs

December 2, 2010

In Unix environment, many scripts take arguments and options from the command line. With Perl 6 it’s very easy to accept those:

    $ cat add.pl
    sub MAIN($x, $y) {
        say $x + $y
    }
    $ perl6 add.pl 3 4
    7
    $ perl6 add.pl too many arguments
    Usage:
    add.pl x y

By just writing a subroutine called MAIN with a signature, you automatically get a command line parser, binding from the command line arguments into the signature variables $x and $y, and a usage message if the command line arguments don’t fit.

The usage message is customizable by adding another sub called USAGE:

    $ cat add2.pl
    sub MAIN($x, $y) {
        say $x + $y
    }
    sub USAGE() {
        say "Usage: add.pl <num1> <num2>";
    }
    $ perl6 add2.pl too many arguments
    Usage: add.pl <num1> <num2>

Declaring the MAIN sub as multi allows declaring several alternative syntaxes, or dispatch based on some constant:

    $ cat calc
    #!/usr/bin/env perl6
    multi MAIN('add', $x, $y)  { say $x + $y }
    multi MAIN('div', $x, $y)  { say $x / $y }
    multi MAIN('mult', $x, $y) { say $x * $y }
    $ ./calc add 3 5
    8
    $ ./calc mult 3 5
    15
    $ ./calc
    Usage:
    ./calc add x y
    or
    ./calc div x y
    or
    ./calc mult x y

Named parameters correspond to options:

    $ cat copy.pl
    sub MAIN($source, $target, Bool :$verbose) {
        say "Copying '$source' to '$target'" if $verbose;
        run "cp $source $target";
    }
    $ perl6 copy.pl calc calc2
    $ perl6 copy.pl  --verbose calc calc2
    Copying 'calc' to 'calc2'

Declaring the parameter as Bool makes it accept no value; without a type constraint of Bool it will take an argument:

    $ cat do-nothing.pl
    sub MAIN(:$how = 'fast') {
        say "Do nothing, but do it $how";
    }
    $ perl6 do-nothing.pl
    Do nothing, but do it fast
    $ perl6 do-nothing.pl --how=well
    Do nothing, but do it well
    $ perl6 do-nothing.pl what?
    Usage:
    do-nothing.pl [--how=value-of-how]

In summary, Perl 6 offers you built-in command line parsing and usage messages, just by using subroutine signatures and multi subs.

Writing good, declarative code has never been so easy before.

Day 19: Whatever

December 19, 2009

Opening the door to today’s present, you find… Whatever. Yes, Whatever is a type in Perl 6, and it stands for whatever it makes sense in the context it appears in.

Examples:

1..*                 # infinite range
my @x = <a b c d e>;
say @x[*-2]          # indexing from the back of the array
                     # returns 'd'
say @x.map: * ~ 'A'; # concatenate A to whatever the
                     # thing is we pass to it
say @x.pick(*)       # randomly pick elements of @x
                     # until all are used up

So how does this magic work?

Some of these uses are easy to see: A * in term position produces a Whatever object, and some builtins (like List.pick) know what to do with it.

The in term position might need some more explanation: Perl 6 parses predictively; when the compiler reads a file, it always knows whether to expect a term or an operator:

say  2 + 4
|    | | |
|    | | + term (literal number)
|    | + operator (binary +)
|    +  term (literal number)
+ term (listop), which expects another term

So if you write

* * 2

it parses the first * as term, and the second one as an operator.

The line above generates a code block: * * 2 is short for -> $x { $x * 2 }. That’s a thing you can invoke like any other sub or block:

my $x = * * 2;
say $x(4);     # says 8

Likewise

say @x.map: * ~ 'A';

is a shortcut for

say @x.map: -> $x { $x ~ 'A' };

and

say @x.map: *.succ;

is a shortcut for

say @x.map: -> $x { $x.succ };

Whatever is also useful in sorting — for example, to sort a list numerically (a prefix ‘+’ means to obtain a numeric value of something):

@list.sort: +*

And to sort a list as strings (a prefix ‘~’ means to get the string value of something):

@list.sort: ~*

The Whatever-Star is very useful, but it also allows some obfuscation (explanation).

The Whatever-Star was also an inspiration for the planned Rakudo Star release. Because we know that release won’t be a complete implementation of Perl 6, we didn’t want to call it “1.0″. But other release numbers and tags also presented their own points of confusion. Finally it was decided to call it “Rakudo Whatever”, which then became “Rakudo Star”. (Our detailed plans for Rakudo Star are kept the Rakudo repository.)

Day 4: Testing

December 4, 2009

Perl authors have a long tradition of shipping test cases with their modules, and in Perl 6 we plan to continue with that nice tradition.

And testing is very easy. The traditional perlish way is to print data in the Test Anything Protocol. But you don’t have to do that yourself, you can use a module for that.

Assume you have written a nice factorial function

 sub fac(Int $n) {
     [*] 1..$n
 }

Currently it doesn’t matter for us how that function works – we want to find out if it does. So let’s test it:

 use v6;

 sub fac(Int $n) {
     [*] 1..$n
 }

 use Test;
 plan 6;

 is fac(0), 1,  'fac(0) works';
 is fac(1), 1,  'fac(1) works';
 is fac(2), 2,  'fac(2) works';
 is fac(3), 6,  'fac(3) works';
 is fac(4), 24, 'fac(4) works';

 dies_ok { fac('oh noes i am a string') }, 'Can only call it with ints';

And let’s run it:

 $ perl6 fac-test.pl
 1..6
 ok 1 - fac(0) works
 ok 2 - fac(1) works
 ok 3 - fac(2) works
 ok 4 - fac(3) works
 ok 5 - fac(4) works
 ok 6 - Can only call it with ints

In detail: use Test; loads the testing module, plan 6; declares that we plan to run six tests. Then five lines of the pattern is $got, $expected, $description follow. is() does string comparison, but since integers always stringify the same way, that’s fine.

Finally with dies_ok { $some_code }, $description we test that calling the function with a non-integer argument is a fatal error.

The output contains the test plan 1..6, followed by one line for each test. That starts with ok (or not ok if the test failed), the test number, space, dash, space and test description.

If you run more tests, you don’t want to look through every test output carefully, but you want a summary. The prove command from Perl 5 gives you such a summary:

 prove --exec perl6 fac-test.pl
 fac-test.pl .. ok
 All tests successful.
 Files=1, Tests=6, 11 wallclock secs ( 0.02 usr  0.00 sys + 10.26 cusr  0.17 csys = 10.45 CPU)
 Result: PASS

You can also put all your test files in a directory, let’s call it t/, and run prove recursively on all .t files in that dir:

 prove --exec perl6 -r t

Putting that line in your Makefile is also nice, so that you can just type
make test to run the tests.


Follow

Get every new post delivered to your Inbox.

Join 36 other followers