Day 19 – Introspection

Perl 6 is an Object-Oriented Programming Language. Well, really, it’s multi-paradigm; but one of the options (that’s used throughout its core) is definitily Object Orientation.

However, as the (current) perl6.org tells us, Perl 6 supports “generics, roles and multiple dispatch”, which are some very nice features, and already covered in other advent calendar posts.

But the one we’re going to take a look at today is the MOP. “MOP” stands for “Meta-Object Protocol”. It means that, instead of objects, classes, etc defining the language; they’re actually a part you can change (or just look at) from the user’s side of things.

Indeed, in Perl 6, you can add methods to a type, remove some, wrap methods, augment classes with more capabilities (OO::Actors and OO::Monitors are two such examples), or you could totally redefine it (and, say, use a Ruby-like object system. Such an example here).

But today, instead, we’re gonna look at the first part: Introspection. Looking at a type after it’s been built, getting to know it, and use these informations.

The module we’re going to build together is based on a need for the Sixcheck module (a QuickCheck-like module): generate some random data for a type, then feed that data to the function we’re testing, and check some post-condition.

So, we write our first version:

my %special-cases{Mu} =
  (Int) => -> { (1..50).pick },
  (Str) => -> { ('a'..'z').pick(50).join('') },
;
sub generate-data(Mu:U \t) {
  %special-cases{t} ?? %special-cases{t}() !! t.new;
}
generate-data(Int);

okay, that’s the first version we wrote. We note a few things:

  • We specify the key type for %special-cases. That’s because, by default, the type is Str. Obviously, we don’t want our types to be stringified. What we actually do is specify they’re going to be subtypes of “Mu” (which is at the top of the types “food chain”)
  • We put parentheses around Int and Str, to avoid stringification.
  • We use the :U in the function’s argument’s type. That means the value has to be undefined. Type objects (as in Int, Str, etc) are undefined, so it serves us well (a different unknown value you’ve probably seen is Nil).
  • Type objects really are… Objects, like any other object (more on that later).
    That’s why we can call .new on them, like here. It’s the same as directly calling Int.new, for example (that’s useful for consistency, and autovivification).
  • We provide a fallback for Int and Str, because calling Int.new and Str.new (0 and “”) would not give us any randomness in the data we create
  • Perl 6 automatically returns the last expression in a function, so we didn’t have to put a return there.

That’s our code to generate the data, fair and square. But we’re gonna need to generate much more than those simple examples.

The least we need to support is classes with properties: we want to look at the list of attributes, generate data for their type, and feed them to the constructor (the only classes we support right now are those with a parameterless constructor).

We need to be able to look inside a class. What we’re going to reach for, in Perl 6 terms, is the Meta-Object Protocol (or MOP for short). First off, let’s define a sample data class for our, huh, blog (the author is sorry for such a lack of imagination).

class Article {
  has Str $.title;
  has Str $.content;
  has Int $.view-count;
}
# we can manually create instances this way:
Article.new(title => "Perl 6 Advent, Day 19",
            content => "Magic!",
            view-count => 0);

But we don’t want to create the article by hand. We want to pass the class Article to our generate-data function, and get an Article back (with random data inside). Let’s go back to our REPL…

> say Article.^attributes;
(Str $!title Str $!content Int $!view-count)
> say Article.^attributes[0].WHAT;
(Attribute)

If you’ve clicked on the MOP link, you shouldn’t be surprised that we get a 3-elements array. If you’re still surprised by the syntax, .^ is the meta-method call. What it means is that a.^b translates to a.HOW.b(a).

If we want to know what’s available to us, we could just ask (removing the anonymous ones):

> Attribute.^methods.grep(*.name ne '<anon>')
(compose apply_handles get_value set_value container readonly package inlined WHY set_why Str gist)
> Attribute.^attributes
Method 'gist' not found for invocant of class 'BOOTSTRAPATTR'

Whoops… Seems like this is a bit too meta. Thankfully, we can use a very nice property of Rakudo: a lot of it is written in Perl 6! To know what’s available to us, we can just look up the source:

#     has str $!name;
...
#     has Mu $!type;

We got the name for the key, and the type to generate the value. Let’s see…

> say Article.^attributes.map(*.name)
($!title $!content $!view-count)
> say Article.^attributes.map(*.type)
((Str) (Str) (Int))

Yep! Seems correct! (If you’re wondering why we get $! (private) twigils back, it’s because $. only means a getter method will be generated. The attribute itself is still private, and accessible in the class).

Now, the only thing we need to build is a loop…

my %args;
for Article.^attributes -> $attr {
 %args{$attr.name.substr(2)} = generate-data($attr.type);
}
say %args.perl;

This is an example of what could be printed:

{:content("muenglhaxrvykfdjzopqbtwisc"), :title("rfpjndgohmasuwkyzebixqtvcl"), :view-count(45)}

You get different results each time you run your code (I don’t think it’ll produce an article worth reading, however…).

Only thing left to do is pass them to Article‘s constructor:

say Article.new(|%args);

(the prefix pipe | allows us to pass %args as named arguments, instead of a single positional argument). Again, you should get something like this printed:

Article.new(title => "kyvphxqmejtuicrbsnfoldgzaw", content => "jqbtcyovxlngpwikdszfmeuahr", view-count => 26)

Yay! We managed to create an Article instance “blindly”, without knowing nothing special about Article at all. Our code can be used to generate data for any constructor that expects its class attributes to be passed. Done!

Before I go, I need to remind you of one things: with great power comes a lot of “WTF”. If you’re only taking a peek like we do here, and gathering information, it should be all fine. But don’t go around and change the internals of Rakudo’s classes, because no one knows what’d happen then :-). As always, have a moderate amount to fun introspecting.

PS: Left as an exercise to the reader is the recursive implementation, move the loop to generate-data, so that we could add a User $.author attribute to Article, and get this one constructed as well. Good luck!

Day 18 – Sized, Typed, Shaped

If you’ve used Perl 5, you probably know PDL (Perl Data Language), the library to manipulate highly efficient data structures.

In Perl 6, we do not (yet) have full-fledged support for PDL (or a PDL-like module), but we do have support for specific features: sized types, shaped variables (and typed arrays).

The first feature, sized types, are low-level types that are composed of a generic low-level type name, and the number of bytes they use. Such an example is int1, int64 (aka int on 64-bit machines), buf8 (a “normal” byte buffer). If you’re interesting in reading the spec, it’s all in S09.

The second feature, shaped variables (still in S09), allow us to specify the fixed size of an array:

my @dwarves[7];

This means the only available indices are 0 to 6 (inclusive). This is an example from the Perl 6’s REPL (Read-Eval-Print-Loop. The input lines start with “>”):

> my @dwarves[7];
[(Any) (Any) (Any) (Any) (Any) (Any) (Any)]
> @dwarves[0] = 'Sleepy';
Sleepy
> @dwarves[7];
Index 7 for dimension 1 out of range (must be 0..6)

The last line of input mentions “dimension 1”. Indeed, Perl 6’s shaped variables can be multi-dimensional (see S09):

> my @ints[4;2];
[[(Any) (Any)] [(Any) (Any)] [(Any) (Any)] [(Any) (Any)]]
> @ints[4;3]
Index 3 for dimension 2 out of range (must be 0..1)
> @ints[4;1]
Index 4 for dimension 1 out of range (must be 0..3)
> @ints[0;0] = 1;
1
> @ints
[[1 (Any)] [(Any) (Any)] [(Any) (Any)] [(Any) (Any)]]

The first output shows us that Perl 6 filled our array with Any at first: this means the whole array is “empty”, and that it can store anything (since Any is a supertype of IntStr, etc).

We might not want that: our array should contain integers (Int). Instead, we can declare a typed array:

> my Int @a[8] = 0..7;
[0 1 2 3 4 5 6 7]
> @a[8] = 8;
Index 8 for dimension 1 out of range (must be 0..7)
> @a[0].WHAT.say # print the type
(Int)

This is all very useful in itself, but there’s a special property we can benefit from, by using everything presented in this post together: contiguous storage!

Natively-typed shaped arrays in Perl 6 are specced to take contiguous memory, meaning we get a very good memory layout, and use little memory

> my int @a[2;4] = 1..4, 5..8;
[[1 2 3 4] [5 6 7 8]]
> @a[0;0]++;
2
> @a.WHAT.say
(array[int])

Because we know how much memory each element takes, and how many elements we have, we can very easily calculate the array upfront, once and for all – or even calculate the size necessary to store the elements: we know that int, on our 64-bit computers (that’s int64), we need 2*4*8 bytes for our 2*4 array.

Wrapping up; that’s 64 bytes total (…almost), and we get to still write Perl 6! We can use the operations we know and love (even meta/hyper-operators) and al – they’re still arrays – with our cool performance boost. Amazing!

(note about the size: we need to add some overhead for the runtime; on MoarVM, that’s something like 16 bytes for GC header, 16 bytes for the dimensions list. That’s a grand total of 16+16+64=96 bytes. pretty cool!)

Day 22 – The Cool subset of MAIN.

In the Unix environment, many scripts take arguments and options from the command line. With Perl 6 it’s very easy to accept those. At least, that’s what Our 2010 advent post on MAIN says.

But today, we’re interested in going further, and we’re going to try to gain more expressiveness from it.
The first Perl 6 feature we’re going to look at here is subsets. Subsets have a very broad range of applications, but they’re even better when used in conjunction with MAIN.

Subsets are a declarative way to specify conditions on your type. If you wanted to translate “A QuiteSmallInt is an Int with a value lesser than 10”, you can write

subset QuiteSmallInt where * < 10;

In case you don’t remember from our 2010 advent post about smart matching or our 2011 advent post about multi-method dispatch, here’s a quick reminder of how you can use subsets with smart matching, multi-method dispatch or typed variables.

say 9 ~~ QuiteSmallInt; # True
say 11 ~~ QuiteSmallInt; # False

multi sub small-enough(QuiteSmallInt) { say "Yes!" }
multi sub small-enough($) { say "No, too big."; }

small-enough(9); # Prints "Yes!"
small-enough(11); # Prints "No, too big."

# You can also use it to type variables
my QuiteSmallInt $n = 9; # Works!
my QuiteSmallInt $n = 11; # Errors out with "Type check failed in assignment to '$n'; ..."

Alright; now that we got this out of the way, what does this buy us?
Well, as we just demonstrated, we can dispatch to different subroutines using those “where” conditions.
That goes for MAIN as well.

Say we wanted to take a filepath as the first argument of a MAIN. We’ll also write two “companions” that’ll give a descriptive error message in case we pass bad arguments.

# a File is a string containing an existing filename
subset File of Str where *.IO.e; # .IO returns an IO::Path object, and .e is for "exists"
subset NonFile of Str where !*.IO.e;

multi sub MAIN(File $move-from, NonFile $move-to) {
  rename $move-from, $move-to;
  say "Moved!";
}
multi sub MAIN($, File $) {
  say "The destination already exists";
}
multi sub MAIN(NonFile $, $) {
  say "The source file doesn't exist";
}

And now, if we try to use it (after saving it as “main.p6”)…

$ touch file1.txt
$ perl6 ./main.p6 file1.txt destination.txt
Moved!
$ perl6 ./main.p6 non-existant.p6 non-existant.txt
The source file doesn't exist!
$ perl6 ./main.p6 destination.txt destination.txt
The destination already exists

Alright, looks good. There’s a little catch, however: the –help will display 3 usages, when there’s only one.

$ perl6 ./main.p6 --help
Usage:
tmp.p6 <move-from> <move-to>
tmp.p6 <Any> (File)
tmp.p6 (NonFile) <Any>

Whoops! We need to find a way to hide them (if you’re wondering, the “<Any>” is because the arguments both are unnamed and untyped).
Luckily enough, Perl6 provides a trait for that: hidden-from-USAGE, added by moritz++ for this advent post (USAGE is the –help message).

Let’s add those to our MAINs.

multi sub MAIN(File $move-from, NonFile $move-to) {
  rename $move-from, $move-to;
  say "Moved!";
}
multi sub MAIN($, File $) is hidden-from-USAGE {
  say "The destination already exists";
}
multi sub MAIN(NonFile $, $) is hidden-from-USAGE {
  say "The source file doesn't exist";
}

And now, the USAGE is correct:

$ perl6 ./main.p6 --help
Usage:
tmp.p6 <move-from> <move-to>

Okay, now, we just need to add a description.

The second Perl6 feature we’re going to use is Pod.
Pod is an easy-to-use markup language, which you can embed to document your code.

#= Rename a file
multi sub MAIN(File $move-from, NonFile $move-to) {
  rename $move-from, $move-to;
  say "Moved!";
}

# You can write it in multiple lines, though in USAGE they'll be collapsed
# (joined by a space)

#={Rename
a
file}
multi sub MAIN(File $move-from, NonFile $move-to) {
  rename $move-from, $move-to;
  say "Moved!";
}

Both versions above will print the same –help:

$ perl6 ./main.p6 --help
Usage:
tmp.p6 <move-from> <move-to> -- Rename a file

And now you have everything you need to create a nice CLI interface :-).