Author Archive

Day 6 – The X and Z metaoperators

December 6, 2010

One of the new ideas in Perl 6 is the metaoperator, an operator which combines with an ordinary operator in order to change its behaviour. There are several of them, but in this post we will concentrate on just two of them: X and Z.

The X operator is one you might already have seen in its ordinary role as the infix cross operator. It combines lists together, one element from each, in every combination:

> say ((1, 2) X ('a', 'b')).perl
((1, "a"), (1, "b"), (2, "a"), (2, "b"))

However, this infix:<X> operator is actually just a shorthand for the X metaoperator applied to the list concatenation operator infix:<,>. Indeed, you’re perfectly at liberty to write

> say ((1, 2) X, (10, 11)).perl
((1, 10), (1, 11), (2, 10), (2, 11))

If you like. So what happens if you apply X to a different infix operator? How about infix:<+>

> say ((1, 2) X+ (10, 11)).perl
(11, 12, 12, 13)

What’s it done? Instead of creating a list of all the elements picked for each combination, the operator applies the addition infix to them, and the result is not a list but a single number, the sum of all the elements in that combination.

This works for any infix operator you care to use. How about string concatenation, infix:<~>?

> say ((1, 2) X~ (10, 11)).perl
("110", "111", "210", "211")

Or perhaps the numeric equality operator infix:<==>?

> say ((1, 2) X== (1, 1)).perl
(Bool::True, Bool::True, Bool::False, Bool::False)

But this post is also meant to be about the Z metaoperator. We expect you may have already figured out what it does, if you’ve encountered the infix:<Z> operator before, which is of course just a shortcut for Z,. If a Haskell programmer understands infix:<Z> as being like the zip function, then the Z metaoperator is like zipWith.

> say ((1, 2) Z, (3, 4)).perl
((1, 3), (2, 4))
> say ((1, 2) Z+ (3, 4)).perl
(4, 6)
> say ((1, 2) Z== (1, 1)).perl
(Bool::True, Bool::False)

Z, then, operates on each element of each list in turn, working on the first elements together, then the second, then the third for however many there are. It stops when it reaches the end of a list regardless of which side that list is on.

Z is also lazy, so you can apply it to two infinite lists and it will only generate as many results as you need. X can only handle an infinite list on the left, otherwise it would never manage to get anywhere at all.

At the time of writing, Rakudo appears to suffer from a bug where infix:<Z> and infix:<Z,> are not identical: the former produces a flattened result list. S03 shows that the behaviour of the latter is correct.

These metaoperators, then, become powerful tools for performing operations encompassing the individual elements of multiple lists, whether those elements are associated in some way based on their indexes as with Z, or whether you just want to examine all possible combinations with X.

Got a list of keys and values and you want to make a hash? Easy!

my %hash = @keys Z=> @values;

Or perhaps you want to iterate over two lists in parallel?

for @a Z @b -> $a, $b { ... }

Or three?

for @a Z @b Z @c -> $a, $b, $c { ... }

Or maybe you want to find out all the possible totals you could get from rolling three ten-sided dice:

my @d10 = 1 ... 10;
my @scores = (@d10 X+ @d10) X+ @d10;

If you want to see some real-world use of these metaoperators, Sudoku.pm in Moritz Lenz’s Sudoku solver.

Day 22: Operator Overloading

December 22, 2009

Today’s gift is something I complain about a lot. Not because it’s a problem in Perl 6, but because Java doesn’t have it: operator overloading, and the definition of new operators.

Perl 6 makes it easy to overload existing operators, and to define new ones. Operators are simply specially-named multi subs, and the standard multi-dispatch rules are used to determine what the most appropriate implementation to call is.

A common example, which many of you may have seen before, is the definition of a factorial operator which mimics mathematical notation:

multi sub postfix:<!>(Int $n) {
  [*] 1..$n;
}

say 3!;

The naming convention for operators is quite straightforward. The first part of the name is the syntactic category, which is prefix, postfix, infix, circumfix or postcircumfix. After the colon is an angle bracket quote structure (the same kind of thing often seen constructing lists or accessing hash keys in Perl 6) which provides the actual operator. In the case of the circumfix operator categories, this should be a pair of bracketing characters, but all other operators take a single symbol which can have multiple characters in it. In the example above, we define a postfix operator ! which functions on an integer argument.

You can exercise further control over the operator’s parsing by adding traits to the definition, such as tighter, equiv and looser, which let you specify the operator’s precedence in relationship to operators which have already been defined. Unfortunately, at the time of writing this is not supported in Rakudo so we will not consider it further today.

If you define an operator which already exists, the new definition simply gets added to the set of multi subs already defined for that operator. For example, we can define a custom class, and then specify that they can be added together using a custom infix:<+>:

class PieceOfString {
  has Int $.length;
}

multi sub infix:<+>(PieceOfString $lhs, PieceOfString $rhs) {
  PieceOfString.new(:length($lhs.length + $rhs.length));
}

Obviously, real-world examples tend to be rather more complex than this, involving multiple member variables. We could also check our pieces of string for equality:

multi sub infix:<==>(PieceOfString $lhs, PieceOfString $rhs --> Bool) {
  $lhs.length == $rhs.length;
}

In which case we’re really just redispatching to one of the built-in variants of infix:<==>. At the time of writing this override of == doesn’t work properly in Rakudo.

One thing you might want to do which you probably shouldn’t do with operator overloading is operating things like prefix:<~>, the stringification operator. Why not? Well, if you do that, you won’t catch every conversion to Str. Instead, you should give your class a custom Str method, which is what would usually do the work:

use MONKEY_TYPING;

augment class PieceOfString {
  method Str {
    '-' x $.length;
  }
}

This will be called by the default definition of prefix:<~>. Methods which have the names of types are used as type conversions throughout Perl 6, and you may commonly wish to provide Str and Num for your custom types where it makes sense to do so.

Thus overriding prefix:<~> makes little sense, unless you actually want to change its meaning for your type. This is not to be recommended, as programmers in C++ and other languages with operator overloading will be aware. Changing the conventional semantics of an operator for a custom type is not usually something which ends well, leads to confusion in the users of your library and may result in some unpleasant bugs. After all, who knows what operator behaviour the standard container types are expecting? Trample on that, and you could be in a great deal of trouble.

New semantics are best left for new operators, and fortunately, as we have seen, Perl 6 allows you to do just that. Because Perl 6 source code is in Unicode, there are a great variety of characters available for use as operators. Most of them are impossible to type, so it is expected that multicharacter ASCII operators will be the most common new operators. For an example of a Unicode snowman operator, refer back to the end of Day 17.

Day 13: Junctions

December 13, 2009

Among the many exciting things in Perl 6, junctions are one of my favourites. While I know I don’t really comprehend everything you can do with them, there are a few useful tricks which I think most people will appreciate, and it is those which I’m going to cover as today’s gift.

Junctions are values which have (potentially) more than one value at once. That sounds odd, so let’s get thinking about some code which uses them. First, let’s take an example. Suppose you want to check a variable for a match against a set of numbers:

if $var == 3 || $var == 5 || $var == 7 { ... }

I’ve never liked that kind of testing, seeing as how it requires much repetition. With an any junction we can rewrite this test:

if $var == any(3, 5, 7) { ... }

How does this work? Right near the core of Perl 6 is a concept called junctive autothreading. What this means is that, most of the time, you can pass a junction to anything expecting a single value. The code will run for each member of the junction, and the result will be all those results combined in the same kind of junction which was originally passed.

In the sample above, the infix:<==> operator is run for each element of the junction to compare them with $var. The results of each test are combined into a new any junction, which is then evaluated in Boolean context by the if statement. An any junction in Boolean context is true if any of its values are true, so if $var matches any value in the junction, the test will pass.

This can save a lot of duplicated code, and looks quite elegant. There’s another way to write it, as any junctions can also be constructed using the infix:<|> operator:

if $var == 3|5|7 { ... }

What if you want to invert this kind of test? There’s another kind of junction that’s very helpful, and it’s called none:

if $var == none(3, 5, 7) { ... }

As you may have guessed, a none junction in Boolean context is true only if none of its elements are true.

Junctive autothreading also applies in other circumstances, such as:

my $j = any(1, 2, 3);
my $k = $j + 2;

What will this do? By analogy to the first example, you can probably guess that $k will end up being any(3, 4, 5).

There is an important point to note in these examples. We’re talking about junctive autothreading, which should give you a hint. By the Perl 6 spec, the compiler is free to run these multiple operations on junctions in different threads so that they can execute in parallel. Much as with hyperoperators, you need to be aware that this could happen and avoid anything which would make a mess if run simultaneously.

The last thing I want to talk about is how junctions work with smartmatching. This is really just another instance of autothreading, but there are some other junction types which become particularly useful with smartmatching.

Say you have a text string, and you want to see if it matches all of a set of regexes:

$string ~~ /<first>/ & /<second>/ & /<third>/

Assuming, of course, you have defined regexes called first, second and third. Rather like |, & is an infix operator which creates junctions, this time all junctions which are only true if all their members are true.

The great thing about junctions is that they have this behaviour without the routine you’re passing them to having to know about it, so you can pass junctions to almost any library or core function and expect this kind of behaviour (it is possible for a routine to deliberately notice junctions and treat them how it prefers rather than using the normal autothreading mechanism). So if you have a routine which takes a value to smartmatch something against, you can pass it a junction and get that flexibility in the smartmatch for free. We use this in the Perl 6 test suite, with functions like Test::Util::is_run, which runs some code in another interpreter and smartmatches against its output.

To finish off, here are some other useful things you can do with junctions. First, checking if $value is present in @list:

any(@list) == $value

Junction constructors can work quite happily with the elements of arrays, so this opens up many possibilities. Others include:

all(@list) > 0; # All members greater than zero?
all(@a) == any(@b); # All elements of @a present in @b?

Go experiment, and have fun!

Day 7: Looping for fun and profit

December 7, 2009

Any programmer who’s ever used a language which has them probably knows that loops are incredibly useful. In languages which provide them, foreach loops for iterating over arrays or lists tend to end up being very common. In Perl 5, these loops were provided by the foreach keyword, although you could also write for, sharing a keyword with the more general C-style for loops.

In Perl 6, that’s all changed.

for is now exclusively for iterating over lists. foreach has disappeared, and C-style for loops are handled by the new keyword loop. We’re not going to discuss those today, but we are going to focus on the new for loop, which combines with some other Perl 6 language features to deliver an enormously flexible and powerful construct.

Let’s look at a basic case.

for 1, 2, 3, 4 { .say }

There some immediately noticeable things about the syntax here. There are no brackets around the list construction, which is something that extends throughout Perl 6. Generally, you need a lot fewer brackety characters than in Perl 5. Much like in Perl 5, the loop variable is $_ by default. The method call of say with no invocant is the same as saying $_.say. Note that in Perl 6, you cannot say say with no argument to default to $_, you have to use the .say form, or specify $_ explicitly.

The block doesn’t have to be an ordinary block. It can be a pointy block, which lets you name your loop variable using the pointy block’s parameter capability.

for 1, 2, 3, 4 -> $i { $i.say }

A pointy block is a bit like an anonymous subroutine, except it doesn’t catch return exceptions. If you call return inside a pointy block, the enclosing routine will return.

Pointy blocks can take more than one parameter in their parameter lists. What happens if you do that?

for 1, 2, 3, 4 -> $i, $j { "$i, $j".say }

Well, if you run it, you get:

1 2
3 4

So what’s actually happened is that you’ve iterated over the list two elements at a time. This works for any number of parameters, with the minimum of one, degenerating to using $_ if you provide no explicit parameters yourself.

Having realised we can do this, what can we do with the generation of the list we iterate over? Well of course, we can use an array variable:

for @array { .say }

Although in many simple cases, we might prefer to use map:

@array.map: *.say;

Or a hyperoperator, if order and sequentiality isn’t important:

@array».say;

But neither of those things are today’s subject.

We might generate a list of numbers using the range constructor &infix:<..>:

for 1..4 { .say }

It’s very common that we want to generate a list of $n numbers beginning with 0, such as array indicies. We could write 0..$n-1, or using a variant of the range constructor 0..^$n, but Perl 6 provides a handy shortcut in the form of prefix:<^>:

for ^4 { .say }

Which will output:

0
1
2
3

One reason people often fall back on C-style for loops in Perl 5 is because they need to know what index in the array they are at for each item, or because they need to iterate over two or more arrays in parallel. Perl 6 offers a shortcut here, with the infix:<Z> zip operator.

for @array1 Z @array2 -> $one, $two { ... }

Assuming the two arrays are the same length, $one will be each element of @array1 and $two will be the corresponding element of @array2. If they are different lengths, iteration will stop when the end of the shorter array is reached.

With this knowledge, and the awareness that Perl 6 has lazy list generators, we can easily include the array index in the iteration:

for ^Inf Z @array -> $index, $item { ... }

Although if infinite lists make you nervous,

for ^@array.elems Z @array -> $index, $item { ... }

will give you the same results, but the most elegant presentation is probably:

for @array.kv -> $index, $item { ... }

@array.kv returns the keys and values interleaved, where the keys of an array are the element indices, so iterating over them two at a time has the desired effect.

Hopefully this post has given you an idea of the flexibility inherent in Perl 6′s for loops and how easy they can be to use for a variety of common tasks. Before we part, I’m going to answer one final question I know somebody’s been thinking.

What, you ask, if I want to iterate over four arrays at once?

for @one Z @two Z @three Z @four -> $one, $two, $three, $four { ... }

That’s a list associative infix operator, that is. Enjoy.


Follow

Get every new post delivered to your Inbox.

Join 36 other followers