Day 4 – Composers, Coercers, and the Case of a Camel’s Curious Corner

“Every programming language has its curious corners, it’s just that some languages’ corners are curiouser than others’.”

When viewed from space, flame wars over programming languages are all about flinging various code or other examples around in order to prompt Yet Another Potentially Awkward Explanation (YAPAE) from the other side.

Even God has to give a YAPAE every once in a while… [image courtesy of xkcd++]

Everyone seems to know that Perl has corners. What is perhaps less famous than these corners themselves is the fact that their YAPAEs by and large make total sense. It may be a very Perlish form of sense, quite particular to Perl 5’s associativity, context, or precedence rules, but the reasoning always works out right. The language itself has a pretty simple set of rules, it’s the interactions of those rules that any good YAPAE code snippet will actually be highlighting.

Yet if the explanation has the potential to make sense, then it is only ever potentially awkward. Behind every YAPAE is the story of a decision in language design, a place where some of the implicit rules of operation suddenly become explicit¹. As such they form the intersecting seams of design choices that were made at the core language level. The explanations are never all that awkward to people who understand those choices and agree with them. I like to think of these seams in the language as “curious corners,” which refers both to the surface-level surprise that comes from a YAPAE as well as my own curiosity in the ways different languages handle their seams that show.

Perl 5 does some pretty incredible things with what syntactically amount to a few rules about precedence, associativity, and context. There’s a lot more to the language, to be sure, but when something blows up in your face it is almost one of these categories that you’ve botched. As such, almost any of syntactical surprise in Perl 5 can be solved with liberal application of parentheses. (If Perl 5 is the duct tape of the internet, then the parenthesis is the duct tape of Perl 5). These usually work together so well that you only notice them where they collide in strange ways, and once you get bitten you start to know better. That said, it was these curious corners which led to the development of a new Perl, which would have different rules and different interactions of those rules. Some of these rules exist precisely to avoid YAPAEs from previous versions.

The original text that I’m using as a source for this blog post began as something of a screed: I thought I had found a curious corner to which I was firmly on the “awkward” side of. However my opinion flipped completely about halfway through the analysis that was to form the basis of my argument. I hope by sharing the process of that reasoning with you that I can convince you of just how low the potential awkwardness of this YAPAE is.

So, join me as we explore a ‘curious corner’ of Perl 6, a journey that will take us straight to the source.

set() which is also Set.new() and ().Set which is also Set()

    > Set("wise men" => 3, "star" => 1, "camels" => 9)
    set(wise men, star, camels)

    > ( "wise men" => 3, "star" => 1, "camels" => 9 ).Set
    set(wise men, star, camels)

    > set("wise men" => 3, "star" => 1, "camels" => 2)
    set("wise men" => 3, "star" => 1, "camels" => 2)

    > Set.new("wise men" => 3, "star" => 1, "camels" => 2)
    set("wise men" => 3, "star" => 1, "camels" => 2)

Well, there’s gotta be more than one way to do it, right? This is Perl, after all…

When I first ran into this, it actually upset me a fair bit. I hack in Perl at my $day_job² and YAPAEs are a part of life when you are bringing new devs into the world of Perl. So I saw that set() and Set() both exist while also both doing different things and bit my fingernails / ranted on #perl6.

If you read that log, you can maybe tell that I’m a bit worked up. Until TimToady stated it simply: composers are not coercers.

Even though I understood the distinction more clearly, I was still a bit unconvinced and went off to investigate.

Composers like Set.new() and set()

Luckily in Rakudo, investigating the internals of the language mainly implies reading how one would implement Perl 6, using Perl 6! All the examples here are taken from their respective files in src/core without modification or truncation.

First we will take a look at some behavior, and then at the code which implements it.

    > set "we wish" => "you", "a" => "merry", "camel" => "ride"
    set("we wish" => "you", "a" => "merry", "camel" => "ride")

That looks exactly like a Perl 6 subroutine invocation: calling set with an argument which is a single List of Pairs. A quick git grep 'sub set' in rakudo.git reveals this declaration:

   #~~( set_operators.pm ) 
   sub set(*@args --> Set) { Set.new(@args) }

It appears to be some syntactic sugar, a sub which takes all of its arguments as a single list (the * prefix is ‘slurpy’) and then passes it to a constructor. I don’t know about you, but that sugar tastes guilt-free to me. Yet our investigation leads us elsewhere.

    #~~( Set.new() ... is in `submethod Build`? )
    submethod BUILD (:%elems) {
        nqp::bindattr(self, Set, '%!elems', %elems);
    }

There’s no .new definition to be found, but what is this submethod business? It sure sounds constructor related. This is a good candidate for looking up in the Perl 6 specification, which tells us that submethods are “infrastructural methods that shouldn’t be inherited by subclasses” (S12). Useful to note is that BUILD is not a phaser, but it shares Huffman coding³ with them: the all-caps is a hint that the code will run automatically.

blessed be the BUILDers

“The bless method automatically calls all appropriate BUILD routines for the current class, which initializes the object in least-derived to most-derived order,” from Synopsis 12: Objects. This is illustrated by the following snippet:

   class Elf {
        submethod BUILD { 
            say "This Elf is busy BUILDing"; 
        }
        method new {
            say "A shiny new Elf";
        }
    }

    role Build { 
        method new { 
            say "Did someone want to BUILD something?";
            self.bless;
        } 
    }

    class Santa is Elf does Build {
       submethod BUILD { 
          say "Even Santa BUILDs";
       }
    }
    Santa.new;
    
    Did someone want to BUILD something?
    This Elf is busy BUILDing
    Even Santa BUILDs

It is the self.bless invocation in Build.new that triggers BUILDALL, which descends the inheritance chain, calling a non-inherited BUILD if it is present. Elf, which does not call self.bless in it’s new method, would only produce the following when constructed:

    > Elf.new
    A shiny new Elf

BUILD is never called here, but it will be if any descendant or role were to mix in a self.bless in a constructor. For this reason, it is generally a good idea to call self.bless when you create a new method, especially if it is in a subclass. Unless, of course, you’ve got other plans for introducing the .bless call.

Back to the investigation

So, the presence of a BUILD in our Set class means that it wants to do something special during initialization. Even though I don’t know NQP, I can guess at what this code might do by asking myself, “what does a set want to do that is so special?”

The presence of the BUILD method is interesting, but as we’ve just seen, it doesn’t do anything without a .new. Just another file over from Set.pm in src/core is Setty.pm. That looks promising, especially if we noticed class Set does Setty in Set.pm and recognized the role declaration syntax.

    #~~( Setty.pm )
    method new(*@args --> Setty) {
        my %e;
        %e{$_.WHICH} = $_ for @args;
        self.bless(:elems(%e));
    }

Well, the %e{$_.WHICH} = $_ for @args line in the new method from is already taking care of one key component of a Set: making a unique list by internally storing each element of the (again slurpy) @args in a hash. That to me implies that the NQP code is taking the other important bit: immutability. Once again we can reach into src/core to see if we can confirm that suspicion. Perl 6 has a mutable form of Set, called SetHash. Let’s check that out.

    #~~( SetHash.pm )
    # No submethod BUILD to be found...

… so it appears that our assumption is correct! Now we know how to implement ‘hard immutability’ for an attribute in a pinch, even if we don’t have a spare second in the heat of that pinch to learn NQP.

Coercers like ().Set and Set()

Coercers are different. They don’t just flatten your lists into a big list, they reduce any Pairs in your list to just the key, or left-hand value, of any given Pair.

    > %( somehow => "special", snow => "flake" ).Set
    set(somehow, snow)

And because Any defines the .Set method, this allows a Set() call with an argument which descends from Any to coerce. If you are curious you can investigate in Synopsis 13: Overloading.

    > Set( (special => "is", as => "does") )
    set(special, as)

Note I didn’t have to turn it into a hash there. The code below shows that it will take all the thingies in the list and extract keys if it can. Otherwise it will “be on its merry,” so to speak.

    #~~( Any.pm )
    method Set()     {     Set.new-from-pairs(self.list) }

    #~~( Set.pm )
    method new-from-pairs(*@pairs --> Setty) {
        my %e;
        for @pairs {
            when Pair {
                %e{.key.WHICH} //= $_.key if .value;
            }
            default {
                %e{.WHICH} //= $_;
            }
        }
        self.bless(:elems(%e));
    }

Et voila! Coercion of a list of Pairs (or a Hash, as in the first example) no longer produces a set of Pairs as it does in the composer, but rather a set of the keys of those Pairs.

    > (special => "is", as => "does", 'a', bag 'of', 
      'snowflakes', 'snowflakes', 'snowflakes').Set
    set(special, as, a, bag(of, snowflakes(3)))

In contrast to earlier versions of Perl, 'a' does not become a key with the value of bag(of, snowflakes(3)). The difference with Perl 6 is that a Pair, denoted by the two terms separated by =>, is it’s very own type — not just a ‘fat comma’ as we have in Perl 5 (just one example of how P5 YAPAEs influenced the design of P6). The behavior is implemented by the when/default behavior: if you are a Pair, we put your key into the set. Otherwise we stuff your object as a whole in, just as we do in the composer.

    > Set( (special => "is", as => "does", 'a' => bag 'of',
      'snowflakes', 'snowflakes', 'snowflakes') )
    set(special, as, a)

Note the extra pair of parentheses when using the Set() form of the coercer. This is because Pairs are used to pass named parameters to subroutines. We can resolve this either by adding another pair of parentheses, as above, or by quoting our keys (which you can see by comparing this last snippet against the very first one in the post).

Conclusion

I hope that this brief, if twisty, tour of composers and coercers in the context of Set has introduced you to a few new dynamics in Perl 6. If nothing else, if it encourages an impulse to pop open src/core to perform your own investigation then I will consider it a success. A merry Christmas, and a jolly curious corner to you all!

Footnotes:

• ¹ Why else would they result in so much cursing?
• ² As opposed to $day-job ;)
• ³ We need a shorthand which represents Huffman Coding. Perhaps we can borrow a character from a logographic language with a similar associated meaning to stand in?