Day 9: Contributing to Perl 6

This time, instead of sharing some cool feature of Perl 6, I’d like to talk about how easy it is to contribute usefully to the project. So I’m going to walk you through the process of making a change to Niecza. It does require a bit of domain knowledge (which the fine folks on #perl6 will be happy to help you with) but it’s definitely not rocket science. It’s not even particularly deep computer science, for the most part.

A few days ago, Radvendii asked on #perl6 if there was a round function in the core. The correct answer is “There should be one”, and it lead to a couple of bug fixes in Rakudo. But it got me to thinking — is Niecza supporting round (and its relatives ceiling, floor, and truncate) correctly?

Perl 6 has a huge suite of tests to see if an implementation is conforming to the spec, including a file for the round tests, S32-num/rounders.t. My first step then was to check the spectests currently being run by Niecza. Just like in Rakudo, this is stored in a file named t/ So

Wynne:niecza colomon$ grep round t/ 
Wynne:niecza colomon$ 

Okay, clearly we’re not running the S32-num/rounders.t test file. (Note, in case you’re getting confused — the links in this post are to the latest versions of the files, which include all the changes I made writing this post.) That’s a sign that something is not properly supported yet. So let’s go ahead and run it by hand to see what happens. Both Niecza and Rakudo use a fudging process, allowing you to mark the bits of a test file that don’t work yet in a particular compiler. So we need to use a special fudging tool to run the code:

Wynne:niecza colomon$ t/fudgeandrun t/spec/S32-num/rounders.t
not ok 1 - floor(NaN) is NaN
# /Users/colomon/tools/niecza/t/spec/S32-num/rounders.t line 16
#    Failed test
#           got: -269653970229347386159395778618353710042696546841345985910145121736599013708251444699062715983611304031680170819807090036488184653221624933739271145959211186566651840137298227914453329401869141179179624428127508653257226023513694322210869665811240855745025766026879447359920868907719574457253034494436336205824

That’s followed by about 15 similar errors, then

Unhandled exception: Unable to resolve method truncate in class Num
  at /Users/colomon/tools/niecza/t/spec/S32-num/rounders.t line 34 (mainline @ 32) 
  at /Users/colomon/tools/niecza/lib/CORE.setting line 2224 (ANON @ 2) 
  at /Users/colomon/tools/niecza/lib/CORE.setting line 2225 (module-CORE @ 58) 
  at /Users/colomon/tools/niecza/lib/CORE.setting line 2225 (mainline @ 1) 
  at <unknown> line 0 (ExitRunloop @ 0) 

Okay, so that’s at least two errors that need fixing.

We’ll go in order here, even though it means tackling what is most likely the most complicated error first. (If you do think this part of the problem is too hard to tackle, please skip ahead, because the last few improvements I made really were incredibly easy to do.) Opening src/CORE.setting, we find the following definition for round:

sub round($x, $scale=1) { floor($x / $scale + 0.5) * $scale }

Okay, so the real problem is in floor:

sub floor($x) { Q:CgOp { (floor {$x}) } }

What the heck does Q:CgOp mean? It means floor is actually implemented in C#. So we open up lib/Builtins.cs and search for floor, eventually finding public static Variable floor(Variable a1). I won’t print the full source code here, because it is on the long side, with a case for each of the different number types. We’re only interested in the floating point case here:

        if (r1 == NR_FLOAT) {
            double v1 = PromoteToFloat(r1, n1);
            ulong bits = (ulong)BitConverter.DoubleToInt64Bits(v1);
            BigInteger big = (bits & ((1UL << 52) - 1)) + (1UL << 52);
            int power = ((int)((bits >> 52) & 0x7FF)) - 0x433;
            // note: >>= has flooring semantics for signed values
            if ((bits & (1UL << 63)) != 0) big = -big;
            if (power > 0) big <<= power;
            else big >>= -power;
            return MakeInt(big);

We don’t actually need to understand how all that works to fix this problem. The important bit is the PromoteToFloat line, which sets v1 to the floating point value which is the input to our floor. If we add a trap right after that, it should fix this bug. A quick C# websearch shows me that Double has member functions IsNaN, IsNegativeInfinity, and IsPositiveInfinity. Looking a bit around the Niecza source shows there is a MakeFloat function for returning floating point values. Let’s try:

if (Double.IsNaN(v1) || Double.IsNegativeInfinity(v1) || Double.IsPositiveInfinity(v1)) {
    return MakeFloat(v1);

One quick call to make later, I can try the test file again:

Wynne:niecza colomon$ t/fudgeandrun t/spec/S32-num/rounders.t
ok 1 - floor(NaN) is NaN
ok 2 - round(NaN) is NaN
ok 3 - ceiling(NaN) is NaN
not ok 4 - truncate(NaN) is NaN
# /Users/colomon/tools/niecza/t/spec/S32-num/rounders.t line 19
#    Failed test
#           got: -269653970229347386159395778618353710042696546841345985910145121736599013708251444699062715983611304031680170819807090036488184653221624933739271145959211186566651840137298227914453329401869141179179624428127508653257226023513694322210869665811240855745025766026879447359920868907719574457253034494436336205824

Progress! Apparently truncate uses a separate method, so we’ll have to fix it separately.

sub truncate($x) { $x.Int }
method Int() { Q:CgOp { (coerce_to_int {self}) } }
    public static Variable coerce_to_int(Variable a1) {
        int small; BigInteger big;
        return GetAsInteger(a1, out small, out big) ?
            MakeInt(big) : MakeInt(small);

Oooo, this is perhaps a little bit trickier. Still a basic variant on the previous method, grabbing boilerplate code from a nearby function:

        int r1;
        P6any o1 = a1.Fetch();
        P6any n1 = GetNumber(a1, o1, out r1);

        if (r1 == NR_FLOAT) {
            double v1 = PromoteToFloat(r1, n1);
            if (Double.IsNaN(v1) || Double.IsNegativeInfinity(v1) || Double.IsPositiveInfinity(v1)) {
                return MakeFloat(v1);

I skipped the HandleSpecial2 bit in the boilerplate, because I’m never quite sure how that works. Luckily, we have the spectests to check and see if I have broken something by doing this.

Now the first 15 tests in rounders.t pass, leaving us with the

Unhandled exception: Unable to resolve method truncate in class Num

error. That should be easy to handle! If we go back to lib/CORE.setting and search for ceiling, we see it appears two times: in the catch-all base class Cool and as a stand-alone sub. If we look at the neighboring subs, we see floor, ceiling, round, and truncate are all defined. If we look in Cool, however, only floor, ceiling, and round defined. That’s the source of our trouble!

The method definitions of the others in Cool are really simple; all they do is forward to the sub versions. It’s very easy to add a truncate that does that:

    method truncate() { truncate self }

And poof! This time when we run rounders.t, we pass all 108 tests.

At this point we’ve got three things left to do. First, now that rounders.t passes, we need to add it to t/ The list of tests there is ordered, so I just find the S32-num section and add S32-num/rounders.t in alphabetical order.

Next I will commit all the changes to my copy of the git repo. (I won’t explain how to do that, there are lots of git tutorials on the web.) Then I run make spectest to make sure I haven’t broken anything with these changes. (Hmm… actually a few TODO passing, bugs elsewhere that this patch has fixed! Oh, and one test broken, but it’s one which we were only passing by accident before, so I won’t feel bad about fudging it.)

Once that is done, you need to send the patch on to the Niecza developers; I believe the easiest way to do this is via github.

I’ve got one more little change to make that popped into my head while I was working on this. One naive way of implementing, say floor would be to convert the input into a floating point value (a Num in Perl 6) and then do Num.floor. That doesn’t work for all numbers, however, as most of the other number types are capable of storing numbers larger than will fit in a standing floating point double. So we probably need tests in the test suite to check for these cases. Let’s add them.

The tests in rounders.t are weirdly organized for my taste. But hey, we can always add our tests at the bottom.

    my $big-int = 1234567890123456789012345678903;
    is $big-int.floor, $big-int, "floor passes bigints unchanged";
    is $big-int.ceiling, $big-int, "ceiling passes bigints unchanged";
    is $big-int.round, $big-int, "round passes bigints unchanged";
    is $big-int.truncate, $big-int, "truncate passes bigints unchanged";

That passes okay in Niecza. (Probably out of courtesy we should check it on Rakudo as well and fudge it appropriately to make sure we’re not breaking their spectest!) We need to remember to add the count of new tests to the plan at the top of the test file. And then we can push that fix to github as well.

In conclusion, contributing to Perl 6 is easy. Anyone who tries writing Perl 6 code and reports problems they have to #perl6 is helping in a very real way. If you can write even fairly simple Perl 6 code, then you can write useful spec tests. It’s only marginally harder to write new methods for the setting in Perl 6. And even when you have to get down and dirty and start dealing with the language the compiler is implemented in, it’s still quite possible to do useful work without any deep understanding of how the compiler works.