Day 5 – Identifiers have hyphens in them

Day 5 – Identifiers have hyphens in them

One day on the #perl6 channel, back in 2009, I stumbled into a conversation where Larry said “it didn’t break any spectests, and that convinced me” or something like that. Maybe it broke a couple of spectests, but they apparently needed breaking anyway.

The change in question was adding hyphens and apostrophes to identifiers. Normally in languages, these are valid identifier names:

my $foo;
my $please_do;

But Perl 6 also allows these:

my $foo-with-hyphens;
my $please-don't;

As usual, I was conservative, and slow to pick up these changes. I didn’t like what it did to my vim highlighting of variables. Whatever.

These days I kind of like the hyphens in identifiers. Mostly I just decide on a project-by-project basis whether I want to use hyphens, but I notice myself deciding to use them more and more often. They just look nicer, somehow. More balanced.

Damian Conway, on the Perl 6 Language mailing list, tried to institute the convention that hyphens and underscores should be used interchangeably — hyphens where you’d use hyphens in a sentence, and underscore where you’d use a space between words. I haven’t seen anyone pick up on that practice. I suspect it is because many of us are not native speakers, but rather speakers of some nebulous Goodenuf English, and we would hesitate between what’s two words and what’s a hyphen-separated compound. Actually, I’m pretty sure native speakers hesitate too sometimes.

Anyway, there’s an obvious parser conflict which you may have spotted: the hyphen is already used for infix minus. Perl 6 disambiguates this by saying that you’re allowed to use a hyphen (or an apostrophe) if it’s then followed by an alphabetic character. Thanks to sigils, this is enough to make it clear what you mean.

my $foo-2;     # variable minus 2 
my $foo-bar;   # a single variable;

Now I want to say two things at once. The first thing is that the apostrophe is an old vestigial thing in Perl 5 identifiers, too. It means the same as the package separator ::. Which is how Acme::Don't can exist, for example. (Look at the actual package name of the module.

The second thing is that Lisp people and Haskell people seem particularly saddened that because of this rule, you’re not allowed to put an apostrophe at the end of an identifier.

my $foo';     # not allowed

Ah, well. There will be slangs. I’m surprised there isn’t already a slang for apostrophes at the end of identifiers. ☺

Day 4 – Going Raw with Rogue Robots

Day 4 – Going Raw with Rogue Robots

DISCLAIMER: accessing or spying on networks without permission to do so is illegal in many jurisdictions. The author does not condone or encourage anyone to break laws. And should this article inspire you to become a cyber-crimefighter and you get caught and killed… well, that’s not a bad way to go.

Agent, we have a mission! The bad guys seem to have set up a server where they are discussing their secrets. We can’t risk being caught and exposed, so you’ll have to design an automated robot to do the job. Here’s the task:

  1. Recon (snoop on the network, to learn the protocol)
  2. Infiltrate (connect to the server)
  3. Put on a disguise (respond to events / use the Perl 6 ecosystem)
  4. Send regular reports to the agency (timed events)

1) Recon (snoop on the network, to learn the protocol)

The bad guys are using an IRC server for communication. Unfortunately, our Lab did not have the time to do the research, so we’ll have to go raw. You’ll need any IRC Client and something that can snoop on the network traffic. We have preliminary results using XChat and WireShark, see if you can replicate them.

Fire up WireShark and enable listening on your network device, on my machine it’s named eth2 (and I had to start Wireshark as root, to get permissions to capture).  Go to Capture -> Interfaces and click Start button for the appropriate interface:

 

Using your IRC client, now connect to the IRC server the bad guys are using—which is irc.freenode.net on port 6667—and join a test channel, say, #perl6-recon. Once that is done, click the Stop Running Live Capture button in Wireshark.

We’re done collecting our data, Agent. Let’s take a look at what we got. Type tcp.port == 6667 in the filter field:

We want to figure out how to make our robot do what we’ve just done: connect to the server and join a channel. Sort the captured data by time and look for what the client is sending to the server. We’ll want to send the same thing:

Ignoring other chatter, it seems we should be successful if we send the following data to the server:

NICK Perl6NotABot
USER Perl6NotABot Perl6NotABot irc.freenode.net :Not a bot
JOIN #perl6-recon

Let’s do just that!

2) Infiltrate (connect to the server)

Fire up your favourite code editor and let’s write some Perl 6. It’s time to infiltrate the system!

 1   my ( $nick, $channel ) = 'P6NotABot', '#perl6-recon';
 2   await IO::Socket::Async.connect('irc.freenode.net', 6667).then({
 3       given .result {
 4           .print(qq:to/END/
 5               NICK $nick
 6               USER $nick $nick irc.freenode.net :Not a bot
 7               JOIN $channel
 8               END
 9           );
10           react { whenever .Supply { .say } }
11       }
12   });

Try this code out on your computer. You should see a whole bunch of output from the server. Let’s break down what the code does:

On line 1 we simply store the name of the spy bot and the channel we’re joining into variables. Line 2 is more interesting: the IO::Socket::Async.connect('irc.freenode.net', 6667) bit creates an asynchronous socket that attempts to connect irc.freenode.net server on port 6667. That returns us a Promise and since we really, really want that socket, we await that promise’s completion right away. When that happens, it means we have a connected socket; we’re moved along to the .then that is given a code block as an argument, which gets executed. Let’s take a closer look at that block (note: if you’re getting errors with line 10, your Rakudo is likely too old; upgrade or use .chars-supply instead of .Supply):

 1   {
 2       given .result {
 3           .print(qq:to/END/
 4               NICK $nick
 5               USER $nick $nick irc.freenode.net :Not a bot
 6               JOIN $channel
 7               END
 8           );
 9           react {
10               whenever .Supply {
11                   .say
12               }
13           }
14       }
15   }

Line 2 is a given block with .result as the given. It’s a bare method call, which means it’s called on the $_ topical variable, which in this case is our socket Promise, thus the given block is operating on the result of that promise, which is our connected socket. Inside the given block, on line 3, we have a .print method executed, again on the $_, which now is our async socket. The qq:to/END/ ... END bit is a HEREDOC—a multi-line chunk of text—that all gets sent to the server. And that bit should look familiar: it’s the same stuff we snooped from the network when connecting using a regular IRC client. We’ve used our nickname on the USER line a couple of times for it to serve us as both user name and anything else the server needs.

On line 9 we have a react block that, unsurprisingly, reacts to events. We’re interested in when some stuff heads our way from the socket, which is why we ask to do stuff whenever we have .Supply. At the moment we simply ask it to print that stuff on screen with the .say method called on the topical variable—this is all the output from the server you saw on screen if you ran this program—but let’s bring out bigger guns and do something more fun, shall we?

3) Put on a disguise (respond to events)

Agent, our spy bot needs to act as if it were a human! We can’t have it sit silently—the bad guys will know right away something is up. Since, for safety, we can’t respond to all queries ourselves, our robot needs to be smart enough to do it on its own. It seems a mammoth task to implement in such a short a time, but luckily, I have a contact who can assist us. They developed a super secret weapon called Text::Markov. Head over to http://modules.perl6.org/ and see if you can locate that weapon. Got it? For the record, if you ever need quick assess to docs and specs, just use the /repo/ part of URL along with the name, for example: http://modules.perl6.org/repo/Text::Markov

Now, install Text::Markov. You should be able to do so by running panda install Text::Markov command. This module will allow our spy bot to respond to any bad guys who attempt to talk to it. Responding means watching for something, so fire up your spy bot again and try talking in the channel its in. Then look at what the server is sending to the bot:

:Baddie!~Bad@localhost PRIVMSG #evil :I have a great plan to do evil stuff!
:Baddie!~bad@localhost PRIVMSG #evil :P6NotABot, hey, who are you?

We’ll guestimate that to send a message, we need to start our line with a colon, send our nick, followed by an exclamation sign, followed by user name, at sign, our hostname, word PRIVMSG, channel name, and the message we want to send prefixed by another colon. And anything said in the channel follows the same format. First, let’s try watching for lines containing PRIVMSG from the server and parse out the actual text said, which we’ll send right back. Here’s our code:

 1   my ( $nick, $channel ) = 'P6NotABot', '#perl6-recon';
 2   await IO::Socket::Async.connect('irc.freenode.net', 6667).then({
 3       my $sock = .result;
 4       $sock.print(qq:to/END/
 5           NICK $nick
 6           USER $nick $nick irc.freenode.net :Really not a bot
 7           JOIN $channel
 8           END
 9       );
10   
11       react {
12           whenever $sock.Supply {
13               .say;
14   
15               /^':' <-[:]>+ 'PRIVMSG ' $channel ' :' (.+)/
16                   and $sock.print(
17                       ":$nick!~$nick@localhost PRIVMSG $channel :You said $0"
18                   );
19           }
20       }
21   });

First, note how we got rid of the given block and are simply storing the connected socket in the $sock variable—this will let us access it more easily later in the code. In the whenever block, along with printing all the data the server is sending us in the terminal (line 13), we’re also doing a regex match that looks for things that look like stuff said in our channel. The (.+) portion captures what was said and we parrot it back into the socket. Since Perl short-circuits conditionals, simply using and on line 16 will cause the $sock.print code to execute only when the regex matches. Try this code out and talk in the channel. The bot should respond to you.

Now, simply parroting back what the bad guys are saying will get our spy-bot spotted and kicked out fast. We need to be smarter, and this is where Text::Markov comes in. Looking at its documentation at http://modules.perl6.org/repo/Text::Markov, we see we need to feed it lines with .feed method and we can get it to produce output via .read method. The plan is this then: we’ll feed the Markov chain all the text messages that occur in the channel and make the bot respond to the channel only when someone addresses it by mentioning its name. The code becomes this:

 1   use Text::Markov;
 2   
 3   my ( $nick, $channel ) = 'P6NotABot', '#perl6-recon';
 4   
 5   my $mc = Text::Markov.new;
 6   /\S/ and $mc.feed($_) for 'story.txt'.IO.lines;
 7   
 8   await IO::Socket::Async.connect('irc.freenode.net', 6667).then({
 9       my $sock = .result;
10       $sock.print(qq:to/END/
11           NICK $nick
12           USER $nick $nick irc.freenode.net :Really not a bot
13           JOIN $channel
14           END
15       );
16   
17       react {
18           whenever $sock.Supply {
19               .say;
20               if /^':' <-[:]>+ 'PRIVMSG ' $channel ' :' $<said>=(.+)/ {
21                   $mc.feed( ~$<said> );
22                   $<said> ~~ /$nick/ and $sock.print(
23                       ":$nick!~$nick@localhost PRIVMSG $channel "
24                       ~ ":{$mc.read.substr(0, 200)}\n"
25                   );
26               }
27           }
28       }
29   });

Let’s break this down. On line 1 we’re useing the Text::Markov module to include its functionality in our code. On line 5, we added a new variable $mc and store the Text::Markov object in it that we obtain by calling .new method on Text::Markov class. Now, normally the bare Text::Markov will take a bit to “learn” new text and until it does so, it’ll do a lot of repeats. To prevent that, I saved a short detective story into a text file called story.txt and on line 6 I’m reading all lines from that file and .feeding the Markov chain all lines that aren’t blank. Much of the following code is the same as before; let’s jump straight to line 20.

Notice the slight change in the regex: I’ve used $<said>=(.+) instead of bare (.+), so that we could have a meaningful name for the captured stuff instead of the cryptic $0. On line 21, I’m feeding the match into the Markov chain (the ~ before the variable forces it into a string). Then on line 22, I have another regex that checks whether the text that was said contains the nick of the bot. If the regex matches, our program proceeds to $sock.print portion of line 22 and outputs the message generated by the Text::Markov module. Line 23 has the prefix the server expects that we’ve been using. On line 24, the ~ is the string concatenation operator. Inside that string, however, notice how we’re actually executing some Perl 6 code! It’s the curly braces { } that allow us to do so. I’m getting a line of text via .read method on our Markov object, and then I’m shortening it to at most 200 characters with .substr method call, since if it’s too long, the IRC server will kick our bot out.

Try this code out (remember to create a file called story.txt and fill it with some text). Try addressing the bot by mentioning its nickname. It should produce some interesting text. You can also try commenting out line 6 and trying to address the bot then. Notice how without having fed the Markov chain some content, the results it produces are uninspiring.

4) Send regular reports to the agency (timed events)

Responding to users on the network is great and all, but we have a job to do, Agent. As a proof of concept, we’ll simply regularly append a time-stamped string into a file, to notify the agency that the bot is still alive and well. Let’s take a look at the code for that:

 1   use Text::Markov;
 2   
 3   my ( $nick, $channel ) = 'P6NotABot', '#perl6-recon';
 4   
 5   my $mc = Text::Markov.new;
 6   /\S/ and $mc.feed($_) for 'story.txt'.IO.lines;
 7   
 8   await IO::Socket::Async.connect('localhost', 6667).then({
 9       my $sock = .result;
10       $sock.print(qq:to/END/
11           NICK $nick
12           USER $nick $nick irc.freenode.net :Really not a bot
13           JOIN $channel
14           END
15       );
16   
17       Supply.interval( 5 ).tap({
18           spurt 'report.txt', "[{DateTime.now}] Still alive!\n", :append;
19       });
20   
21       react {
22           whenever $sock.Supply {
23               .say;
24               if /^':' <-[:]>+ 'PRIVMSG ' $channel ' :' $<said>=(.+)/ {
25                   $mc.feed( ~$<said> );
26                   ~$<said> ~~ /$nick/ and $sock.print(
27                       ":$nick!~$nick@localhost PRIVMSG $channel "
28                       ~ ":{$mc.read.substr(0, 200)}\n"
29                   );
30               }
31           }
32       }
33   });

If you’re not seeing much difference, it’s because there isn’t! Lines 17–19 is all we added. We’re .tapping a Supply that emits an event every five seconds. The code block we give to .tap uses spurt in :append mode to append a string to file named report.txt. The string it spurts uses DateTime type’s method .now to obtain the time stamp. And there you have it—doing stuff every five seconds!

Conclusion

You’ve now seen how easy it is to do event loops in Perl 6, connect to a network resource, read from and write to files, as well as use code libraries developed by third parties. In just 33 lines of liberally-written code, we have something that connects to an IRC server and respond to specific messages, while doing work in intervals as well.

There’s more evil in the world, Agent! Be sure to read all the documentation referenced throughout this blog post. See if you can improve your robot.

Together with the power of Perl 6… We’ll save the world.

 

Day 3 – Atom Editor support

Day 3 – Atom Editor support

If there is one thing in abundance in the world of programming, it’s text editors. It seems like at least once a month a new one will be trending on HN or Reddit.  While many are often just pet projects or slow to add features, Github’s Atom Editor has rapidly gained a strong feature-set and has risen far above the pack and now competes with Vim, Emacs, and especially Sublime Text. Though many see Atom as a text editor for web developers, it actually has packages available for pretty much any language (the Atom team has even built a program to convert TextMate packages to Atom packages, which made it easy to seed their ecosystem).

Atom’s Perl 6 support

Out-of-the-box, Atom supports Perl 6 via the language-perl package included with the base install.  However, the highlighter has many shortcomings and doesn’t illustrate (pun intended) the breadth of features that Perl 6 provides. So I endeavored to write a syntax highlighter that would be much more vibrant and let the language truly shine! And here is a glimpse of language-perl6fe: Read more

Day 2 – 2 bind or !2 bind

Day 2 – 2 bind or !2 bind

Is a question one may want to answer while programming in Perl 6. Let me explain with the help of a Proxy and a script.

11 sub BOS() {
12 	my Str $c-value; # closure variable
13 	return Proxy.new(
14 		FETCH => method () { $c-value },
15 		STORE => method ($new-value) { 
16 			die $?LINE ~ ' Semper invicta!' if $new-value.tc ~~ any <Supermutant Ghoul Synth>; 
17 			$c-value = $new-value;
18 		}
19 	);
20 }
21 
22 # Let's prefix &say with the line number it's called from
23 my sub say(**@args is raw) { 
24 	print callframe(1).line; 
25 	print ' ' ~ .Str ~ $*OUT.nl-out for @args
26 }
27 
28 my Str $who-container = BOS;
29 my Str \who-raw = BOS;
30 my Str $who-bound := BOS;

Now we have three variables, of which one is a buildin container. All of them refer to a ‘magic’ variable. Via the methods given to Proxy.new, we can control what values are allowed to be stored. As you likely have guessed already, our magic variable doesn’t like Supermutants and other unhumanly creatures. There is a catch. The container that is still a container wont do what we expect.

43 $who-container = 'supermutant';
44 say $who-container;
45 
46 try {
47 
48 	who-raw = 'ghoul';
49 	say who-raw;
50 
51 	CATCH {
52 		when X::AdHoc { warn $_.Str }; # line 52
53 	}
54 }
55 
56 who-raw = 'Cait';
57 say who-raw;
58 
59 try {
60 
61 	$who-bound = 'synth';
62 	say $who-bound;
63 
64 	CATCH {
65 		when X::AdHoc { warn $_.Str }; # line 65
66 	}
67 }
68 
69 $who-bound = 'Dogmeat';
70 say $who-bound;

By handling X::AdHoc we can turn a fatal die into a harmless warn. For $who-container we don’t need to do that. Let’s see how they differ.

79 say $who-container.VAR.^name; # Scalar
80 say who-raw.VAR.^name; # Proxy
81 say $who-bound.VAR.^name; # Proxy

With VAR we get access to the typeobject of the container we introduce with my or our. Binding allows us to set the type of the container, what is not the same thing as the type of the value stored in the container. Let’s have a look at the type checks.

92 say so BOS() ~~ Proxy; # False
93 say so BOS() ~~ Str; # True
94 
95 say so BOS().VAR ~~ Proxy; # True
96 say so BOS().VAR ~~ Str; # False

The type of the returned value is the type of $c-value. The container type is Proxy. We can use that to decide at runtime if we need to bind.

105 my Str $foo;
106 $foo = BOS if BOS().VAR ~~ Scalar;
107 $foo := BOS if BOS().VAR ~~ Proxy;
108 say $foo.VAR.^name; # Proxy

There is no way for a function to force binding on its returned value. So we have to be careful with returned values. Luckily Perl 6 provides us with enough introspection so we can handle it at runtime.

A bound Proxy can help us to probe Rakudo a little.

120 my $c-zeta;
121 my $counter;
122 my $zeta := Proxy.new(FETCH => { $counter++; $c-zeta }, STORE => -> $, $new { $c-zeta = $new } );
123 
124 $zeta = 'how often will it be fetched?';
125 say $zeta;
126 say $counter; # 5

On Rakudo Beta 1 FETCH is called 5 times just to .say the value. Let’s see how far our probe will reach.

135 f($zeta, $zeta, $zeta);
136 
137 sub f($c, $c-rw is rw, \r){
138 	say $c.VAR.^name; # Scalar
139 	say $c-rw.VAR.^name; # Proxy
140 	say r.VAR.^name; # Proxy
141 }

The default for sub and method parameters is ro, what does what is said on the tin. Both is rw and a sigilless parameter are using binding.

150 constant lover = BOS();
151 
152 say lover.VAR.^name; # Proxy
153 
154 lover = 'Piper';
155 say lover; # Piper

If we declare a constant Perl 6 will also force a binding. Allowing sigillessness may give it away. Proxy isn’t all that constant though. So we can turn a constant value into a variable. You just can’t trust those dynamic languages.

166 constant fibonacci = 0, 1, *+* ... *;
167 
168 say fibonacci[^10]; # (0 1 1 2 3 5 8 13 21 34)
169 say fibonacci.VAR.^name; # Seq

But then a sequence isn’t really constant. It’s values are calculated at runtime, for as many values we ask for.

178 try {
179 	fibonacci[11] = 0;
180 	CATCH { when X::Assignment::RO { say .Str } }
181 }

If we want to take advantage of type checks, we have to make sure a call to FETCH does return a default value of the right type. That’s easy to do with a type capture. If we omit the type in STORE we can still cheat to our heart’s content. If we don’t want to cheat, we could use ::T in STORE‘s signature.

193 sub typed(::T $type = Any){ # you may not want to default to Any, here we do
194 	my T $c-typed-value;
195 	my $c-typed-value-type-string = $c-typed-value.WHAT.perl;
196 	return Proxy.new(
197 		FETCH => method () { $c-typed-value },
198 		STORE => method ($new-value) { 
199 			$c-typed-value = $new-value ~~ Str 
200 				?? ( $new-value.comb>>.ord.sum / $new-value.chars )."$c-typed-value-type-string"()
201 				!! $new-value;
202 		}
203 	);
204 }
205 
206 my Int $typed-container := typed(Int);
207 say $typed-container = 11;
208 $typed-container = 'FOO';
209 say $typed-container;

If we do fancy container magic, we have to implement readonlyness by hand.

217 my $constant-variable := Proxy.new( 
218 	FETCH => { q{Merry 1.0!} }, 
219 	STORE => -> $, $ { die X::Assignment::RO.new(typename => 'none really') } # line 219 
220 );
221 
222 say $constant-variable;
223 
224 $constant-variable = 'The Grudge stole Christmas!'; # this will die in line 219 
225 say $constant-variable;

Binding is a powerful tool to expose the dynamic nature of Perl 6 and allows us to take advantage of that nature. No matter if you bind or not I wish you a Merry 1.0 and a happy new year!

Day 1 – The State of Perl 6 in 2015

Day 1 – The State of Perl 6 in 2015

Please fasten your seat belt for your annual dive into Perl 6.

As has been customary the last six years, we start with a brief overview of the state of Perl 6.

Last year's State of Perl 6 anticipated an upcoming production release of Perl 6. That is still scheduled for this Christmas. Last year's summary also identified major areas of work to-be-done for this release.

The 2015.05 release of Rakudo introduced NFG or "Normal Form Grapheme", which means that strings in Perl 6 are not based on Unicode codepoints anymore, and instead on grapheme clusters. Grapheme clusters can consist of base characters, like the latin lower-case c, and combining characters, like the "combining cedilla" character. Together, they form a single, visual character "c with cedilla", ç. This character also exists as a single codepoint, but other combinations of base character and combining character don't. Yet with this new feature, Perl 6 still treats the cluster of base character plus one (or more) combining characters as a single character, so regexes matches and substr won't tear them apart.

In September, Rakudo shipped with the GLR or Great List Refactoring in place. This mostly means that the rules for using and accessing nested data structures are now much simpler and more consistent. Under the hood we also have a sane and powerful iterator model, and a new type Seq for lazy value streams that don't necessarily memorize old values on iteration.

Finally, the September release introduced native, shaped arrays (or NSA). This allows you to write

    my int32 @matrx[4;5]

which allocates a continuous block of 20 32-bit values, but is still usable as a two-dimensional matrix in Perl 6 land. This paves the road towards memory efficient linear algebra (and other applications, of course).

But, development didn't stop there. The upcoming December release of Rakudo brings us automatic precompilation of modules and installation repositories.

Not only the compiler progresses. Where the Perl 6 Modules page showed around 270 modules a year ago (end of November 2014), we are now at about 460 modules. I'm also happy to report that there are two module installers now, panda and zef.

We also now have decent documentation coverage, at least on built-in types; other areas such as tutorials and material on language features are still a bit sparse. Other documentation efforts such as Learn Perl 6 in Y minutes and perl6intro.com have sprung up, and enjoy popularity.

I'm sure there has been more Perl 6 activity that would be worth reporting, but the excitment for the upcoming release makes a calm retrospective a bit harder than usual.

Stay tuned for more, and have fun!

Advent 2015

Advent 2015

Advent is a time of preparation. A time to make ready for Christmas. But this Advent is special. This Advent, the Christmas we are preparing for is the Christmas. The one where Perl 6 is “official” and “released” and ready for the world.

In a way, it has been a year long advent for the Perl 6 developers. There have been many improvements to the Rakudo compiler over the past year. Two really big features on our way to Christmas were Normal Form Grapheme (NFG) and the Great List Refactor (GLR) , which, after the September release were largely finished.

I say “largely finished” rather than just “finished” because there are always lingering issues in any endeavor and that’s why Christmas is so important to Perl 6. With the Christmas release of Perl 6, a larger community of programmers will be poking and prodding our Perl 6 compilers and using them for real world tasks. Moreover, there may be a few hidden corners in the language specification that could lead to futures changes in the specification which would cascade to the compiler implementations. With more people trying Perl 6, it’s likely that the community of people associated with Perl 6 will also receive a boost in numbers. There may even be an influx of new Perl 6 developers. It’s an exciting time.

So, over the next 24 days, enjoy some of the many gifts that Perl 6 has to offer. Christmas is just the beginning. :-)

Day 24 – Seeing Wrong Right

Day 24 – Seeing Wrong Right

There are all kinds of things I could’ve written about here, but none of them felt quite right. So I decided to write a few wrongs instead.

This year I had the privilege of experiencing a triple retinal detachment. Fortunately, they did two surgeries to correct this. Unfortunately, despite the surgeries, I ended up with a blind spot in half my macula and a paralyzed iris, stuck halfway open. Fortunately, this was all in my “bad” eye, so if I look out the right eye, meaning the wrong eye, things look a little wrong, but if I look out the right eye, meaning the left eye, things still look fine. It’s all very confusing, unless you’re the one looking out the eyes in question, in which case everything is completely clear, except for the fuzzy bits. From the outside, I just look a little crazy, but that’s nothing new.

Similarly, people inside the Perl 6 community have a different viewpoint from people outside the community. We often fret about this “echo chamber” effect, but that’s a problem that will become self-correcting as Perl 6 comes into more widespread use. But for now, it does induce a few distortions in outsiders’ perceptions of our community.

One thing that has never been quite clear to folks outside the community is how our design process works. To the people inside the community, it’s clear that the process works quite well, and that’s why Perl 6 just keeps getting better all the time. Every month we make a new release that is more performant and closer to the ideal of Perl 6 as expressed in the design docs and spec tests, even as those design docs and spec tests continue to converge on an implementable and optimizable design. The whirlpool methodology works. Eventually.

But this whirlpool that we find so strangely attractive depends on a kind of feedback loop that many projects do not achieve, and I think that part of the problem is that these projects get trapped by an inadequate view of the nature of wrongness. They see wrong wrong, as it were. In contrast, we try to see wrong right. Or at least righter.

And that’s really the first step, learning not to ask either/or questions. Most design decisions are not binary, because most ideas are not completely right or wrong. Instead, their degree of rightness is kinda truthy. Which makes them righty or wrongy, I guess. So the first step to figuring out right and wrong is to see how they are mixed together in various contexts. And maybe we have an unfair advantage here. This was a relatively easy principle for the Perl 6 community to learn, because as the original RFCs pointed out, Perl 5 is obviously a mixture of righty and wrongy ideas. That’s why we decided to do Perl 6, after all.

Now, Santa Claus knows that most kids are really a mixture of naughty and nicey. So the next step is to zoom out from ideas or features and also allow the people in your project to be a mixture of wrongy and righty, naughty and nicey. Several decades ago people used to talk a lot about “egoless” programming, but that’s not really what we’re doing. We all have our egos here, but somehow we’ve managed (mostly) to attach those egos to higher community ideals rather than to individual ideas or implementations. This frees people to experiment.

The social contract, frequently expressed as “forgiveness > permission”, really means that you have permission to try things out that might turn out right, as long as the rest of us have permission to back them out again if they turn out to be not-so-right. This operates on different scales, from small bug patches to complete attempts at implementation. We don’t promise to use a particular idea or module or implementation, but we do promise to learn from it, and to learn why it is suboptimal if we don’t use it.

It seems our modern western culture always has to find someone to blame, and even in our community we don’t promise not to blame anyone if something goes haywire. On the other hand, in our community, blame is merely a reason for considering someone human, and forgiving them. It’s not a mechanism for determining pecking order.

So we argue a lot on our IRC channel, but many of us take great delight in arguing both sides of any issue. Eventually we either reach a consensus, or decide we don’t know enough yet, so we’d better argue again later when we know more. Occasionally I have to invoke rule #1, “Larry is always right”—but not as often as you might think. Note that we also have rule #2, “If Larry changes his mind, he’s still right”—which is about letting Larry make mistakes, because Larry knows his brain is a mixture of wrongy and righty too.

So in a sense, this entire Perl 6 process is just a large, complicated experiment that we expect to have mixed success with, because we know our own mental limitations. And we hope this also gives us some empathy with our end users.

Once you’ve decided that mistakes are part of the creative process, you can pretty much view the entire evolution of Perl as a dialectic succession of mistakes and corrections. The original Perl design was a mistake. Letting Perl get typecast as a “CGI scripting language” in order to prototype the Web was a mistake. Letting the community try to redesign Perl was a mistake, hence most of the RFCs were mistakes. Or at least many of the proposed solutions in the RFCs were mistakes, and there was no coherence at all to ideas for fixing Perl 5.

The various attempts at implementation can be viewed as unsuccessful experiments. The early implementors prohibited me from working on the implementation. This may have been a mistake. Nowadays the implementors allow me to work on the implementations, which is certainly a mistake. :)

Another way things go wrong in a design is to pick the wrong principle for guidance. Most of the design principles we derived from thinking about the RFCs can be easily be abused when taken to extremes. And we’ve got lots of principles to pick from.

Just for the fun of it, this last year I finally went back through the 361 original RFCs and reread them to see how we did. (Maybe that’s why my retina detached…) What struck me was how many of Perl 6’s current design principles trace directly back to one or more of the RFCs. Of course, many of these principles came from elsewhere, but we decided to apply them in the new design. After I’d annotated my list of RFCs, I tried to give a talk on them, but there were just too many principles to talk about. Here is a list of principle we’ve used in the design, in no particular order:

☞ Don’t just do the first thing that occurs to you.
☞ YAGNI. (You Ain’t Gonna Need It)
☞ Self-clocking code produces better syntax error messages.
☞ Band-aids won’t regrow your limbs.
☞ Torment the implementors on behalf of the users.
☞ One-pass parsing is darn near mandatory for an extensible, braided language.
☞ Kill two birds with one stone.
☞ Avoid lists of exceptions the user has to memorize.
☞ Think ahead. (You think that’s cute this year…)
☞ Unifications can simplify things, sometimes at little cost.
☞ There’s no reason for an error message to be LTA (Less Than Awesome).
☞ Watch out for XY problems.
☞ There are not enough sigils in ASCII to classify everything. Choose wisely.
☞ Choose the right default, the one that enhances readability when omitted.
☞ Larry is not omniscient.
☞ Stick with Perl’s operator-rich philosophy.
☞ Discourage unrelated overloading of existing operators; define new ops instead.
☞ Don’t buy that special tool you see on TV.
☞ DIHWIDT. (Doctor, it hurts when I do this…)
☞ Hang things on the right peg—and if there isn’t one, install one.
☞ If you’re gonna generalize, do it harder, but not too hard.
☞ Pragmas are like vice grips: they’re the wrong tool for any job, but they’ll do any job.
☞ Languages with Do-What-I-Mean autodeclarations too often Don’t-What-I-Mean.
☞ People really get attached to their preferred methods of sloppy thinking.
☞ Question authority, especially your own.
☞ Avoid accidental genericity. Intentional genericity is okay though.
☞ Late binding sometimes causes your program to be late.
☞ Major features should fix lots of things.
☞ Grammatical categories can create a system of related precedence droppers.
☞ If you’re going to reduce the power of a construct, maybe remove it instead.
☞ Little languages (such as regexes) don’t have to violate one-pass parsing.
☞ Replace weak features with stong ones that degenerate to the weak case.
☞ Plan to know more someday.
☞ Don’t multiply entities without reason.
☞ Sometimes it’s worth making a feature pervasive, such as topicization.
☞ Don’t reinvent OO poorly.
☞ “Everything is an object” doesn’t mean it’s all method calls.
☞ Operational definitions are often weaker than declarative definitions.
☞ Even declarative definitions are implemented by operations at compile time.
☞ There are always tradeoffs.
☞ Good ideas go nowhere without a champion.
☞ Don’t punish the innocent with the guilty.
☞ Math just is. Don’t make people declare it.
☞ The identity is the most important part of the API; the right method on the wrong object is always wrong.
☞ People are like sheep, and will wander out any open gate.
☞ Save your money for power tools.
☞ No battle plan survives first contact with the enemy, or the second, or the third…
☞ The waterbed theory of complexity means if you push here, it goes up over there, unless there’s a leak.
☞ Generalizing is often cheaper than specializing.
☞ Bad features are sometimes not intrinsically bad, but just in the wrong context.
☞ Introspection is nearly always a design smell.
☞ You can’t be both eager and lazy simultaneously. Duh.
☞ Conserve your brackets. ASCII has so few of them.
☞ Encourage Unicode operators, yet provide ASCII workarounds.
☞ Sometimes it’s worthwhile to provide more than one way to do it; sometimes one way is enough.
☞ Fundamental unifications don’t have to hurt. If they do, maybe you’re persecuting the user.
☞ Easy things should be easy, and hard things should be possible (aka “Huffman coding”).
☞ The degenerate case should make sense.
☞ Remove artificial discontinuities (such as special-casing negative subscripts).
☞ Sometimes it’s better to leave where you are and approach the design from a different direction.
☞ Don’t plan to confuse things without a really good reason.
☞ Allow last-ditch escape hatches such as BEGIN and EVAL, but try to take away all their use cases.
☞ Premature failure (always throwing exceptions) is often a kind of premature optimization.
☞ Sometimes you have to retarget your shallow principles to a deeper level.
☞ Perl is not intended so much as a first language as it is a last language.
☞ All generalizations are false, including this one.

Arguably it was also a mistake to adopt so many principles. :)

Despite all these mistakes, a bunch of stubbornly optimistic people keep working on Perl 6. And somehow, after all the years of mistakes, we’ve managed to come out with something rather nice.

And now we’re talking about releasing Perl 6.0.0 “officially” sometime in 2015. After all these years of refusing to name a date, maybe this is a big mistake. On the other hand, maybe NOT announcing would be a mistake.

Either way, it doesn’t really matter, because there’s no stopping Perl 6 now. After a few final tweaks, sometime in 2015 we’ll finally be at a point where we can just decide when to tag the next release as 6.0.0.beta. And maybe we’ll keep it in beta until 2015-12-25 or so.

Of course, I could be wrong.

But what is certainly right is this: someday the Whos will start singing, all the Grinches of the world will be surprised, and perhaps a few of them will decide to be not quite so wrong. And they will be welcomed to our song, because we really do believe in forgiveness.

It’s beginning to look a lot like Christmas.