I’ll explain. What generally happens is this:

1. I start a new job, and meet a whole new bunch of people.
2. After about six weeks, someone suddenly mentions “hey, you’re that MegaHAL guy”!

It happened just today. My new boss, at a job I started six weeks ago, suddenly made the connection when a good friend of his published an interesting post about cobe, an optimised and much improved reverse-engineered rewrite of MegaHAL, written in Python and using SQLite as a storage back-end. A nice piece of work, although I’m a bit ashamed that someone had to wade through my 16-year-old vanilla C code to try to figure our what the bloody hell is going on :)

Of course, this will probably help to knock me out of my summertime hiatus. Things have been quite on the blogging front lately, what with moving into our new house just prior to Christmas and starting a new job first thing in the New Year. Now that our new place is looking more like a home and less like a cardboard box wholesaler, it’s time to sit back, take stock and figure out what’s next.

For me, that’s as follows:

• Send out a final “Occasional Jason” missive to my mailing list, winding up my six-month sabbatical.
• Work on a commercial version of the well-received Professor Lazybones and the Cellular Automata Game Engine.
• Release MegaHAL.10.
• Get LaggyDash, my Global Game Jam entry, to some level of completeness.

Although, really, the first item on the list should really be “get more sleep, and exercise”.

Exciting times!

MegaHAL.10 generates sentences using a second-order Markov model. That means they tend to be random walks; you start out with a blank slate, and you never know where you’ll end up. The only guarantee is that any sequence of three symbols will have previously been observed by the model.

These kinds of random generations may be quite amusing, but they’re not too useful when you’re trying to simulate a conversation, which requires that each generation adheres to the current context. What would be much better is if you could force the model to generate a sentence that has the desired Markovian properties, but which contains pre-determined keywords.

With the original MegaHAL I did this by starting at the keyword and using two Markov models to generate the sentence, one proceeding from the keywords forwards to the end of the sentence, and the other going in the opposite direction. That was a bit of a hack, and could only guarantee that the sentence contains a single keyword.

For MegaHAL.10 I’ve been experimenting with other strategies, including a Fractal slot-filling model and a family of Markov models, each of which generates towards a specified target symbol. Both of these took a template sentence as input, which specified the keywords and their order, and both suffered from several problems, such as the fact that it may be impossible to complete the template, and that the local grammaticalness condition may need to be violated to make the keywords join up.

Recently I had a brainwave: what if we generated with the standard Markov model, but weighted the generations based on another model that specified the distance to the desired keyword. We could then generate a directed random walk, similar to an A* pathfinding algorithm. We may not be able to guarantee that the resulting sentence contains the keywords we want, but chances are it will contain at least a few.

Some bugs still need to be ironed out (and some wrinkles squished), but my tests so far indicate that this method may be the missing link. I trained MegaHAL.10 on the entire collection of Sherlock Holmes stories, and then I asked it to generate a random sentence, specifying these ten keywords in advance:

adventure foot wild scandal street twenty death finger villain rock

This is the sentence I got (I’ve highlighted the keywords):

“Twenty had changed their position, and the street by the death of the adventure of the ROCK OF GIBRALTAR. From the dead scientist.  His wild flight to escape from her finger into the garden.  Perhaps the villain was able to take counsel how far we can hardly put my foot over the case of this question. I saw that a very remarkable man, that was this William doing there is to avoid scandal.”

To me that’s completely amazing. “Give me a sentence containing these ten words”, I said, “making sure that each sequence of three words in your sentence appears somewhere in the collected Sherlock Holmes stories.” And MegaHAL.10 delivered.

Just to prove it wasn’t a fluke, here are a few more, using the same keywords:

“Twenty or thirty, I have seen death in the street by a strong family resemblance about misdeeds, and if you know now and so I dipped my finger on the ledge above the other hand had dropped his mask of his foot than might at any time before he was wild, wayward, and to our young companion looked at the young diplomatist’s window. There was this dark business which had been in the young lady.”

“Villain! Scoundrel!” cried the old quarters. At the foot of the death of the street. Holmes looked eagerly round for it was twenty before her in the dead scientist.  His wild eyes, stained cheeks and He put a triumphant conclusion.

“Death from some little distance down the street by a very simple chain: 1. You had a wild thrill as this was at the foot of the rock on each side of the adventure as a great scandal threatened to raise a scandal may well be cleared along those lines. Mr.  Douglas has spent a large coil from under the three strips. No finger impressions–no! Well, he is not here as I was at the other night.”

I’m planning to spend next week improving this algorithm, fixing various bugs, and optimising for memory and speed. I’ll then set to work creating a web version of MegaHAL.10, which will hopefully be going live before the end of the month.

Why is the #golang community so much more holier-than-thou than the #ruby community? Harumph to all sanctimonious hackers.

This got a concerned response from someone who works at Google as a “Go Gopher”, whatever that means. It also made me feel like a dirty troll, which wasn’t my intention at all: I was just venting after a frustrating day.

When Google’s Go Programming Language was announced late last year I was fascinated. I loved the decisions they’d made regarding code formatting (the language enforces a particular standard, making debates about the right way of formatting code a moot point), dependency management (it’s a compile-time error to include something that you don’t use; streamlining your dependencies to speed up builds in C++ is a nightmare by comparison), and language features (defer, iota, interfaces and goroutines are all very cool). I read through the documentation, watched the videos and toyed around with the language a bit, but I didn’t dive in and get my hands (really) dirty straight away.

Recently, I’ve started working in earnest on the back-end for MegaHAL.10, an online chatterbot that can learn to talk in any language by example. That project has a few requirements that make Go an ideal language for the server-side code, including:

1. The need to be fast, and to manipulate a lot of data in memory. This makes C or C++ an obvious choice, and discounts languages such as Ruby or Python. Go is a lot closer to C++ in terms of efficiency, so it’s a candidate for the implementation language.
2. The need to work with all human languages, meaning it’s important that the chatterbot engine supports UTF-8 end-to-end. All strings in Go are UTF-8 by design, whereas it’s fiddly to properly support UTF-8 in C++.
3. The server will need to handle many simultaneous requests, and the concept of goroutines is ideally suited to satisfying this requirement.

So I started coding things up, and eventually ran into some problems that I couldn’t answer by reading through the provided documentation.

For example, consider the provided documentation for the sort package. The description is a brief “the sort package provides primitives for sorting arrays and user-defined collections”. Just what I need! Unfortunately, the documentation for the Sort method is no more than its specification, “func Sort(data Interface)“. That is all.

You can click through to see the implementation of Sort, revealing that it just calls through to quickSort. Great! I’ll look at the documentation for that. Unfortunately, there is none; quickSort is private (because it starts with a lowercase character), meaning that it doesn’t appear in the documentation at all, although its implementation is right there in the source.

So they’re hiding stuff from you in the docs, while at the same time recommending that you treat the source as documentation.

Now, this is a contrived example. I didn’t really need to look up documentation for Sort, but I did have several problems of this category with other packages and functions that I wanted to use. What I’m trying to illustrate here is how difficult they’ve made it for a newcomer to the language to get acquainted with things. Contrast this, for example, with the documentation for sort in Ruby:

array.sort → an_array

array.sort {| a,b | block } → an_array

Returns a new array created by sorting self. Comparisons for the sort will be done using the <=> operator or using an optional code block. The block implements a comparison between a and b, returning -1, 0, or +1. See also Enumerable#sort_by.

   a = [ "d", "a", "e", "c", "b" ]
   a.sort                    #=> ["a", "b", "c", "d", "e"]
   a.sort {|x,y| y <=> x }   #=> ["e", "d", "c", "b", "a"]

It’s succinct and easy to understand, with clear examples and cross references to related functions. Exactly what you need to dive in and get started on something.

I’ve just finished working on FAQoverflow, a fun little project that was implemented in two weeks using Ruby, and I rarely needed to look outside of the provided documentation when writing a fairly complicated API spider. When I did, I found the community polite, welcoming and useful. Consider, for example, this question to ruby-talk, made soon after the first version of Ruby was released in the west:

Ok,  I'm having trouble with an extremely simple class.
Here's my example:

class ABC
    def initialize( _v )
        @v = _v
    end
end

a = ABC ( 50 )
print a.v            # produced an ERROR

How would I access the variable "v" ?

This got an immediate reply from Matz, the creator of the language:

Example:

class ABC
    def initialize( _v )
        @v = _v
    end
    attr_accesser :v
end

a = ABC ( 50 )
print a.v
a.v = 25
print a.v

Now, that’s really helpful! Here’ by contrast, is the first question I ever saw on golang-nuts (when searching to find out how the ternary operator worked):

How can you call your self a C-like language and NOT have the ternary

operator? But seriously, why isn’t it in Go? This could be a deal-

breaker for me, as it’s often more succinct and clear to use a ternary

operator than an if/else, *especially* if you require curly braces

even for single-line blocks.

And here’s the first answer:

Then “go” write your own language or use a language that makes use of

your precious ternary operator.

Ouch! That answered my question, but it certainly rubbed me the wrong way.

When you’re deeply focused on writing code, and jump to a browser to do a quick search, and the first thing you hit is a mean-spirited reply to a reasonable question, then it breaks your stride and leaves you with a foul taste in your mouth.

Of course, if this just happened a few times it wouldn’t be a problem. What concerned me is that throughout the day, whenever I searched for an answer to a question I had about Go, I always got a mean-spirited answer to a fair question. Sure, I admit I might have had an unlucky streak, and I admit that I usually read the first reply to each question due to my familiarity with Stack Overflow, whereby the first reply tends to be the best. But I just don’t encounter that with the Ruby community. And it was enough to make me stop looking to the forums for answers, which is a shame.

Some more examples of less-than-useful answers (paraphrased) that I encountered on the day I was working with Go:

1. Q: “Having optional arguments to functions would be useful.”

   A: “In the rare instances you need them, make wrapper functions and give them unique names.”

   Coming from Python and Ruby, I’d have to say that optional arguments, or arguments with sensible defaults, are far from a rarity, and are often extremely useful. And, no, I don’t believe that it’s good practice to create eight differently-named wrappers for a function that you’d like to have three optional arguments.

2. Q: “Why does strconv have functions to parse int and int64 but not int32?”

   A: “Because making a function for every possible integer type is tedious and clutters the interface.”

   Clutters the interface? Really? Do you mean that the generated documentation gets a bit longer, meaning you have to scroll through it? Do you mean that you really shouldn’t need to specifically convert int8 and int32 from their string representation, ever? Or that if you do you should write that functionality yourself?

3. Q: “I was following the tutorial on the website, and it said… (a simple misunderstanding)?”

   A: ” I don’t get the point.”

   Thanks for sharing, but why not correct the obvious misunderstanding or be more specific about what it is that you don’t get?

4. Q: “The library code is … hard to learn from. I’d rather see short example programs.”

   A: “Rule one of problem solving: break it down into smaller problems, see rule zero.”

   Right, so rather than provide short, concise usage examples in the documentation, you want me to dive into the source and spend time understanding the implementation of the functions I want to use?

5. Q: “I’m sure this is totally obvious, but I can’t seem to find it anywhere in the docs…”

   A: “One way to get answers to questions is to search the golang-nuts mailing list.”

   Hmmm… dare I ask any questions at all?

Now, yes, perhaps I’m being harsh. But when you’re struggling with something unfamiliar and new, it’s a godsend to find a superb resources such as Programming Ruby or The Ruby User’s Guide or why’s (poignant) guide to Ruby or the Standard Library Documentation, and these resources exist because the community cared enough to want to help newbies to understand why they fell in love with the language. Guess what? It works.

I just don’t get that feeling with the Go community at all. At the moment, it feels defensive and argumentative. Now, it may well be that Go is suited to a different class of problems than Ruby, and that the language is therefore going to be niche, appealing to systems programmers  only, and that it’s still immature and not really suited to production code. Fine. My problem is that it sure wasn’t marketed that way. The Go team claim that their language is an expressive, concise, clean, and efficient language designed to make programmers more productive. Sounds good to me! Now if only I could learn to use it as such without banging my head against defensive, unhelpful, critical answers on golang-nuts.

So, Go Community, where should I be looking?

Sandbox + narrative? Image by GameInformer.

On the other hand, I’d love to see a sandbox adventure game, which places the player in a vast world that they are free to explore. Such a game would render the usual approach of playing an adventure game (go everywhere, examine everything, pick up everything, talk to everyone) impractical. It would be interesting to see what would happen if you were free to explore an enormous environment in which a story could be told based upon what you do, not in spite of it.

Strangely enough, I once worked on such a game. When I worked on L.A. Noire I was particularly interested in the problem of bringing story to the player in large sandbox environments, and of procedurally generating a massive population of NPCs who went about their daily business in realistic ways. I would love to write a text adventure game that used some of these techniques to produce an experience truly different to anything else out there.

Yes, I’m the kind of guy that just hung out and watched a shopkeeper go about his business in Shenmue for the entire day-night cycle, and who tails pedestrians in GTAIV to see where they go. I love that kind of stuff.

Stack Overflow. Image by Jeff Atwood.

Joel and Jeff reused the Q&A Engine they developed for StackOverflow to power a couple of related websites, ServerFault and SuperUser, and then attempted to monetise the whole shebang by offering StackExchange, providing hosted Q&A websites on any topic to whomever was prepared to pay a steep monthly hosting fee. That didn’t work out so well.

Not ones to give up, Joel and Jeff tried Plan B, visiting a whole bunch of VCs in a whirlwind tour, raising US$6m in two weeks. They then returned to the drawing-board, coming up with StackExchance2.0, which is totally free, and whereby new Q&A sites are born via a community proposal process, which is how the USENET newsgroups of yore were created. It seems to be going swimmingly, and there are now about a dozen new Q&A websites in open beta on topics that include Cooking, Bicycles, Home Improvement and Personal Finance.

Many of the questions and answers across all of the websites in the StackExchange family are interesting to browse. As a StackOverflow user, I’ve been creating accounts on all of the new sites as they go into open beta, but, unless you’re really motivated to ask questions or write answers (and they do their darndest to motivate you, with XBLA-style achievements, points and leaderboards), you’ll find it difficult just to browse the sites for enjoyment.

Late last week I discovered two things that I hope will change that.

1. They’ve released a public API that’s super clean and powerful. It’s easy to browse if you have the JSONview Firefox extension, and really easy to access from script.
2. Amazon have, finally, implemented default root object capability for CloudFront, making it possible to serve a static website entirely from S3.

I’m currently writing a Ruby script that will spider all of the StackExchange Q&A websites, with the intention of finding the 100 best question-answer pairs for each site (as determined by a metric that takes into account many things, such as the brevity of the question and answer, the number of upvotes the question received, and the difference between the upvotes received by the first and second answers). These question-answer pairs will be grouped into ten buckets, based on the tags associated with the questions. All of this will then drive a new website, FAQoverflow, which purports to contain “great answers to questions about everything”. The site will be easy to browse (even on your iPhone), using styling inspired by Readability. And there’ll be PDF and eBook editions of the FAQ available for download. It’ll all be built automagically via a script that publishes a new edition every week or so, and will be user-supported (I’ll ask for donations from the community to pay the minimal hosting bill).

This explains why I started doing some vanity searches this afternoon, digging up an old web-page of mine from the mid-1990′s. Yes, it used TABLEs for formatting (back then, it was common to UPPERCASE your HTML). Yes, it included a count of page views at the bottom of each and every page. And, yes, it even included a link to my “hotlinks” which I could access when using “Netscape” from home, over my swanky 14.4 dialup connection (presumably when I could drag myself away from playing with my PlayStation).

One of my old web pages was about number problems. I used to have fun writing programs to solve puzzles such as these by brute force:

Which numbers satisfy the following property… the sum of the individual digits of the number raised to the power of that digit is equal to the number itself? For the purpose of this problem, we define 0^0=0.

For example, the number 3435 is known to have this property, since 3^3 + 4^4 + 3^3 + 5^5 = 3435.

This problem proved rather easy to solve. We know that no numbers above 10*9^9 (about 4e9) can have this property, simply because the sum will always be less than the original number. Therefore we can solve the problem by brute force, using unsigned 32-bit integers.

I wrote such a program, and it took a few hours to execute on our SGI Onyx computer. The solution set is { 0, 1, 3435, 438579088 }.

And here’s my program:

#include <stdio.h>

typedef unsigned long INT;

int main(int argc, char **argv)
{
	INT number;
	INT sum;
	INT num;
	INT power[]={0,1,4,27,256,3125,46656,823543,16777216,387420489};

	for(number=0; number<4e9; ++number)
    {
		sum=0;
		num=number;

		do {
			if((sum+=power[num%10])>number) break;
		} while(num/=10);

		if(number==sum) printf("%d\n", number);
	}

	return(0);
}

It’s almost unbelievable that such a program used to take “a few hours” to run on a machine that was, at the time, a veritable power horse. Today I compiled and ran it on my grungy lappy. It burned one core for a few minutes to find the same solution. All the while, my computer remained snappy, and I actually became impatient waiting for the answer. What have we become? Why aren’t we doing more, now that we all carry around supercomputers with us?

Funny thing is, my website was averaging 1800 hits per day back in the mid-90′s. This blog, by comparison, barely manages 10. Well, phooey.

Update: Optimisations

On my desktop at work, this same program ran in 150 seconds, using a quarter of the total processing power of the machine. That’s something like 26 million numbers checked every second. That would have seemed like magic 13 years ago, when the Onyx was checking half a million numbers each second.

Beetlefeet asked about the optimisations mentioned in my comment below. Each optimisation sped up processing by a factor of 10 or so. I’ve re-run them on my desktop. The first completes in 45 seconds, and the second completes in around 5 seconds, so they still count for something!

The first optimisation, by David Conrad, worked by keeping a running total of the current value, modifying it each time around the loop based on how the digits in the number changed. It printed the result whenever the number equalled this running total. Here’s the code:

#include <stdio.h>

int digit[10] = {0,0,0,0,0,0,0,0,0,0};
long diff[10] = {-387420489,1,3,23,229,2869,43531,776887,15953673,370643273};

int main(void)
{
    unsigned long x = 0, val = 0;
    int i;
    while (x < 4e9) {
        if (x == val) printf("%lu\n", x);
        i = 0;
        digit[i] = digit[i] == 9? 0 : digit[i]+1;
        val += diff[digit[i]];
        while (digit[i] == 0) {
            i++;
            digit[i] = digit[i] == 9? 0 : digit[i]+1;
            val += diff[digit[i]];
        }
        x++;
    }
    return 0;
}

The second optimisation, by Geoff Bailey (AKA Fred the Wonder Worm), was really clever; it maintained a running total too, but worked by looping over 10 variables, one that represented each digit. Here’s the code:

#include <stdio.h>

typedef unsigned long   ulong;

long diffs[] = {
    1, 3, 23, 229, 2869, 43531, 776887, 15953673, 370643273, -387420489
};

main()
{
    ulong       sum, n;
    int         i0,i1,i2,i3,i4,i5,i6,i7,i8,i9;

    sum = 0;
    n = 0;

#define LOOP(var, lim, code)	\
    for (var = 0; var < lim; sum += diffs[var++])	code

    LOOP(i9, 4,
    LOOP(i8, 10,
    LOOP(i7, 10,
    LOOP(i6, 10,
    LOOP(i5, 10,
    LOOP(i4, 10,
    LOOP(i3, 10,
    LOOP(i2, 10,
    LOOP(i1, 10,
    LOOP(i0, 10,
    {
        if (n == sum)
            printf("%lu\n", n);
        n++;
    }
    ))))))))))
    return 0;
}

I guess that optimisation and obfuscation are closely related (although you don’t always get faster code by making it harder to understand).

I went into GGJ with the intention of developing a Flash game, even though I’d never done that before, and didn’t know my ActionScript from my Objective C. It was a long haul; I worked around 40 hours in a single weekend (the equivalent of a normal working week), and I did all of the design and implementation myself (although Pazu and Sizzle helped with graphics, and I used some other freely available assets).

So, without further ado, I humbly request that you check out and vote for Bogus Quest.

A journey is best measured in friends, rather than miles.

In addition, each game needs to incorporate, somehow, at least one of these three keywords:

• Sheep
• Scorched
• Reflection

Thinking!