Skip to content

FAQoverflow Has Launched!

I’m very happy to announce that FAQoverflow has launched! Yay! Please take a look, by visiting www.faqoverflow.com, and please help to spread the word in whichever way you can.

FAQoverflow

FAQoverflow Has Launched!

FAQoverflow is the coffee table book version of the StackOverflow (SO) family of Q&A sites. It contains great answers to questions about everything, and is ideal for flipping through.

Our crazy algorithms automagically generate FAQoverflow, selecting the best questions and answers from each Q&A site (there’s about 20 of them so far, on topics from Cooking and Home Renovations to Advanced Mathematics and Computer Programming). These questions and answers are then rearranged into a traditional FAQ format, with a bit of Javascript to make everything readable from the home page.

There is a dedicated webapp version of FAQoverflow for the iPhone. It’s free, and it’s easy to install.

In the next week or so we’ll be releasing PDF and eBook versions of FAQoverflow, idea for offline reading. When they’re available, you’ll be able to get them here.

I had the idea for FAQoverflow just over two weeks ago, and I quickly set about making it real. All up, implementation has taken about 60 hours so far. This is what I did:

  1. Registered the faqoverflow.com domain name.
  2. Set up an Amazon Web Services account.
  3. Set up S3 (Amazon’s file hosting service) and CloudFront (Amazon’s content-distribution network).
  4. Quickly got a “Coming Soon” page online
    • Hand-coded HTML5 and CSS
    • Incorporated Google Analytics
    • Chose suitable Typekit fonts
    • Set up site for Google Webmaster Tools (and Yahoo! and Bing as well)
    • Made sure site works well on the iPhone
  5. Wrote a Ruby script to spider the Stack Exchange API
    • Made sure the script recovers gracefully if it crashes out, as spidering takes a long time
    • Made sure the script doesn’t hit the SO servers too hard
    • Developed a crazy algorithm for finding and categorising the best questions and answers
  6. Wrote a Ruby script to generate a static website from the spidered content
  7. Tweaked the CSS to make sure each question and answer is beautifully formatted
    • Used Google’s “Prettify” library for code formatting
    • Used MathJax to format LaTeX-style maths equations
    • Hooked up a PayPal donation button (in the hope the community can fund ongoing hosting)
  8. Wrote some snazzy JavaScript so you can navigate all content without leaving the home page.
  9. Wrote a Ruby script to upload all of the generated content to S3, thereby setting the site live :)
    • Compact HTML with HTML Tidy
    • Compact JavaScript with the Closure Compiler
    • Compact CSS with the YUI Compressor

Spidering the content is relatively easy. It works as follows.

  1. Make a request to the Stack Auth API to find all SO sites that are publicly visible, excluding the “meta” sites.
  2. For each site, request the top 1000 questions, sorted by votes.
  3. For each question, request the top 2 answers, sorted by votes.
  4. Exclude any questions that are closed, or that don’t have any answers.
  5. For each question, calculate a quality score. This takes into account the following factors:
    • The number of votes the question received.
    • The difference between the number of votes of the top two answers (thereby favouring questions that have one standout answer).
    • The length of the question (preferring questions that are a few sentences long).
    • The length of the answer (preferring answers about a page long).
  6. For a particular site, sort the questions by quality.
  7. Divide the questions up into sections, as follows:
    • Consider the question at the top of the list (i.e. the one with the highest quality score).
    • Create a tentative section for each combination of tags for that question, but don’t create a tentative section if the FAQ already contains that section. So, for example, if a question is tagged “Ruby” and “Web”, then create three tentative sections (“Ruby”, “Web” and “Ruby and Web”).
    • Walk through the list, adding questions to the tentative sections if they have the requisite tags. Stop adding questions to a tentative section once it contains ten questions.
    • Discard any tentative sections that contain less than 5 questions by the end of this process (we want each section of the FAQ to contain between 5 and 10 questions).
    • Of the remaining tentative sections, choose the one with the greatest number of tags. If there is more than one such section, choose the one with the largest average quality score. Add the chosen section to the FAQ, and remove the questions in that section from the list. Repeat this process!
    • If no tentative sections remained after this process, add the question to the “Miscellaneous” section. Once that fills up, we’re done!
  8. Sort the sections by their average quality, but always place the Miscellaneous section last.
  9. Sort the sites (which we now call “chapters”) by their average quality, but always place the “About FAQoverflow” chapter at the top.

The entire process is automated, and recovers from errors. At the moment we kick things off manually, but we plan to start up a server so that the FAQ spidering and generation process runs automatically, around the clock, which will allow me to update FAQoverflow every few days.

Yes, that’s right – we currently do not run a server at all. The entire site is static, and hosted by CloudFront. It look like it’ll cost us a few cents a month to run. Not only that, we leverage Amazon’s content distribution network, so the site is really fast regardless of where you are in the world, as it serves files from “edge locations” in the US (9 datacentres across the country), Europe (four countries across the EU) and Asia (Singapore, Hong Kong and Japan), depending on where the requests are coming from. Finally, it’s super reliable and robust, so that we fully expect to survive any level of Slashdottery that we may be subjected to. Great stuff!

We believe that this is the direction that webapps are moving in; with the smarts migrating from the server back-end to the client and to offline script processing, and we anticipate experimenting with this approach a whole lot more.

Anyway, I’ve spent hours reading FAQoverflow, so it works for me. Hope you’ll get some pleasure out of it too!

Procedural Adventures

Apart from text adventures, I love point-and-click adventure games (of the Monkey Island ilk), and the more modern Japanese interpretations of this genre (such as Hotel Dusk and Another Code). These latter games tend to be more focused on telling a story, and the interface offered to the player is subsequently stripped down (with less emphasis on navigating the environment and using objects on everything to see what might happen).

L.A. Noire

Sandbox + narrative? Image by GameInformer.

On the other hand, I’d love to see a sandbox adventure game, which places the player in a vast world that they are free to explore. Such a game would render the usual approach of playing an adventure game (go everywhere, examine everything, pick up everything, talk to everyone) impractical. It would be interesting to see what would happen if you were free to explore an enormous environment in which a story could be told based upon what you do, not in spite of it.

Strangely enough, I once worked on such a game. When I worked on L.A. Noire I was particularly interested in the problem of bringing story to the player in large sandbox environments, and of procedurally generating a massive population of NPCs who went about their daily business in realistic ways. I would love to write a text adventure game that used some of these techniques to produce an experience truly different to anything else out there.

Yes, I’m the kind of guy that just hung out and watched a shopkeeper go about his business in Shenmue for the entire day-night cycle, and who tails pedestrians in GTAIV to see where they go. I love that kind of stuff.

XYZZY

I really love text adventure games. I’ve always had a fascination with natural language processing (my postgrad research was about computational grammatical inference after all), so part of the fun was figuring out how the parser works, and applying that knowledge to write my own text adventures in BASIC on the C64.

Zork

A classic text adventure. Image by Peter Scheyen.

These days there are great tools for writing text adventure games. The best would have to be Inform 7, which lets you write text adventures that are compatible with the Infocom Z-Machine (and thereby playable on a whole bunch of platforms, including the iPhone). Code written in Inform looks like this:

The wood-slatted crate is in the Gazebo. The crate is a container 

Insane, right? You should try it out. It’s pretty amazingly great.

If you’re not interested in writing your own text adventures, but you do enjoy playing them, then you’ll be pleased to know that there’s a vibrant community that continues to produce wonderful games (and who would prefer it if they were referred to as “interactive fiction”, thank-you very much). If you want to play the very best of these modern text adventures (sorry), then you could do worse than check out the winning entries of the Interactive Fiction Competition, which has been running for the past fifteen years. Heck, you can even play them in your browser, with Parchment (a z-machine interpreter written in JavaScript).

Speaking of text adventures, I’ve recently received my copy of Get Lamp, the text adventure documentary. Perhaps a screening is in order?

Looking Back

I clicked the Readability bookmarklet by accident when viewing my profile page on Twitter and was surprised to get a single, nicely formatted page of every tweet I’ve ever written (all 500-ish of them). I had fun taking a look back at the last 31 months. Here’s a selection.

2008

  • My next-door neighbours are involved in making a feature film, shooting tonight nearby.
  • Conan the Barbarian or Super Mario Galaxy? Just one of life’s puzzles. Who am I kidding? Mario beats Arnold hands-down.
  • Listening to TMBG’s new children’s album… after E. has gone to bed. Is it bad that I enjoy it as much as her?
  • Just saw Eastern Promises. Great movie, great eye-stabbing scene.
  • Four chicken frames -> eight litres of chicken stock, frozen, to last us for the next few weeks.
  • Time to spend some time in the curious village.
  • How would YOU matchmake a bunch of humans into groups that then play each other at some game? Huh?
  • Basking in the glory of a freshly cleaned car.
  • Bowling is easier on the Wii. Beginning to suspect same for boxing.
  • Car stolen, crashed and written off. Up since 4am with the police.

2009

  • Back home; squeeze in an hours coding before bed at 3:30am, then wake up at 8:30am to get back to the Jam.
  • Been indoors for 8 days. Finally starting to feel on the mend. Was quite worried there for a bit – thought we were goners. Swine flu sucks!
  • Dana wants to push up to jump. Eighties child can’t handle more than one button. A to blow, B to jump… how hard can it be?
  • Final Twin Peaks sesh tonight. Dinner: T-Bone with Diane sauce, potato au gratin, salad, beer. Snacks: showbag goodies, beer.
  • Just tried to get into the wrong car. For five minutes.
  • Coffee and croissant at Perth airport, waiting for flight to Melbourne. Go!
  • Whisky poisoning not too bad. Coffee and shower, then off for Jaffe’s keynote.
  • Farbs: fan e-mail is a gateway drug to self-Googling.
  • Best advice for startups from the big publishers: “don’t build an MMO”.
  • Yahtzee says Project Natal is a “creepy little boy simulator”.
  • Dinner at Attica… Rob Sitch and Eddie Maguire just walked in. Only in Melbourne!
  • Plan to polish & release KranzkyEngine, and develop PostalWorker, which was devised at the hotel bar.
  • What’s up with the Hawkeye 4wd driving around with like 20 cameras mounted all over it? Seen in Northbridge.
  • Great party last night; 21yo scotch, good cheese, and arthouse cinema in the back garden. All while the kids went berserk. In a good way.
  • I ran 14345m before hitting a wall and tumbling to my death on my iPhone.

2010

  • Mixed a brandy-and-dry, and drank while munching green olives with pimientos. I think my hair will turn a swept-back grey soon.
  • Frankly, you should quite frankly prototype your GUI. Frankly, I say!
  • Channel 9 commentary: “he’s universally known in America”
  • Note to self: don’t type “anal” into the address bar as a shortcut to Google Analytics.
  • Counted dozens of smashed car windows walking to train station. Some brand new luxury cars in dealerships too.
  • Arrived at beautiful holiday house on the beach in Eagle Bay. Now, to figure out the optimum way to relax…
  • Played San Juan at 399bar last night, and I must say it was pretty cool. More German board games in small bars!
  • “I saw it on YouTube” not a valid defense for making daughter Cheerios fried in butter. But it seems so American! And we had no popcorn!
  • Curse you, Desktop Dungeons, for stealing my Friday night.
  • Chicken livers and ladies fingers at The Prophet. Good friends, good wine, good times. Great party at Tee’s afterwards. Eh, what kids???
  • Oh man that was totally fucked up. In the most awesome of ways. Breaking Bad, you’re the best thing ever.
  • Off tonight for a beef injection (1.5kg 400+ day dry aged wagyu ribeye to share, with roasted bone marrow and potatoes gratin). And wine!
  • Meat Stylus.
  • I am the mayor of crumpet (unfortunately NOT a euphemism).
  • Just realised I’m sharing my iTunes library at work. Trouble is, I’ve only used this laptop to assemble a playlist for my mum’s 60th.

Hmmm… what a strange way to look back on your life! But, I gotta say, I prefer myself in retrospect :)

Zarch

My three favourite games of all time are Exile, The Sentinel and Zarch.

Zarch is the name of a game on the Acorn Archimedes; I played the Amiga port, which was called Virus. In fact, Zarch itself was a full-fledged version of an Archimedes demo called Lander. David Braben (yes, he who, along with Ian Bell, created Elite) wrote it in three months in 1987. Virus, the Amiga port, didn’t look as good (no light-sourced polygons), ran slower, and was released a year later, but I still loved it.

Virus

Virus on the Amiga. Image by Wikipedia.

Lander became known for its fast, smooth, solid-3D graphics. In 1987 this was an almost unbelievable achievement. ACE magazine (which was known for rating games out of 1000 points, and for publishing a lastability graph with each review that extimated how much you’d still be enjoying the game after an hour, a day, a week, a month and a year) featured it on it’s cover, along with the headling: “Solid 3D – The Future of Games?”

I loved playing Virus for its procedurally generated landscapes, but what really clicked for me was the controls. It was the first game I played where a mouse was the preferred control method for movement. Moving the mouse rotated the ship around two axes, while the right mouse button was used to generate thrust. First time players would instantly crash and burn, due to the precision and sensitivity of the controls, and the learning curve was steep. Persistence paid off, however, and after plenty of practice it was possible to skim low over the landscape and turn on a dime. Sweet.

The left mouse button, of course, was used to fire your weapon. The gameplay consisted of ridding the landscape of all polluting alien craft before they could complete their task of seeding the verdant land with their evil red virus. Simple, really.

Post Scriptum

Some people have asked how to get hold of The Sentinel, Exile and Zarch (aka Virus). Here’s how:

  1. Purchase the Plus edition of Amiga Forever, which includes licensed versions of all Amiga Operating System ROM and 100 games of various quality. It costs US$30, and it comes with Exile included.
  2. Search Google to try to find versions of The Sentinel and Virus that you can run under emulation via Amiga Forever, but that may not be available with the permission of the copyright owner.
  3. Alternatively, search Google for free remakes of these games (although I cannot vouch for their quality).

Exile

My three favourite games of all time are Exile, The Sentinel and Zarch.

Exile is an “arcade adventure” (a now defunct term; these days you’d call it an “exploration platformer”) that features realistic physics and AI, and which has a large map that was mostly procedurally generated (but then touched-up by hand). How could I not love it?

Exile

The Chimps of Ni. Image by: Hall of Light

I discovered Exile when a playable demo was included on the cover of Amiga Power in 1991. I played the demo for hours on end, and eventually wound up buying the full game (reaching the limits of the demo took quite a lot of playing). I just loved the exploration, and the ways that items interacted with one another. You could pick up a glass jar, carry it into a pool of water to fill it up, then gently transport it across the map to use to extinguish a fire, for instance.

Much of the game involved experimenting with the world, and solving subtle puzzles to open up new areas for exploration, all interspersed with moments of frantic combat.

The physics engine was quite advanced for the day. In fact, it was a rarity to play a game that had any kind of physics simulation at all, so it was quite a novelty. It was fun to pick up and throw objects of all sorts, just to see what would happen.

The AI was also impressive. You discover and interact with many different kinds of creatures in the caverns beneath the planet’s surface, including anthropomorphic monkeys who say “ni” and steal items from you, and a friendly robot who responds to your whistle, and which you can use to solve puzzles.

Exile was conceived and written by Peter Irvin and Jeremy Smith (Jeremy, RIP, also wrote the classic C64 game Thrust). How good is it? Well, Edge Magazine has awarded only three perfect 10/10 ratings to games that were released before the magazine started publication. Those games are Elite, Super Mario Bros. and Exile. Which means you should play it. Now.

The Sentinel

My three favourite games of all time are Exile, The Sentinel and Zarch.

The Sentinel is an extremely weird game, but I loved it from the moment I bought the Commodore 64 version at KMart in 1987. I went on to love the Amiga version even more.

The Sentinel

A typical view in the game. Image by: C64-Wiki

It was conceived and written by Geoff Crammond, who also wrote REVS (a great driving game on the C64), Stunt Car Racer (an insanely great game on the Amiga, which was best enjoyed by linking two Amigas together with a null-modem cable for multiplayer mayhem), and the Grand Prix series (a realistic driving simulation that started life as Formula One Grand Prix on the Amiga in 1992).

For me, The Sentinel is an outstanding game. It features a first-person view of a solid-3D landscape, which was incredible at the time. The ten-thousand levels are procedurally generated, which is something I’ve loved, from the days of Mike Singleton’s The Lords of Midnight and Midwinter through to Derek Yu’s Spelunky and Asobo’s Fuel (which Shamus Young has deconstructed at length).

The game mechanic is stealth; you must creep up on and ultimately destroy the all-seeing Sentinel, which occupies the highest square of the chessboard terrain, and which slowly rotates its gaze. You, of course, start each level standing on the lowest square. And, most unfortunately, you cannot move; only rotate.

To vanquish the Sentinel, you must absorb energy from the trees scattered around the level, and then use it to create robots that you may transfer your consciousness into, thereby teleporting yourself from square to square around the terrain. You gain height by creating stacks of boulders, and then creating a robot on top of them. In this sense, The Sentinel is a game of resource management, with trees, boulders and robots representing one, two and three units of energy respectively.

The trick is that you can only absorb energy from a tree, boulder or robot (when teleporting, it always pays to absorb the robot that you’ve teleported from) if you can see the square that it’s standing on. You must therefore work to increase your height, opening up lush new fields of tree-energy to mine. All the while, the Sentinel does the same thing; it absorbs energy from all bounders and robots it can see (including you), and uses it to create new trees at random places around the level. I loved this aspect of the game; it satisfies the first law of thermodynamics!

You can imagine how gut-wrenching it can be playing a level, with the gaze of the Sentinel upon you, watching your energy slowly drain away while you frantically scramble to create a new robot in a safe location, teleporting there only to watch your former self transform from robot to boulder to tree as the Sentinel consumes its energy.

Things get worse. If the Sentinel can see you, but not the square that you stand upon, then it transforms a nearby tree into a Meanie, which, when it rotates to face you, forces you to hyperspace. This consumes three units of energy, and transports you to a random square on the level that is lower than the one you were standing on. You could also initiate a hyperspace yourself, as a last-ditch attempt to avoid the full gaze of the Sentinel.

Victory was sweet. Standing above the Sentinel, immune to its gaze, you’d spend some time gloating before absorbing its energy to win the level. You’d then go on to a harder level, with the exact level number determined by the amount of energy you had remaining (so you’d jump more levels if you finished with more energy to spare). And because the levels were procedurally generated, there was no need to save the game. You’d simply write down theĀ  8-digit code for the level you were on, and type it in the next time you wanted to resume your game (or, sneakily, try to reverse-engineer the algorithm, or enter level codes that you’d got from a friend at school, or seen printed in a magazine).

Later levels featured Sentries, which had all of the power of the Sentinel, but which stood on lower squares around the level. This made things even more frenetic.

Zzap!64, the premier games magazine at the time, refused to award The Sentinel a score in its review of the game, stating that such a wholly original game is in a class of its own, and therefore defies rating. Over twenty years later this is still the case. Everyone should play this classic game.

FAQoverflow

StackOverflow is a Q&A website for professional programmers. Joel Spolsky and Jeff Atwood developed the concept as a direct competitor to Expert Sex Cha… errr, Experts Exchange, and I now find that answers on StackOverflow are often in the top few results when I search Google for geeky, programmer-type stuff. They’re doing some good things; I liked the podcasts they recorded while the site was being developed, and I’m excited that all user-contributed content is cc-wiki licensed, allowing it to be shared and remixed.

Stack Overflow

Stack Overflow. Image by Jeff Atwood.

Joel and Jeff reused the Q&A Engine they developed for StackOverflow to power a couple of related websites, ServerFault and SuperUser, and then attempted to monetise the whole shebang by offering StackExchange, providing hosted Q&A websites on any topic to whomever was prepared to pay a steep monthly hosting fee. That didn’t work out so well.

Not ones to give up, Joel and Jeff tried Plan B, visiting a whole bunch of VCs in a whirlwind tour, raising US$6m in two weeks. They then returned to the drawing-board, coming up with StackExchance2.0, which is totally free, and whereby new Q&A sites are born via a community proposal process, which is how the USENET newsgroups of yore were created. It seems to be going swimmingly, and there are now about a dozen new Q&A websites in open beta on topics that include Cooking, Bicycles, Home Improvement and Personal Finance.

Many of the questions and answers across all of the websites in the StackExchange family are interesting to browse. As a StackOverflow user, I’ve been creating accounts on all of the new sites as they go into open beta, but, unless you’re really motivated to ask questions or write answers (and they do their darndest to motivate you, with XBLA-style achievements, points and leaderboards), you’ll find it difficult just to browse the sites for enjoyment.

Late last week I discovered two things that I hope will change that.

  1. They’ve released a public API that’s super clean and powerful. It’s easy to browse if you have the JSONview Firefox extension, and really easy to access from script.
  2. Amazon have, finally, implemented default root object capability for CloudFront, making it possible to serve a static website entirely from S3.

I’m currently writing a Ruby script that will spider all of the StackExchange Q&A websites, with the intention of finding the 100 best question-answer pairs for each site (as determined by a metric that takes into account many things, such as the brevity of the question and answer, the number of upvotes the question received, and the difference between the upvotes received by the first and second answers). These question-answer pairs will be grouped into ten buckets, based on the tags associated with the questions. All of this will then drive a new website, FAQoverflow, which purports to contain “great answers to questions about everything”. The site will be easy to browse (even on your iPhone), using styling inspired by Readability. And there’ll be PDF and eBook editions of the FAQ available for download. It’ll all be built automagically via a script that publishes a new edition every week or so, and will be user-supported (I’ll ask for donations from the community to pay the minimal hosting bill).

Effortlessness

My daughter Eliza, who is four-and-a-bit, is starting to help around the house. I’m proud that she, without being asked, removes the depleted toilet paper roll from the holder and fits on a fresh replacement before either throwing the old cardboard tube in the bin or, more likely, using it as a didgeridoo or decorating it with stickers.

I’m particularly proud because someone in the building where I work isn’t capable of doing this, and they’re all growed up.

Almost Empty. Photo by Eastwood/Getty.

Rather than going through the simple, obvious steps (unhook toilet roll holder, slide off used cardboard tube, pick up fresh roll from the convenient stack within arms reach, thread it on the holder, rehook the holder, take the empty cardboard tube out when you’re done to throw in the bin), they’ve chosen some not-so-obvious ones (start rationing the last remaining scraps of paper in the hope that you’ll avoid the hassle of being the person to replace the toilet roll by leaving one tiny dangling ripped piece of paper on the roll, thereby signalling that it’s Not Quite Finished Yet, or, in the worst-case scenario, retrieve a fresh roll only to balance it precariously on top of the fucking empty cardboard tube, resulting in a completely non-functional toilet paper dispenser).

This obviously annoys me, as small things often do. It’s annoying mostly because it’s not obvious why this strategy benefits the individual whom practices it. You see, it doesn’t require any less effort than the strategy which has a beneficial outcome for every (man) in the building, so why do it at all? When presented with two actions of comparable cost, why not choose the one with greater utility? Isn’t there an Occam’s Razor for responsible human behaviour?

Equally annoying are drivers who choose to signal that they’re changing lanes, turning the corner or moving into the freeway exit ramp while they’re in the act of doing it. I can understand why people choose not to signal at all – they’re actually saving themselves a modicum of effort. But why go to the same effort (of moving your hand to the signal lever and pushing it up or down) while, at the same time, completely removing any beneficial effect to other drivers on the road (that is, actually giving us fair warning of your intentions)? I know that when you were seventeen you were taught how to move the signal lever by moving it while turning the steering wheel (up for anti-clockwise, down for clockwise), but that doesn’t mean you should still be doing that. It’s not a rule, you know.

I try to give these people the benefit of the doubt, I really do, but it’s of no use. I think they’re just passively evil.

Pseudo Intelligence as Entertainment

Research can be speculative or applied. Artificial Intelligence research is often both, trying to solve real-world problems while at the same time testing theories about how the human brain works.

Man Playing Game

Game Face. Photo By: Phillip Toledano

A branch of the AI research crowd are interested in games both as a testbed for theoretical work and as a market for applied AI. Unfortunately, these are conflicting goals.

People play games to be entertained, and any AI present in the game must contribute to this. I personally think that AI enhances player enjoyment when it is both surprising and relevant. That is, it should result in an experience which feels new, yet which is consistent in the current context.

This regrettably suggests that AI is synonymous with NPCs, which is a mistake that both game players and researchers make. There are plenty of opportunities for non-NPC AI in games, and yet there is scant research being done in these areas. I’m referring to things such as

  • a cinematic camera that responds realistically to game world events and player movement;
  • dynamic set pieces, including chase sequences and fights;
  • story events that fit the overarching narrative but which adapt to a sandbox environment;
  • an audio score that foreshadows unscripted events and announces the presence of hero characters;
  • large-scale crowd and vehicle simulation;
  • adaptive character animation and movement;
  • accurate matchmaking algorithms for multiplayer online games;
  • elegantly handling dropouts with automatic AI takeover;
  • automatic navmesh generation from a polygon soup;
  • predicting player behaviour to counteract controller and network lag; and
  • automatic exploit detection and prevention.

The problem is that the role of the (usually lone) AI programmer on a game development team often involves many tasks that get in the way of performing research, including asset acquisition, audio and animation integration, data production, tool implementation and support, multithreading support, optimisation, debugging and so on, leaving a perfect opportunity for academia to supply the research chops. What’s needed are robust, efficient, designer-tweakable techniques that are easy to debug, and which scale with available CPU and memory. Sadly these requirements are not a priority for researchers, and yet researchers remain perplexed that game developers don’t use some of the inefficient, unpredictable techniques that they develop.

You see, the problem is that your neat little algorithm might perform well 95% of the time, which may be a great improvement over the state-of-the-art, and which may justify publication, but 95% is not good enough when you have an audience of 5 million game players (as hundreds of thousands of them will see broken behaviour).

But the biggest point of contention between game developers and researchers is that we gamedevs think that cheating is acceptable. After all, a game is just a Turing Test, with the player deciding whether intelligence exists based on the behaviour they perceive, so why not use all available information to deliver on that promise, instead of placing artificial restrictions on what data can be used based on whether or not it would be available to a human player? It just doesn’t matter how the behaviour is achieved – we’re not looking for insights into how the human brain works – it’s all down to player experience. This behavioural approach is out of favour with researchers (and has been ever since Chomsky defeated Skinner), but is the core of pragmatic game design. Perhaps never the twain shall meet.