Tuesday, November 17, 2015

Neuroevolution in games

Neuroevolution - the evolution of weights and/or topology for neural networks - is a common and powerful method in evolutionary robotics and machine learning. In the last decade or so, we have seen a large number of applications of neuroevolution in games. Evolved neural networks have been used to play games, model players, generate content and even enable completely new game genres. To some extent, games seem to be replacing the small mobile robots ubiquitous in evolutionary robotics and simple benchmarks used in reinforcement learning research.

Sebastian Risi and I have written a survey on neuroevolution in games, including a discussion of future research challenges. The main reason is that there was no survey of neuroevolution in games in existence; the other reason was that we wanted a tutorial overview to hand out to the students in our Modern AI for Games course.

A while back we asked the community to send us comments and suggestions for important work we might have overlooked. We received a lot of useful input and incorporated most of the suggested work. Thank you for very much for the help!

Now we are happy to announce that the paper will finally be published in the IEEE Transactions on Computational Intelligence and AI in Games (TCIAIG). We hope you like it!

TCIAIG early access:

A preprint of the manuscript is available here:

Wednesday, October 07, 2015

What if videogames had actual AI?

Is there any artificial intelligence in a typical videogame? Depends on what you mean. The kind of AI that goes into games typically deal with pathfinding and expressing behaviors that were designed by human designers. The sort of AI that we work on in university research labs is often trying to achieve more ambitious goals, and therefore often not yet mature enough to use in an actual game. This article has an excellent discussion of the difference, including a suggestion (from Alex Champandard) that the "next giant leap of game AI is actually artificial intelligence". And there's indeed lots of things we could do in games if we only had the AI techniques to do it.

So let's step into the future, and assume that many of the various AI techniques we are working on at the moment have reached perfection, and we could make games that use them. In other words, let's imagine what games would be like if we had good enough AI for anything we wanted to do with AI in games. Imagine that you are playing a game of the future.

You are playing an "open world" game, something like Grand Theft Auto or Skyrim. Instead of going straight to the next mission objective in the city you are in, you decide to drive (or ride) five hours in some randomly chosen direction. The game makes up the landscape as you go along, and you end up in a new city that no human player has visited before. In this city, you can enter any house (though you might have to pick a few locks), talk to everyone you meet, and involve yourself in a completely new set of intrigues and carry out new missions. If you would have gone in a different direction, you would have reached a different city with different architecture, different people and different missions. Or a huge forest with realistic animals and eremites, or a secret research lab, or whatever the game engine comes up with.

Talking to these people you find in the new city is as easy as just talking to the screen. The characters respond to you in natural language that takes into account what you just said. These lines are not read by an actor but generated in real-time by the game. You could also communicate with the game though waving your hands around, dancing, exhibiting emotions or other exotic modalities. Of course, in many (most?) cases you are still pushing buttons on a keyboard or controller, as that is often the most efficient way of telling the game what you want to do.

Perhaps needless to say, but all the non-player characters (NPCs) navigate and generally behave in a thoroughly believable way. For example, they will not get stuck running into walls or repeat the same sentence over and over (well, not more than an ordinary human would). This also means that you have interesting adversaries and collaborators to play any game with, without having to resort to waiting for your friends to come online or have to resort to being matched with annoying thirteen year olds.

Within the open world game, there are other games to play, for example by accessing virtual game consoles within the game or proposing to play a game with some NPC. These NPCs are capable of playing the various sub-games at whatever level of proficiency that fits with the game fiction, and they play with human-like playing styles. It is also possible to play the core game at different resolutions, for example as a management game or as a game involving the control of individual body parts, by zooming in or out. Whatever rules, mechanics and content are necessary to play these sub-games or derived games are invented by the game engine on the spot. Any of these games can be lifted out of the main game and played on its own.

The game senses how you feel while playing the game, and figures out which aspects of it you are good at as well as which parts you like (and conversely, which parts you suck at and despise). Based on this, the game constantly adapts itself to be more to your liking, for example by giving you more story, challenges and experiences that you will like in that new city which you reached by driving five hours in a randomly chosen direction. Or perhaps by changing its own rules. It's not just that the game is giving you more of what you already liked and mastered. Rather more sophisticatedly, the game models what you preferred in the past, and creates new content that answers to your evolving skills and preferences as you keep playing.

Although the game you are playing is endless, of infinite resolution and continuously adapts to your changing tastes and capabilities, you might still want to play something else at some point. So why not design and make your own game? Maybe because it's hard and requires lots of work? Sure, it's true that back in 2015 it required hundreds of people working for years to make a high profile game, and a handful of highly skilled professionals to make any notable game at all. But now that it's the future and we have advanced AI, this can be used not only inside of the game but also in the game design and development and process. So you simply switch the game engine to edit mode and start sketching a game idea. A bit of a storyline here, a character there, some mechanics over here and a set piece on top of it. The game engine immediately fills in the missing parts and provides you with a complete, playable game. Some of it is suggestions: if you have sketched an in-game economy but have no money sink, the game engine will suggest one for you, and if you have designed gaps that the player character can not jump over, the game engine will suggest changes to the gaps or to the jump mechanic. You can continue sketching, and the game engine will convert your sketches into details, or jump right in and start modifying the details of the game; whatever you do, the game engine will work with you to flesh out your ideas into a complete game with art, levels and characters. At any time you can jump in and play the game yourself, and you can also watch a number of artificial players play various parts of the game, including players that play like you would have played the game or like your friends (with different tastes and skills) would have played it.

If you ask me, I'd say that this is a rather enticing vision of the future. I'll certainly play a lot of games if this is what games will look like in a decade or so. But will they? Will we have the AI techniques to make all this possible? Well, me and a bunch of other people in the CI/AI in Games research community are certainly working on it. (Whether that means that progress is more or less likely to happen is another question...) My team and I are in some form working on all of the things discussed above, except the natural interaction parts (talking to the game etc).

If you are interested in knowing more about these topics, I recently wrote a blog post summarizing what I've been working on in the last few years. Last year, I also co-wrote a survey paper trying to give a panoramic overview of AI in games research and another survey paper about computational game creativity. Also, our in-progress book about procedural content generation covers many of these topics. You might also want to look at the general video game playing competition (and its results) and the sentient sketchbook and ropossum AI-assisted level design systems. For work on believable NPC behavior, check out the Mario AI Turing Test competition and procedural personas.

Finally, I've always been in favor of better collaboration between AI researchers and game developers. I wrote a post last year about why this collaboration doesn't always work so well, and what could be done about that.

Thursday, July 30, 2015

Revolutionary algorithms

Edit: Apparently it is not clear to everyone that the following post is satire. Well, it is.

You have surely heard about evolutionary algorithms and how they, inspired by Darwinian evolution in the natural world, are excellent general-purpose search and optimization algorithms. You might also know about neural networks, which are learning algorithms inspired by biological brains, and cellular automata, inspired by biological developmental processes. The success of these types of natural computation has spurred other attempts to base algorithms on natural phenomena, such as particle swarm optimization, ant colony routing, honey bees algorithm, and cat swarm optimization. These algorithms are popular not only for their biological inspiration but also for their proven performance on many hard computational problems.

However, in an era where unfettered market forces force bankruptcy upon liberal-democratic countries as a result of bank bailouts dictated by the global financial elite, the neoliberal ideological basis of such algorithms can be called into question. After all, they are based on a model of individual betterment at the expense of the weaker members of population, an all-consuming "creative" destruction process where disenfranchised individuals are ruthlessly discarded. "Survival of the fittest" describes the elimination process by which the invisible hand strangles the weak; "self-organization" is the capitalist excuse for exploiting non-unionized labor. Common evolutionary algorithm operators like survivor selection represent the violence inherent in the system.

But there is an alternative: algorithms for socially just optimization based on models of the workers' struggle and the liberation of the oppressed. While rarely discussed in major  (corporate-sponsored) conferences, revolutionary algorithms have certain similarities with  the better-known evolutionary algorithms. The basic structure of a revolutionary algorithm is as follows (Marks and Leanin 2005):

  1. Initialize the population with n individuals drawn at random.
  2. Evaluate all individuals to assign a fitness value to them; sort the population according to fitness.
  3. Remove the most fit part of the population (the "elite").
  4. Calculate the average fitness in the population; assign this fitness to all individuals.
  5. Increase fitness of the whole population linearly according to a five-generation plan.
  6. Repeat step 5 until maximum fitness has been reached.

As you surely understand, this simple scheme does away with the need for elimination of lower-performing individuals while assuring orderly fitness growth according to plan. Just like in evolutionary computation, a number of modifications to the basic scheme have been proposed, and proponents of the various "schools" that have grown up around specific types of algorithms do not always see eye to eye. Here are some of the most important new operators:

  • Forced population migration (Sztaln 2006): While in evolutionary computation much effort is is spent on diversity maintenance, in revolutionary computation it is important to counteract the damaging effects of diversity. Forced population migration moves whole parts of the population around in memory space, so as to counteract any dangerous clustering of similar individuals.
  • Continuous anti-elitism (Polpotte 2008): While standard revolutionary algorithms only cull the elite in the initialization phase, the radical scheme suggested by Polpotte eliminates the most-fit part of the population every generation. When no fitness differences can be discerned, which individuals to remove can be determined based on arbitrary factors.
  • Great leap mutation (Maocedong 2007): This modification of the basic scheme is particularly useful when the initial population has a very low average fitness. Here, the population is sorted into small "villages" and each village is told to accomplish its development goals on its own, including creating its own search operators.

More recently a newer generation of researchers have questioned some of the basic assumptions underlying revolutionary computation, such as the stable identity of individuals and the boundaries of the population array. The replacement of some parts of the population with others has been decried as a form colonialism. Revolutionary algorithms of the poststructuralist variety therefore eschew strict divisions between individuals and practice adding random variables to instruction pointers and array indexes. This naturally meets resistance from antiquated, orthonormative models of computation and operating systems. In this context, it is important to remember that "segmentation fault" is just a form of norm transgression.

In the end, those algorithms that are most efficient will win; society cannot afford substandard optimization. And in the same way as the success of evolutionary algorithms is predicated on the success of Darwinian evolution in nature, the success of revolutionary algorithms is predicated on the success of the ideologies and movements that they are modeled on.

(This post was inspired by discussions with Daan Wierstra, Mark Nelson, Spyros Samothrakis and probably others.)

Sunday, July 26, 2015

How not to review a paper

On occasion of several paper reviews I've received recently, and a few I've written, I'd like to give some useful tips for how to review a paper. That is, how to review a paper if you want to do a really, really bad job of it. Note that I work in the AI and Games field, so somewhat different advice might apply to screwing up a paper review in another field.

First of all, be vague. Say what you think about the paper in very abstract terms, and at all costs avoid pointing out specific flaws with the paper so that they could be easily fixed.

This applies most of all to any comments about the literature review. It's fine to point out that the literature review is missing important related work, but by no means include any references to said work. Ideally, say that the paper cannot be accepted because of glaring omissions in the references, and fail to provide a single paper they should have referenced.

If by any chance you think the paper you are reviewing should have referenced one of your own papers, then you should definitely not say so. Your papers are obviously of such brilliance that everybody already know about them by virtue of them being published somewhere. Instead, treat this omission of citation as a personal insult, and add a passive-agressive slant to your review.

If you find it hard to be abstract enough in your review, then you may consider doing the opposite: only talk about details. Talk at length about verb forms and the possible inclusion of semicolons, and if you have substantial comments about the methodology or results bury them as deep as you can in a wall of text. For maximum effect, use a stream-of-consciousness style where you jot things down as you read the paper, often digressing into reflections on various topics that reading the paper reminded you of. It's great if some of your later comments contradict your earlier comments. At the end of it all, issue an arbitrary accept/reject recommendation without explaining which of the numerous comments made you come to this conclusion.

It's imperative that you don't write any summary of your review, or the effect is lost.

If the language and tone of the paper is not exactly how you would have written it, urge the author(s) to enlist the help of a native English speaker to correct their English. This comment works best if you can tell that the authors are native English speakers, and if you carefully add some grammar and spelling mistakes to your review.

Speaking of how you would have written the paper, it's a good idea to evaluate the paper from the perspective of what you would have done if you wrote the paper. Say for example that the authors present an algorithm and are mostly interested in the algorithm's correctness, whereas you would personally be more interested in its runtime. Then it's perfectly fine to reject the paper because they studied the correctness and not the runtime of the algorithm, and they clearly should have studied the runtime instead. After all, you are the one reviewing the paper, so you should decide what it should be about.

In the same vein, don't just accept any definitions of terms or scoping of the investigation that the paper might contain. Read the paper using whatever meaning of the words in it that you find convenient. And if the authors state that they are not concerned with topic X, that is no reason for you to not go on at length about how important topic X is and why they should have included it.  It's your freedom to read the paper any way you want and assign any kind of meaning to it you like.

This brings to our final and perhaps most important piece of advice. It's likely that there is some part of the paper you don't understand, because like everybody else you are occasionally frequently out of your depth and these authors write like morons anyway. If this happens - don't admit it! Don't lose face by explaining that you don't understand the paper! Your reputation as an anonymous reviewer is at stake. Instead, simply pick at random some interpretation of the part of the paper you don't understand, preferably some interpretation that makes very little sense. Then write your review as if that interpretation was true. Hilarity ensues, at least on your side.

If you heed all this advice, you will surely be able to produce the kind of reviews that one frequently receives after submitting to some famous conferences and well-respected journals.

Friday, June 19, 2015

What I've been working on lately

It has been remarked upon that I publish a lot of papers on a number of topics. It might not be clear how they all fit together. What is it that I work on, really? Of course, I have no problems seeing how the various strands of my research inform each other and contribute towards a small set of common goals. But that might not be so easy for you to see.

So here's an incomplete selection of themes from my recent research, together with links to some relevant papers. For simplicity and for the sake of the length of this text vs your attention span, I will limit myself to work that I've published in the last two years. Note that though I've written code for some of the research systems myself, contributed with conceptual development to all of these papers, and written parts of almost all of them, most of the work below was done by my various students and other collaborators. That's why I'll use the academic "we" in the examples below, referring to my various groups of collaborators. I'm of course very happy that so many talented and hard-working people feel like attaching my name to the author lists of their papers, for one reason or another.

Generally, my research has the dual aim of using AI to enable better (or more interesting) games, and on using games to enable better AI. This means coming up with better game-playing algorithms, with algorithms that can perhaps also play games in a human-like manner, methods for generating complete games or part of games, studying games and players, and for using games to test AI algorithms. This all comes together, for example you cannot design a game without being able to play it and knowing how humans play it, and you can't advance your game-playing AI without having suitable games and levels to try it out on. Ultimately, we're aiming towards general AI that can not only play any game but also design any game, including the game that you seem to want to play right now. That's very general, so let's be more specific.

Procedural content generation

PCG, creating game content with algorithms, is sort of hot right now. For several reasons: if the computers create levels, items, quests etc we don't have to do it ourselves, so games could be endless, adapt to the player, be cheaper to produce etc. Also, creating a content generators is about defining and understanding a particular aesthetic. My various collaborators and me have been working on PCG for quite some time now (actually, since before it was hot) in particular exploring how to best use evolutionary algorithms for generating various forms of game content. We call this search-based PCG. Some examples of recent work includes basing Super Mario level generation on design patterns, evolving maps to advantage particular game-playing algorithms, and multiobjective and multimodal methods for strategy game map generation.  We've also introduced new algorithms like constrained novelty search and repurposed older methods such as n-grams for level generation. Understanding the output of these generative methods is very important, and for that reason we have developed ways of characterizing level generators, and generic metrics for level design. A major effort was to edit and write the first textbook on procedural content generation in games; earlier we wrote about goals and challenges for PCG research.

Mixed-initiative and AI-assisted game design

While having the computer create levels, items, quests etc all by itself is great, there's also room for collaboration with humans. Computers are good at some things and humans at others, and human designers might want to interfere at various points in the content creation process. Mixed-initiative PCG systems implement a dialogue between a designer and computer, where PCG processes assist the human designer while retaining the designer's freedom. Sentient Sketchbook is one such system, where humans design strategy game maps while algorithms critique the designs and offer suggestions; it also learns the designer's style. Another of our systems is Ropossum, which can generate complete levels for the physics puzzler Cut the Rope and also assist designers. It uses a combination of grammatical evolution, reasoning and tree search, but we have recently experimented with using object path projections for playability testing and with creating levels based on evolved (fictive) playtraces.

Data games

There's more digital available than ever before, including large amounts of open data that anyone can access. This includes geographical, economical and demographical data amongst other forms. Data games are games that are built on such data, in the sense that the game's content is generated from open data. We're trying to create ways to improve the generation of meaningful game content by seeding PCG with real-world data, but also make data exploration more playful. Our work involves data-based content generation for existing games such as Open Data Monopoly and Open Data Civilization, using game mechanics for data exploration and visualization such as in Open Trumps and Bar Chart Ball, and data-based procedural generation of complete games as in Data Adventures.

Generating complete games

The logical endpoint of PCG is generating the whole game, not just the levels and textures but all the rules and mechanics and everything else that's not the game engine. Even what the game is about and what the goal is. If we are to understand game design properly, we need to build systems that can generate good games; and if we want to test general AI properly we need an infinite supply of new games. We have argued that those game types that would be most realistic to try yo generate are classical board games and simple 2D arcade games; this is also what we have been attempting to generate earlier. Recently, we have invented ways of representing and generating card games, by searching through a space of card games that includes well-known games such as Texas Hold'em. We have also designed a Video Game Description Language which can be used to define simple card games, and invented ways of automatically evaluating the quality of such games and generating new games. It is also interesting to see how different games can be generated by simply varying the parameters of a simple existing game – in our work on generating unique Flappy Bird variants we found that plenty of playable yet different games can emerge.

MCTS for video games

In order to be able to generate games you need to test them, and to test them automatically you need AI that can play the games. Being able to play games is of course also important for other reasons, such as providing good opponents and collaborators to the player. Monte Carlo Tree Search is a statistical tree search algorithm that has been very successful in playing board games such as Go. We are trying to figure out how to adapt this algorithm to video games, that have very different requirements from board games - for example continuous space and time, as well as lack of guarantee that random actions will lead to a terminal state. In the course of this, we have developed a number of MCTS modifications and MCTS-inspired algorithms for e.g. Super Mario Bros, car racing and general video game playing; the further success of MCTS variants can be seen in the first General Video Game Playing Competition, where the objective is to not just play one game but a whole set of different games.

Behavior imitation and procedural personas

Playing a game well is not all there is to playing a game – there's also the issue of playing style to consider. We've seen numerous cases where the best-playing AI plays the game in a decidedly non-humanlike manner. If we want to test a game or a level, or provide interesting NPCs in a game, we need to create AI that can play that game in the style of a particular human, or maybe humans in general. One approach is to train neural networks on gameplay traces as we've tested with Super Mario Bros. A more involved approach is to model the player as a ``procedural persona'', assuming bounded rationality and a set of objectives. This conceptual framework has been combined with q-learning and neuroevolution to play a roguelike dungeon crawler game in various styles. These procedural personas have also been used to evaluate levels in level generation algorithms. We also organized a Turing Test competition for Super Mario Bros-playing agents, where the objective was to fool judges into believing your bot was a human.

Game data mining and player modeling

The vast amount of data generated by modern games can be used to understand both games and their players. We call attempts to make sense of this data game data mining. Our efforts include crowd-sourcing platform game level preferences, and using this to study which micro-structures in the game levels best predict certain player responses. We have also found out that we can predict people's life motives from how they play Minecraft, and detect sexual predators from game chat logs. Another question that we have investigated with data mining is how game mechanics create meaning. Of course, much of the behavior imitation work above could be seen as game data mining as well. You could perhaps take this further if you allow the game to select what the player plays so as to learn more about the player.

Music and games

There's many similarities between music and games. For example, games are often experienced as quasi-linear structures with variations in intensity and "mood"; music is often also used to accompany and heighten the emotional expression of a game. We worked on affect-expressing music generation for games and on bidirectional communication between games and music so that playing becomes composing and composing becomes level designing. Given all the work that has been done in music generation, it seems reasonable that some of the methods and representations used there can be used for game content generation; here, we have investigated using functional scaffolding for level generation.

Surveying and organizing the research field

As I've been involved in the field of artificial and computational intelligence in games since it was just a baby (or at least just a symposium), I've had a chance to get some perspective on what we are doing, why and perhaps where we should be going. Our Panorama of CI/AI in games is an attempt to give a high-level overview of all the different research directions within the field and how they can inform and benefit each other. In some ways, it is a much longer (oh no!) version of this blog post with more pretentious language (really?) and also talks about other people's work. You should read it. We have also written surveys about computational creativity and games and neuroevolution in games. On top of that we recently organized a Dagstuhl seminar on the future of the field.


There are are so many interesting things to do – ars longa, vita brevis. So when a nice idea comes by it's always a good idea to try to implement it, run some experiments and turn it into a paper, even though it might not fit perfectly into the current research direction that you've told yourself you're pursuing. Some of my "other" recent papers which I still consider very interesting deals with community structure detection in complex networks and geometric differential evolution. On particular note is our DeLeNoX system for computational creativity, which I think is really cool and should be used for... something.

Finally, just a note that we are not done yet. We don't have AI that can play any game or that can design a large variety of good, novel and meaningful games, so the job is not done. And when the job is done this very likely means that we'll have solved general AI and there is no more job to do at all for anyone. But until then: more work to do, come join.

Wednesday, March 11, 2015

EvoStar paper previews

Why not use the blog to post links to new papers from me and my group? At EvoGames (part of the EvoStar conference), in April in Copenhagen, my collaborators will be presenting the following papers:

Thorbjørn S. Nielsen, Gabriella Barros​, Julian Togelius and Mark Nelson​: General video game evaluation using relative algorithm performance profiles.
This is groundwork towards generating complete games in VGDL, the language used to describe games in the general video game playing competition. We show that random games can be separated from human designed games by testing a portfolio of algorithms on playing them.

Marco Scirea​, Mark J. Nelson and Julian Togelius: Moody music Generator: Characterising control parameters using crowdsourcing.
Marco has been working on his music generator for games for a while, and here we perform the first evaluation of its ability to express emotions. One of the main novelties of the paper is the evaluation method, which analyzes free-text descriptions.

Antonis Liapis​, Christoffer Holmgård Pedersen​, Georgios Yannakakis​ and Julian Togelius: Procedural Personas as Critics for Dungeon Generation.
For a while, Christoffer has been working on modeling player behavior in a bounded rationality framework. Here, we use those models in an evaluation function for search-based level generator.

Mohammad Shaker​, Noor Shaker​, Julian Togelius and Mohamed Abou-Zliekha: A Progressive Approach to Content Generation.
This work follows up on Mohammad's work on the AI-assisted design tool "Ropossum" for the physics-based puzzler Cut the Rope. In this paper, the generation problem is turned on its head, as play-sequences are first evolved and then content is found that could allow those sequences to be played.

Mohammad Shaker, Noor Shaker, Mohamed Abou-Zliekha and Julian Togelius: A Projection-Based Approach for Real-time Assessment and Playability Check for Physics-Based Games.
Another addition to the Ropossum toolbox, this paper presents a way of testing and visualizing playability through a form of occlusion modeling, where level components cast shadows of unplayability.

Monday, February 23, 2015

How to write me an email (and get a response)

My inbox has as many mails as the night sky has stars. That's why I did not respond to your mail. Not because I didn't want to and not because I didn't like you. I'd love to be able to hit that reply button right after I read your mail. Could you please help me with this? Below are some tips.
  1. Write short. Response time is approximately proportional to mail length. This includes any attachments to your mail.
  2. State your request clearly. What do you want an answer to? Make it easy to answer your request. Best case: a yes/no question.
  3. Skip the formality. Skip the boilerplate. "Hi Julian" is a good start, your name is a good way to finish the mail. Everything in between should be unique.
  4. Do tell me how your day was if you feel like it. If I know you, I care about you and appreciate you telling me. Don't ask me how my day was. I will feel I need to reply and it will take me longer. It's also perfectly fine to not tell me how your day was.
  5. If the purpose of your mail is general chitchat, maybe we should talk on the phone instead.
  6. Don't be rude. Your email deserves an answer, unless you tell me explicitly that it deserves an answer.
  7. If I don't answer within a few days, do send me a reminder in the same mail thread. See if you can state your request more succinctly in your reminder.
  8. Don't contact me on Facebook, SMS, Skype or similar with the same request as your mail. These services don't provide an overview of outstanding messages and I therefore don't treat them as a task list. Your message will be lost in cyberspace.
I'm aware that this post may make me sound like a self-important and condescending person. I really hope I am not. I'm just someone who want to spend less time reading and writing emails, and avoid making people angry because I don't respond to their emails. I guess you want the same.

Thursday, February 12, 2015

Six things that cannot be listed - #4 is totally non-trivial!

There are many lists of things out there right now. But not everything can be listed. Here is a list of some things that can not be listed. In a particular order:

The real numbers

You might think it's easy to list the real numbers. Start at 0, and then you have 0.1, 0.2, 0.3... But wait, between 0 and 0.1 you have 0.01, 0.02 and so on. And between 0 and 0.01 you have 0.001, 0.002... In fact, it is impossible to get anywhere at all, because there is always a finer-grained way of listing the numbers. Unlike the natural numbers (positive integers), which you can simply list as 1, 2, 3... Georg Cantor proved that this was the case. And that's why the real numbers are in a sense more infinite than the natural numbers.

Cantpr later tried to prove that nothing could at the same time be more infinite than the natural numbers and less infinite than the real numbers. He couldn't, so he went crazy and ended up in a mental asylum.

The list of all lists that do not contain themselves

Sure, there are many lists that do not contain themselves. Your laundry list is presumably one of those, even if you write the list on a piece of paper you forget in your jeans (you'd only be washing the paper, not the list). An example of a list that does contain itself is the list of all lists of lists. And of course the list of all lists also contains itself. But the question is whether the list of all lists that do not contain themselves should list itself or not. Because if it does, it shouldn't, and if it doesn't, it should. Bertrand Russell spent a number of years puzzling over this so you don't have to.

This was important, because lists of lists are necessary in order to show that mathematics are true. Having thus destroyed attempts at founding mathematics on logic, Russell went on to be imprisoned for pacifism.
Thinking about infinity might make you want to hide.

All the animals

Jorge Luis Borges provided the following list of animals, in a fictional text about a text:
  • Those that belong to the emperor
  • Embalmed ones
  • Those that are trained
  • Suckling pigs
  • Mermaids (or Sirens)
  • Fabulous ones
  • Stray dogs
  • Those that are included in this classification
  • Those that tremble as if they were mad
  • Innumerable ones
  • Those drawn with a very fine camel hair brush
  • Et cetera
  • Those that have just broken the flower vase
  • Those that, at a distance, resemble flies
It's clear that this is not getting us anywhere and we should stop now.

Borges later lost his eyesight, but apparently never his virginity.

There are some pretty weird animals out there.

All statements that are true in a formal system of sufficient power

Kurt Gödel thought a lot about truth. In particular, about which statements were true and which were not. To find out which statements were true, he invented a way to list all the statements in a system. Even more impressively, using this list he could figure out whether the statement was true just by looking at the number of it. Because the statements could be about anything within the system, there must be a statement in the list which talks about whether this statement itself is true - a statement that contains the proof of itself. As with all other statements, the number of this statement says whether it is true or false. Gödel showed that this statement is false. This means that is is impossible to list all true statements in a formal system, or all false statements for that matter. Rather disheartening really if you want to believe that the truth is in there.

Gödel, always a somewhat troubled fellow, later starved himself to death.

All steps you take as you follow a coastline

Following the coastline of Britain in larger and smaller steps. Sorry, no animals. Image from Wikipedia.
Benoit Mandelbrot wrote a paper with the amazing title "How Long Is the Coast of Britain?". You might think that this should have been a very short paper. I mean, it's something like 12000 kilometers or so. But actually the question is complicated. There's another figure you can find online which is something like 19000 kilometers. How can they be so different? You see, you can only measure length in straight lines, and a coast is never completely straight (neither are most things in the natural world). In order a measure a coastline, you need to approximate it with a series of straight lines. Think of when you want to measure the length of a path by a walking it: you simply count the steps you take. Perhaps surprisingly, the length you measure depends on how long steps you take. You see, with smaller steps you can follow the path more closely and take more turns, and you will arrive at a higher number. The same goes for coastlines. If you want to measure the coast of Britain, you can do this in different ways. For example, you can choose to measure it by fitting straight lines of 10 kilometers to it and get one number. Or by fitting straight lines of 1 kilometer and get a much higher number. If you fit lines of a hundred meters you get an even higher number... and so on. At some point you will be measuring around individual grains of sand, and Brighton beach itself will probably be hundreds of kilometers long. In fact, you can't list the number of steps you would need to take, for the same reason you can't list the real numbers: there's always a smaller step possible.

Mandelbrot escaped the nazis and went on to live a long (how long?) and apparently happy life.

(Note to self: should illustrate this text with pictures of puppies, kittens and/or smiling people. That way, people might actually read it. Or at least click on it.)

Wednesday, November 05, 2014

Game-based AI competitions

So you've built an artificial intelligence, or some kind of learning system. How do you know if it's any good? Well, you test it. And what do you test it on? You let if play games of course! Games are excellent testbeds, because they were made to test human thinking skills of various kinds - in fact, they are fun to play largely because they exercise your brain so well - so therefore they will test adequate aspects of your AI's intelligence. Also, games are excellent testbeds because they execute much faster than robots, don't need any expensive hardware, can be understood by everyone and are fun to both watch and participate. In fact, it is hard to understand why you would ever use an AI testbed that was not a game. To make things easier for you, there are now a number of competitions running at the major AI/games conferences, which allow you to submit your AI agents and tests them on their capacity to play games such as car racing, Super Mario Bros, Othello, StarCraft, Ms. Pac-Man, Go or Unreal Tournament.

Alright, great! Sounds like we have AI benchmarking all sorted, right?

But hold up! There's something wrong with the picture I paint above. And it's not that games are excellent testbeds; they truly are. And I am certainly not meaning to demean the set of game-based AI competitions on offer (how could I, having started some of them myself?).

Instead, what's wrong with the picture is the idea that you first build an AI and then test it on some game. More likely, a researcher will look at the game, play it for a bit, and try to imagine how it could be played by a computer program. The researcher will then try to build an AI with the game in mind, and tune it to play this game specifically. The result, in many cases, is a piece of software that plays a game very well (in many cases better than the researcher who built it) - but who might not be good at anything else.

So what do we learn about artificial intelligence - in general - from creating software that is good at playing a particular game? How could you design game-based competitions that make you learn more about AI? In fact, how could you design game-based competitions that are easy and fun to participate in yet are scientifically meaningful? And how could you write up the results of these competitions so that they add to the sum of scientific literature, while giving everybody who worked on the project due credit? Because running, and participating in, game-based AI competitions is a form of AI research too. It's just one we have not discussed very much yet from a methodological or epistemological standpoint, or even from a standpoint of giving researchers credit for their work.

I explore these issues in a new article in TCIAIG; the article is available here (or use this direct link if you don't have access to IEEE Xplore). The article is based on my tutorial at CIG 2013, and the comments I received afterwards; this tutorial is in turn based on my experience of running a number of these competitions.

Also, the article can be seen as a travesty of a particular "literary" genre. See if you can find all the genre tropes!

Wednesday, October 22, 2014

New York, New York

In January, I will move to New York City and take up a position as Associate Professor at New York University (to be more specific in the Department of Computer Science and Engineering at the Polytechnic School of Engineering).

I am extremely excited about this. I'll be joining a growing games programme at NYU, and will be affiliated with the Media and Games Network (MAGNET) and the NYU Game Center. My new department wrote a short story about my arrival, which emphasises my work in procedural content generation and interdisciplinary collaborations. This is also my plan - continue my current main research directions (PCG, general video game playing, player modelling, game generation) while also starting new collaborative projects with the excellent people at NYU.

My five and a half years at ITU have been excellent and I've learned a lot from my amazing colleagues and developed as a researcher. I'd like to point out that I'm not moving in order to leave ITU. I'm moving because it is time to move on and seek new challenges and opportunities, and because of all the great people I will work with here and the great city I will be living in.

Maybe you want to come here as well? I am looking for two new PhD students, to start in September. You need to apply soon. And you should definitely mail me first to discuss research proposal ideas. (Hint: have a look at some of my recent papers to see what my current research interests are; these include general video game playing/generation and mixed-initiative PCG.) More information about the procedure can be found here.