Busy days in Hong Kong in June

If you're going to WCCI2008 in Hong Kong in June, you might run into me. Chances are, however, that I'll just keep on running, as I've got a busier schedule than I've ever had in a conference. So just don't take it personally, OK?

To begin with I'm giving a tutorial together with Simon Lucas and Tom Runarsson on "Learning to play games". Preliminarily, Tom will talk about evolution versus other types of reinforcement learning, Simon will talk about different sorts of function approximators, including his n-tuple classifier, and I will discuss different ways in which such techniques can be used in games, with examples. Not only car racing, is the plan...

Then, I will participate in a panel discussion about Computational Intelligence and Games, together with some of the people that are defining this young field.

Staying on the topic of games, I will then present the results of the car racing competition, together with Daniele Loiacono and Pier Luca Lanzi. By the way, have you considered participating? We now have proper software packages available, so it's easy to get started with developing controllers using your favourite brand of learning algorithms.

Yes, I do have some papers as well. However, these are not (primarily) about games this time...

The first paper, a collaboration with my good friends Alberto Moraglio and Renzo De Nardi, is called Geometric PSO + GP = Particle Swarm Programming. The idea is to combine Alberto's geometric particle swarm optimization algorithm with genetic programming. The good thing with GPSO is that it can be used in any search space where a distance between two solutions can be defined, and a weighted recombination operator for these solutions can be devised; in contrast, standard PSO only works in continuous spaces. We have previously successfully used GPSO to solve Sudoku problems, to show this really works. In our new paper, we invent a few weighted recombination operators for expression trees, and test the PSP (GPSO + GP) algorithm on two standard GP benchmarks. The results are decent, but not as good as we would have wanted them; we think this is down to the recombination operators still needing some more work. Nevertheless, we think this is the first time PSO has been applied directly to GP.

The other paper, my first collaboration with Faustino Gomez and Juergen Schmidhuber (my new colleagues at IDSIA) is called Learning what to ignore: Memetic climbing in topology and weight space. It's about "memetic neuroevolution", an idea I've had for a while, and which someone else really should have thought about. The key algorithm here (the "memetic climber") is incredibly simple: evolve topology of weights of a neural network at different time scales. Topology mutations are almost always very destructive, so after each topology mutation, do local search (hill-climbing) in the space of neural network weights do find the potential of the new topology. If the fitness at the end of the local search is not at least as high as it was before the topology mutation, the topology mutation is discarded. This algorithm turns out to work really well, especially for problems where an agent has to handle high-dimensional input arrays. In such problems, which is really the ones i'm most interested in, most neuroevolution algorithms get stuck in local optima if you don't manually specify a good topology for the neural net. So I have big plans for this little algorithm in the future.

Mentioned at the GDC

A paper by me, Renzo and Simon was recently mentioned as "one of the top ten research finding in games studies" at the Game Developers Conference. This is really cool, as GDC is the most important conference for the games industry, attended by all the important developers and industry heads and tons of journalists.

The presentation can be found here:
http://www.avantgame.com/top10.htm

A short description and some discussion on Raph Koster's site:
http://www.raphkoster.com/2008/02/20/gdc2008-game-studies-download-30/

And the paper:
http://julian.togelius.com/Togelius2007Towards

Quite surprising to see that I'm doing "games studies", I thought I was doing artificial intelligence...

Call for participation: IEEE WCCI 2008 Simulated Car Racing Competition

*** SIMULATED CAR RACING COMPETITION @ IEEE WCCI 2008 ***
*** DEVELOPING CONTROLLERS FOR A REALISTIC CAR RACING SIMULATOR ***

Homepage: http://cig.dei.polimi.it/?page_id=5

Deadline: May 25th, 2008
Organizers: Daniele Loiacono, Julian Togelius and Pier Luca Lanzi.
Email: carracing@gmail.com

The car racing competition, organized in association with the 2008 IEEE World Congress on Computational Intelligence WCCI 2008, is now open for submissions. Competition rules and further details can be found on the competition web site:

http://cig.dei.polimi.it/?page_id=5

while recent news can be found at

http://cig.dei.polimi.it/?cat=4

*** GOAL & SCOPE***

The goal of the competition is to develop racing car controllers that can compete on a set of tracks against the timer or other controllers.

There are no limitations on the type of controller, handcoded controllers are also welcome, although the focus of the competition, given the venue, will be on controllers using computational intelligence techniques (e.g. genetic programming, neuroevolution, td-learning, policy gradient search, fuzzy logic, etc.).

*** ORGANIZATION ***

The competition is organized as the successful car racing competitions held in association with CEC2007 and CIG2007, but it is based on "The Open Racing Car Simulator" (TORCS), a realistic car racing simulator, and structured as a client-server architecture.

A racing server will manage the race, while controllers will run as different processes. Communication between the racing server and the controllers is done through TCP/IP. The race is therefore in *real time* and the controller's reaction time is likely to be an important factor in the design.

We will score every submitted controller on the distance raced in a fixed amount of time when driving on its own on a set of tracks. At the end of the competition, the best few controllers will race against each other on a different set of tracks, validating that the controllers perform well in the presence of other cars and that their performance generalizes to other tracks than those they were trained for. The winner of the final competitive races will get to present their controller at WCCI2008 <http://www.wcci2008.org>, and will have their registration fee reimbursed.

*** SOFTWARE ***

The competition software can be downloaded from:

http://cig.dei.polimi.it/?page_id=5

where detailed installation instructions, a description of the sensors model, and examples of controllers are available; more example trainers and interfaces in different programming languages will be added soon.

On industrial-academic collaboration in game AI

Today was the kick-off event day for the UK Artificial Intelligence and Games Research Network. I wasn't there, but my ex-supervisor is one of the organizers, so I heard about it beforehand.

This is the first blog post I've seen about the kick-off; the author seems to have left the event with a pretty pessimistic view of the prospects for industrial/academic collaboration. His main complaints is that academics don't understand games, and the specific needs of game developers. Well, then tell us! I would love to hear about specific problems in game development where evolution or some other form of machine learning or computational intelligence could matter.

Alex J. Champandard, in a comment on the same blog post, develops the point further. He asks:

So why do you need government funding for [applied games research]? It's a bit like admitting failure :-)

On the other hand, if [academics are] doing research for the sake of research, why do they need input from industry?

These questions can be asked for just about any research project in the interface between academia and industry. And yet companies happily keep funding PhD students postdocs, and even professors in a huge number of research fields, from medicinal chemistry to embedded systems design to bioinformatics. In some cases these collaborations/funding arrangements definitely seem strange, but apparently it makes economic sense to the companies involved.

I once asked an oil company executive (at a party! Now, stop bothering me about what sort of parties I go to...) why his company funds a professor of geology. His answer was roughly that it was good to have expert knowledge accessible somewhere close to you, so you know who to ask whenever you need to. Plus, a professor's salary wasn't really that much money in the grand scheme.

Now, game companies and oil companies are obviously very different sorts of creatures. I think the main opportunity for game companies would be to outsource some of their more speculative research - things that might that not be implementable any time in the near future, either because the computational power is not there yet, or because the technique in question would need to be perfected for a couple of years before deployment. Having a PhD student do this would be much more cost-efficient than assigning a regular employee to do it (especially with government funding, but probably also without), and frees up the employee for actual game development. In addition, the company's own developers might very well be too stuck in the way things currently work to try radically new ideas (of course, academics might also be stuck in old ways of thinking, but there are many academics around and if you offer some funding you can typically select which academic you want to work for you).

This argument assumes that game companies do any sort of research into technologies that lie more than one release cycle away. I'm not stupid enough to claim that no game companies do this - e.g. Nintendo obviously does - but I venture to guess there are many that don't.

As for the other part of Alex's question, "if we do research for the sake of research, why do we need input from industry?", the answer is more obvious. Because even if we do research because we love the subject itself and really want to find out e.g. how to best generalize from sparse reinforcements, we also want to work on something that matters! And fancy new algorithms look best together with relevant problems. It's that simple.

New webpage at IDSIA

I've now set up a parallel home page at my new workplace, IDSIA. It's currently mostly a quick overview of the various things I'm involved in academically, but I plan to set up pages with a bit more detail about the projects in that domain as well (for those who don't feel like going straight for the papers).

My primary home page will still contain my publication list, CV and such formal stuff.

A task force and an interview

More games-related news today. The IEEE Computational Intelligence Society has just spawned a Task Force on Computational Intelligence in Video Games, chaired by Ken Stanley (of NERO and NEAT fame) and which I am an inaugural member of. From the mission statement:

"We are aiming to become a repository of information on CI in video games, a networking resource for those in the field, and the spearhead for initiatives in the area. We will also attempt to bridge academia and industry by including members from both. Thus ideally we can become a focal point for discussion and action that will facilitate further progress in the field."

This is a very good initiative in my opinion, and being backed by a such a powerful organisation as the IEEE is certainly not bad. As the web site is only just up, the member list is far from complete yet. The task force is looking for information on interesting research groups and projects, so if you want your project featured, contact them!

Over at Alex J. Champandard's blog, "Game AI for Developers", we find an interview with none other than yours truly. Personally, I think it's an interesting read, of course... Thanks for the opportunity, Alex!

One of the things I suggest in the interview is that game developers initiate contacts with academic researchers interested in CI in games. The above mentioned task force could come in very handy for such purposes, as soon as the member list is expanded to include everyone who should be there!

Confessions of an academic crack smoker

Look, I got some attention again. This time from Christer Ericson at Sony Santa Monica, "the God of War team". His blog post is a scathing critique of most of what I've been doing for the last three years, without going into any detail whatsoever, and devoid of constructive suggestions.

I'll try to be less rude.

Christer's argumentation consists in showing one of my early videos with two cars on a track, and pointing out that the AI is not very impressive, as the cars behave erratically and crash into walls. He also makes fun of question I posted to Slashdot, where I was genuinely wondering about what people perceive as being the flaws of current game AI. From this, he implies that my contribution to game AI is null and that I could as well stop what I am doing.

Now, if someone from industry came and argued that what I'm doing is completely useless for game developers, I would take this seriously. Even if he was right, it seems that at least some of what I do is appreciated by the CI community, which is at least equally important to me, so I could accept developer' thinking my ideas were all stupid. However, I would only take such criticism seriously from someone who had actually read my papers and knew what I was doing, and bothered to come up with some suggestions on how to improve my work. None of this is true for Christer's rant.

It's true that the cars in the video don't seem to be driving very well. That was never the objective. Instead, the video is from a series of experiments where I manipulated the fitness function in order to produce interesting driving behaviour. Evolution of controllers that drove a particular track better than any tested human was already reported in our very first car racing paper. It's also true that the cars never learned to recover from some wall crashes. I had wanted this to emerge from the overall progress-based fitness function, which it didn't, and I might get back to work on this later; however, it would be straightforward to either add crash recovery as a specific learning objective, or add a hard-coded function for this. After all, normal game AI is 100%hard-coded.

In short, it would help if Christer either judged my experiments based on their actual objectives, or told me in what way I needed to change my objectives.

It would also help if he looked at some of the work that I myself consider more useful for game development, at least conceptually. (I'm not an expert in graphics, physics, or for that sake real-time collision detection, and don't profess to be one.) Especially the experiments on player modelling and track evolution, but also generalization and specialization for quickly creating drivers for any track, and co-evolution of diverse sets of opponents.

If he read these, and came back and still thought it all stank, I would be very happy to listen to his ideas on how to make my research more relevant for hard-working game developers like him. In the meantime, I'll continue my vacation.

And by the way, I don't smoke.

CEC 2007 Conference Report

So, the 2007 IEEE Congress on Evolutionary Computation is now over. Actually, it's been over for the last ten days. Sorry for taking such time to update my blog, I'm out backpacking at the moment to celebrate finishing my PhD, and I try not to spend all my vacation in front of a computer (even though it's hard fighting that Internet addiction)!

Overall, CEC was an excellent event this year as well. A generous supply of on average really good keynote and invited speakers, so many parallel sessions that there was always something interesting going on, and a superb organization. The only things I would have done differently is spreading the conference out on five or six instead of four days, and not charging money for the tutorials (in fact, many of the tutorials are the same as are included in the general registration for Gecco or PPSN). But those are really minor issues. (A major issue that CEC shares with Gecco and some other conferences is the too low entry barriers / too high acceptance rates, but that's stuff for another blog post.)

Simon's keynote on Evolutionary Computation and Games went down really well, it seems. Apparently, more and more EC researchers are warming up to the idea of using games as testbeds for their algorithms. Simon plugged the car racing competition as well, and there were lots of people talking to me about it in appreciative terms both before and after I presented the results. It seems we have quite a momentum for these kinds of activities at the moment.

Hugo de Garis' invited talk was interesting in a very different way. Actually, it was quite sad. de Garis is known for his huge ambitions and provocative statements, (evolving "artificial brains" as complex as those of kittens, or was it even humans this time around?) so I was looking forward to bold new theories on how such grand aims should be achieved. What followed was some very conventional neuroevolution stuff, and a complete failure to appreciate the real challenge in putting all his evolved neural modules together. Most importantly, he has absolutely no empirical results to show. Predictably, the audience gave him a hard time during the question round.

Other interesting talks included those of Jong-Hwan Kim, the father of RoboCup, on evolvable artificial creatures for ubiquitous robotics, and of Marc Schoenauer on how modern bio-inspired (and population-based) continuous optimisation algorithms such as CMA-ES and PSO now often outperform the orthodox optimisation algorithms used by the applied maths people, on their own benchmark problems. Quite cool.

By the way, did I point out that the organization was superb? Anyway, it deserves saying again. The Stamford convention centre is not only lavishly, but also tastefully, decorated and conference delegates were continuously tended to by an army of servants making sure that we always had something to eat and drink and knew where the venue for the next talk was. The food was simply fantastic, the night safari at the end of the conference was a very nice event, and the conference banquet had nine (!) courses. I can't imagine how our conference fees can have paid for all this - some of the sponsors must have contributed serious money. Rooms were generally easy to find, and most importantly, there was plenty of places where you could just bump into old and new people and have those all-important corridor chats. In all, a very rewarding experience.

Thesis online

My thesis corrections have now been approved, and the final version is online at http://julian.togelius.com/thesis.pdf

Now I'm off to Singapore to attend CEC, present two papers and the competition results, and have a bit of vacation!

Just passed my viva!

Only minor corrections, will take me a few days to sort out, and them I'm a PhD! External examiner was professor Peter Cowling, University of Bradford (who has a research group on computational intelligence and games), and internal examiner was John Gan.

Yes, it feels fantastic... now we're going out to party! See you!