Academics constantly complain about being overworked, yet they also praise their freedom to decide how and when to work as one of the great advantages of their job. This seems contradictory. If they have such freedom, why are they so overworked?
Part of the answer is of course that you are never really done. You can always do better. Write more and better papers, get better research results and deliver better teaching. But this is true for most white-collar jobs these days - you could always work more on your next budget, product description, market report, technical drawing or whatever it is you do. If you are good at what you do, it should be enough to work normal hours, go home and get some good rest and return refreshed next day to do more work. Few successful academics do this. There seems to be some extra factor that affect academics, contributing to that they always seem stressed out - and that you very rarely see successful academics below the age of 50 working reasonable hours.
That extra factor is increasing marginal work utility. Simply put, the more you work, the more you get out of each extra work hour. This gives you a very strong incentive to work as much as possible, which is clearly not a very good idea if you also want to live your life. Let me explain.
We begin by imagining I work the statutory work week of around 40 hours. If I work efficiently, I will then have time to prepare and conduct my teaching, supervise my masters students, do my administrative duties (including answering lots of mails), review papers for some conferences and devote the minimum acceptable time to supervising my PhD students. These are all tasks that I must do - not doing them would mean that I am simply not doing my job. This amount of work is clearly not enough, as I do not get time to do any research - though I could of course try to take credit for the work of PhD students that I don't supervise enough.
If I put in some more time, I have time to do some of the research where I am not in a "driving" role. I could participate some more in what my PhD students are doing, help my masters students write up papers based on their theses (which I have supervised), do my part in joint research projects with people in other institutions, and serve on organising committees of conferences. Also, write parts of collaborative grant proposals. These are activities where I am mostly fulfilling obligations to other people, though they are also activities where I get relatively much visibility from limited effort. But it does not mean doing any research on my own.
So let's say I put in even more time. I can now advertise some of the research I (and my collaborators) have been doing and help shape the directions of my own and others' research, for example by going to give invited talks, or write parts of survey articles. I can start new initiatives, e.g. competitions, discussion groups or grant proposals. These are things that make a lot of sense to do and I enjoy doing them, but it is still not the same as actually doing my own research.
Finally, when I'm done with all of the above and there are no urgent mails awaiting reply in my inbox, I could sit down with my own idea, write some code of my own, run my own experiments and write a paper with myself as first author. In other words, do some research of my own. This rarely happens, as I rarely get to this point and when I do I rarely have any energy left.
The utility of that sixtieth work hour is so much higher than the tenth or twentieth because I can use it do my own research. If I work 60 hours rather than 40, I don't get 50% more of my own research done, but almost infinitely much more, as I would otherwise get almost none of my own research done. Given that I am interested in doing research of my own, there is a very strong incentive to work very long hours. It is not that I am uninterested in any of the other parts of my job - I enjoy all of them, except grant writing and meetings with management - but I am more interested in research.
You could compare the job of an academic to having a teaching and administration job and having research as a hobby. Except that the "hobby" is the presumed core of the job as advertised, and the reason you applied for the job.
An interesting detail is that it wasn't always like this. Back when I was a PhD student, and to a large extent still when I was a postdoc, my marginal work utility was basically flat as I spent most of my time on my own research. But as I was good at research I got promoted to a position where I had to spend most of my time doing something else. (Incidentally, I have seen quite a few excellent researchers who are less than excellent at teaching and administration.)
Finally, let me point out that I am not complaining, just explaining. And I am certainly not blaming anyone, especially not my wonderful colleagues and students. After all, I love my job and I would not trade it for any other job. I just hope to have made the logic clear behind why I, and probably many others like me, work such long hours.
Thursday, February 06, 2014
Tuesday, December 17, 2013
Against "selective" conferences
Computer science differs from most other academic fields in that conference papers are counted as real, citable publications. While journals are generally seen as more important, it is perfectly possible to get a tenured faculty position without ever publishing a journal paper.
This is mainly a good thing. The relatively low time from initial submission to publication (compared to traditional journals) makes sure that research gets published relatively timely. The deadlines make sure that people get their act together and get the paper submitted. Not all computer science papers are super-polished, but who cares? It's more important that people get their ideas and results out there for others to build on.
However, it has also had the result that many computer science conferences are hard to get into. In computational intelligence it's pretty common to accept about half of the submitted papers. Other fields are much harsher. In American academia, it is common to require that a conference accepts at most 30% of submitted papers in order to be counted as "selective". Many conferences are even stricter than that, with acceptance rates in the 10% range.
Why are acceptance rates so low? The often stated reason is that conference attendees don't have enough time to see lots of bad research being presented, and therefore only the best papers should be presented. However, this assumes that all attendees see all talks. If there are many talks (or posters) in parallel at a conference, people can choose which talk/poster they want to see. This is after all how it works in fields such as medicine and physics, where conference presentations are based on abstracts, and all or most abstracts are accepted.
The real reason is that conferences want to be prestigious through being exclusive. If a conference only accepts 13% of submitted papers, getting in is seen as an accomplishment for the author, something to put in your CV to impress various committees with. Paper acceptance becomes instant gratification for the scientist. For conferences, being an exclusive conference means that you get more submissions and more attendees, and as every organisation wants to perpetuate itself there is an incentive for conferences to be exclusive.
So what is the problem with this? There are several problems. To begin with, peer review is notoriously noisy. Most conferences have three or four reviewers per paper. It is very often the case that reviewers disagree with each other. One reviewer might think a paper is excellent, another thinks it is boring and derivative, a third thinks it is off-topic, and a fourth does not understand the paper and therefore thinks its badly written and/or wrong. How do you make a decision based on this? Being the program chair of a conference means making decisions based on conflicting reviewer judgements, knowing that your decisions will often be wrong. If you have a low acceptance rate, it is extremely probable that you will reject papers with one or two negative reviews, therefore rejecting many good papers.
Rejecting good papers is bad, because you are holding up science. Good papers, whose research should be out there for the world to see, don't get published. Authors don't get feedback, and get dissuaded from doing further research.
Why are reviewer opinions so conflicting? Part of the reason is certainly that there are few incentives to do a good job when reviewing papers, or even to review papers at all, so why bother? But more fundamentally, it is not always possible to tell the good papers from the bad ones. It is often as hard to spot greatness as it is to spot errors and misconduct. Many groundbreaking papers initially got rejected. There are probably many other groundbreaking results that the world doesn't know about, because the papers were never published.
If conferences are selective, reviewers will become nasty. It is a well-known fact that reviewers in many parts of computer science are more negative than in other sciences. This is probably because they might themselves submit to the same conference, or have submitted in the past and gotten rejected, and they don't want the paper they are reviewing to be treated better than their own paper was or will be. This breeds a culture of nastiness.
People respond to incentives. With selective conferences, researchers will start writing papers to maximise the likelihood of acceptance, and start doing research that can be written up in such papers. This is a disastrous consequence, because the easiest way to get into a selective conference is to write a paper which makes a small incremental advance, and which is not wrong in any way. The easiest part of a reviewer's job (certainly so for a nasty reviewer) is to find faults in the paper under review. I personally feel I have done a good job as a reviewer when I've found many faults in the paper I'm reviewing. Papers that make drastic claims or bold hypothesis are the easiest to shoot down. It is much harder to reject a paper because it is "not exciting". Thus selective conferences could conform to the Law of Jante, even though that is nobody's intention.
I suspect that this effect is even greater for people entering some sub-field of computer science, without being hand-led by someone who is already an insider. A newcomer to the field does not know which buzzwords to use, important people to cite and theories to subscribe to, making it very easy to shoot down an uncomfortable paper.
To sum this up, selective conferences are bad for science because good papers get rejected, because they perpetuate the myth that we can accurately judge the quality of a paper before it is even published, because they breed a culture of nastiness, and because they can reward mediocre research rather than ground-breaking research.
I see no reason why we should go on playing this game. Instead, we should have inclusive conferences, that accept all papers that are good enough. This could 10%, 50% or even 100% of them. Good enough could be defined as being on topic, substantially correct, intelligibly written and making some sort of contribution. This would of course mean that we can no longer judge a researcher based on what conferences he or she gets papers accepted in.
So, if we can't judge papers based on what conference they have been published in, how should we judge them? Well, there are two ways. The first is to actually read them. Shocking as this suggestion might seem, reading a paper is the only way to really know its value. Of course, it requires that you know the research field well enough to understand the paper, and that you are prepared to spend the time it takes to read it.
The other way is to wait a few years and see if the paper influences other peoples' research, and therefore gets cited. In addition to citation count, we could have some sort of voting mechanism, where attendants of a conference or members of a research community get to vote on the most important papers of conferences three of five years before. The problem with this is of course that you have to wait a few years.
But there is not really any way around it, if you don't want to or can't read the papers. Research takes time, and hiring or tenure decisions should not be based on the sort of incomplete information you can get from low-quality metrics such as in which venue a paper was published.
This is mainly a good thing. The relatively low time from initial submission to publication (compared to traditional journals) makes sure that research gets published relatively timely. The deadlines make sure that people get their act together and get the paper submitted. Not all computer science papers are super-polished, but who cares? It's more important that people get their ideas and results out there for others to build on.
However, it has also had the result that many computer science conferences are hard to get into. In computational intelligence it's pretty common to accept about half of the submitted papers. Other fields are much harsher. In American academia, it is common to require that a conference accepts at most 30% of submitted papers in order to be counted as "selective". Many conferences are even stricter than that, with acceptance rates in the 10% range.
Why are acceptance rates so low? The often stated reason is that conference attendees don't have enough time to see lots of bad research being presented, and therefore only the best papers should be presented. However, this assumes that all attendees see all talks. If there are many talks (or posters) in parallel at a conference, people can choose which talk/poster they want to see. This is after all how it works in fields such as medicine and physics, where conference presentations are based on abstracts, and all or most abstracts are accepted.
The real reason is that conferences want to be prestigious through being exclusive. If a conference only accepts 13% of submitted papers, getting in is seen as an accomplishment for the author, something to put in your CV to impress various committees with. Paper acceptance becomes instant gratification for the scientist. For conferences, being an exclusive conference means that you get more submissions and more attendees, and as every organisation wants to perpetuate itself there is an incentive for conferences to be exclusive.
So what is the problem with this? There are several problems. To begin with, peer review is notoriously noisy. Most conferences have three or four reviewers per paper. It is very often the case that reviewers disagree with each other. One reviewer might think a paper is excellent, another thinks it is boring and derivative, a third thinks it is off-topic, and a fourth does not understand the paper and therefore thinks its badly written and/or wrong. How do you make a decision based on this? Being the program chair of a conference means making decisions based on conflicting reviewer judgements, knowing that your decisions will often be wrong. If you have a low acceptance rate, it is extremely probable that you will reject papers with one or two negative reviews, therefore rejecting many good papers.
Rejecting good papers is bad, because you are holding up science. Good papers, whose research should be out there for the world to see, don't get published. Authors don't get feedback, and get dissuaded from doing further research.
Why are reviewer opinions so conflicting? Part of the reason is certainly that there are few incentives to do a good job when reviewing papers, or even to review papers at all, so why bother? But more fundamentally, it is not always possible to tell the good papers from the bad ones. It is often as hard to spot greatness as it is to spot errors and misconduct. Many groundbreaking papers initially got rejected. There are probably many other groundbreaking results that the world doesn't know about, because the papers were never published.
If conferences are selective, reviewers will become nasty. It is a well-known fact that reviewers in many parts of computer science are more negative than in other sciences. This is probably because they might themselves submit to the same conference, or have submitted in the past and gotten rejected, and they don't want the paper they are reviewing to be treated better than their own paper was or will be. This breeds a culture of nastiness.
People respond to incentives. With selective conferences, researchers will start writing papers to maximise the likelihood of acceptance, and start doing research that can be written up in such papers. This is a disastrous consequence, because the easiest way to get into a selective conference is to write a paper which makes a small incremental advance, and which is not wrong in any way. The easiest part of a reviewer's job (certainly so for a nasty reviewer) is to find faults in the paper under review. I personally feel I have done a good job as a reviewer when I've found many faults in the paper I'm reviewing. Papers that make drastic claims or bold hypothesis are the easiest to shoot down. It is much harder to reject a paper because it is "not exciting". Thus selective conferences could conform to the Law of Jante, even though that is nobody's intention.
I suspect that this effect is even greater for people entering some sub-field of computer science, without being hand-led by someone who is already an insider. A newcomer to the field does not know which buzzwords to use, important people to cite and theories to subscribe to, making it very easy to shoot down an uncomfortable paper.
To sum this up, selective conferences are bad for science because good papers get rejected, because they perpetuate the myth that we can accurately judge the quality of a paper before it is even published, because they breed a culture of nastiness, and because they can reward mediocre research rather than ground-breaking research.
I see no reason why we should go on playing this game. Instead, we should have inclusive conferences, that accept all papers that are good enough. This could 10%, 50% or even 100% of them. Good enough could be defined as being on topic, substantially correct, intelligibly written and making some sort of contribution. This would of course mean that we can no longer judge a researcher based on what conferences he or she gets papers accepted in.
So, if we can't judge papers based on what conference they have been published in, how should we judge them? Well, there are two ways. The first is to actually read them. Shocking as this suggestion might seem, reading a paper is the only way to really know its value. Of course, it requires that you know the research field well enough to understand the paper, and that you are prepared to spend the time it takes to read it.
The other way is to wait a few years and see if the paper influences other peoples' research, and therefore gets cited. In addition to citation count, we could have some sort of voting mechanism, where attendants of a conference or members of a research community get to vote on the most important papers of conferences three of five years before. The problem with this is of course that you have to wait a few years.
But there is not really any way around it, if you don't want to or can't read the papers. Research takes time, and hiring or tenure decisions should not be based on the sort of incomplete information you can get from low-quality metrics such as in which venue a paper was published.
Thursday, July 18, 2013
On best paper award selection through reviewer nomination followed by attendee vote
At the IEEE Conference on Computational Intelligence on Games, we decide on our annual best paper award through reviewer nomination followed by attendee vote. Concretely, this means that we select all papers with an exceptionally high average review score, and all papers with a high score where several reviewers have separately indicated that the paper should be nominated for a best paper award. The nominated papers (for CIG 2013, there are 9 nominated papers) are presented in one or two plenary sessions, and the conference attendees are asked to vote for the best paper award at the end of the presentations by way of secret ballot. The paper with most votes wins.
To paraphrase someone famous, I think that while this system for deciding a best paper award might not be perfect, it is better than all the other systems that have been tried. However, we recently received a mail questioning this policy, and suggesting that we instead select a best paper award by a special awards committee. I wrote a longish answer, which I'm reproducing below (slightly edited):
While some conferences decide awards in a small committee, other conferences (including some reputable ones like Gecco, Foundations of Digital Games and EuroGP) decide on their awards like us, through reviewer nomination and votes among conference attendees. There are several reasons for why one might want to do it this way:
* It makes the award more legitimate and avoids many potential conflicts of interest. If the award is decided on by a small committee, who is either secret or otherwise conducts their work in secrect, suspicions can always arise about why the committee decided like it did. This is especially true for a relatively small community where the conference organisers know many of the attendees personally. (It is worth noting that the chairs of CIG has this year not selected any papers directly at all; the 9 nominees are selected purely based on a cutoff in terms of reviewer scores and number of best paper nominations.)
* It ensures that a larger number of people with different expertise get to weight in on the award. This could be seen as ensuring quality through "wisdom of the crowds", or simply as ensuring that a set of experts whose fields of competence and interest that reflect the conference attendees get to decide. In a small committee, some fields of expertise are bound to be missing.
* It engages the audience. Attendees are more likely to pay close attention in a session where every attendee is expected to provide feedback, especially if this feedback has real impact.
* It incentivises good presentations. Twenty minutes is enough to present the core ideas of any conference paper. However, many researchers do not put sufficient effort into preparing and revising their presentations, and as a result conferences are often filled with poor presentations. Knowing that getting the best paper award or not depends partly on how well you can bring your message across tends to have a great effect on presentation quality.
As a personal anecdote, earlier this year I attended the best paper session at EuroGP in Vienna. The winner was a paper that was complex, unintuitive and challenged core notions of how genetic programming works. The presenter had gone to great lenghts to prepare a presentation that most of the audience actually understood - and walked off with a very well-deserved award. To me, that presentation was worth as much as rest of the conference together.
To paraphrase someone famous, I think that while this system for deciding a best paper award might not be perfect, it is better than all the other systems that have been tried. However, we recently received a mail questioning this policy, and suggesting that we instead select a best paper award by a special awards committee. I wrote a longish answer, which I'm reproducing below (slightly edited):
While some conferences decide awards in a small committee, other conferences (including some reputable ones like Gecco, Foundations of Digital Games and EuroGP) decide on their awards like us, through reviewer nomination and votes among conference attendees. There are several reasons for why one might want to do it this way:
* It makes the award more legitimate and avoids many potential conflicts of interest. If the award is decided on by a small committee, who is either secret or otherwise conducts their work in secrect, suspicions can always arise about why the committee decided like it did. This is especially true for a relatively small community where the conference organisers know many of the attendees personally. (It is worth noting that the chairs of CIG has this year not selected any papers directly at all; the 9 nominees are selected purely based on a cutoff in terms of reviewer scores and number of best paper nominations.)
* It ensures that a larger number of people with different expertise get to weight in on the award. This could be seen as ensuring quality through "wisdom of the crowds", or simply as ensuring that a set of experts whose fields of competence and interest that reflect the conference attendees get to decide. In a small committee, some fields of expertise are bound to be missing.
* It engages the audience. Attendees are more likely to pay close attention in a session where every attendee is expected to provide feedback, especially if this feedback has real impact.
* It incentivises good presentations. Twenty minutes is enough to present the core ideas of any conference paper. However, many researchers do not put sufficient effort into preparing and revising their presentations, and as a result conferences are often filled with poor presentations. Knowing that getting the best paper award or not depends partly on how well you can bring your message across tends to have a great effect on presentation quality.
As a personal anecdote, earlier this year I attended the best paper session at EuroGP in Vienna. The winner was a paper that was complex, unintuitive and challenged core notions of how genetic programming works. The presenter had gone to great lenghts to prepare a presentation that most of the audience actually understood - and walked off with a very well-deserved award. To me, that presentation was worth as much as rest of the conference together.
Thursday, January 10, 2013
CfP: PCG workshop 2013
Call for Papers
The fourth workshop on Procedural Content Generation in Games (PCG 2013)
Organized in conjunction with the International Conference on Foundations of Digital Games (FDG 2013)
Important Dates
Full paper submission: March 4
Decision notification: March 25
Camera-ready deadline: April 1
Workshop held: between May 14 and 17
Website: http://pcg.fdg2013.org/
Procedural content generation (PCG) in games, a field of growing popularity, offers hope for substantially reducing the authoring burden in games, improving our theoretical understanding of game design, and enabling entirely new kinds of games and playable experiences. The goal of this workshop is to advance knowledge in PCG by bringing together researchers and fostering discussion about the current state of the field. We invite contributions on all aspects of generating game content, using any method. Both descriptions of new algorithms, theoretical or critical analysis and empirical studies of implementations and applications are welcome.
We solicit submissions as either full papers about results from novel research (8 pages) or short papers describing works-in-progress (4 pages). Papers may be about variety of topics within procedural content generation, including but not limited to:
Offline or realtime procedural generation of levels, stories, quests, terrain, environments, and other game content
Case studies of industrial application of procedural generation
Issues in the construction of mixed-mode systems with both human and procedurally generated content
Adaptive games using procedural content generation
Procedural generation of game rulesets (computer or tabletop)
Techniques for procedural animation
Issues in combining multiple procedural content generation techniques for larger systems
Procedural content generation in non-digital games
Procedural content generation as a game mechanic
Automatic game balancing through generated content
Techniques for games that evolve and/or discover new game variants
Player and/or designer experience in procedural content generation
Procedural content generation during development (e.g. prototyping, playtesting, etc.)
Theoretical implications of procedural content generation
How to incorporate procedural generation meaningfully into game design
Lessons from historical examples of procedural content generation (including post-mortems)
Authors are especially encouraged to submit work that has the potential to be adopted by the digital games industry.
Organizers
Alex Pantaleev, SUNY Oswego
Gillian Smith, Northeastern University
Joris Dormans, Amsterdam University of Applied Sciences
Antonio Coelho, Universidade do Porto
The fourth workshop on Procedural Content Generation in Games (PCG 2013)
Organized in conjunction with the International Conference on Foundations of Digital Games (FDG 2013)
Important Dates
Full paper submission: March 4
Decision notification: March 25
Camera-ready deadline: April 1
Workshop held: between May 14 and 17
Website: http://pcg.fdg2013.org/
Procedural content generation (PCG) in games, a field of growing popularity, offers hope for substantially reducing the authoring burden in games, improving our theoretical understanding of game design, and enabling entirely new kinds of games and playable experiences. The goal of this workshop is to advance knowledge in PCG by bringing together researchers and fostering discussion about the current state of the field. We invite contributions on all aspects of generating game content, using any method. Both descriptions of new algorithms, theoretical or critical analysis and empirical studies of implementations and applications are welcome.
We solicit submissions as either full papers about results from novel research (8 pages) or short papers describing works-in-progress (4 pages). Papers may be about variety of topics within procedural content generation, including but not limited to:
Offline or realtime procedural generation of levels, stories, quests, terrain, environments, and other game content
Case studies of industrial application of procedural generation
Issues in the construction of mixed-mode systems with both human and procedurally generated content
Adaptive games using procedural content generation
Procedural generation of game rulesets (computer or tabletop)
Techniques for procedural animation
Issues in combining multiple procedural content generation techniques for larger systems
Procedural content generation in non-digital games
Procedural content generation as a game mechanic
Automatic game balancing through generated content
Techniques for games that evolve and/or discover new game variants
Player and/or designer experience in procedural content generation
Procedural content generation during development (e.g. prototyping, playtesting, etc.)
Theoretical implications of procedural content generation
How to incorporate procedural generation meaningfully into game design
Lessons from historical examples of procedural content generation (including post-mortems)
Authors are especially encouraged to submit work that has the potential to be adopted by the digital games industry.
Organizers
Alex Pantaleev, SUNY Oswego
Gillian Smith, Northeastern University
Joris Dormans, Amsterdam University of Applied Sciences
Antonio Coelho, Universidade do Porto
Thursday, November 29, 2012
Call for Expressions of Interest: Hosting IEEE Conference on Computational Intelligence and Games 2015
The IEEE Conference on Computational Intelligence and Games is the premier annual event for researchers applying computational and artificial intelligence techniques to games. The domain of the conference includes all sorts of CI/AI applied to all sorts of games, including board games, video games and mathematical games. Recent editions have been held in Granda, Spain (2012) and Seoul, Korea (2011). The next CIG will be held in Niagara Falls, Canada (2013), very likely to be followed by Dortmund, Germany (2014). Since the start of the conference series in 2005, there has been a trend towards higher numbers of both submissions to and attendants at successive conferences.
We are now looking for expressions of interest for people willing to host CIG 2015. Given the IEEE Computational Intelligence Society policy that conferences alternate between Europe, North America and Asia, we are looking for an Asian location for CIG 2015.
Expressions of interest should be sent to Julian Togelius (julian@togelius.com) by December 15; questions about the procedure should be directed to the same address. All expressions of interest will be forwarded to members of the Games Technical Committee for discussion and a straw poll, and the winning submitter will be invited to submit a formal application to host the conference to the IEEE Computational Intelligence Society's Conference Committee.
An expression of interest should be a text document of one or a few pages. Apart from the proposed location and dates, it should include the names and short biographies of a general chair and preferably some other proposed organisation committee members, e.g. program chair and local chair. It should also include a brief description of the proposed site in terms of facilities available, touristic attractions and communications. No budgetary information is necessary at this stage.
Past CIG conferences: http://www.ieee-cig.org/
Games Technical Committee: http://cis.ieee.org/games-tc.html
Saturday, October 06, 2012
Call for papers: FDG 2013
Foundations of Digital Games 2013
Call for papers, workshops, panels, experimental games and participation
14-17 May 2013
Chania, Crete, Greece
http://www.fdg2013.org/
We invite researchers and educators to submit to FDG 2013 and share insights
and cutting-edge research related to game technologies and their use. FDG 2013
will include presentations of peer-reviewed papers, invited talks by
high-profile industry and academic leaders, panels, and posters. The conference
will also host a technical demo session, a Research and Experimental Games
Festival, and a Doctoral Consortium. The technical demo session will include
novel tools, techniques, and systems created for games. The Research and
Experimental Games Festival will showcase the latest experimental and research
games. The Doctoral Consortium serves as a forum for Ph.D. students to present
their dissertation research, exchange experiences with peers, discuss ideas for
future research and receive feedback from established games researchers and the
wider FDG community.
Important dates
---
Workshop proposals:
* Submission: 28 October 2012
* Notification: 11 November 2012
Papers, panel proposals, doctoral consortium:
* Submission: 10 December 2012
* Notification: 1 March 2013
* Camera-ready: 18 March 2013
Research and experimental game festival:
* Submission: 13 January 2013
* Notification: 22 February 2013
* Camera-ready: 18 March 2013
Posters and demos:
* Submission: 4 March 2013
* Notification: 18 March 2013
* Camera-ready: 31 March 2013
Full papers
---
Full papers must not exceed 8 pages in length. Authors should submit to either
the general conference or one of the following tracks:
* Game studies, social science track (games, players, and their role in society
and culture)
* Game studies, humanities track (aesthetic, philosophical, and ontological
aspects of games and play)
* Game design (methods, techniques, studies)
* Serious games (building and evaluating games for a purpose, learning in games)
* Game education (preparing students to design and develop games)
* Artificial intelligence (agents, motion/camera planning, navigation,
adaptivity, content creation, dialog, authoring tools)
* Game technology (engines, frameworks, graphics, networking, animation)
* Interaction and player experience (game interfaces, player metrics, modeling
player experience)
Panels
---
Panel submissions should be in the form of a 2-page extended abstract
describing the focus of the panel, providing a list of confirmed speakers, and
indicating their areas of expertise relative to the topic. We encourage both
debate-style panels that include representatives advocating several positions
on a topic of disagreement, and emerging-area style panels that consolidate and
explain recent work on a subject of interest to the FDG community.
Research and experimental games festival
---
The Festival is designed to showcase playable games that are experimental or
have a research component. Submitted games could be significant because they
are designed to answer a research question or experiment with the design
process, or because their technological components represent research
advancements. Works in progress are permitted, but the game will ideally
include at least one playable level (or comparable unit of play time). Works
that have not yet reached this stage may be more suitable for the conference
demo track. In addition to submitting the game, submissions should also include
a 2–4 page writeup of the project. The text should outline the game's research
context, and how the work demonstrates rigor in methodology and a contribution
to knowledge. Submissions should also include a link to the game hosted on your
own server or one of your choosing. We welcome and encourage works exploring a
variety of disciplinary approaches and methodologies, including
interdisciplinary collaborations. It is the responsibility of the contributor
to ensure all necessary information is accessible at all times during the
judging period (13 January 2013 to 22 February 2013).
Posters and demos
---
The poster and demo track provides a forum for late-breaking and in-progress
work to be presented to the community. Submissions should be in the form of a
2-page extended abstract. The interactive technical demo event will showcase
the latest tools, techniques, and systems created for games by academic or
industrial research groups. (Playable games should instead be submitted to the
Research and Experimental Games Festival.)
Workshop proposals
---
The conference workshops are full-day and half-day sessions focused on emerging
game-related topics. These workshops provide an informal setting for new
developments to be presented, discussed and demonstrated. We are particularly
interested in topics that bridge different communities and disciplines. Concise
workshop proposals (2 pages) should include: an extended abstract, the
objectives and expected outcome of the workshop, the planned activities, the
background of the organizer(s), the anticipated number of participants, and the
means for soliciting and selecting participants.
Doctoral consortium
---
We invite PhD students to apply to the Doctoral Consortium, a forum to provide
PhD students with early feedback on their research directions, from fellow
students, researchers, and experienced faculty in the area. The consortium is
intended primarily for PhD students who intend to pursue a career in academia,
who will soon propose, or have recently proposed, their research. To apply,
doctoral students should submit a CV, a 3-page extended abstract describing
their proposed research, and a support letter from their PhD advisor. The
abstract should address the goals of your research, the proposed approach and
how it differs from prior work, any results you may have, and your plans for
completing the work. Invited Doctoral Consortium students will give a
presentation and present a poster at the conference.
On behalf of the organizing committee:
General chairs: Georgios N. Yannakakis and Espen Aarseth
Program chairs: Kristine Jørgensen and James Lester
Proceedings chair: Mark J. Nelson
Workshops chair: Julian Togelius
Industrial relations chair: Alessandro Canossa
Local chairs: Kostas Karpouzis and Alexandros Potamianos
Track chairs: Kevin Kee, Rilla Khaled, Olli Leino,
R. Michael Young, Jose Zagal (more to be announced)
Call for papers, workshops, panels, experimental games and participation
14-17 May 2013
Chania, Crete, Greece
http://www.fdg2013.org/
We invite researchers and educators to submit to FDG 2013 and share insights
and cutting-edge research related to game technologies and their use. FDG 2013
will include presentations of peer-reviewed papers, invited talks by
high-profile industry and academic leaders, panels, and posters. The conference
will also host a technical demo session, a Research and Experimental Games
Festival, and a Doctoral Consortium. The technical demo session will include
novel tools, techniques, and systems created for games. The Research and
Experimental Games Festival will showcase the latest experimental and research
games. The Doctoral Consortium serves as a forum for Ph.D. students to present
their dissertation research, exchange experiences with peers, discuss ideas for
future research and receive feedback from established games researchers and the
wider FDG community.
Important dates
---
Workshop proposals:
* Submission: 28 October 2012
* Notification: 11 November 2012
Papers, panel proposals, doctoral consortium:
* Submission: 10 December 2012
* Notification: 1 March 2013
* Camera-ready: 18 March 2013
Research and experimental game festival:
* Submission: 13 January 2013
* Notification: 22 February 2013
* Camera-ready: 18 March 2013
Posters and demos:
* Submission: 4 March 2013
* Notification: 18 March 2013
* Camera-ready: 31 March 2013
Full papers
---
Full papers must not exceed 8 pages in length. Authors should submit to either
the general conference or one of the following tracks:
* Game studies, social science track (games, players, and their role in society
and culture)
* Game studies, humanities track (aesthetic, philosophical, and ontological
aspects of games and play)
* Game design (methods, techniques, studies)
* Serious games (building and evaluating games for a purpose, learning in games)
* Game education (preparing students to design and develop games)
* Artificial intelligence (agents, motion/camera planning, navigation,
adaptivity, content creation, dialog, authoring tools)
* Game technology (engines, frameworks, graphics, networking, animation)
* Interaction and player experience (game interfaces, player metrics, modeling
player experience)
Panels
---
Panel submissions should be in the form of a 2-page extended abstract
describing the focus of the panel, providing a list of confirmed speakers, and
indicating their areas of expertise relative to the topic. We encourage both
debate-style panels that include representatives advocating several positions
on a topic of disagreement, and emerging-area style panels that consolidate and
explain recent work on a subject of interest to the FDG community.
Research and experimental games festival
---
The Festival is designed to showcase playable games that are experimental or
have a research component. Submitted games could be significant because they
are designed to answer a research question or experiment with the design
process, or because their technological components represent research
advancements. Works in progress are permitted, but the game will ideally
include at least one playable level (or comparable unit of play time). Works
that have not yet reached this stage may be more suitable for the conference
demo track. In addition to submitting the game, submissions should also include
a 2–4 page writeup of the project. The text should outline the game's research
context, and how the work demonstrates rigor in methodology and a contribution
to knowledge. Submissions should also include a link to the game hosted on your
own server or one of your choosing. We welcome and encourage works exploring a
variety of disciplinary approaches and methodologies, including
interdisciplinary collaborations. It is the responsibility of the contributor
to ensure all necessary information is accessible at all times during the
judging period (13 January 2013 to 22 February 2013).
Posters and demos
---
The poster and demo track provides a forum for late-breaking and in-progress
work to be presented to the community. Submissions should be in the form of a
2-page extended abstract. The interactive technical demo event will showcase
the latest tools, techniques, and systems created for games by academic or
industrial research groups. (Playable games should instead be submitted to the
Research and Experimental Games Festival.)
Workshop proposals
---
The conference workshops are full-day and half-day sessions focused on emerging
game-related topics. These workshops provide an informal setting for new
developments to be presented, discussed and demonstrated. We are particularly
interested in topics that bridge different communities and disciplines. Concise
workshop proposals (2 pages) should include: an extended abstract, the
objectives and expected outcome of the workshop, the planned activities, the
background of the organizer(s), the anticipated number of participants, and the
means for soliciting and selecting participants.
Doctoral consortium
---
We invite PhD students to apply to the Doctoral Consortium, a forum to provide
PhD students with early feedback on their research directions, from fellow
students, researchers, and experienced faculty in the area. The consortium is
intended primarily for PhD students who intend to pursue a career in academia,
who will soon propose, or have recently proposed, their research. To apply,
doctoral students should submit a CV, a 3-page extended abstract describing
their proposed research, and a support letter from their PhD advisor. The
abstract should address the goals of your research, the proposed approach and
how it differs from prior work, any results you may have, and your plans for
completing the work. Invited Doctoral Consortium students will give a
presentation and present a poster at the conference.
On behalf of the organizing committee:
General chairs: Georgios N. Yannakakis and Espen Aarseth
Program chairs: Kristine Jørgensen and James Lester
Proceedings chair: Mark J. Nelson
Workshops chair: Julian Togelius
Industrial relations chair: Alessandro Canossa
Local chairs: Kostas Karpouzis and Alexandros Potamianos
Track chairs: Kevin Kee, Rilla Khaled, Olli Leino,
R. Michael Young, Jose Zagal (more to be announced)
Monday, May 28, 2012
Human intelligence is overrated
People often wonder when computers will become as intelligent as humans. This question assumes that being as intelligent as a human is a worthy goal for an artificial system. But really, are humans all that smart? I don't think so.
Humans are quite stupid in many ways, compared to computers. Let's start with the most obvious: they can't count. Ask a human to raise 3425 to the power of 542 and watch them sit there for hours trying to work it out. Ridiculous. The same goes for a number of other trivial tasks, such as calculating the average age in a population of 300 million. Shouldn't take more than a couple of seconds. Unless you are a human, in which case it'll probably take you years, and even then you would have made a number of errors.
Humans have almost no memory, either. Ask a human to give you the correct name and current address for a randomly chosen personal registration number (or national insurance number, or whatever the equivalent is in your country). Even if they have all the information in whatever format they prefer, it will still take them at least several seconds - and most humans would not even know where to get the information. Or ask a human to produce 100 URLs to websites talking about artificial intelligence, or even a complete list of everything that happened to them yesterday. Humans talk about "goldfish memory", but from the perspective of a computer the human and the goldfish aren't that far apart, capability-wise.
At this point, many humans will be protesting wildly and saying we are being terribly unfair to them. We are only choosing tasks that computers excel at, and ignoring those where humans have an advantage, such as motor control and pattern recognition.
Right. Computers can land a jet plane and fly an helicopter. In fact, almost any computer can do it, if you just load the right software. Very few humans can land a jet plane and even fewer can fly an helicopter. Sometimes humans fail spectacularly at these tasks. (It's hard to understand why anyone would want to be in a plane flown by a human, now that there are alternatives.) Computers can drive regular cars on-road and off-road, obeying all traffic regulations. There are many humans that can't even do that.
Speaking of pattern recognition, it's true that humans can recognise the faces of their friends with quite high accuracy. But then, humans only have a couple of hundred friends, at most. The face recognition software that Facebook uses can tell the faces of millions of people apart. Other pattern recognition algorithms can successfully match a scan of a human thumb to the right fingerprint in a database of millions.
Now let's take another activity that humans should be good at: game-playing. Games were invented by humans in order to entertain themselves, and as humans seem to find it entertaining to exercise their learning, motor and reasoning capabilities, games should be perfectly tailored to human intelligence. Humans should excel at game-playing, right? Well, no. Since 1997, the world Chess champion has been a computer. The situation is even more extreme for Checkers: computers have computed how to play the game perfectly, meaning that it is mathematically impossible to win over a computer unless you follow the very same strategy. Moving from board games to video games, we have a similar situation. There are AI players that play most levels of Super Mario Bros better than any known human, and computers kick human ass in first-person shooters like Unreal Tournament because humans have such lousy reaction times. Mind you, all these are games that were designed by humans for humans. It would be very easy to invent games that were so complicated that only computers could play them; computers could even invent such games automatically.
Other things that have been cited as pinnacles of human achievement are tying shoelaces and self-reproduction. But tying shoelaces is sort of pointless, it's an obsolete technology even for humans; why would you need shoelaces if you're a robot? And humans don't really know how to reproduce themselves. They know how to have a sex, which is quite a different thing and really sort of easy. The actual reproduction is down to various biochemical processes that humans don't even understand yet, much less can they replicate them.
Now, some humans reading this will be all up in arms, and accusing me of dishonesty and sophistry in trying to degrade human intelligence. I'm giving all these examples of computers being good (and humans bad) at performing very specific tasks and solving very specific problems, when the hallmark of real intelligence is to be able to perform well in a large variety of situations. According to this definition, the human would say, computers are not very intelligent: a chess-playing program cannot land a jet plane, and a face recognition program cannot play Super Mario Bros, nor can it do exponentiation.
This is true. But the argument goes both ways here as well: take an arbitrary human (such as yourself, if you happen to be human) and try placing this human in the cockpit of a landing jet plane, in a semiconductor factory, in the oval office of the White House, in the kitchen of a gourmet restaurant, on a horseback in Siberia, or equipped with only a spear in the middle of the Amazonas jungle. There are humans that have been programmed to do well in each of these situations, but it is very unlikely that the human you were thinking of (perhaps yourself) would know what to do in more than at most one of these situations.
So, compared to humans, computers seem to be doing quite well indeed. If man is the measure of all things, artificial intelligence research should be declared a success already. But that would be like saying that the development of the car would have been finished when cars first became as fast and reliable as horse-drawn carriages. As we know now, it was certainly possible to do better than that.
It will take some time, and much hard work, before we can say that computers are truly generally intelligent. But, to be fair, we should be careful with ascribing general intelligence to humans as well - humans are just not that smart. In particular we should be very careful with using human intelligence as a measuring yard for machine intelligence, and we should stop asking when computers will become as intelligent as humans. It seems that man is the measure in this particular case simply because we didn't know any other type of intelligence when we started thinking about the problem of artificial intelligence. Hopefully that will change soon.
Monday, February 20, 2012
Playing with games
Let's talk about digital games (computer games), their relation to play, their relation to you and to artificial intelligence. One way they could be divided up is the following: there are games you play, games you play with, games that play you and games that play with you. The first category is very common, the second rather common, the third a current research topic and the fourth an unknown territory awaiting (or perhaps provoking) new advances in artificial intelligence.
Games you play
These are the games that most people come to think of when you mention computer games, and that usually dominate sales charts. Games like Doom, Asteroids, Tetris, StarCraft, Angry Birds and Call of Duty. Games of progression (Juul) where there are goals and there is a clear sense of what it means to perform the game better or worse. Games where "instrumental play" (Taylor), i.e. playing the game like you were performing a sport, is possible (but not necessary). In these games, the rules are set and enforced by the game and it's up to you as a player to do your best according to what the game thinks you should do and within the possibility space the game affords.
It is very common that these games feature some sort of artificial intelligence, typically in the role of controlling the opponents and allies in order to provide a challenge. But there are also examples of AI-based dynamic difficulty adjustment and other forms of experience management, such as the AI Director in Left 4 Dead. Additionally, AI techniques can be used during the design and development phases of such games, especially for procedural content generation.
Games you play with
Some games have less explicit goals, less clear sense of progression and perhaps less structure overall. These are games which are sometimes describes as more of toys than games, and which when replayed from scratch may play out completely differently. Games of emergence, like Sim City, Minecraft and The Sims. As these games try to hard to make you feel like you can do whatever the hell you want, you are encouraged to express your own desires, dreams and whims using the mechanics of the game. Of course, no games have so far been made where you can actually do whatever you want; therefore, you are likely to run up against the limits of the simulation and game mechanics sooner or later. (Sooner, if you are of the "non serviam" player type.) Of course, most of the games that are meant to be played can be played with with varying degrees of success. For example, you can decide to build sculptures instead of clearing lines in Tetris, and you can enact little dances in Doom. The very fuzzy boundary beetwen games you play and games you play with can be further illustrated with how hard it is to classify Civilization or Skyrim as one or the other of these.
There are more potential roles for artificial intelligence in games you play with than there are in games you play. Because of the impossibility of scripting a free-form game, they are often crucially relying on a complicated simulation of the game world, many parts of which could be called AI. A good example is the use of "smart objects" that match characters' needs with provided services in The Sims. Given the risk of such free-form simulations getting out of hand due to emergent unbalance, there is also a role for experience managers here (e.g. causing an earthquake if your city is doing too well).
Games that play you
You can become a challenge for the game. That is, if the game sees as its objective to make you happy, to keep you challenged, to make you sad, to keep you playing or perhaps to make you stop playing. You might not be inclined to do as the game wants you to do, and then the game needs to manipulate you. Luckily (for the game) the game knows quite a lot about you, as you keep feeding it with information all the time, via the keyboard, mouse or gamepad interface you use to interact with the game. (The game might also have other nefarious ways to glean information about what you feel and think, such as watching what you do via a Kinect device or webcam, or measuring your stress level via galvanic skin response.) Thus, the game can take actions by introducing new elements, new levels, changing music and art, distributing rewards, changing difficulty level and behaviour of in-game characters etc, and measure the success of these actions by your behaviour. This is isomorphic to how you take actions such as jumping, shooting, looting and choosing dialogue options when you play a game, and measure the success of these actions through score, progress meters, achievements etc.
Games that play you is at the center of our AI research program here at IT University of Copenhagen. Georgios and me recently wrote a survey/position paper that outlines our view of how to achieve such games. We call our approach "Experience-driven Procedural Content Generation". This is a very AI-heavy approach, where various AI techniques are used both for modelling what you think and feel while playing the game, predicting what you will do under various circumstances, and creating new game content that will bring the game closer to its goal according to the derived model. An example of this approach is our work with a version Super Mario Bros with adaptive levels. This game wants to keep you entertained. Therefore it has let hundreds of players play pairs of different levels, and asked the players which of the levels were most entertaining. When you play it, it will match your play style with its model of player experience, and deliver a freshly created level which it thinks will keep you optimally entertained.
Games that play with you
Remember the games you used to play as a kid? Not the digital games or board games, but the ones where you never quite knew in advance what the game was about or what the rules were, because the basic premise was "to play". The ones where you went "let's pretend I'm an astronaut, and this is our spaceship" and your friend went "ok, but I'm a space knight and I ride next to your spaceship on my space horse!". Now that you're a grown-up you don't do this very often, except if you engage in improvisational theatre. In particular, you don't do this while playing games on your computer. The basic idea of a digital game is that you play against or with a stable set of rules that is more or less predictable and therefore "fair"; if you fail (or things play out differently than expected) it was your own fault or simply bad luck. The game enforces these rules with such strictness that anything that works is correct. Any change in the rules during the game session (such as removing your jump ability or making your friends your enemies) is carefully scripted.
There are some attempts to undermine the status of rules in computer games, for example the game B.U.T.T.O.N (see Wilson's article) where the rules are underspecified. Players are required to invent new rules, which are not observable or enforceable by the game, as they play. However, what this game does is simply to reduce the role of the computer in the game, and mixing in more of classic non-digital play. The game does not express any desire on its own, and is even more of a mindless automaton than most of the classic games you play. This is of course unappealing from the perspective of an AI aficionado interested in technological progress, as is the fact that the game needs to be played in physical space with other humans.
Imagine that we could replace these people with AI, and integrate this system into the game itself. Imagine a game that is actually playful. A game that wants to play with you, not because it wants to fulfil some sort of goal (keeping you playing, or entertained), but because it wants to keep itself entertained, and explore the possibility space of the play session together with you. A game that treats you as a means, not as an end. That could refuse to play with you if you are too boring or hard to play with. That gives you the feeling of really playing with someone, not just against a ruleset.
Unfortunately, we don't really know how yet how to create a game that plays with you. How can we implement a desire in the game for having fun and a lust for exploring the possibilities of interaction with a player? How can we implement the concrete mechanisms where the game co-creates the rules with you and plays along, both constructively and obstructively? Actually, this is not unfortunate at all. It means that we have a new research problem for AI and games, and yet another angle of attack on the elusive problem of artificial intelligence. For we could surely not have any real artificial intelligence that was not capable of playing and enjoying itself.
One idea for how to approach this problem is Schmidhuber's theory of artificial curiosity, where agents choose what to learn so that they optimise their predicted learning rate. There is also related work in evolutionary and developmental robotics. Another strand of work which could potentially inform the development of games that play with you is work in mixed-initiative procedural content generation, such as Tanagra and Sketchaworld. In these systems, the designer and the software take turns to work on the game content, typically so that the software acts as an resourceful and inventive assistant to the designer. Last year, I published a paper outlining a simple system that tries to do mixed-initiative rule generation while you're playing the game. Or perhaps you're playing the system, and the game emerges from the play session. The system tries to understand what rules you're playing according to, and then enforcing these rules, which might force you to play differently (especially if it has misunderstood you) and see the flaws in your implicit design concept. What is missing from this system is intentions and desires on part of the system; the game is still trying to please you, rather than itself. I should get to work right away on changing this.
This blog post was inspired in part by Sicart's article Against Procedurality, and the opposition it sets out between rule-centric play/interpretation and player-centric play/interpretation. I have nothing against any of these, and think that both are interesting. However, I think that it's important to point that just because something is procedural, it's not necessarily rigid, predictable and intentionally designed; it could be adaptive, emergent and have a will of its own.
Games you play
These are the games that most people come to think of when you mention computer games, and that usually dominate sales charts. Games like Doom, Asteroids, Tetris, StarCraft, Angry Birds and Call of Duty. Games of progression (Juul) where there are goals and there is a clear sense of what it means to perform the game better or worse. Games where "instrumental play" (Taylor), i.e. playing the game like you were performing a sport, is possible (but not necessary). In these games, the rules are set and enforced by the game and it's up to you as a player to do your best according to what the game thinks you should do and within the possibility space the game affords.
It is very common that these games feature some sort of artificial intelligence, typically in the role of controlling the opponents and allies in order to provide a challenge. But there are also examples of AI-based dynamic difficulty adjustment and other forms of experience management, such as the AI Director in Left 4 Dead. Additionally, AI techniques can be used during the design and development phases of such games, especially for procedural content generation.
Games you play with
Some games have less explicit goals, less clear sense of progression and perhaps less structure overall. These are games which are sometimes describes as more of toys than games, and which when replayed from scratch may play out completely differently. Games of emergence, like Sim City, Minecraft and The Sims. As these games try to hard to make you feel like you can do whatever the hell you want, you are encouraged to express your own desires, dreams and whims using the mechanics of the game. Of course, no games have so far been made where you can actually do whatever you want; therefore, you are likely to run up against the limits of the simulation and game mechanics sooner or later. (Sooner, if you are of the "non serviam" player type.) Of course, most of the games that are meant to be played can be played with with varying degrees of success. For example, you can decide to build sculptures instead of clearing lines in Tetris, and you can enact little dances in Doom. The very fuzzy boundary beetwen games you play and games you play with can be further illustrated with how hard it is to classify Civilization or Skyrim as one or the other of these.
There are more potential roles for artificial intelligence in games you play with than there are in games you play. Because of the impossibility of scripting a free-form game, they are often crucially relying on a complicated simulation of the game world, many parts of which could be called AI. A good example is the use of "smart objects" that match characters' needs with provided services in The Sims. Given the risk of such free-form simulations getting out of hand due to emergent unbalance, there is also a role for experience managers here (e.g. causing an earthquake if your city is doing too well).
Games that play you
You can become a challenge for the game. That is, if the game sees as its objective to make you happy, to keep you challenged, to make you sad, to keep you playing or perhaps to make you stop playing. You might not be inclined to do as the game wants you to do, and then the game needs to manipulate you. Luckily (for the game) the game knows quite a lot about you, as you keep feeding it with information all the time, via the keyboard, mouse or gamepad interface you use to interact with the game. (The game might also have other nefarious ways to glean information about what you feel and think, such as watching what you do via a Kinect device or webcam, or measuring your stress level via galvanic skin response.) Thus, the game can take actions by introducing new elements, new levels, changing music and art, distributing rewards, changing difficulty level and behaviour of in-game characters etc, and measure the success of these actions by your behaviour. This is isomorphic to how you take actions such as jumping, shooting, looting and choosing dialogue options when you play a game, and measure the success of these actions through score, progress meters, achievements etc.
Games that play you is at the center of our AI research program here at IT University of Copenhagen. Georgios and me recently wrote a survey/position paper that outlines our view of how to achieve such games. We call our approach "Experience-driven Procedural Content Generation". This is a very AI-heavy approach, where various AI techniques are used both for modelling what you think and feel while playing the game, predicting what you will do under various circumstances, and creating new game content that will bring the game closer to its goal according to the derived model. An example of this approach is our work with a version Super Mario Bros with adaptive levels. This game wants to keep you entertained. Therefore it has let hundreds of players play pairs of different levels, and asked the players which of the levels were most entertaining. When you play it, it will match your play style with its model of player experience, and deliver a freshly created level which it thinks will keep you optimally entertained.
Games that play with you
Remember the games you used to play as a kid? Not the digital games or board games, but the ones where you never quite knew in advance what the game was about or what the rules were, because the basic premise was "to play". The ones where you went "let's pretend I'm an astronaut, and this is our spaceship" and your friend went "ok, but I'm a space knight and I ride next to your spaceship on my space horse!". Now that you're a grown-up you don't do this very often, except if you engage in improvisational theatre. In particular, you don't do this while playing games on your computer. The basic idea of a digital game is that you play against or with a stable set of rules that is more or less predictable and therefore "fair"; if you fail (or things play out differently than expected) it was your own fault or simply bad luck. The game enforces these rules with such strictness that anything that works is correct. Any change in the rules during the game session (such as removing your jump ability or making your friends your enemies) is carefully scripted.
There are some attempts to undermine the status of rules in computer games, for example the game B.U.T.T.O.N (see Wilson's article) where the rules are underspecified. Players are required to invent new rules, which are not observable or enforceable by the game, as they play. However, what this game does is simply to reduce the role of the computer in the game, and mixing in more of classic non-digital play. The game does not express any desire on its own, and is even more of a mindless automaton than most of the classic games you play. This is of course unappealing from the perspective of an AI aficionado interested in technological progress, as is the fact that the game needs to be played in physical space with other humans.
Imagine that we could replace these people with AI, and integrate this system into the game itself. Imagine a game that is actually playful. A game that wants to play with you, not because it wants to fulfil some sort of goal (keeping you playing, or entertained), but because it wants to keep itself entertained, and explore the possibility space of the play session together with you. A game that treats you as a means, not as an end. That could refuse to play with you if you are too boring or hard to play with. That gives you the feeling of really playing with someone, not just against a ruleset.
Unfortunately, we don't really know how yet how to create a game that plays with you. How can we implement a desire in the game for having fun and a lust for exploring the possibilities of interaction with a player? How can we implement the concrete mechanisms where the game co-creates the rules with you and plays along, both constructively and obstructively? Actually, this is not unfortunate at all. It means that we have a new research problem for AI and games, and yet another angle of attack on the elusive problem of artificial intelligence. For we could surely not have any real artificial intelligence that was not capable of playing and enjoying itself.
One idea for how to approach this problem is Schmidhuber's theory of artificial curiosity, where agents choose what to learn so that they optimise their predicted learning rate. There is also related work in evolutionary and developmental robotics. Another strand of work which could potentially inform the development of games that play with you is work in mixed-initiative procedural content generation, such as Tanagra and Sketchaworld. In these systems, the designer and the software take turns to work on the game content, typically so that the software acts as an resourceful and inventive assistant to the designer. Last year, I published a paper outlining a simple system that tries to do mixed-initiative rule generation while you're playing the game. Or perhaps you're playing the system, and the game emerges from the play session. The system tries to understand what rules you're playing according to, and then enforcing these rules, which might force you to play differently (especially if it has misunderstood you) and see the flaws in your implicit design concept. What is missing from this system is intentions and desires on part of the system; the game is still trying to please you, rather than itself. I should get to work right away on changing this.
This blog post was inspired in part by Sicart's article Against Procedurality, and the opposition it sets out between rule-centric play/interpretation and player-centric play/interpretation. I have nothing against any of these, and think that both are interesting. However, I think that it's important to point that just because something is procedural, it's not necessarily rigid, predictable and intentionally designed; it could be adaptive, emergent and have a will of its own.
Thursday, July 14, 2011
Help us test new strategy game ideas
We have two new playtests/surveys going on, this time about strategy games. Those of us working on AI in the Center for Computer Games Research are interested in creating adaptive mechanisms for games, and automating the generation of various forms of game content. For strategy games, we are working on automatic map generation, automatic rule generation and AI techniques for competently playing strategy games with little domain information.
What we want from you is some help with evaluating the results of our recent endeavours.
The first survey is about automatically generated rulesets and general AI. If you take this survey, you will play a tutorial scenario and then two different (very short) scenarios in a strategy game you have never seen before. You will then be asked which one you prefer, which opponent AI was best, and a couple of other relevant questions. It will take you about 10 minutes. Click here to participate in the playtest survey.
You can also read the paper describing some of the previous research leading up to the system used in the current playtest. However, you are only allowed to read the paper after you've taken the survey. No cheating!
The second survey is about automatically generated maps for the classic real-time strategy game StarCraft. If you take this survey, you will be asked to look at ten different StarCraft maps and judge their relative qualities. It helps if you have played StarCraft, but it's not necessary. This survey will take you about 5 minutes to complete. Click here to take part in the StarCraft survey.
When you're finished, you could reward yourself with reading a paper on how the maps were created (though the current version of the system has evolved a bit). But only after you've taken the survey, or you'll bias the results...
Seriously though: thanks a lot for helping us with this!
What we want from you is some help with evaluating the results of our recent endeavours.
The first survey is about automatically generated rulesets and general AI. If you take this survey, you will play a tutorial scenario and then two different (very short) scenarios in a strategy game you have never seen before. You will then be asked which one you prefer, which opponent AI was best, and a couple of other relevant questions. It will take you about 10 minutes. Click here to participate in the playtest survey.
You can also read the paper describing some of the previous research leading up to the system used in the current playtest. However, you are only allowed to read the paper after you've taken the survey. No cheating!
The second survey is about automatically generated maps for the classic real-time strategy game StarCraft. If you take this survey, you will be asked to look at ten different StarCraft maps and judge their relative qualities. It helps if you have played StarCraft, but it's not necessary. This survey will take you about 5 minutes to complete. Click here to take part in the StarCraft survey.
When you're finished, you could reward yourself with reading a paper on how the maps were created (though the current version of the system has evolved a bit). But only after you've taken the survey, or you'll bias the results...
Seriously though: thanks a lot for helping us with this!
Tuesday, July 12, 2011
On the Game AI versus traditional AI debate
Luke Dicken wrote a very nice blog post the other day on the differences between how AI is viewed in academia (and among students) and in the game industry. He focuses on the real-time requirements of game AI, and how little processing power most games make available for AI. He also talks about how NPC behaviour in games needs to be entertaining, not just high-performing, and reminds the reader that in striking contrast to e.g. AI for robots, game AI is allowed to cheat if it makes the game better.
Kevin dill wrote a response post where he points out that "traditional AI" is simply trying to solve very different problems than those faced by AI in games. Far from being simplistic and primitive, the techniques devised specifically within game AI are well suited to their specific purpose: reliably providing interesting NPC behaviour, while being understandable and moddable by designers. Perhaps academic AI research should take a hint or two from industrial game AI, rather than the opposite.
I am not fundamentally disagreeing with anything Luke or Kevin say. I think they both make several good points. However, I'd like to point out that I have a rather different and, I think, broader perspective on what game AI is. Both posts make implicit assumptions on what games are and what AI is, which I think are limiting.
The main game examples used by Luke and Kevin are Red Dead Redemption, Dragon Age, Battlefield Bad Company 2 and Left 4 Dead. While these are impressive games, they are all representatives of a pretty small subspace of gaming: AAA first-person story-driven games with a real-time component and graphics that require them to run on a home console or computer. They are also all targeted at a classic "hardcore gamer" audience. These are the sort of games that we typically talk about when discussing games, and these are the sort of games that everybody wants to work on. But not the games that most people play. It's like if in the automotive industry, everybody would want to work on the next Porsche, while most people drive a Toyota.
Bejeweled, FarmVille and Diner Dash don't have a first-person perspective, don't have complex graphics and don't have a story in the same sense as the games above. Yet as far as I know, they have more players than those games. Importantly, they don't have NPCs that need to be controlled by AI, but still they present a number of interesting AI problems. Even traditional hardcore strategy games like Civilization, StarCraft or Total War present hard AI problems which are only insufficiently solved by the techniques used in the game industry.
The other limiting assumption is that AI is used for controlling NPC behaviour. In fact, this is only one of many applications for the bountiful toolbox of techniques found in artificial intelligence. AI techniques can also be used to generate game content (levels, maps, rules, puzzles etc), model players, adapt various aspects of the game (such as the difficulty or the reward schedule), match players in online games, control artificial economy, debug game mechanics or implementations, and so on.
Between all the myriad types of games out there and the multitude of interesting AI problems within them, I feel there's more than enough to work on even for an academic like me. Real-time pathfinding and planning for FPS and RTS games is all great, and I look forward to playing the results, but I'm happy to see someone else doing that specific work.
If you're interested in the "other" game AI work I'm involved, you might want to read our recent survey papers on generating game content and on adapting games based on player models.
Kevin dill wrote a response post where he points out that "traditional AI" is simply trying to solve very different problems than those faced by AI in games. Far from being simplistic and primitive, the techniques devised specifically within game AI are well suited to their specific purpose: reliably providing interesting NPC behaviour, while being understandable and moddable by designers. Perhaps academic AI research should take a hint or two from industrial game AI, rather than the opposite.
I am not fundamentally disagreeing with anything Luke or Kevin say. I think they both make several good points. However, I'd like to point out that I have a rather different and, I think, broader perspective on what game AI is. Both posts make implicit assumptions on what games are and what AI is, which I think are limiting.
The main game examples used by Luke and Kevin are Red Dead Redemption, Dragon Age, Battlefield Bad Company 2 and Left 4 Dead. While these are impressive games, they are all representatives of a pretty small subspace of gaming: AAA first-person story-driven games with a real-time component and graphics that require them to run on a home console or computer. They are also all targeted at a classic "hardcore gamer" audience. These are the sort of games that we typically talk about when discussing games, and these are the sort of games that everybody wants to work on. But not the games that most people play. It's like if in the automotive industry, everybody would want to work on the next Porsche, while most people drive a Toyota.
Bejeweled, FarmVille and Diner Dash don't have a first-person perspective, don't have complex graphics and don't have a story in the same sense as the games above. Yet as far as I know, they have more players than those games. Importantly, they don't have NPCs that need to be controlled by AI, but still they present a number of interesting AI problems. Even traditional hardcore strategy games like Civilization, StarCraft or Total War present hard AI problems which are only insufficiently solved by the techniques used in the game industry.
The other limiting assumption is that AI is used for controlling NPC behaviour. In fact, this is only one of many applications for the bountiful toolbox of techniques found in artificial intelligence. AI techniques can also be used to generate game content (levels, maps, rules, puzzles etc), model players, adapt various aspects of the game (such as the difficulty or the reward schedule), match players in online games, control artificial economy, debug game mechanics or implementations, and so on.
Between all the myriad types of games out there and the multitude of interesting AI problems within them, I feel there's more than enough to work on even for an academic like me. Real-time pathfinding and planning for FPS and RTS games is all great, and I look forward to playing the results, but I'm happy to see someone else doing that specific work.
If you're interested in the "other" game AI work I'm involved, you might want to read our recent survey papers on generating game content and on adapting games based on player models.
Subscribe to:
Posts (Atom)
