Losing my religion

 In spring 2025 I had a crisis of faith. I thought about what the technology I'm helping to create might do to our future, and got scared. My worries kept intruding on my daily activities to the point where I needed to seek professional help. During this period, I tried to capture my worries with words. I wrote a text called “Losing my religion”, which I circulated among a small group of friends. But I never published it, because it was too dark. The lack of redemption arc jarred with my typical optimism and cheerful demeanor. Frankly, I worried that people would think I was going crazy.

I’m feeling better now. My fears are still there, but I have updated my belief system and am trying to turn what I feel into positive action. So I’ve decided to publish what I wrote last year, followed by some new commentary. I’m doing this partly because I think many others might be going through what I went through, and my perspective might help.

Losing my religion, as I wrote it in summer 2025

I never had a religion in any conventional sense of the term. You would have to go at least two generations back to find anyone in my family who regularly went to church or even believed in a god. My parents are artists. The reason I’m not one is not only that I like getting a paycheck every month, but also that it felt more meaningful to add to the sum of human technological and scientific knowledge.

You see, for as long as I can remember I have felt that the ultimate purpose of humanity is to deepen our understanding of the universe and increase our power over it. Advancing knowledge was what felt most meaningful. In other words, scientific, technological progress was my religion. I guess you could have called me a scientist, or a progressive, if those terms were not already taken.

I chose to become an artificial intelligence researcher mostly because I was good at it. AI was what my thoughts naturally gravitated towards. I liked thinking about thinking. I didn’t have the patience for maths, and computers made sense in a way that chemistry and biology did not. (Perhaps because computers are built by humans.) But it also seemed like AI was a good way to advance technology in general. In a sense, AI seemed like the ultimate technology, as it could potentially advance every other technology. When I started my PhD more than 20 years ago, working on AI seemed like one of the most important things I could do with my life. 

That was back in the days when practical successes in AI were rare. We had plenty of theories about how to create intelligence, but our methods only worked in specific circumstances, with much ado. Then things started working better. But that didn’t spark joy.

I was once excited when I learned of major advancements in AI. Gradually, I became more ambivalent, and these days I’m worried. Fearful, even. I wince when I see announcements of new models that break all the benchmarks. I find myself rooting for the problems rather than the solutions. As Leonard Cohen sang, “you loved me as a loser, now you’re worried that I just might win”.

I know that I am not alone in feeling this way. When I ask them in private, many of my colleagues say that they wish AI models would stop getting better already. That we could just have a break from this seemingly neverending onslaught of new capabilities. Many of them also say that working on AI no longer feels fun and exciting like it used to.

The specter that is haunting me is that we could one day have machines that could do all intellectual or creative work we do, as well as us or better. Many of us derive much of the meaning of our lives from our intellectual or creative endeavors. To some extent, it is what we live for. We want to matter. To those around us, and to humanity. Some of us are lucky enough to work with what interests us most, others have hobbies. But still, even those who are not inherently interested in what they work with tend to take pride in doing a good job.

Now, imagine that there was technology that could do what you do best, and do it better, cheaper, and faster. Whether it’s fiction writing, accounting, business strategy, psychotherapy, or interior design, the machine could do it at least as well as you. It would no longer make sense to pay you to do your work. To the extent you practice your art at all, it would be as a hobby or sport. Your output would always be judged by whether it is as good as the machine’s, and if it’s good, people might suspect that you just passed off the machine‘s output as your own. You would no longer be needed. Or, more accurately, your knowledge and skills would no longer be needed. 

Nobody wants this. At least, no ambitious person who has really thought about it wants this scenario to happen. Personally, I find the idea horrifying. It keeps me awake at night. This is not a figure of speech; I sometimes wake up at 3 am, heart pounding, from the dread of a future where human talent, knowledge, and even genius does not matter. Imagine that we might one day wistfully look back to the age of humans doing science, art, and engineering the way we now look back to a time where humans hunted their food with bow and arrow.

As an exercise, take some product of human ingenuity you really admire. Maybe it’s a poem, a scholarly book, an algorithm, a product design, a piece of architecture, or something else. Now imagine that there would be no point for humans to do things like this anymore, because software could do it faster, cheaper, and better. But please don’t overdo this exercise. I’m serious. I can’t stop thinking about this, and it’s been ruining my mood for months now.

Creative and intellectual redundancy is not only devastating on a personal level–the top levels of Maslow’s hierarchy of needs would forever be out of reach–but also on a societal level. Our civilization depends on us mattering to each other. Relying on each others’ work, admiring each others’ skill and creativity, and working hard to make ourselves better. Meaningful effort is the glue of society. A world where most of us do not meaningfully contribute would mean an atomized quasi-society at best. And history tells us that societies where large parts of the population are barred from contributing do not flourish. Most people do not simply accept fading into irrelevance. Nor should they.

Now, I don’t think the nightmare scenario of full human redundance will happen soon. Maybe (hopefully) never. The techniques we currently call “artificial intelligence” are just too different from us. It is a surprise to all of us that large language models work as well as they do; to me, it suggests that the linguistic layer of cognition is somehow thinner and less grounded in experience than we imagined. Therefore, this type of technology will probably always have a different way of doing things than we do, leaving room for meaningful human contribution. There are also more prosaic technical issues with current approaches that may turn out to be fundamental limitations. I keep hoping we will hit some serious roadblock soon. Maybe we will, maybe not.

I also hope that if technological progress happens at a manageable pace, this leaves room for society’s defense mechanisms to cocoon the intruding technology, making room for human agency around it. This may reduce most of us to mere operators or wrappers of AI technology, but that is better than being irrelevant.

Even if the nightmare scenario of full human redundance is not imminent, the fact that a considerable number of people claim that this is their goal is unsettling. This view is perhaps most succinctly stated by the startup Mechanize, which aims to “fully automate the economy”. As a participant in the economy, I find this goal deeply objectionable. I think that trying to fully replace all human labor is not a morally acceptable goal to have. It’s frankly horrifying. To be clear, “fully automate the economy” means “destroy society as we know it”. Unfortunately, similar goals are articulated by quite a few inside and outside Silicon Valley. I think the callousness of such objectives should be called out whenever they are encountered.

The great irony of this is that those who claim to want to automate all human labor are typically the kind of competitive, smart, high-agency persons who absolutely need to have a purpose and something to build. They would hate to be redundant. Yet, here they fly, like so many moths to a flame.

Yes, I have also said in the past that I wanted to develop AI systems that can do anything a human can do. But that was in the past. Back when AI did not work well, and few seemed to have a vision for going beyond specific, well-defined applications.

I guess you can see now why I am having a crisis of faith. I used to believe that deeper understanding of the world and greater technological capability was, in a sense, our purpose. But our greater technological capability might lead us to a world where we can no longer make a difference, and there is little point in us understanding more. Perhaps we get abundance, but at the price of redundance. Again: nobody actually wants this. And I, a mere AI researcher, am no longer sure about what to do. The thought of simply quitting the field has occurred to me, but I’m not sure that would make the world better, or even exorcise my personal demons. Besides, AI research is the one thing I know how to do well. And I need that paycheck.

So I’m left with the Cartesian task of trying to reconstruct my belief system, starting from what I really value.

Well, I believe in humans. I believe in individual human brilliance and creativity. But I also believe in society. I believe that we matter for each other, and that we must matter for each other. The world I want is one where we can all fulfil our individual potential, but where we need and admire each other.

First of all, throughout all of history we humans have adapted to and mastered the technologies we invent. Some may say we have merged with our technologies. We are so dependent on our technologies that the whole idea of us humans ourselves having general intelligence only really makes sense in the context of all the tech that we have embedded ourselves in. We feel so intelligent because we have built a world that empowers us so much.

So far, so good. But is not this new technology we call artificial intelligence fundamentally different from writing, steam engines, electricity, and telephones? It is. But these technologies were all very different from each other. Yet, we adapted to each of them. We learned how to build with them. Completely new domains of human excellence appeared that we couldn’t even imagine before new technology had made them possible. Because cars were invented, you could become great at designing, repairing, selling, racing, or writing about them. In 2025, you can also shine by streaming your Minecraft creations on TikTok. Who knows what new things you can be good at in 2045?

Then, we can note that there is a lot of scientific and technological progress that we actually want. I want us to be able to travel to the stars, cure all diseases, and make better video games. There are also so many things we don't know yet - every academic field has more questions than answers. The world is full of wonder. So let’s discover and invent all the things they have in Star Trek, minus the murderous aliens. But let's make sure we do the inventing and discovering, with the help of the machines. (It is also worth noting that Star Trek is so good because it always centers human agency and cooperation, which the showrunners accomplish partly by limiting what the computers can do.)

You may have noted that I am placing myself in a long tradition of people worrying about how new technologies will dehumanize us and ruin society someway. Your eyes may have been rolling over several times as you read this text. Basically every technology of note - from writing to telephones to video recorders - have come accompanied by commenters warning about how they will somehow end the world as we know it. Why would thinking machines be any different? I can only say that I hope that you are right and I am wrong. I really do. I want nothing more than returning to the optimism for the future I used to have.

When I'm in particular need of optimism, I look at my young children. They will come of age in a world different from mine. To them, whatever machine capabilities they grew up with will seem perfectly natural, just as desktop computing and early computer networks seemed to me. I hope they will have opportunities to contribute to society by doing something they are really good at, better than the machines. If not, they will barely have known any other world. Above all, I hope they will be happy.

Redefining my religion, summer 2026

I wrote the text above just over a year ago. Clearly, AI capabilities have not stopped advancing since. Quite the opposite. So how can I be less worried today?

Well, first I started going to a therapist. We were not a match, so I moved on and found me a psychiatrist, who I got on much better with. He prescribed me an SSRI, which has helped me get the peace of mind to focus my thinking. One thing we puzzled over together was why these worries gripped me now, given that the thoughts themselves are by no means new to me. I’ve worked on AI all my adult life, read boatloads of science fiction, and have a degree in philosophy. I remember thinking about the singularity and intelligence explosion concepts as an undergraduate in my early 20s, and how I already back then thought that the assumptions made in those arguments were so simplistic as to make them essentially meaningless. For most of my adult life, I’ve been an even-keeled person with a mostly sunny disposition. So why the anxieties now?

The triggering event in my case was probably an episode in 2024, where I was provisionally diagnosed with a very serious disease. The diagnosis turned out to be wrong, but for almost two months I lived with a fear of death that blanketed all my waking moments. When I laughed, it felt hollow. Moments of happiness were drained of meaning because of the ubiquitous thought that life could soon be over and all this be gone. I very distinctly remember standing outside the Mount Sinai hospital in Manhattan after learning that I did not have the disease they first thought I had, and that I was probably going to live a long and healthy life after all. I had no idea what to do with all my emotions. My psychiatrist suggested that this monthslong episode had awakened the fears I used to have as a child, and the worrying predisposition I had back then. Now that I was no longer acutely fearing death from disease, my fears needed something new to latch on to, and it was right there in front of me: artificial intelligence. The happy and stable person I had been all my adult life had turned into a worryer for mostly incidental reasons.

My psychiatrist also suggested, half jokingly, that it was useful for me to get my existential crisis in early, so I would be ready to help when others get to this stage. Not a bad idea, I thought. That’s why I’m writing this.

Let us talk about perspective. I have found that putting my worries in perspective is great for my sanity. Most people I know do not worry about AI rendering us intellectually irrelevant, although some do. Some do not really worry very much, others worry, but about other things. A close friend of mine is consumed by worry about climate change. Many others worry about the rise of fascism and populism, and the risk that the US and perhaps other countries turn into full-fledged autocracies. Some worry about war, nuclear and conventional.

I don’t mean to belittle these concerns. Climate change is real and is going to cost the whole world dearly. The political situation is not great in much of the world, and it’s not trending the right way. As a kid, I was in constant fear of nuclear war, and those nukes are still there. There are multiple wars around the world, including a major land war in Europe. Still, none of these very real issues grip me emotionally in the way intellectual obsolescence because of AI does. The latter is the only one that can keep me from sleeping.

Reading this, you might think that I am being ridiculous. How can I worry about AI making us intellectually redundant, but not worry as much about fascism, war, or global warming? You may also worry about something completely different that I haven’t even considered worrying about, such as the dominance of some particular ethnic or religious group, or vaccine conspiracies, or aliens. Or for that matter, you may worry that a superintelligent AI will break free and kill us all, which I also don't really worry about. The point here is perspective. Why do I worry about this particular thing, and not all the other things that other people worry about? There are so many things I could potentially worry about, and worrying all the time is clearly no way to live. That I‘m stuck on this particular issue and not something else is probably because I work in AI and drink from the firehose of AI discourse all day. I know, it’s not healthy, which is why I’ve made efforts to diversify my information diet.

I've found it particularly useful to read plenty of history, in particular on the various upheavals that have followed intellectual and technological change. Like with all of history, it is not always uplifting reading. But it does drive home the points that society changes at human pace even when technology changes much more rapidly (as also argued in e.g. Narayanan and Kapoor's excellent "AI as Normal Technology"), and that humans always keep coming up with new things to do. Also, for what it's worth, these changes tend to result in interesting art.

Why didn't I just quit AI research? My predicament would seem like that of a vegan butcher, or a monk who makes money on Onlyfans. But me quitting and becoming an Uber driver would not make the world better. The pace of AI progress would clearly not slow down noticeably. And I assure you, I still love AI research, even if I sometimes hate what AI does to the world. I'm not even very good at anything else. AI is what I do. And I think that I can do more good by trying to steer my field in a good direction than if I became an Uber driver. So I’d rather think of myself as akin to a hypochondriac doctor, or a pilot with a fear of heights.

What about my religion? Did I find my way back to it? Yes, but after a complete update to version 2.0. I still believe that the progress of science and technology is deeply meaningful. But it has become clear to me that this must be research done and technology invented by humans, for humans, and understood by humans. If humans are not in the loop and at the steering wheel, it is not progress. Of course we should use technology to help us, as we have always done, but it must be we who use the technology rather than the other way around. It does not matter how amazing progress we make if we are not the ones who make it.

To some of you readers, the paragraph above might seem trivial, almost like a tautology. Did it really take me 47 years to understand something so obvious as that science and technology must be for humans, by humans? Some kind of researcher I am. To other readers, I am just plain wrong. For example, at NeurIPS 2025, I voiced some critical concerns at a panel in AI for science. I wrote up these comments as a post called "please, don't automate science" which briefly made me Twitter famous in the wrong way. Basically, what I said was that cutting humans out of science would be terrible for humanity, no matter how much research could be sped up. Several hundred people (well, several hundred accounts) accused me of being wrong, stupid, and/or evil. Many thought that it was unavoidable that humans would be automated out of science. Some even thought it was desirable! To my critics, faster technological progress would be worth giving up all agency and even understanding. To them, I am anti-progress. To me, they are technologically defeatist or even anti-human.

I am in some sense a man of action, although action in my case often consists in typing words into a document. My parents, who are artists (I said this earlier, but the text is long and it might have fallen out of your context window by now) told me that the way they handle their fears is by making art out of it. So the best thing I can do about what scares me is the thing I know how to do: AI research, starting with the theoretical framework.

To that end, I've developed my thinking around what I call complementary intelligence. This is a continuation of a perspective I've developed over a while and written about in various places, including my Artificial General Intelligence book. The basic tenet is that there is not a single thing called intelligence; there is a highly complex set of behaviors and capabilities that have evolved to fit the particular ecological niche of each species. When it comes to humans, we have built our own niche; most of the things most of us interact with every day are designed by us, for us, to amplify our agency. That's why we think we are so intelligent.

From this perspective, the history of AI is a history of attempts to mimic the specific combination of behaviors and capabilities that humans have; most of them successful in some way, but all of them quite different to humans. The onslaught of LLMs becomes a push in a particular capability direction. Understanding that direction becomes crucial to figuring out which types of human intellectual patterns and capabilities will become more important in the future. Where the new domains of human excellence will appear. But this understanding can also help us develop different types of AI that are more complementary to what humans can do and like to do. Seeing intelligence as a scalar, where machines can overtake humans, is a recipe for paralysis; dissolving this faulty notion gives us the freedom to act. More of this argument in the article I linked above; what's important here is that there are things that can be done. Indeed, there's a lot to do. Such as building mechanisms for meaningfully incorporating humans in creative search processes, and open-ended learning and discovery processes that are not based on imitating what humans do.

But not everything has a technical fix. Norms, structures, and laws are probably more important. I still think that trying to automate humans out of the processes that give them, and our civilization, meaning is immoral. We need to build counter-narratives, and be vocal that "fully automating the economy" is not an acceptable goal to work towards. We also need to push hard to counter the centralization of power that so easily comes with lavishly funded tech companies trying to achieve monopolies on some layers of the AI stack. Our best bet for a future where we all matter is one with a myriad different forms of intelligence, open and accessible for all to use as tools for our natural intelligences. Let's get to work.

What if

What if the superpower of the future is self-discipline. The discipline to write enough code yourself to understand coding well enough to effectively outsource coding to agents. The discipline to learn math by proving theorems yourself. The discipline to write your own texts, so you learn to think. The discipline to pay attention. The discipline to sit with emotionally uncomfortable situations until you understand those emotions, yours and the other's. The discipline to read old long books. The discipline to repeatedly bang your head against the wall. The discipline to be bored. So that you can think beyond the obvious. Beyond the next token.

What if the way to become an AI super-user is to learn to resist AI almost all the time. 

What if the way to win at Chess is not to move the pieces as fast as you can. What if the way to find true love is not to enter a new relationship every week. What if Pythia only allowed you to ask one question.

What if the schools of the future require you to sit straight and write in your book with a graphite pencil, in total silence, for hours at a time. What if the mark of a successful person in that future is an unhurried demeanor and a steady gaze, lending you their full attention for as long as you deserve it. What if the capital of the future is not San Francisco.

Congratulations, you have reached the end of this post. Feel free to keep scrolling.

Complementary Intelligence

 In the following, I will try sketch a way of thinking about human intelligence and human nature through emphasizing its difference from the various methods and systems we call artificial intelligence. This is rooted in my strong belief that talking about a single-dimensional “intelligence” that one can have more or less of, and the obvious extension to asking whether humans or machines are “more intelligent”, is actively harmful for understanding both human and machine intelligence. What matters is the qualitative difference. Between humans and machines, but also between different approaches to AI. We can even think of the different approaches in a geometric framework, with the implication that any type of intelligence must have a direction as well as a magnitude. This perspective, which I call Complementary Intelligence, also suggests a positive research program, that seeks to find the types of intelligence that we do not currently have but that could be interesting to humans, rather than simply imitating human intelligence.

Ok, this was a lot. Let’s rewind the tape, and start by looking at the history of artificial intelligence and the ways we think about it.

You can understand the human mind by looking at the history of our failures at modeling it. For a very long time, we have tried to make machines in our image. After we invented the digital computer, this development sped up and we called it Artificial Intelligence. During the last 70 years or so, the AI research community invented a number of clever ways to make computers do things that so far only humans could. Usually, we set ourselves some problem – play Chess, prove mathematical theorems, translate from Russian to English, or something like that - that humans were good at. Then, we came up with some way of making a computer perform the task. Success!

But when we look closer, we find that the way the computer does the task is typically quite different from how humans do it. The computer may be much better than humans in some ways, and much worse in other ways, and in general just different. So we conclude that we didn’t really achieve “real” artificial intelligence after all. Maybe we were trying to solve the wrong task? So we find another task to solve, and another way of making the computer solve it, and try again. As a result, the study of artificial intelligence has contributed a wide range of technologies, many of which are crucial to our technological civilization. In our urge to talk about AI as a single thing, it is often underappreciated how many these technologies are, and also how different they are. Path finding, object-oriented programming, and optical character recognition are quite different things, but they are all outcomes of AI research. They are also in use in myriads of places and the world would grind to a halt if they disappeared.

Another result of this process is that we know more about what we are not. Every time we realize that the successful solution we have built is fundamentally different than us, we learn something about ourselves. We learn that we are not like that technology, or not only like that technology. However good that machine is at identifying traffic signs or playing Pac-Man, it does so in a fundamentally different way than we do it.

In a sense, this is just a continuation of our long history of using the defining technology of whatever age we are in as a metaphor or lens for understanding ourselves. Descartes, living in an age recently transformed by the mechanical measurement of time, thought of animals and humans as being like clockworks. Freud thought of our drives as producing something like pneumatic pressure requiring outlets, much like the steam engines that pulled trains and powered factories. The telephone switchboard was a popular metaphor in early 20th century neuroscience. But of course, if you try to actually build a mind that functions like a clockwork, a steam engine, or a telephone switchboard, you rapidly realize that there’s a lot missing. And from the incompleteness of the metaphor, you conclude that we are much more, and quite different.

Let us therefore try to sketch a history of AI focusing on not only its successes, but its complement: what we have learned about what we are not. This will by necessity be a very potted history.

The earliest successes of what is now known as AI were based on planning. This includes early Chess and Checkers players, and automated theorem provers such as Logic Theorist. The basic idea is to start at some state (such as a board position in Chess, or an axiom from which you want to derive a theorem) and consider the various possible actions available from there (moves in Chess, transformations in theorem proving). As considering all possible consequences of all possible actions recursively becomes computationally intractable for all but trivial problems, much of the art of planning is in the heuristics for which actions to consider at each point. Already in the 1950s we had theorem proving systems that could rediscover some previously discovered theorems much faster than humans, and in the next decades we saw major successes. In 1996 the Robbins conjecture, a long-open problem, was proven by a search-based theorem prover. Similarly, planning approaches led to superhuman play in classic board games such as Checkers and Chess.

In particular for board games, there has been quite a bit of work comparing humans to planning algorithms for game play. It turns out we are not alike. The algorithms explore many, many more potential moves and board states. Humans tend to explore just a few move sequences, but be much better at evaluating the positions.

In the 1970s and 1980s, expert systems were one of the main foci of AI research. The idea was to encode the knowledge of human experts in a form amenable to logical reasoning, and then let the computer do the reasoning rather than the human expert. Alas, it was not so easy.

Extracting the requisite knowledge from human experts turned out to be an enormous time sink. Humans, experts or not, seemed to have a hard time expressing what they knew. In particular, they found it very hard to express procedural knowledge (how to do things) in a way that could be formalized as rules. The finished systems often turned out to be brittle and inflexible, needing humans to look through decisions, which obviously severely limits the usefulness of the system. An often-used example is MYCIN, which was developed at Stanford in the early 70s to diagnose blood diseases. It took years to encode the 500 or so rules that the system used, and despite good performance in trials, MYCIN was never used in clinical practice.

The most reasonable explanation for the limited success of expert systems is that humans do not store their knowledge as a set of logical statements and rules. This might seem like a pretty obvious thing. Did anyone ever think that our brains operated this way? Surprisingly, yes. A long history of thought–ever since Aristotle–has postulated logics not just as a normative ideal about how we should think, but as a theory of how we actually think. More explicitly, the computer metaphor of the mind that become popular with the rise of cognitive psychology and the advent of cognitive science explicitly compares the functioning of the human mind to a standard computer, von Neumann architecture and all. The best interpretation of the relative failure of classic expert systems is that the computer metaphor cannot literally be true, at least at the level of how we encode knowledge.

Neural networks in one form or another underlie almost all modern AI. That much is generally known. Less often mentioned is that the earliest computer models of neural networks were proposed back in the 1940s, and the backpropagation algorithm that is the direct predecessor of the optimizers used in modern deep learning was invented in the 1970s. While there have been numerous minor and medium-size inventions in neural networks since, the remarkable success of the neural network approach is to a large extent due to us having more data and more compute so we can train larger networks. 

Much has been said about how neural networks mimic some features of human learning, such as learning hierarchies of representations. Less is said about how profoundly different they are to human brains. To begin with, there is no evidence of anything like backpropagation going on in the brain. This is reflected in how differently neural networks learn. In most settings, a neural network must see many more training examples than a human to learn the same concept. And when a concept is learned, it seems to be brittle. For any given model, it seems to be possible to find “attacks”, where changing a few tiny elements of the image completely throws the neural network, making it classify a panda as a gibbon or an abstract pattern of yellow and black as a school bus. Modern foundation models keep being susceptible to jailbreaks and prompt injection attacks. For all their proficiency at recognizing patterns, neural networks clearly do this in a different way than we do.

Similar things can be said about reinforcement learning. Seemingly miraculously, we can train neural networks to play games or control robots based only on feedback on their behavior. It’s astonishing that this works at all; it’s essentially trial-and-error on a massive scale. But why is such massive scale necessary? DeepMind’s classic experiments on learning to play Atari games with deep reinforcement learning saw each game being played for an equivalent of 38 days of game time. In contrast, a human can usually learn to play such games in less than an hour, sometimes in mere minutes. Of course, humans do this partly based on their familiarity with other games, as well as a lifetime of learning other visuomotor skills, from hopscotch to chopping onions. Artificial reinforcement learning systems are not good at this. Typically, they struggle to generalize beyond the narrow setting they have been trained on. Those networks that spent 38 simulated days to learn a simple Atari game? If you make a tiny change to the game, such as remapping the colors, or changing a few pixels here and there, they become utterly helpless.

There are other paradigms within AI that contextualize our own intelligence in other ways. Such as evolutionary computation. By (often crudely) mimicking Darwinian evolution, we can solve a large variety of problems. Evolution can come up with new designs for antennas, surprising but lucrative trading portfolios, useful software, and many other things. Evolution can also be used for supervised learning and reinforcement learning, often with more or less equally good results as the more commonly used gradient descent methods. But isn’t this weird? How can we get such good results through a completely different type of algorithm? Clearly, the currently dominant paradigm of AI is not the only way of solving the various problems we use AI to solve. 

The viability of evolutionary computation also reminds us about the perhaps greatest product of natural evolution: us. We are evolved beings, with an evolved culture. The main reason that we perceive ourselves as being generally intelligent is that we have built a world tailored to our shared cognitive capabilities. These capabilities have evolved over hundreds of millions of years. When we come to the world and start thinking, we are not blank slates; we build on an intricate neurophysiology and a vast repertoire of skills, instincts, and perspectives, some of which might at some point have helped our ancestors pick non-poisonous fruit, outwit crocodiles, or predict when the rain would come. In contrast, a machine learning model is quite literally a blank slate before training starts. Or rather, a blank matrix. Unlike all AI in existence, we are “trained” in a multi-timescale distributed process, encompassing our whole phylogenetic lineage as well as our whole culture.

Which brings us to present day. We now have large language models, and they are like the mind of god. At least according to breathless hypesters and accelerationists. More sober commentators still recognize that they are some of the most impressive technology we have ever seen, and they may well turn out to be almost uniquely consequential. We have all been humbled by LLMs doing something we didn’t think they could. Some of us multiple times. What are their shortcomings?

To begin with, they are good at tasks largely in proportion to how easily these tasks can be represented as strings. If the input is text and the output is text, chances are the LLM can solve the task very well. Modern multimodal models are also now very good at generating and classifying images, which are internally represented as strings of tokens. But spatial reasoning and interaction is another matter. Currently, huge resources are spent on trying to make these models confidently interact with graphical user interfaces. Granted, they are getting better at it. But they are still atrociously bad at, for example, playing video games. (Unless the game is very well known and you build an elaborate harness for it.)

It is likely that multimodal models will soon get much better at spatial interaction, at least for tasks that are economically relevant. The bigger issue is the lack of memory and continual learning. The current state of LLM memory is like the protagonist of the movie Memento, an amnesic man who can’t form new memories, and therefore has to write little notes to himself (or tattoo notes on his body) to remind himself who he is and what he is doing. This is because an LLM does not modify its parameters as you interact with it. All the little numbers that define it remain frozen in time. Instead, it keeps a short-term memory of its interaction in its context, but the length of this context is necessarily limited. To achieve something akin to long term memory, the harness around the LLM will at intervals summarize its context as a text file and store it away in a kind of database, which it can then access in the future. Rather like writing little notes to itself. This is likely to be a fundamental limitation of LLMs, not in the sense that it cannot be overcome, but in the sense that the solution will look quite different to an LLM as we know them today.

There are other ways in which LLMs differ from us which have not yet understood fully, because the technology is so new. For example, just like other forms machine learning, LLMs appear to have a strong bias towards problems that are in its training set. One way this manifests is a curious lack of novel insights stemming from LLMs recombining existing knowledge. Very much, if not most, of human creativity comes from recombining existing knowledge. Now, LLMs have a broader range of “expertise” than any human ever. Which actual human would simultaneously have detailed knowledge of peat bogs, pupillometry, polymerization, Pasadena, and Paul Krugman? In fact, frontier LLMs have had this staggering breadth of knowledge for at least 3 years, since GPT-4. A human with such range would surely make a stream of unexpected connections. Yet, few if any truly novel insights are directly attributable to LLMs. Why? We don’t know. We also don’t know whether this is a fundamental limitation of this approach to AI.

So far, we have only talked about intelligence in a relatively abstract information-processing sense. But we are not just brains, we are whole bodies. As you may have noticed, the way you think is strongly affected by whether you are hungry, horny, angry, or something else. And much of your thinking involves your body in some way, whether it is walking, tying your shoelaces, or typing on a keyboard. Some argue that all of your thinking is rooted in your body. Opinions diverge within cognitive science as to how important the body is to thinking. But what is plain to see is that physical robots are far, far behind non-embodied AI. Robots struggle to do things that are trivial for us, such as opening door handles. This is not a new issue: it has been the case for the whole history of AI, and much commented on.

Replaying the history of AI this way, we can sketch a different understanding of human intelligence and human nature than what we would get from using the AI we built as a metaphor for ourselves. More precisely, we can paint a picture of intelligence that emphasizes the parts which our AI systems are not good at, or which they do in a very different way to us. We can emphasize the complementary part.

Let us consider the difference. If we did use AI as a metaphor for our own intelligence, or as a lens for understanding it, similar to how previous generations used clockworks or steam power, we would arrive at what could be described as a rather classicist picture. Human intelligence operates by considering a large range of alternatives, tries to solve specific tasks that have well-defined rewards and can be clearly separated from other tasks, learns each skill on its own, starts from a blank slate when learning, and sees task descriptions and world descriptions largely as text. This picture has echoes not only of philosophies of past centuries, but also of modern management thinking and the kind of postmodern thinking which sees everything as a “text”.

The complementary intelligence view is instead that we are creatures that are deeply rooted in our history, both our evolutionary history as a species, our cultural history, and our personal history. Context is what we excel at. Most of what we do cannot easily be stated as separate tasks with well-defined rewards. We learn and reason slowly in terms of clock time, but effectively in terms of number of examples we need to see. We almost never operate according to logical rules, though we may tack them on as justification for what we did. Text is just one of our modalities, and somewhat “tacked on” compared to for example sight, smell, or proprioception. The body plays an important role in our thinking, and fine manipulation is another thing we excel at.

Neither human nor artificial intelligence is “general” in anything but a trivial sense, and could never be. The reason we believe we have general intelligence is that we live in a world we have constructed over the course of our civilization to fit our capabilities perfectly. Our societies, technologies, and built environments are scaffolding and support systems for our very particular type of intelligence. This makes us feel very smart and powerful. But thinking that the particular capabilities that the world we constructed test and amplify is all there is to intelligence is a very parochial view.

Looking at human intelligence this way gives us perspective on the rapidly advancing capabilities of AI. It is often asked when AI will overtake human intelligence. But this assumes that intelligence is a single-dimensional quantity. The various types of machine intelligence we have created can instead be seen as vectors pointing in different directions. Classic symbolic planning is one vector, LLMs are another, and fuzzy logic also another. Human intelligence is yet another direction. Moving further along in one direction (increasing the magnitude of the vector) may have limited bearing when projected on other intelligence vectors.

So, to directly answer the question about when artificial intelligence will surpass human intelligence: it did so long time ago, many times, and it never will. Various technologies that we refer to as artificial intelligence have surpassed humans at calculating, planning, solving logical puzzles, factual recall, and many other things. Yet, it is extremely unlikely that any technology would have exactly the same intelligence vector as human intelligence. Because these machines are not humans, their intelligence will always point in different directions.

Interestingly, the history of AI can be seen as a sequence of attempts at approximating the human intelligence vector. The moving goalpost phenomenon then becomes a game of finding a particular point on this vector, only describing it in one or a few dimensions, and trying to invent something that reaches this point. We then reach that point, only to discover that we did so by following a completely different vector than then we tried to imitate. So we find a new point, and repeat the procedure.

We can use complementary intelligence as a term to describe this view, but we can also see it as a positive research program. Complementary intelligence as a direction is, basically, to lean into the difference. We should not try to eliminate it, instead we should support it. Recognize the strengths of the human intelligence vector and build AI systems that amplify it.

At the same time, we should try to move away from trying to approximate the human intelligence vector. For example, plenty of AI researchers around the world are currently working on how to augment LLMs with continual learning, because they realize that’s not something that will be found along the intelligence vector of current LLMs. I think we should take our efforts elsewhere. Simply imitating human capabilities is perhaps the least interesting way of building artificial intelligence. It’s so unimaginative. And it leaves so many potential capabilities on the table. You could even argue that fully imitating human intelligence is immoral. We don’t actually want artificial intelligence that has all the capabilities we have, and is better at all of them. Because we want to matter. And we don’t want to be replaced.

Instead we should seek to amplify machines’ capabilities at tasks that we humans are not particularly good at, or do not want to do. In the best case, tasks that are not done at all, because we can’t or won’t do them, even though we might want to. We want AI that lets us focus on the things that we want to be good at, and give us abilities we didn’t have before.

To take some very quotidian examples: high-frequency trading is an example of complementary intelligence, because we can’t trade that fast. It is literally physically impossible for humans. AI deployed inside a video game, to generate levels, control non-player characters, or something like that, is another example of complementary intelligence, because you could not have humans implementing every NPC, and they would probably be very bored if they were asked to follow the rules that NPCs follow. AI methods for helping us make sense of modalities we are not attuned to, from WiFi reflection to gravitation fields are also good examples, as they expand our perceptual space. Then there is of course the boring but immensely impactful technologies that make the modern world possible and does not replace any cognitive work people actually want to do, such as databases and web search.

But beyond this, there is a virtual infinity of new types of intelligence we could develop, and new tasks we could discover, and new solutions we could invent. Taking the metaphor of intelligence vectors seriously, we could envision a hypersphere of capabilities on which every possible intelligence vector, at its maximum magnitude, could only reach a particular point. By definition, we have only explored an infinitesimal part of the interior volume of this hypersphere. There is so much more to do.

Most of these types of intelligence would likely be uninteresting and indeed incomprehensible to us humans and our society. But there is likely to be a practically infinite number of directions we could appreciate and build exciting new capabilities around if we first invented them. I don’t know what these capabilities would be, because they have not been invented. But I think the semi-automated open-ended search for new types of intelligence and associated tasks that would likely be of interest to humans to be the most exciting direction for AI I can imagine. This will definitely require new thinking about open-ended search, discovery of new ways of measuring search space, and clever measures of what humans find interesting.

As you can tell, there’s a lot to be worked out here. I’m thinking I should write my next book about Complementary Intelligence, so I get a chance to work some of it out. What do you think, should I?

Computers and me

I read things, write things, and talk to people. The proportions vary, but that’s essentially what I do. Or rather: those are my observable activities. I also think. The thinking often happens when I read, write, or talk, but also when I walk, drive, or take the subway or elevator. And when I shower! That exclamation mark! I should shower more.

Once upon a time I also programmed. I even considered that my craft, on par with writing. The last time I wrote non-trivial code was 2015. Long before that, I’d stopped keeping up with modern toolchains and software development practices, or even languages. That’s actually partly why I stopped programming: nobody uses Java for AI research or SVN for version control, and I think Python is unacceptably sloppy and git is incomprehensible.

Of course, the main reason I don’t program anymore is that I’m busy reading, writing, and talking. And thinking. I enjoy those things more. It’s not that I didn’t like programming: I enjoyed it a lot. And I was quite good at it. But there are lots of enjoyable activities you don’t easily find time for when you have two jobs and two kids. Even activities you’re good at.

Before I programmed, I took things apart. First, my toys. My room was full of useless thingamagogs that had once been part of fully functioning toys. I was no good at putting them back together again, or I didn’t have the patience. Or the interest. At some point, I graduated to computers, and built various PCs from parts I bought cheaply from flea markets or badgered my mom’s friends to give me. I destroyed a lot of those parts in the process. It was a lot of fun.

The PC-XT clone I bought with the proceeds from my first summer job (as a gardener) when I was 13 had a Turbo Pascal IDE on its 20 Mb hard drive. I decided to learn to program so I could make games. I copied and pasted things and tried to figure out what worked through trial and error. I learned a thing or two. Later on, I also spent a lot of time composing music on a 486 I built myself, and learned the basics of website building on the same machine. I never even fastened the hard drive to the chassis, and the computer had blinking lights and some kind of glitch so that you might get an electric shock from touching it.

These days, I don’t want to see the insides of my computers. I use Macs, and I want them pristine. No stickers, clean desktop, and no unnecessary applications. As few customizations as possible. It’s like I’m not even interested in computers anymore.

In sum, I’m a bad computer user. I do not let my computers fulfill their potential. Basically, I use the computer for reading and writing. Anything I actually use my computer for could be done on a 20 year old machine. If it could connect to the internet, I could do what I do on a 40 year old computer.

Yet, I keep buying new computers. I happily hand over my employers’ money to Apple in exchange for swanky new gear with waaaay more power than I need. And I don’t feel bad about it. I tell myself that I need an M5 Max with max memory so I can run local LLMs, and that is in fact a minor hobby of mine, but not really important to my actual work. Most of the time I use my computer for reading and writing emails, or reading papers or web pages, or having Zoom calls. My jacked monster of a swole M5 processor must be really bored.

I think I like computers mostly for aesthetic reasons. I’m like a rich old man who buys a Ferrari only to drive it around town and never exceed the speed limit. I just want to hear the menacing growl of the V8 and admire those aerodynamic lines. Except I’m not rich, and not that old, so I buy computers instead.

I’ve been thinking about this recently because computers are finally learning to use computers. Fat harnesses around frontier models help them navigate various applications, and this means you can increasingly just ask your computer to do things for you. Language models can also write code really well now, so you can (sometimes) conjure functioning new software just by calling it by its true name. Thus, it’s all the rage to make your AI agents do things for you. Writing code, reading reports, answering emails, other computer things.

Some seem to want to automate all of their digital life. Some seem to think it’s a good idea to install OpenClaw and give it root access to their computer and logins to all their computers. These kinds of people remind me of myself when I was 16, deeply into building weird things that rarely worked just for the sake of it, customizing every piece of software and interface because it’s cool, and caring not for safety nor security. I try to keep in mind that I was also once like that, because that allows me to understand these people. They just love technology in the way I once did.

Anyway. I am allegedly an “AI researcher”, a type of “computer scientist", and this comes, I think, with the obligation to at least occasionally act like one. So I try to use all these frontier models like I was Buffalo Bill. Often, it involves looking hard for some need I barely have that might be satisfied by a language model. This task is getting harder and harder. For what do I actually need? What should I use these things for?

A friend of mine suggested I vibe-code some unique software just for me. What kind of software, I asked. He said he had made some software for himself that keeps track of his exercise routine just the way he wants it. But I don’t want that! The point of going to the gym is to not have to care about such things, and instead put on the headphones and zone out while incinerating calories and letting the mind wander. Also, I don’t want to have to take care of maintaining a piece of software, even if it’s just for myself. Unnecessary stress. In fact, I want less software, not more. There’s far too much software in the world already. For any given need, there’s probably an app for that already, but I don’t want to have to look for it and I don’t want to install even more apps. That my local lunch restaurant has its own app and pesters me to install it is proof that too much software is being written.

What else could I have the models do for me? Write for me? But the whole point of my writing is that it’s mine. It’s not so much that it would be immoral to put my own name to something an LLM wrote (it certainly would), but that it doesn’t even make sense. It just wouldn’t be my writing. Please don’t tell me I need to explain this to you.

Could I have the models think for me? But I thought we already established that I am in this job because I like thinking. If you want to avoid thinking, you should not become an academic. Imagine going to a restaurant, ordering food, and then paying extra for the waiter to eat the food as well. That’s right, eating the food yourself is kind of the point. (My original metaphor was more striking, but this is a family-friendly blog.)

The more I think about it, the more the advent of AI agents has made me realize that I’m not much of a computer user. I don’t care for the vast majority of things you could make a computer do, and I don’t want to bother with new software if I can avoid it. Please don’t bother me with your buzzwordladen productivity catalyst. Give me a text editor and shut up already.

Maybe I would rather not even use computers. The computers can use themselves now, so maybe we can get on with our lives? Computers aren’t real anyway. So let me valet my shiny laptop. What I really need is some good books, some good friends, and a typewriter. And some good wine. So I can read, write, talk. And think.

Saving peer review from AI slop requires getting rid of anonymous submissions and reviews

The scientific ecosystem is struggling to deal with AI-written papers, and this is a great opportunity to revisit how we publish, where, and why. As many have noticed, a properly prompted modern LLM can produce complete papers that look like real science to qualified scientists in many fields. Yes, really. Whether these papers are actually correct, novel, interesting, insightful, and get their scholarship right will vary depending on the scientific field, the AI model, the observer, the type of paper, and of course the prompter. Better not get into specifics here, especially as the situation is evolving rapidly. The point is that it is now easy to produce what looks like good papers with little human effort. 

So, how do publishers, journals, conferences, professional societies, faculty admissions committees and peer reviewers (that is, you and me) deal with this? Not very well.

Scientific publishing is really badly configured for this challenge. For a while now, we have had a movement towards ever larger publication venues and a more anonymous process. I'll talk here about computer science, but I think the winds have been blowing in the same direction in other fields. The largest computer science conferences (such as NeurIPS, CVPR, and AAAI) now have many thousands of published papers at each conference, with tens of thousands of attendees and submissions. Reviewing is at least double-blind: reviewers don't know the identity of the authors, and authors don't know who the reviewers are. The area chairs, who make the first round of decision recommendations, also don't know who the authors are.

This is partly done for reasons of equality and fairness. There is this beautiful idea that anyone from anywhere in the world, from an elite university in Tamil Nadu or a rural high school in Tennessee, with or without powerful mentors, can just submit a paper and have it judged on its merits alone. As we all know, who you know and who knows you matters for your exposure. But the multiple anonymity principle was supposed to counteract this. It was also meant to make it possible to speak truth to power, so that high school student in Tennessee can point out the errors of my ways just as much as the professor in Tamil Nadu.

The concentration of academic publishing into ever larger venues, on the other hand, is probably mostly due to academic bean counting. It's genuinely very hard to gauge the strength of a researcher who is not in your own narrow specialty. But we often have to do that, because we need to decide on hiring and promoting researchers. So we need metrics. Citations are one such metric, though it has many problems, including that it takes time to get cited. Therefore it is common to look at the prestige of a publication venue. Conferences become prestigious by being very large and rejecting most submissions. There are other reasons as well why conferences have ballooned like this; I've written about this phenomenon and my dislike of it before.

The end results of this combination of idealism and economic incentives has led to a breakdown of scientific community. Think about it. Why would you, my peer, review a paper? You don't get any recognition for it, it's not worth mentioning on your CV (unless you're a masters student), and you certainly don't get paid to do it. The authors of the papers you review don't know about the effort you put in, and your parents, friends, spouse, and kids don't care. They just wonder why you are spending your Saturday afternoon tearing apart some paper you don't care about submitted by some people you may never meet instead of hanging out with your loved ones. You say you do it "for the community". Which community? You are just an anonymous cog in the machinery. 

This is not a theoretical concern. It has become steadily harder to find reviewers for at least a decade. (I've seen this from a bunch of different angles, including as a reviewer myself, as part of who knows how many committees, and as the editor in chief of a scientific journal of some repute.) In case of the larger conferences, they are basically vacuuming every nook and cranny for anyone remotely competent to review, including bright undergraduates. It is now common to require authors of papers at large conferences to review a number of papers themselves, as a kind of tax for submitting. Obviously, reviewers who are conscripted this way are not highly motivated to do a good job. The consequences of half-assing it are essentially zero. So the temptation is to just ask Gemini or Claude to write the review, changing a few words, and calling it a day and go out and play. Sticking it to the man. But the man, the machine, is the whole scientific system that we carry on our shoulders.

Into this already dysfunctional mess of a system enters a new factor: AI-written papers. En masse. If it's this easy to write papers, you can just bombard the system. Buy many tickets to the lottery. Peer review is so broken that some of them are likely to get through. And you're anonymous. If you get rejected, you never have to reveal your name.

Alright. How can we patch up this sinking ship? We must, because we are on the ship.

First of all, I am not arguing that we should ban the use of AI in the publication process. I think that in the future, most research will be done by humans with the help of an assortment of AI systems. In some cases, the AI systems might have contributed work that would have taken humans extreme amounts of time and effort to do; this is not in principle different from how computer-aided research has been done for decades. The exact amount and character of AI involvement will vary. But for any paper worth publishing, the whole text should have been written (in some revision) by a human, and checked (in its most recent revision) by a human. Here, the general principle of "don't make me read what you didn't write" applies. If the authors of the paper couldn't be bothered to write it, they are not publishing in good faith.

When you think about it, it is kind of odd that peer review works at all. We, and journalists, and to some extent the general public, take the fact that a paper is peer reviewed as a sign that it is true. But when we read the paper, we mostly just believe the statements in it. Sure, we hunt for really bad logic and bad scholarship, but we usually just accept factual statements of the type "algorithm X was faster than algorithm Y, p=0.04". What if the authors just lie to us? In some cases, we can run the code ourselves from an anonymous GitHub, but it's really quite rare that people do that. And running the code rarely answers all the questions. Instead, we just assume that the authors are honest people. Why do we this, again? Because the author are our peers? Are they?

Cue scene of a mortgage broker at Lehman Brothers circa 2007, fresh from buying billions in bad loans, rolling their eyes at the gullibility of the scientific community. Like, these professors and researchers with PhDs just accept the statements of anonymous people on the internet? And we thought they were smart? 

What it all comes down to is accountability. A human should be accountable for every piece of research, and stake their name that the research is theirs, produced in good faith, and, as well as they can judge, correct. This should apply at all levels of the publication cycle, at the time of initial submission as well as for the final published paper. As a reviewer, you deserve to know who wrote the paper, because you need to know whether you can trust them. Their name should be a key reason that you trust the paper. And when someone submits bad science, or lies, this should have negative repercussions on their name.

Getting rid of anonymity as the default can help save reviewing as well. Think about it. Why do you review, again? Because of some kind of abstract commitment to the scientific community. But, as we've seen, this is not working very well. What would work is to pay reviewers in the same currency as academics always get paid in: recognition. (Yes, academia is full of narcissists, the same way fields where you get paid real money are full of greedy people.) Simply attach the reviewer's name to the review, so they can brag about it. Even better, they will have an incentive to actually do a good job.

Sure, there would need to be some kind of anonymous review option, so that the child can point out the naked emperor. But it should not be the default. There is also the concern of "review rings", where authors coordinate the boost each others' work. I think those are best fought by shining a light on them. If submissions and reviews are public, people who boost each others' substandard work will look like the fools they are.

All of this relies on the idea that your reviews and paper submissions can have negative consequences as well as positive ones, if they are bad. You may object that this is unlikely as long as we are all atomized participants sampling from an almost infinite width stream of papers. If you see a bad paper with authors you've never heard of, you have no incentive to do anything about it. You'd rather just keep scrolling.

The solution to this is to actually take scientific community seriously. You should be making and defending your name in a community of at most a few hundred people. This is why primary submission venues should be of such size that participants who have attended for a few years can realistically personally know a large proportion of attendees. This is my experience with venues like IEEE Conference on Games, AAAI Artificial Intelligence and Interactive Digital Entertainment, and ACM Foundations of Digital Games. These are the conferences I prefer submitting my work to, and also going to. They usually have between 100 and 300 attendees. Unlike supersized conferences like NeurIPS, ICML, and IJCAI, fun-size conferences allow you to actually get to know the research community. I, like most other repeat participants, have my own opinions of who does research I care about, novel research, high-quality research, and so on. I think this is how it should be. Your own manageably sized research community should be the first port of feedback on and judgement of your research output.

This does not mean that you can't post your work online for anyone in the world to read. On the contrary, I very much think you should. But everyone does this anyway; in 2026, if someone writes a paper and does not submit a preprint of it in some publicly accessible place (such as arXiv, GitHub, or their own website) that's just weird. Maybe a little shady. As if they had something to hide. People should obviously keep posting their work publicly for their world to read, but approval by their own research community should be a strong signal that it is worth reading. And in a research community where people know your name, and you attach your name to your submissions and reviews, you can't get away with bullshit, AI-powered or not. In case the research community gets corrupted and starts letting its members get away with bullshit, the standing of the whole community would drop and people would cease to trust it.

Let's get back to the AI disruption. What if the machines get so good at research that they start producing papers that are actually good and novel? Then it becomes even more important that a human acts as owner and guarantor of the research. As I've (somewhat controversially) argued in the past, it is essential that we retain human control of the scientific process.

More generally, the fact that AI systems are getting better at various tasks with the research process is a good reason to re-examine the role of humans within the research process. And it is becoming ever more clear that research is a long game. AI systems excel at limited-duration tasks that can be clearly specified and evaluated. The rule of humans will increasingly shift into the really thorny stuff, questions without clear answer or evaluation. Such as: what research are you doing and why? Research is a long game. Gemini 3 has a context length of a million tokens, but your context length is your entire career. Your whole life, really, taking into account those childhood experiences that turned you into the weirdo you are, obsessed with whatever obscure questions you care about. In light of this, it's more clear than ever that the individual research paper is not the level at which your research should be judged. So let's make the scientific process more personal, relational, community-based, and human. 

Math and me

For most of my adult life, I was too cowardly to write this text, never mind posting it. I was worried about what people would think, and the repercussions on my career. Would people still take me seriously? But I’m now a whole full Professor of Computer Science at a top university, with all kinds of fancy metrics and titles to point to. Time to stop being such a pussycat.

Here’s the thing: I’ve always been terrible at math. How bad? Tell me to solve a quadratic equation, or differentiate something, and I would have no idea where to even start. I usually skip right past the equations when I read a paper because I don’t understand them. Last time I proved a theorem was approximately never.

I also always hated math. Not the abstract idea of math, but math as it actually exists. In particular, the activity of doing math, and trying to get stuff right. I hate math because I’m so bad at it, but clearly my negative feelings towards the topic is not helping me get better at math.

I almost failed maths in high school, and all my memories of math class in high school are of me staring out the window, talking to friends, writing weird stories, or programming my calculator. Anything to avoid those detestable math problems. During my undergrad, I had to take an introductory calculus class in order to take some computer science class I wanted to take. I failed the exam for that calculus class four times, and only passed on the fifth try because I realized that one of the professors was reusing his old exams with very minor changes. I learned basically nothing from that course. And not only do I not know how to differentiate anything, I also never learned things such as matrix multiplication or other parts of linear algebra that are supposed to be crucial for AI researchers like me.

Our PhD program requires my PhD students to take some theory courses that I’m pretty sure I couldn’t pass myself. I’m not even sure I could make it through our required undergrad theory courses. Some kind of computer scientist I am. The reason I could get a bachelors degree is that my undergrad is in Philosophy, though I did take a bunch of CS classes.

Which brings us to the question everyone asks, even though they often don’t believe my answer. The question is: how the hell can I be a successful AI researcher without knowing math? The implication is that I’m lying, or at least grossly exaggerating, because we all know that machine learning is very mathematical. It must be, because those GPUs are multiplying matrices all day. I’ll try to answer this below. Please bear with me, I’m trying to be as honest as I can here.

My first instinct is to say that mathematics is not important to the research I do. I never need to prove a theorem or even rewrite an equation. The details of how the matrices get multiplied don’t matter to me. I deal in ideas and code. Not math.

I remember when I taught myself programming using a Turbo Pascal IDE I discovered on the used computer I had bought when I was 13. As I blundered my way through the intricacies of Pascal, mostly by trial and error, I felt that a beautiful new world was opening up to me. It was hard, but I could learn it, and I had talent for it. Writing program code felt pretty much like writing natural language. And I was always good at writing. One of the things I learned about was variables. Some time after that, we were introduced to variables in school. I was excited, as here was a concept I actually knew something about! I was pleasantly surprised that I seemed to understand variables better than anyone else. But this didn’t help with the mind-numbingly boring stuff we did in maths class, all these exercises up and down the page.

In my undergrad, after two years of philosophy and psychology, I started taking computer science classes. I was naturally good at computer science. I understood the concepts and I became a cracked programmer. It was a lot of hard work but that was not a problem, because it was so fun. It was very different to studying philosophy, where I would just read the book and ace the exam. Mathematics, on the other hand, was all hard work and no understanding, and I couldn’t pass the exam at all.

In short, I was good at writing, philosophy, programming, and most aspects of computer science, and saw these subjects as intimately related. At the same time, I was terrible at math. So you may understand how I can see maths as largely unrelated to what I do.

And yet, I often use mathematical concepts when I talk about my research. Actually, when I do research as well. A recent project of ours focuses on embedding programs represented as syntax trees into a latent space that can the be searched efficiently. This involves considerations such as keeping the dimensionality of the space low enough to allow covariance matrix calculation and how to regularize the search to stay within the training distribution. That’s a bunch of mathematical terms there. And they mean something, because reasoning with them is how we got the method to work so well. But please don’t ask me to write down the equations.

So, how do I reason with mathematical concepts if I cannot do the symbol manipulation? Mostly visually. There are these little images of these things going on in my head, like a search blob moving against a gradient in a latent space. The images are somehow incomplete and clearly misleading–it is impossible to visualize a 128-dimensional space, so you have think of it as two-dimensional–but they are useful. But I also sometimes think of them in terms of program code, and the program code often comes out as animations, e.g. I see the program counter looping in a for-loop. It’s not clear to me how being able to to do the symbol manipulation (e.g. rewriting the equation for for the encoder function in some other form) would be of any help in reasoning about the algorithm. But that might just be because I don’t know how to do the symbol manipulation. If I did, maybe I would see new possibilities.

There are other uses of mathematical concepts which are possibly even fuzzier. A key skill in designing algorithms is understanding approximately how they scale in time and space. This basically boils down to figuring out what operations take time and which data takes space, and then having a mental picture of how many of them there are. Quite often, you’re counting loops. I learned the basics of doing this formally back in undergrad, but I haven’t done a formal analysis of an algorithm since. But I do loose, very informal analyses a lot when thinking and talking about algorithms. They help. But please don’t ask me to write them down.

Could it have been different? Could I have become the kind of person who was genuinely good at maths, enjoyed it, and perhaps even published papers with mathematical results of my own?  Who knows. The closest I ever came to thinking I understood math was during a discrete maths course in my undergrad, which I found myself actually enjoying, although it was a lot of work. For a little while I felt like math might actually be for me. I’m not sure if this was because of the topic, as discrete maths felt discontinuous with all the continuous maths I’d learned to not learn so far. Maybe it was mainly my very inspirational teacher, Thore Husfeldt. In either case, the feeling dissipated as soon as I encountered that analysis class, the one that I failed four times. 

As I write this, I keep fighting the impulse to brag about how successful a researcher I am. “Trust me, I’m a good researcher even though I don’t know math, see, I published so-and-so many papers and got so-and-so many citations and won this-and-that award.” I hate being that guy. So I’ll keep fighting that impulse. But it speaks to how deeply the impostor syndrome has taken root. Enough people have told me that I cannot possibly do what I’m doing without knowing a lot of math so that I’ve somehow think I can’t do what I do.

If you’ve read this far, you may wonder where I’m going. Who am I writing this for, and what am I trying to say? Let’s discuss some alternatives. 

I’m definitely not saying that you shouldn’t study math. If you like mathematics, go ahead and study it. It’s useful (I know) and beautiful (they say). I have a lot of respect for theoreticians and wish I could do what they do.

Another thing I don’t want to do is to blame my teachers. Maybe it was my teachers who taught me that math was boring and that I was bad at it. Maybe it was their curriculum they had to follow. Maybe it was me. Other people seemed to enjoy those same math lessons, after all. Dear teachers, thank you for trying to teach me; I don’t think you and I were good fits for each other, but that’s not your fault.

More likely, I’m writing this for those of my colleagues who are in the same boat as me, who somehow became successful computer scientists despite sucking at math. I’m like you, guys. We exist. I also write it for those of my colleagues who actually do know a lot of math, to explain how I work.

But I also write it for myself, because I genuinely don’t understand. Do I actually know a decent amount of math? I use those concepts all the time. But I certainly can’t solve any exercise problems. What does it mean to know math, anyway? I think the idea that you need to start from the basics and solve all those boring exercises to even learn about the more interesting concepts is male-cow-excrement. Or maybe that is one way of approaching mathematics, but far from the only one.

Most of all, I write for those who have been thinking of learning computer science, but are afraid to try because they don’t like math or are bad at it. You can certainly do it. You can become a very good computer scientist despite sucking at math. If anyone tells you that you can’t learn, say, machine learning because you don’t have the “mathematical fundamentals” tell them to go to Helsinki. In the winter.

There are some strong feelings involved here, and I should perhaps stop writing now before I get more explicit. And I should post this before I go back and re-read it and start toning it down. Better post it fresh and raw, like sushi.