I’m currently in the process of recruiting a whole new research lab, as I’m moving to a new university. I’m sifting through several hundred PhD student and postdoc applications in order to hire a handful of people initially, and more soon. The quality of the candidates is excellent, and several of them have impressed me deeply in interviews. (It also makes me wonder how I ended up getting a PhD position back in 2004; I was way less prepared then than my top candidates are now.) The selection work is hard but fun, as work should be.
I’m doing this against a backdrop of things moving fast, to say the least. Two years ago AI agents were mediocre at coding and terrible at maths; today they are, in some senses, superhuman at both. You can now simply give an AI agent a research idea, and it goes away and does the work and gives you a finished paper. You can even ask the damn thing to come up with the research idea itself. In my field and experience, the result is typically a fairly solid but also quite boring paper, reporting some incremental progress on some minor problem with good experimental statistics but terrible writing. Nothing that would win any awards, but the kind of work that used to be a respectable first paper from a PhD student. Now, you can get it with a prompt. And, again, things are moving fast. I’m not sure how much the writing abilities of AI agents will improve, but their ability to design and execute complex research pipelines is likely to be much better next year, and then next year again…
What does this mean for an incoming PhD student? And what does this mean for a PI, responsible for hiring students and postdocs, setting research direction, managing a lab, and advising students, which means being everything from domain expert to brainstorming partner to secretary to therapist? I will try to reason about this here. My field is AI for games/creativity/open-endedness, but I guess what I say will apply to other fields to some extent.
Let us first agree on what a PhD is supposed to be about. During a PhD you learn to do research, meaning that you learn to produce or uncover new knowledge. New, as in new to humanity. The way you learn this is through a form of apprenticeship: you do research on your own and together with others, advised by one or more senior, experienced researchers. Typically, you are part of a research group led by a professor who is also your advisor. In my experience, your advisor and research group, and how well you get along with them, matter a lot to the quality of the research you do and your general enjoyment and mental health. Much more than the prestige of the university.
In general, you develop skills, factual knowledge, and understanding during a PhD. As a viral tweet stated, you can outsource thinking but not understanding. I think this is an excellent observation. Understanding is what really matters. The more work AI agents can do, the more you need to understand in order to guide them. Also, understanding is arguably the main goal of science (as opposed to technology). Still, skills, factual knowledge, and understanding cannot be easily disentangled. Implementing an algorithm brings not only skill, but also a level of understanding that you cannot get from simply importing it, or having an agent implement it for you.
There is a phenomenon which I call understanding debt, by analogy to technical debt. Understanding debt is when you have had your agents do a bunch of stuff for you, and you no longer really understand what’s going on. Then you’re faced with the prospect of either trying to understand in depth what’s happening, or just going on anyway. But if you go on anyway, you will notice that the quality and relevance of your inputs drop, because you no longer understand enough to make informed decisions. This is something that I think we will all need to learn and re-learn as agents get better.
I’m not saying I’ve solved this for myself. I have various half-finished projects sitting around that will probably never get published because I am not ready to put in the work to understand them well enough to write them up. Writing them up is a good test; you need to really understand a project to be able to write a good paper about it. And you should definitely write it yourself, at least the first draft. If you weren’t interested enough to write it, why would anyone be interested enough to read it? Pushing slop on your peers is rude. On the other hand, using agents to critique and polish your drafts is useful.
Which brings us to a key question for a beginning researcher, just like for everyone else in a knowledge-intensive profession: When and how should you outsource your work to AI agents? The answer is not “as often as possible”.
Why not? Because your effort and your choices matter. You can see this in project-based classes, as well as in workplaces which use AI heavily. Even when everyone has access to the same AI tools, the quality and character of people’s contributions vary widely. (Not so much the quantity, because it’s so easy to produce things now.) Some students in my classes turn in barely adequate work, whereas others repeatedly shine. Typically, those who turn in the best work also understand it the best. Just like it’s always been; access to amazing tools does not erase the value of talent and dedication. Perhaps it even amplifies it.
Here’s another reason: Just because you can get to lots of places rapidly and easily by flying does not mean that you should never drive, or bike, or even walk. There is lots to be gained by seeing the landscape up close. If your idea of visiting a country is to go only to the major cities with international airports, you are going to get a very skewed idea of that country. Most of the US is really not like New York, and most of Sweden is not like Stockholm. From the car window, you can see farms, factories, and small towns; you can stop to talk to people and get their views. If you walk or bike, it takes even longer to get to your destination, but you’ll notice way more details. You’ll also be able to take little detours, perhaps find new paths that aren’t even on the map yet. Perhaps you will be the ones to put these paths on the map, and lay the first stone for the airport, so that people can one day fly there. It’s exciting to be an explorer! In fact, as a researcher, it’s your job to take the roads less traveled. Anyone can buy a ticket to the big swanky international airports and sip espresso in a lounge. It’s your job to do the uncomfortable and uncertain: make your way through the jungles/mountains/swamps/ruins so that we can go somewhere we couldn’t go before.
Nothing wrong with getting carried away by a metaphor. In fact, it can be a good way to ideate. See for example bio-inspired algorithms, where a lot of good ideas have come from taking biological metaphors very seriously. A lot of bad ideas as well. Luckily, we can run experiments to help us tell the bad ideas from the good ones. Running experiments is now easier than ever, and you will be doing a lot of it as a PhD student, to test your ideas and develop your understanding.
Given how easy it now is to implement and test ideas, many researchers will probably be spending their time in tight loops with agents, feeding them ideas to test, and trying to make sense of the results as they come in. I personally think it is important to also spend plenty of time offline reading and writing before handing instructions to the agents, but research and thinking styles vary. We are all learning and re-learning how to work with these new tools available to us, and I think it is important to also experiment with how we work.
I think PhD programs in technical subjects are going to become more similar to art schools, in the sense that the focus will increasingly be on developing your own style of asking and answering questions. And of course your taste in questions. During an MFA (Master of Fine Arts, typically the terminal degree in an art school) the student learns to develop and express a cohesive artistic identity, a process which includes plenty of conceptual argumentation. Craft, in contemporary art, is often taken as a given, or as something that can be outsourced. Interestingly, this describes successful senior researchers as well: they are known mostly for their artistic identity, as the actual craft of their research is largely carried out by their postdocs and PhD students. The artistic identity of the PI is what gives their lab a research direction, and attracts attention, students, and funding. What has changed recently is that outsourcing of the craft of research (to AI agents) can happen further down the line; the PI might now need to spend more time and effort supporting the development of their PhD students’ researcher identities and research taste rather than technical skills.
Now, some people might disagree with how I paint research as akin to art. Science, they would say, is about truth; however great Picasso, Virginia Woolf, or Charlie Parker were, they were not searching for truth. I would rather set aside the complex question of whether art seeks truth, and instead emphasize that many of the greatest scientists and inventors operate with something like an artistic worldview, with their taste in problems rooted in visions only they fully understand, based on their idiosyncratic combinations of interests and understandings. See: Einstein, Ramón y Cajal, Lovelace, McClintock, Feynman, Tesla. Artists. But they helped uncover new truths, because they generated ideas that could be tested experimentally. With AI agents potentially automating so much of research execution, even junior researchers could have the resources to develop their unique vision.
Things will change when it comes to the rituals, norms, and technical infrastructure that govern research dissemination and collaboration. Take academic publishing. People have been complaining about the academic publishing machinery for a long time now. Well, good/bad news: AI agents seem to be pushing it to some kind of breaking point this time. As it is trivial to just generate new papers that may or may not have scientific value, conferences and journals are getting flooded by shameless papermaxxers and jolly amateurs. Nobody wants to review this flood of slop, so people just ask AI agents to do it for them.
Reviewing will have to evolve, taking into account that AI agents are now very good at many aspects of the review process, including replication and fact-checking. The human part of the review process might deal more with critical commentary and judging what is interesting, perhaps with the fact that people chose to review your paper at all becoming an important signal. Yes, this is another way in which technical research may come to resemble arts more.
And yes, your papers need to be interesting. Your job as a researcher is not just to produce research, but to produce interesting research. This remark may seem trivial and obvious to some, and borderline blasphemous to others. There are those who think that human value judgment should be minimized in the research and publication process. Papers should be accepted purely on technical merits in an as-blind-as-possible process. I think we should go the other way. As more and more of the research process can be automated, the scarce resource is your own vision, narrative, and personality. There are infinitely many things you can do at any given point, and infinitely many questions you can address. Your job is to choose the next step to take towards some goal that perhaps only you understand. Then, your job is to explain to the world what you are trying to do and why that is important.
It is sometimes said that you should write the novel you want to read, make the music you want to listen to, or paint the paintings you want to see. Good advice, methinks. Well, you need to be capable of doing the work as well; some authors would rather write like Shakespeare or Dostoyevsky, but find success writing teen soaps or movie tie-in novels. As for myself, I’m neither Turing nor von Neumann, but I’m doing alright in my niche. Anyway. You should write the papers you want to read. To do that, do the research you are interested in. In general, you must do work that interests you.
This leads to an interesting problem: how do you develop your interest, so that you find the right things interesting? You must know things first to find new things interesting. I would not be interested in measures of skill depth through agent differentiation, tradeoffs of tokenizer design, or arguments against functionalism in the philosophy of mind if I had not spent a lot of time reading, writing, thinking about, and even experimenting in those domains. Did I simply follow my own interests all the way there? No, not simply. Sometimes I’ve done and read really tiresome things because I thought it would advance my interests. And by sometimes, I mean often.
Now for the most obvious thing I’m saying in this text: to be an interesting person, with interesting interests, you must be interested in multiple different things, and be able to see the connection between them. Because that’s where the magic happens, and that’s what makes people want to talk to you at the party. (I’m talking about parties where you don’t need to shout, civilized ones where you sit in armchairs and sip good wine.) Look, there is a near-infinite number of things to know in the world. But humanity only knows a small sliver of these things, and a good LLM in a good harness knows a shockingly large proportion of that sliver. But the number of combinations of things you can know is vastly larger than the number of things there are to know (elementary combinatorics), and it is entirely possible to have a unique set of specialized knowledge, and a unique combination of interests. This means that you, by seeing the connection between your unique set of interests, can take research in a unique direction. If you only studied the standard CS curriculum and your interests are all discussed on AI Twitter, then you need to broaden your horizons. Go to a different section of the bookstore, buy some books you wouldn’t normally think of reading, and read them. Talk to people on the train about what they do at work all day (most people are very happy to talk about their work if you ask them, and their jobs are way more diverse than you think). Talk to your family.
As I’m apparently in the habit of saying obvious things, let me continue down that path. Obviously, hard work matters, just like it always did. There is a certain allure in simply sending off tasks to agents, feeling good about having “done” something, and calling it a day. But then you are not adding much of yourself. And you’re not learning much; there pretty much needs to be friction for effective learning. When I recruit for my lab, I’m looking for the people who run towards friction, those who actually like to learn and to put in the effort. People with a high need for cognition. This is as important now as it ever was, but the AI era has led to a nice litmus test, as some people like to say things like “my work is so easy now that AI can do it for me”. Those are people I don’t want to hire as PhD students, or for any other intellectual job. Work should be hard but fun.
Let me now try to summarize what I’ve been writing as terse advice: Be yourself. Be unique. Become an interesting person by broadening and deepening your interests. Do hard work, seek out the friction that leads to learning. Do things that interest you, write things that interest others. Understand, formulate, and develop your identity as a researcher, which is much like being an artist. Seek to do the research that only you would do, in your own style. Learn how you best work with these marvelous new things we call AI agents by thorough experimentation, and how to manage understanding debt. Read a lot, and always write the first draft of whatever you’re writing yourself. If it’s not worth your time to write, it’s not worth anyone else’s time to read.
Note that while I very much think we should all experiment with AI-enhanced workflows, I am emphatically not saying that we should, in Gretzky’s words, simply “skate to where the puck is going to be”. Some Silicon Valley people believe they know what the future looks like, and that that future looks like us replacing our own thinking with AI systems as much as we can. Well, Silicon Valley is an incestuous monoculture; listener beware. Nobody knows where the puck will be. The puck might be like a wave function that never really collapses; there might be lots of pucks. And we are not passive observers, we can steer the puck by methodically scrubbing the ice. A bit like how we can steer an extended sports metaphor from ice hockey to curling.
Finally, remember that we’re all in this together. AI agents are changing things rapidly, and we are all trying to figure out, and shape, the new rules. Welcome to the club! Let’s get to work.
No comments:
Post a Comment