The year is 2027. Powerful artificial intelligence systems are becoming smarter than humans, causing chaos to the global order. Chinese spies steal American AI secrets, and the White House is rushing to retaliate. Within the major AI labs, engineers discover that models are beginning to deceive them, increasing the likelihood of becoming fraudulent.
These are not scenes written in sci-fi scripts. They are scenarios envisaged by nonprofits in Berkeley, California, and are called the AI Futures Project. This has been spent the past year trying to predict what the world will look like in the coming years as more and more powerful AI systems are developed.
The project is led by former Openai researcher Daniel Kokotajlo, who left the company last year over concerns that he was acting recklessly last year.
While he was in Openai, where he was on the governance team, Kokotajlo wrote a detailed internal report on how competition for artificial general information or AGI (a vague term for human-level mechanical intelligence) unfolds. After leaving, he worked with Eli Lifland, an AI researcher with a track record of predicting world events accurately. They worked to predict the next wave of AI.
The result is “AI 2027”, and the report and website released this week explain in a detailed fictional scenario, what happens when an AI system outweighs human-level intelligence.
“We expect AIS will continue to improve until by the end of 2027, it becomes a completely autonomous agent who is superior to all humans,” Kokotajlo said in a recent interview.
There is no shortage of recent AI speculation. San Francisco is seized by AI enthusiasm, and the Bay Area technology scene has become a collection of fighting tribes and shards.
Some AI predictions take the form of manifests, including a 14,000-word essay written by humanity chief executive Dario Amodei last year, or a 14,000-word essay written by Dario Amodei, a “situational awareness” report by Dario Amodei, a widely read in policy.
The people on the AI Futures project designed it as a predictive scenario. Essentially it is part of a rigorously studied science fiction that uses the best guesses about the future as a plot point. The group has spent nearly a year honing hundreds of predictions about AI and brought in author Scott Alexander, who writes the blog's Astral Codex 10, to help turn their predictions into stories.
“We tried to incorporate what we thought would happen and make it appealing,” Lifland said.
Critics of this approach may argue that fictional AI stories are better at giving birth than educating people. Also, some AI experts will undoubtedly oppose the group's central claim that artificial intelligence will overtake human intelligence.
Ali Farhadi, CEO of Allen Artificial Intelligence Laboratory, an AI lab in Seattle, said he was impressed after reviewing the “AI 2027” report.
“I'm all about prediction and prediction, but this prediction doesn't seem to be based on scientific evidence or the reality of how things are evolving with AI,” he said.
There is no doubt that some of the group's views are extreme. (For example, Kokotajiro told me last year that he believes there is a 70% chance that AI could destroy or devastating humanity. It has been giving disastrous warnings about AI for years.
But it's worth noting that some of Silicon Valley's biggest companies are planning a world beyond AGI, and many of the crazy predictions made about AI in the past look crazy, such as the view that machines pass the Turing test, and the thought experiments that determine whether machines can communicate like humans.
In 2021, before ChatGpt was released, Kokotajlo wrote a blog post entitled “What 2026 looks like,” outlining his views on how AI Systems will progress. Many of his predictions proved foresight, and he was convinced that this kind of prediction was worth it and that he was good at it.
“It's an elegant and convenient way to share your opinions with others,” he said.
Last week, Mr. Kokotajiro and Mr. Lifeland invited me to their office. This is a small room in a coworking space in Berkeley called Constellation, where many AI safety organizations hang pieces of iron.
Dressed in Tan's military style jacket, Kokotajiro grabbed the marker and wrote four abbreviations on a large whiteboard: sc>sar>siar>asi. Each of them explains and represents a milestone in AI development.
First, he said in early 2027 that AI will become a superhuman coder if there is a current trend. Then, by mid-2027, it will become a superhuman AI researcher. This is an autonomous agent that allows you to oversee a team of AI coders and make new discoveries. Then, in the second half of 2027 or early 2028, it becomes a close AI researcher. This is machine intelligence that knows more than we do about building advanced AI, allowing us to automate our own research and development and build a smarter version inherently. From there, he said it was artificial superintelligence, or a short hop to the ASI. All bets are off at that point.
If all this sounds fantastic… well, that's right. What Kokotajlo and Lifland are forecasting is not what AI tools today predict.
However, we believe that these blind spots will shrink rapidly as AI systems require sufficient coding to accelerate AI research and development.
Their report focuses on OpenBrain, a fictional AI company that builds powerful AI systems known as Agent-1. (They ruled out certain AI companies as single and instead decided to create composites from major American AI labs.)
Once Agent 1 is good at coding, it starts to automate much of the engineering work in OpenBrain. This will help the company move faster and help build Agent-2, an even more capable AI researcher. By the end of the scenario in late 2027, Agent-4 has made a year's worth of AI research breakthrough each week, threatening to be fraudulent.
I asked Mr. Kokotajiro what he thought would happen afterwards. For example, did he think life in 2030 is still recognizable? Will the streets of Berkeley be filled with humanoid robots? Are people texting their AI girlfriends? Are all of us doing our job?
He stared out the window, admitting that he wasn't sure. If the next few years go smoothly and we continue to control AI, he said he can imagine a future where most people are still the same, but a nearby “special economic zone” filled with highly efficient robotics factories will eliminate everything they need.
And what if the next few years don't work out?
“Maybe the sky is filled with pollution and people are dead?” he casually said. “That kind of thing.”
One of the risks of dramatizing AI predictions in this way is that, without caution, measured scenarios can turn into apocalyptic fantasies. The other is that by trying to tell a dramatic story that will catch the attention of people, you risk missing out on more boring consequences, such as scenarios where AI is common and not causing major trouble for anyone.
I agree with the author of “AI 2027” that powerful AI systems are coming soon, but I'm not sure that superhuman AI coders will automatically pick up other skills needed to bootstrap towards general intelligence. And we note predictions that assume that AI progress is smooth and exponential, with no major bottlenecks or obstacles along the way.
However, I think this kind of prediction is worth making, even if you don't agree with some of the specific predictions. If a powerful AI is really round the corner, we all need to start imagining a very strange future.
