“Transformative AI will generate a genius supply shock: abundant, cheap, and fast agents that can outperform human beings across many domains. But society is likely to adapt too slowly to this remarkable but unfamiliar new capability,” wrote University of Toronto professors Ajay Agrawal and Joshua S. Gans in the introduction to their essay, “Transformative AI and the Increase in Returns to Experimentation: Policy Implications.” Their essay, published in Volume 2 of The Digitalist Papers, explores policies “that can help companies, regulators, and individuals learn how to use these powerful new tools and put them to effective use.”
What Is the Genius Supply Shock?
The authors cite a few concrete examples of what they mean by a genius supply shock. “Humanity’s Last Exam” is a test developed over the past couple of years by a team of researchers who claim it’s the hardest test ever administered to an AI system. The exam includes roughly 3,000 multiple-choice and short-answer questions designed to test AI systems’ abilities in areas ranging from analytic philosophy to rocket engineering.
“Unlike traditional AI evaluations, which test for narrow capabilities in isolated tasks, this benchmark simulates the challenge of a PhD qualifying exam merged with a generalist’s oral defense,” wrote Agrawal and Gans. “Questions are long-form and open-ended. To score well, an AI must not only know, but understand. Success requires what we typically associate with our highest-functioning minds: flexible reasoning, conceptual abstraction, and the capacity to transfer knowledge across domains. Until recently, no machine had come close to passing.”
In 2024, researchers gave the Humanity’s Last Exam to six leading AI models. “All of them failed miserably,” said a NY Times article. The AI model with the highest score got only 8.3 percent correct answers. The percentage of correct answers improved significantly over the next two years, with a few models achieving accuracies above 40 percent, — putting them within striking distance of high-performing postgraduate scientists.
In their essay, Agrawal and Gans noted that the advances realized in 2025 “were anticipated by, among other experts, Dario Amodei, cofounder and CEO of AI foundation model company Anthropic, who described forthcoming systems as providing a country of geniuses in a datacenter.” Each such genius, Amodei, added, would be “smarter than a Nobel Prize winner across most relevant fields — biology, programming, math, engineering, writing, etc.” These systems do not simply automate routine tasks, he speculated; they synthesize knowledge across domains, propose and critique solutions, and do so at digital scale. Unlike human experts, they are cheap, abundant, and tireless. (more…)