The 2025 AI Index Report was released in early April by the Stanford Institute for Human-Centered Artificial Intelligence, — it’s eigth annual analysis of the impact, progress, and trends of AI. Led by an interdisciplinary group of experts from across academia and industry, the AI Index report aims to be the world’s most authoritative source for data and insights about AI.
“What a year 2024 has been for AI,” wrote the report’s co-directors Yolanda Gil and Raymond Perrault. “The recognition of AI’s role in advancing humanity’s knowledge is reflected in Nobel prizes in physics and chemistry, and the Turing award for foundational work in reinforcement learning. The once-formidable Turing Test is no longer considered an ambitious goal, having been surpassed by today’s sophisticated systems. Meanwhile, AI adoption has accelerated at an unprecedented rate, as millions of people are now using AI on a regular basis both for their professional work and leisure activities. As high-performing, low-cost, and openly available models proliferate, AI’s accessibility and impact are set to expand even further.”
“AI is no longer just a story of what’s possible — it’s a story of what’s happening now and how we are collectively shaping the future of humanity,” Gil and Perrault added. At over 450 pages and eight chapters, the 2025 AI Index is recognized globally as one of the most comprehensive and authoritative resources on artificial intelligence, referenced in hundreds of academic papers and used by policymakers and government agencies around the world.
- AI performance on demanding benchmarks continues to improve. In 2023, researchers introduced new benchmarks — MMMU, GPQA, and SWE-bench — to test the limits of advanced AI systems. Beyond benchmarks, AI systems made major strides in generating high-quality video, and in some settings, language model agents even outperformed humans in programming tasks with limited time budgets.
- AI is increasingly embedded in everyday life. From healthcare to transportation, AI is rapidly moving from the lab to daily life. In 2023, the FDA approved 223 AI-enabled medical devices, up from just six in 2015. On the roads, self-driving cars are no longer experimental: Waymo, one of the largest U.S. operators, provides over 150,000 autonomous rides each week, while Baidu’s affordable Apollo Go robotaxi fleet now serves numerous cities across China.
- Business is all in on AI, fueling record investment and usage, as research continues to show strong productivity impacts. In 2024, U.S. private AI investment grew to $109.1 billion—nearly 12 times China’s $9.3 billion and 24 times the U.K.’s $4.5 billion. Generative AI saw particularly strong momentum, attracting $33.9 billion globally in private investment — an 18.7% increase from 2023. AI business usage is also accelerating: 78% of organizations reported using AI in 2024, up from 55% the year before.
- The U.S. still leads in producing top AI models — but China is closing the performance gap. In 2024, U.S.-based institutions produced 40 notable AI models, compared to China’s 15 and Europe’s three. While the U.S. maintains its lead in quantity, Chinese models have rapidly closed the quality gap. China continues to lead in AI publications and patents.
- The responsible AI (RAI) ecosystem evolves — unevenly. AI-related incidents are rising sharply, yet standardized RAI evaluations remain rare among major industrial model developers. Among companies, a gap persists between recognizing RAI risks and taking meaningful action. In contrast, governments are showing increased urgency.
- Global AI optimism is rising — but deep regional divides remain. In countries like China (83%), Indonesia (80%), and Thailand (77%), strong majorities see AI products and services as more beneficial than harmful. In contrast, optimism remains far lower in places like Canada (40%), the United States (39%), and the Netherlands (36%).
- AI becomes more efficient, affordable, and accessible. Driven by increasingly capable small models, the inference cost for a system performing at the level of GPT-3.5 dropped over 280-fold between November 2022 and October 2024. At the hardware level, costs have declined by 30% annually, while energy efficiency has improved by 40% each year.
- Governments are stepping up on AI—with regulation and investment. In 2024, U.S. federal agencies introduced 59 AI-related regulations — more than double the number in 2023 — and issued by twice as many agencies. Alongside growing attention, governments are investing at scale.
- AI and computer science education is expanding — but gaps in access and readiness persist. Two-thirds of countries now offer or plan to offer K-12 CS education — twice as many as in 2019 — with Africa and Latin America making the most progress. In the US, the number of graduates with bachelor’s degrees in computing has increased 22% over the last 10 years.
- Industry is racing ahead in AI — but the frontier is tightening. Nearly 90% of notable AI models in 2024 came from industry, up from 60% in 2023, while academia remains the top source of highly cited research. Model scale continues to grow rapidly — training compute doubles every five months, datasets every eight, and power use annually.
- AI earns top honors for its impact on science. AI’s growing importance is reflected in major scientific awards: Two Nobel Prizes recognized work that led to deep learning (physics) and to its application to protein folding (chemistry), while the Turing Award honored groundbreaking contributions to reinforcement learning.
- Complex reasoning remains a challenge. AI models excel at tasks like International Mathematical Olympiad problems but still struggle with complex reasoning benchmarks like PlanBench. They often fail to reliably solve logic tasks even when provably correct solutions exist, limiting their effectiveness in high-stakes settings where precision is critical.
Science and Medicine
I would now like to further discuss the report’s chapter on the key trends in AI-driven Science and Medicine. This is an area I’m particularly interested in given my long been involvement with the use of supercomputers in scientific research.
As a physics graduate student at the University of Chicago in the 1960s, my research mostly involved the use of increasingly powerful computers in atomic and molecular calculations. Then a few decades later, in the early 1990s, I led IBM’s new parallel supercomputing initiative, as part of which I worked closely with leading edge users in business, government, and universities.
A recent issue of The Economist included a special focus on “How AI Can Revolutionize Science. The issue’s lead article noted that AI could help humanity solve some of its biggest and thorniest problems “by radically accelerating the pace of scientific discovery, especially in areas such as medicine, climate science and green technology.”
“AI tools and techniques are now being applied in almost every field of science, though the degree of adoption varies widely,” said the Economist. “AI is being employed in many ways. It can identify promising candidates for analysis, such as molecules with particular properties in drug discovery, or materials with the characteristics needed in batteries or solar cells.” In drug discovery, for example, after searching through millions of candidate compounds, an AI model helped find new antibiotics, salicin and abaucin, for use against two of the most dangerous known antibiotic-resistant bacteria. Once the AI developed a shortlist, the scientists tested them in the lab and identified their antibiotics.
Then a few weeks ago, I wrote a blog based on “Artificial Intelligence, Scientific Discovery, and Product Innovation,” a research paper by Aidan Toner-Rodgers, a second year PhD student in MIT’s Economics Department. Toner-Rodgers came up with a way of estimating the impact of AI on scientific research and product innovation by leveraging the randomized introduction of new, AI-based materials discovery tools to over one thousand scientists in the R&D lab of a large U.S. firm. The AI-assisted researchers came up with 44% more potential new materials, which led to a 39% increase in patent filings, and 17% more products prototypes based on the new materials. Research productivity increased by 13% to 15%. Overall, the new materials exhibited novel physical structures which in turn led to more radical innovations.
Each of the 2025 AI Index eight chapters starts out by highlighting ten notable AI milestones discussed in the chapter. Let me conclude by listing the ten notable milestones discussed in the Science and Medicine chapter.
- Bigger and better protein sequencing models emerge. In 2024, several large-scale, high-performance protein sequencing models, including ESM3 and AlphaFold 3, were launched. Over time, these models have grown significantly in size, leading to continuous improvements in protein prediction accuracy.
- AI continues to drive rapid advances in scientific discovery. AI’s role in scientific progress continues to expand. While 2022 and 2023 marked the early stages of AI-driven breakthroughs, 2024 brought even greater advancements, including Aviary, which trains LLM agents for biological tasks, and FireSat, which significantly enhances wildfire prediction.
- The clinical knowledge of leading LLMs continues to improve. OpenAI’s recently released o1 set a new state-of-the-art 96.0% on the MedQA benchmark—a 5.8 percentage point gain over the best score posted in 2023. Since late 2022, performance has improved 28.4 percentage points. MedQA, a key benchmark for assessing clinical knowledge, may be approaching saturation, signaling the need for more challenging evaluations.
- AI outperforms doctors on key clinical tasks. A new study found that GPT-4 alone outperformed doctors—both with and without AI—in diagnosing complex clinical cases. Other recent studies show AI surpassing doctors in cancer detection and identifying high-mortality-risk patients. However, some early research suggests that AI-doctor collaboration yields the best results, making it a fruitful area of further research.
- The number of FDA-approved, AI-enabled medical devices skyrockets. The FDA authorized its first AI-enabled medical device in 1995. By 2015, only six such devices had been approved, but the number spiked to 223 by 2023.
- Synthetic data shows significant promise in medicine. Studies released in 2024 suggest that AI-generated synthetic data can help models better identify social determinants of health, enhance privacy-preserving clinical risk prediction, and facilitate the discovery of new drug compounds.
- Medical AI ethics publications are increasing year over year. The number of publications on ethics in medical AI nearly quadrupled from 2020 to 2024, rising from 288 in 2020 to 1,031 in 2024.
- Foundation models come to medicine. In 2024, a wave of large-scale medical foundation models were released, ranging from general-purpose multimodal models like Med-Gemini to specialized models such as EchoCLIP for echocardiology, VisionFM for ophthalmology, and ChexAgent for radiology.
- Publicly available protein databases grow in size. Since 2021, the number of entries in major public protein science databases has grown significantly, including UniProt (31%), PDB (23%), and AlphaFold (585%). This expansion has important implications for scientific discovery.
- AI research recognized by two Nobel Prizes. In 2024, AI-driven research received top honors, with two Nobel Prizes awarded for AI-related breakthroughs. Google DeepMind’s Demis Hassabis and John Jumper won the Nobel Prize in Chemistry for their pioneering work on protein folding with AlphaFold. Meanwhile, John Hopfield and Geoffrey Hinton received the Nobel Prize in Physics for their foundational contributions to neural networks.
Comments