Irving Wladawsky-Berger

How Can Companies Figure Out How to Best Apply Generative AI?

“Business leaders are struggling to understand how seriously they should take the latest phenomenon in the world of artificial intelligence: generative AI,” wrote Andrew McAfee, Daniel Rock, and Erik Brynjolfsson in their recent Harvard Business Review (HBR) article “How to Capitalize on Generative AI.”  

“On one hand, it has already displayed a breathtaking ability to create new content such as music, speech, text, images, and video and is currently used, for instance, to write software, to transcribe physicians’ interactions with their patients, and to allow people to converse with a customer-relationship-management system. On the other hand, it is far from perfect: It sometimes produces distorted or entirely fabricated output and can be oblivious to privacy and copyright concerns.”

“Is generative AI’s importance overblown? Are its risks worth the potential rewards? How can companies figure out where best to apply it?,” the authors asked. To illustrate how to best answer these questions, the article references a recent research paper by Brynjolfsson and two collaborators, Generative AI at Work, which shows that there are ways to both reap the benefits of generative AI and contain its risks.

How Can AI Accelerate the Pace of Scientific Discovery?

A recent issue of The Economist included a special focus on “How AI Can Revolutionize Science,” with three articles on the topic. “Debate about artificial intelligence (AI) tends to focus on its potential dangers: algorithmic bias and discrimination, the mass destruction of jobs and even, some say, the extinction of humanity,” noted the issue’s lead article. “As some observers fret about these dystopian scenarios, however, others are focusing on the potential rewards. AI could, they claim, help humanity solve some of its biggest and thorniest problems. And, they say, AI will do this in a very specific way: by radically accelerating the pace of scientific discovery, especially in areas such as medicine, climate science and green technology.”

“Could they be right?,” asked the The Economist, reminding us that in the 1990s we had high hopes that the internet would reduce inequality and eradicate nationalism by transforming the world into a connected global village. Instead, internet-based social media platforms have been blamed for fueling political and ethnic polarization around the world.

“But the mechanism by which AI will supposedly solve the world’s problems has a stronger historical basis, because there have been several periods in history when new approaches and new tools did indeed help bring about bursts of world-changing scientific discovery and innovation.”

Winners and Losers in the Emerging Age of AI

“The development of AI is as fundamental as the creation of the microprocessor, the personal computer, the Internet, and the mobile phone,” wrote Microsoft co-founder Bill Gates in “The Age of AI has begun,” a March, 2023 blog. “It will change the way people work, learn, travel, get health care, and communicate with each other. Entire industries will reorient around it. Businesses will distinguish themselves by how well they use it.”

Over the past two centuries, major transformative technologies like AI have led to both winners and losers. Kodak, Blockbuster, Digital, Wang, and BlackBerry are among the corporate giants who were once leaders in their fields, but were unable to survive the disruptive technologies of the past few decades. Amazon, Alphabet/Google, Microsoft, Facebook/Meta, and Apple have been among the big winners in the internet-based digital economy.

A recent issue of The Economist explored this question. We may all assume that given the continuing pace of technology and business change, no company is safe from disruption. But, in “America’s corporate giants are getting harder to topple,” The Economist questioned these assumptions. “America Inc has experienced surprisingly little competitive disruption during the internet age. Incumbents appear to have become more secure, not less.”

Working With AI’s Human-Like Alien Intelligence

“When OpenAI unleashed its humanlike ChatGPT software on the world last year, one thing was clear: These AI systems are coming for our jobs. But don’t write off the humans just yet.” said “The New Jobs for Humans in the AI Era,” a WSJ article published on October 5. As has been the case with major technologies over the past two centuries, — e.g., steam power, electricity, the internet,  — AI will both threaten and lead to the creation of new jobs. While still in the early stages, “AI is already creating new opportunities,” said the article, mentioning a few of those new opportunities.

I found two of those new jobs particularly intriguing: prompt engineer and AI psychotherapist. Generative AI and large language models (LLM) are a new kind of human-like alien intelligence that even the people building them don’t really understand how they work. Researchers are attempting to unlock the AI black box in hopes of understanding how to best work with these powerful, alien technologies.

Let me discuss each of these two jobs.

Could Generative AI End Up Reducing the Growing Inequality in Labor Markets?

Over the past three decades, high skill jobs requiring expert problem solving and complex communication skills significantly expanded, with the earnings of the college educated workers needed to fill such jobs rising steadily. Conversely, opportunities and wages declined for less educated middle skill blue-collar manufacturing jobs and white-collar administrative jobs whose careers have been upended by automation.

“But the new kind of automation — artificial intelligence systems called large language models, like ChatGPT and Google’s Bard — is changing that,” noted a recent NY Times article, “In Reversal Because of A.I., Office Jobs Are Now More at Risk.” “These tools can rapidly process and synthesize information and generate new content. The jobs most exposed to automation now are office jobs, those that require more cognitive skills, creativity and high levels of education. The workers affected are likelier to be highly paid, and slightly likelier to be women, a variety of research has found.”

The NYT article referenced two that analyzed the occupations most exposed to AI, one from the Pew Research Center, the other from Goldman Sachs Research.

AI and the Future of Healthcare Systems

In 2009 I participated in a panel at an MIT Symposium on complex systems where I gave a talk on complex social organizations. The panel included talks on financial, energy, and healthcare systems by three eminent leaders in their fields. I was particularly impressed by the talk given by Denis Cortese, who at the time was the president and CEO of the Mayo Clinic, and who after retiring in 2010, joined Arizona State University as Foundation Professor and director of its Center for Healthcare Delivery and Policy.

Dr. Cortese explained that healthcare is a system of coupled systems. Healthcare is composed of three interconnected domains: knowledge, — the domain of medical research, where new ideas, inventions and medical approaches are developed; care delivery, — the domain of physicians and hospitals where patients are treated; and payers, — the domain of insurance and governments that pay for the delivery of healthcare. Each of these domains is quite large and complex in its own right, but their convoluted interactions is one of the main reasons why healthcare systems are so incredibly complex.

Could AI now help us better deal with the inherent complexity of healthcare systems? After decades of promises and hype, AI has finally become the defining technology of our era. Over the past two decades, the necessary ingredients have come together to propel AI beyond universities and research labs into the broader marketplace: powerful, inexpensive computer technologies; advanced algorithms and models; and huge amounts of all kinds data. Data has been the key element in the major AI advances over the past 20 years, including big data and data science in the 2000s, machine and deep learning in the 2010s, and Large Language Models (LLMs) and generative AI in the past few years.

In trying to understand the potential use of AI in healthcare systems, I came across the work of John Halamka. Dr. Halamka is president of the Mayo Clinic Platform, an organization that aims to drive innovations in diagnosis, treatment, and operational improvements in healthcare systems. He’s also a prolific writer, having authored a number of articles and books on the subject.

How the Original Promises of the Open Source Movement Now Apply to AI Systems

“AI systems don’t operate like traditional software – they require distinct development processes and rely on specialized and costly resources currently pooled in the hands of a few large tech companies,” wrote David Gray Widder, Meredith Whittaker, and Sarah Myers West in a recent research paper, “Open (for Business): Big Tech, Concentrated Power, and the Political Economy of Open AI” (Open AI). “Even so, many of the original promises of the open source movement, which were made in reference to open source software, are now being projected onto ‘open’ AI. From the promise that open source could democratize software development, to the view that many eyes on open code could ensure it did what it said it did and was free of vulnerabilities, to the perspective that open source levels the playing field, allowing the most innovative to triumph. Open source software did many of these things, to varying degrees. But ‘open’ AI is a different story.”

I’ve been personally involved with open source, open standards, and related initiatives for the past few decades. In the mid-late 1990s, I led IBM’s Internet Division, and let's remember that a major part of the explosive growth of the internet and World Wide Web was due to the widespread availability of open source implementation of their key open standards. In the early 2000s I led IBM’s Linux initiative, which played a leading role in organizing a consortium of companies to support the development of Linux which later became the Linux Foundation (LF). And, for the past two years, I’ve been a member of the Advisory Board of the Linux Foundation Research division.

In trying to understand how the original promises of the open source movement now apply to this new generation of AI systems, I found the Open AI paper quite helpful. Let me summarize some the key points discussed in the paper.

Jobs Expected to Grow and Jobs Expected to Decline Through 2030

“The COVID-19 pandemic disrupted labor markets globally during 2020,” said a February 2021 McKinsey report on “The future of work after COVID-19.” “The short-term consequences were sudden and often severe: Millions of people were furloughed or lost jobs, and others rapidly adjusted to working from home as offices closed. Many other workers were deemed essential and continued to work in hospitals and grocery stores, on garbage trucks and in warehouses, yet under new protocols to reduce the spread of the novel coronavirus.”

Two and a half years later, a recently published report by the McKinsey Global Institute (MGI), “Generative AI and the future of work in America,” paints a radically different picture: “the US job market has come roaring back from its sudden drop. The nature of work has changed as many workers have stuck with remote or hybrid models and employers have sped up their adoption of automation technologies. More recently, the accelerated development of generative AI, with its advanced natural language capabilities, has extended the possibilities for automation to a much wider set of occupations.”

The US labor market is now undergoing a rapid evolution in the way people work and the work people do. “Some 8.6 million occupational shifts took place from 2019 through 2022. Now even more change is in store. We expect an additional 12 million occupational shifts by 2030.”

The Critical Value of Human-Created Knowledge to the Future of Generative AI

“In early 2021, a Wikipedia editor peered into the future and saw what looked like a funnel cloud on the horizon: the rise of GPT-3, a precursor to the new chatbots from OpenAI,” wrote author and journalist Jon Gertner in “Wikipedia’s Moment of Truth,” a recent NY Times Magazine article. “When this editor — a prolific Wikipedian who goes by the handle Barkeep49 on the site — gave the new technology a try, he could see that it was untrustworthy. The bot would readily mix fictional elements (a false name, a false academic citation) into otherwise factual and coherent answers. But he had no doubts about its potential.”

The article caught my attention because I’ve long been relying on Wikipedia as my go-to website for looking up topics I want to learn more about, as well as using links to Wikipedia articles as references in the weekly blogs I’ve been posting since 2005.

Over the past decade, Wikipedia has become much more than an encyclopedia, — “a kind of factual netting that holds the whole digital world together.” Google, Bing, Siri, Alexa, and other search engines and digital assistants often rely on Wikipedia for the information needed to answer users’ questions. More recently, Wikipedia has been one of the largest sources of the data, — currently estimated at around 3% to 5%, — for training Large Language Models (LLMs) and related chatbots. Wikipedia has played a major role in the digital world because its large amounts of data are free, easily accessible, high quality, and well-curated.

The World Economic Five Year Jobs and Skills Outlook

Earlier this year, the World Economic Forum (WEF) published “The Future of Jobs Report 2023.” The WEF has been publishing Future of Jobs Reports since 2016 to explore how socio-economic and technology trends are likely to shape the evolution of jobs and skills over the following 5 years.

“In 2023, labour-market transformations driven by technological breakthroughs, such as the coming of age of generative artificial intelligence (AI), are being compounded by economic and geopolitical disruptions and growing social and environmental pressures,” wrote WEF Managing Director Saadia Zahidi in the Preface of the 2023 Report. Encompassing almost 300 pages, the report explores how jobs and skills are likely to evolve in the 2023-2027 period. Its trends and predictions are based on a survey of over 800 large companies which collectively employ over 11.3 million workers across 27 industry clusters and 45 economies from all around the world.

Let me discuss a few of the report’s key findings and predictions.

Answers from Gartner Experts to Key Enterprise Questions about Generative AI

“Worldwide IT spending is projected to total $4.7 trillion in 2023, an increase of 4.3% from 2022,” said Gartner in its July 19 forecast of IT spending. “As CIOs continue to lose the competition for IT talent, they are shifting spending to technologies that enable automation and efficiency to drive growth at scale with fewer employees.” The forecast adds that “While generative artificial intelligence (AI) is top of mind for many business and IT leaders, it is not yet significantly impacting IT spending levels. In the longer-term, generative AI will primarily be incorporated into enterprises through existing spending.”

The forecast includes a link to an excellent article, “Gartner Experts Answer the Top Generative AI Questions for Your Enterprise.” The overall assessment of the Gartner experts is that “Generative AI isn’t just a technology or a business case — it is a key part of a society in which people and machines work together.” Let me summarize their answers to some key questions.

What is Generative AI? In answer to a question or prompt, generative AI can create an impressive variety of content based on the characteristics of its training data, including images, videos, music, speech, text, software code and product designs. Generative AI is based on foundation models, - the name coined by the Stanford Institute for Human-Centered Artificial Intelligence to large machine learning models that are trained on vast amounts of data, and can thus be adapted to a wide range of tasks.

Productivity Gains of Generative AI for Software Developers

In June of 2023, McKinsey published “The Economic Potential of Generative AI: The Next Productivity Frontier.” Based on its analysis of 63 business use cases, and estimates of its effect on the productivity of the global workforce, McKinsey’s study projected that generative AI could add an additional $6.1 trillion to $7.9 trillion annually, boosting the overall impact of artificial intelligence on the global economy by an annual $17.1 trillion to $25.6 trillion.

The McKinsey study found that while generative AI use cases could have an impact on most enterprise business functions, four in particularly could account for approximately 75% of the total annual value from generative AI use cases: customer operations, marketing and sales, research and development, and software engineering.

Digital technologies, including advanced analytics and machine learning algorithms, have been applied to the first three functions for the past few decades, so it’s easier to understand how GenAI would build on previous work as well as give rise to innovative new advances. But that hasn’t quite been the case with software engineering.

In a recent article, “Quantifying the Productivity Gains of Generative AI for Developers,” IBM Fellow Jerry Cuomo nicely explained the role of generative AI in enhancing the productivity of software developers by examining the potential benefits for developers of varying experience levels, from novice to experts. Over the years, Cuomo has played a major role in a number of major software advances, from IBM’s WebSphere Application Server to open source Hyperledger blockchain technologies. He’s currently CTO in IBM's Consulting business unit where he’s responsible for AI-based automation technologies and strategy.

“The productivity gains enabled by the introduction of generative AI allow for a shift to value and create opportunities for increased innovation, improved quality, enhanced customer experience, and faster time-to-market,” wrote Cuomo. “Specifically, we estimate that generative AI can lead to a 15–20% increase in the number of new products or features developed, a 10–15% reduction in the number of bugs found in production, a 5–10% increase in customer retention and loyalty, and a 10–15% reduction in time-to-market for new products or features.”

What Small Children Can Do that AI Doesn’t Know How to Do (Yet)

“It’s natural to ask whether large language models like LaMDA (short for Language Model Dialogue Application) or GPT-3 are really smart — or just double-talk artists in the tradition of the great old comedian Prof. Irwin Corey, ‘The World’s Greatest Authority,’ wrote UC Berkeley cognitive and developmental psychologist Alison Gopnik in ‘What AI Still Doesn’t Know How to Do,’ a July, 2022 WSJ essay. “But I think that’s the wrong question. These models are neither truly intelligent agents nor deceptively dumb. Intelligence and agency are the wrong categories for understanding them.”

“Instead, these AI systems are what we might call cultural technologies, like writing, print, libraries, internet search engines or even language itself. They are new techniques for passing on information from one group of people to another. Asking whether GPT-3 or LaMDA is intelligent or knows about the world is like asking whether the University of California’s library is intelligent or whether a Google search knows the answer to your questions. But cultural technologies can be extremely powerful — for good or ill.”

A recent article in The Economist, “How AI could change computing, culture and the course of history,” made a similar point. That article aimed to give us a sense of the world-changing transformations we might expect from generative AI by discussion two of of the most transformative cultural technologies of the past few hundred years: the printing press and the web browser.

Why Transformative AI Is So Hard to Achieve

“Humans have a good track record of innovation,” wrote Arjun Ramani and Zhengdong Wang in an essay which systematically explains “Why transformative artificial intelligence is really, really hard to achieve.” “The mechanization of agriculture, steam engines, electricity, modern medicine, computers, and the internet — these technologies radically changed the world. Still, the trend growth rate of GDP per capita in the world's frontier economy has never exceeded three percent per year.”

“Yet many people are optimistic that artificial intelligence is up to the job. AI is different from prior technologies, they say, because it is generally capable — able to perform a much wider range of tasks than previous technologies, including the process of innovation itself. Some think it could lead to a Moore’s Law for everything, or even risks on a par with those of pandemics and nuclear war.”

The essay consists of three sections: the transformational potential of AI is constrained by its hardest problems; major technical hurdles remain; and social and economic hurdles may limit AI’s impact. Let me summarize a few of the key arguments in each section.

What’s the Likely Long Term Evolution of AI?

Since the advent of the Industrial Revolution, general purpose technologies (GPTs) have been the defining technologies of their times. Their ability to support a large variety of applications can, over time, radically transform economies, and social institutions. GPTs have great potential from the outset, but realizing their potential takes large tangible and intangible investments and a fundamental rethinking of firms and industries, including new processes, management structures, business models, and worker training. As a result, realizing the potential of a GPT takes considerable time, often decades. Electricity, the internal combustion engine, computers, and the internet are all examples of historically transformative GPTs.

For example, after the introduction of electric power in the early 1880s, it took companies about four decades to figure out how to restructure their factories to harness electric power with manufacturing innovations like the assembly line. It took even longer to develop highly popular electric household products like refrigerators, toasters, washing machines, and air conditioners.

The internet’s precursor, ARPANET,  was initially developed by the US Department of Defense in the late 1960s for military and research applications. It wasn’t until the 1990s that the internet was opened up to companies and consumers in the wider commercial marketplace, becoming the most significant platform for innovation the world has ever since, and ushering a new 21st century digital economy.

Similarly, artificial intelligence first came to light in the mid-1950s as a promising new academic discipline that aimed to develop intelligent machines capable of handling human-like tasks like natural language and playing chess. AI became one of the most exciting areas in computer sciences in the 1960s, ’70s, and ’80s, but after years of unfulfilled promises and hype, a so called AI winter of reduced interest and funding that nearly killed the field set in everywhere.

Risks and Opportunities in an Increasingly Complex AI World

On July 13 I participated in an online panel on Board Governance: Risks and Opportunities in a Complex World, sponsored by eCornell, Cornell University’s executive education unit. The panel was organized and moderated by LizAnn Eisen, acting professor of the practice at the Cornell Law School, and former corporate partner at Cravath.

In addition to me, panel members included Bill Priestap, former assistant director of the FBI, head of the FBI Counterintelligence Division and now founder of the risk consulting startup Trenchcoat Advisors; Noah Phillips, former commissioner at the FTC and now anti-trust partner at Cravath; and David Kappos, former director of the USPTO and now IP partner at Cravath.

The panelist discussed a range of topics that board members should pay attention to when considering risks and opportunities. Each of us foced on our respective areas of expertise. I discussed the potential business impact of artificial intelligence; Phillips focused on regulatory issues that are arising as AI is used in an increasing number of business applications, like the need for consumer protection; Kappos discussed the extensive IP and legal implications posed by AI; and Priestap focused on the challenges posed by an increasingly dangerous global geopolitical environment.

A recording of our panel can be found here.

The Era of AI Is Here: Why the Panic?

“The era of Artificial Intelligence is here, and boy are people freaking out,” wrote technologist, entrepreneur and VC investor Marc Andreessen in the opening sentence of his article, Why AI Will Save the World  posted in his personal website. The article argues that not only will AI not destroy the world, but that it might in fact help save it.

While a computer sciences student at the University of Illinois, Andreessen was the leader of the team that developed the Mosaic browser in the early 1990s, the first easy-to-use graphical web browser that could be ported to a wide range of computers. Soon after its release in 1993, millions started using Mosaic to access the fast growing World Wide Web. After getting his degree in 1993, Andreessen moved to Silicon Valley and became a co-founder of Netscape, one of the first companies focused on bringing the fast growing World Wide Web to the commercial world.

The Mosaic browser and Netscape played a major role in the explosive growth of the internet in the 1990s. To give us a sense of the world-changing transformations we might expect from generative AI, a recent essay in The Economist cited the browser, along with the printing press as two major historical analogues to generative AI. Given his accomplishments, Andreessen is clearly someone whose opinions we should pay attention to when discussing the potential impact of generative AI, large language models, chatbots, and related AI technologies.

His article is organized in a number of sections. The first two sections give a detailed explanation of why he believes that AI can make everything we care about better, and why in contrast to his positive view, so many people are panicking. He then systematically discusses and refutes each of the major AI risks that are causing the fear and panic. He adds that the real risk is not aggressively pursuing AI in the US and the West in general, and proposes a simple plan on what’s to be done. 

Let me summarize a few of the key points in each section.

Data-Centric AI: the Systematic Engineering of Data to Build AI Systems

After decades of promise and hype, artificial intelligence has finally become the defining technology of our era. Over the past few decades, the necessary ingredients have come together to propel AI beyond universities and research labs into the broader marketplace: powerful, inexpensive computer technologies; advanced algorithms, models, and systems; and huge amounts of all kinds data.

Like the steam engine, electricity, computers, and the internet, AI will have a historically transformative impact across economies and societies. But, over the past two centuries we’ve learned that historically transformative technologies have great potential from the outset, but realizing that potential requires major complementary investments including business process redesign; innovative new products, applications and business models; and the re-skilling of the workforce; as well as a fundamental rethinking of organizations, industries, economies, and societal institutions.

In the early 1990s, for example, it was clear that something big and exciting was taking place with the advent of the internet and World Wide Web, but it wasn’t at all clear where things were heading, how to sort out the growing hype from reality, and the implication to industries, companies and jobs. As was the case with the dot-com bubble of the 1990s, there’s now a combination of rapid technology adoption, new standards-based open source infrastructures, exciting new applications, — some of which will turn out to be truly innovative and some rather silly, — and a speculative frenzy that is likely to end with the bursting of the current AI bubble.

However, while sharing many attributes with previous historically transformative technologies, AI is in a class by itself, especially when it comes to the kinds of concerns, fears and uncertainties it’s been generating.

The Economic Potential of Generative AI

In September of 2018, McKinsey published a report on the impact of AI on the world economy. The report’s overriding finding was that AI has the potential to incrementally add 16 percent or up to $15.4 trillion by 2030 to annual global economic output, — an average contribution to productivity growth of about 1.2 percent. McKinsey’s 2018 report was published before the advent of generative AI and related technologies lik large language models (LLMs) and chatbots like ChatGPT that have been taking AI to a whole new level in the past few years.

In June of 2023, McKinsey published, “The Economic Potential of Generative AI: The Next Productivity Frontier.” This new research study aims to take a first look at generative AI’s economic impact over and above the projections in its 2018 report. The 2023 report study evaluated the potential economic impact of generative AI by identifying 63 concrete use cases of generative AI that would lead to measurable outcomes. In addition, the study estimated how generative AI would affect the productivity of the global workforce by analyzing its potential impact on each of more that 2,100 detailed work activities across 850 occupations.

Not surprisingly given that generative AI is still in its early stages, the new report explains that “While generative AI is an exciting and rapidly advancing technology, the other applications of AI discussed in our previous report continue to account for the majority of the overall potential value of AI. Traditional advanced-analytics and machine learning algorithms are highly effective at performing numerical and optimization tasks such as predictive modeling, and they continue to find new applications in a wide range of industries. However, as generative AI continues to develop and mature, it has the potential to open wholly new frontiers in creativity and innovation.”

Machine Learning Has an AI Problem

“Machine learning has an AI problem,” wrote author Eric Siegel in a recent Harvard Business Review (HBR) article, “The AI Hype Cycle is Distracting Companies.” “With new breathtaking capabilities from generative AI released every several months — and AI hype escalating at an even higher rate — it’s high time we differentiate most of today’s practical ML projects from those research advances. This begins by correctly naming such projects: Call them ML, not AI. “Including all ML initiatives under the AI umbrella oversells and misleads, contributing to a high failure rate for ML business deployments. For most ML projects, the term AI goes entirely too far — it alludes to human-level capabilities.”

Siegel’s article brought to mind similar concerns many of us had in the 1990s given all the frenzy and hype surrounding the then fast growing Internet which eventually led to what became known as the dot-com bubble. Let me explain.

The 2023 MIT Sloan CIO Symposium

The 2023 MIT Sloan CIO Symposium took place on May 15 and 16 at the Royal Sonesta Hotel, overlooking the Charles River, and a short walk from the MIT campus in Cambridge, MA.  Livestream tickets were also offered to help better engage the Symposium’s worldwide audience.

This year’s overall theme was “Driving Digital Resilience in a Turbulent World.” It was a very appropriate theme given that business and economies are still bouncing back from the major disruptions brought about by the COVID-19 pandemic, trying to understand and adapt to transformative technology advances like Generative AI, and coping with the rising volume of cybersecurity threats in an increasingly dangerous world.

2023 marked the 20th anniversary of the MIT CIO Symposium. I’ve been involved with the MIT Symposium for most of those 20 years, having first participated in the 2009 Symposium where I moderated a panel on The Virtual Organization. In those earlier years, CIOs and other senior business executives from around the world came to MIT to explore how leading-edge academic research on information technologies could help address the challenges faced by their companies in the 21st century, Internet-based digital economy.

Are Open AI Models Safe?

A recent NY Times article, “In Battle Over A.I., Meta Decides to Give Away Its Crown Jewels,” reported that Meta, — the technology giant that owns Facebook, Instagram, and WhatsApp, — made the decision to open source LLaMA, its state-of-the-art large language model (LLM), making the code available to academics, government researchers and others once they’d been properly vetted. Meta’s actions are quite different from those of Google and OpenAI, its chief AI rivals, who’ve been keeping the software underpinning their AI systems proprietary.

“Driven by its founder and chief executive, Mark Zuckerberg, Meta believes that the smartest thing to do is to share its underlying A.I. engines as a way to spread its influence and ultimately move faster toward the future,” said the article. Meta’s chief AI scientist, NYU professor Yann LeCun noted in a recent interview that the growing secrecy of Google and OpenAI is a “huge mistake,” arguing that consumers and governments will refuse to embrace AI if it’s under the control of a couple of powerful American companies. LeCun, along with Geoffrey Hinton and Yoshua Benjio, received the 2018 Turing Award, — often considered the “Nobel Prize of Computing,” — for their pioneering work on deep learning.

Meta’s open source approach to AI isn’t novel. After all, the infrastructures underlying the Internet and World Wide Web were all built on open source software, as has the widely used Linux operating system. And, in September, 2022, Meta contributed its PyTorch machine learning framework to the Linux Foundation to further accelerate the development and accessibility of the technology.

The Current State of AI: A Tale of Two Worlds

What’s the current state of AI? I’ve been closely following AI over the past several years, and based on everything I’ve read and written about in my blog, the question brings to mind Charles Dickens’ A Tale of Two Cities.

We have in fact a tale of two very different AI worlds. Ever since OpenAI released ChatGPT a few months ago and encouraged the general public to try it, we’ve seen article after article about its emergent unexpected capabilities and potential existential risks. But, at the same time, most recent surveys of the state of AI in the enterprise have concluded that the majority of surveyed companies are still in the early stages of deployment and have not achieved significant outcomes.

For example, Stanford’s recently published 2023 AI Index Report found that industry was now leading the development of state-of-the-art large language models like GPT-4 and chatbots like ChatGPT. But at the same time, the Stanford report references the most recent McKinsey global survey on AI which found that while leading edge firms continue to pull ahead, the number of companies adopting AI has plateaued.

The global survey, “The State of AI in 2022 — and a Half Decade in Review,” marks the fifth consecutive year in which McKinsey has analyzed AI’s impact on companies around the world. The 2022 report is based on a survey of almost 1,500 firms from a variety of regions, industries, and company sizes. Only about half of respondents said that their organization have adopted AI in at least one function.

Let me summarize the report’s key findings.

The Evolution of Global Supply Chains in the 21st Century Digital Economy

“The societal and economic spasms of the early 2020s highlighted the crucial role of world-spanning supply chains in the modern global economy, as well as the growing role of digital technology, including A.I. and automation, in the future economy,” wrote MIT professor Yossi Sheffi in the Preface of his new book The Magic Conveyor Belt: Supply Chains, A.I. and the Future of Work. In the Preface, Sheffi succinctly stated that “The central purpose of supply chains is to fulfill the needs and wants of humanity — delivering all the food, medicine, energy, apparel, and other worldly goods need by the eight billion people in the planet.”

The 1990s ushered a golden age of globalization. The world seemed to be coming together. Thomas Friedman’s The World is Flat became an international best-seller in 2005 by nicely explaining what globalization was all about, including the key forces that contributed to flattening the world, - from the collapse of the Berlin Wall in November of 1989 to the rise of outsourcing, offshoring and global supply chains.

“The golden age of globalisation, in 1990-2010, was something to behold,” wrote The Economist in a January, 2019 article.  “Commerce soared as the cost of shifting goods in ships and planes fell, phone calls got cheaper, tariffs were cut and the financial system liberalised.” But, ever since the 2008 global financial crisis, globalization and global trade started to slow down. “Globalisation has slowed from light speed to a snail’s pace in the past decade for several reasons,” said The Economist.

Three major shocks have further reshaped globalization: the increasing trade wars and tariffs of the past five years, especially between the US and China; the disruptive impact of Covid-19 on global supply chains; and more recently, the war in Ukraine, which threatens to further decouple the world’s economy into a Western and a Chinese trading bloc. Mohamed El-Erian, one of the world’s most influential economic thinkers, said in a recent podcast interview, that the rewiring of global supply chains is one of the major drivers of major structural changes in the global economy.

AI Need Not Be World-Ending to Be World-Changing.

“How to worry wisely about artificial intelligence” was the overall theme of the April 22 issue of The Economist, with several articles on the subject. “Rapid progress in AI is arousing fear as well as excitement. How worried should you be?,” said the issue’s lead article. “In particular, new ‘large language models’ (LLMs)—the sort that powers Chatgpt, a chatbot made by Openai, a startup—have surprised even their creators with their unexpected talents as they have been scaled up.”

“Proponents of AI argue for its potential to solve big problems by developing new drugs, designing new materials to help fight climate change, or untangling the complexities of fusion power. To others, the fact that AIs’ capabilities are already outrunning their creators’ understanding risks bringing to life the science-fiction disaster scenario of the machine that outsmarts its inventor, often with fatal consequences.”

The Economist issue included an essay, “How AI could change computing, culture and the course of history,” by European business editor Ludwig Siegele in close collaboration with briefings editor Oliver Morton. I found the essay quite interesting, and recognized that it had been written by Siegele, with whom I’ve long been discussing advanced technologies given his previous positions as technology correspondent for The Economist. Let me summarize the essay’s key points.

The 2023 AI Index Report: AI Has Moved Into Its Era of Deployment

The 2023 AI Index Report was released in early April by the Stanford Institute for Human-Centered Artificial Intelligence, it’s sixth annual analysis of the impact, progress, and trends of AI. Led by an interdisciplinary group of experts from across academia and industry, the report aims to be the world’s most authoritative source for data and insights about AI.

Last year’s report concluded that 2021 was the year that AI went from an emerging technology to a mature technology, no longer just a subject of speculative lab research but something with real-world economic impact, both positive and negative.

“AI has moved into its era of deployment; throughout 2022 and the beginning of 2023, new large-scale AI models have been released every month,” wrote the report’s co-directors Jack Clark and Ray Perrault about the 2023 report. “Policymakers are talking about AI more than ever before. Industry leaders that have integrated AI into their businesses are seeing tangible cost and revenue benefits. The number of AI publications and collaborations continues to increase. And the public is forming sharper opinions about AI and which elements they like or dislike.”

How to Leverage AI for Substantial Business Value

“AI initiatives at many organizations are too small and too tentative,” wrote Babson professor Tom Davenport and Deloitte principal consultant  Nitin Mittal in “Stop Tinkering with AI,” a recently published HBR article. The article is adapted from their book All-in on AI: How Smart Companies Win Big with Artificial Intelligence which was published earlier this year.

AI technologies have significantly advanced over the past few years. But, while leading edge firms are placing AI at the center of their business strategies, a number of recent surveys continue to show that the majority of enterprises are still in the early stages of AI experimentation and deployment and risk being left further behind. For example, the latest Deloitte survey on the “State of AI in the Enterprise” reached out to over 2,800 executives from advanced economies and found that 28% of respondents were deploying AI at scale and achieving high outcomes, but 46% were still in the early stages of deployment with no significant outcomes.

In another recent survey, Accenture reached out to over 1,600 C-suite executives of the world’s largest companies and found that only 12% had the strategic and operational AI capabilities needed to achieve superior growth, while the majority of firms, 63%, were still at the experimenting stage and had only average AI capabilities.

This is not surprising. As we’ve learned over the past two centuries, even after achieving broad acceptance, it takes considerable time, - often decades, - for transformative technologies like AI to be widely embraced across firms and economies. These technologies have great potential from the outset, but realizing their potential requires major complementary developments including business process redesign, innovative products, and services, new business models and strategies, the re-skilling of the workforce, and a fundamental rethinking of the overall organization.

The Potential Impact of Large Language Models on Jobs

Throughout the Industrial Revolution there were periodic panics about the impact of automation on jobs, going back to the Luddites, - textile workers who in the 1810s smashed the new machines that were threatening their jobs. But each time those fears arose in the past, technology advances ended up creating more jobs than they destroyed.

Automation fears have understandbly accelerated in recent years, as our increasingly smart machines have been applied to activities requiring intelligence and cognitive capabilities that not long ago were viewed as the exclusive domain of humans. Over the past decade, powerful AI systems have matched or surpassed human levels of performance in a number of tasks such as image and speech recognition, skin cancer classification, breast cancer detection, and highly complex games like Go. More recently, large language models (LLMs) and chatbots like ChatGPT are taking AI-based automation to a whole new level.

 “OpenAI’s ChatGPT is the latest advance in a steady march of innovations that have offered the potential to transform many occupations and wipe out others, sometimes in tandem,” wrote journalists Lydia DePillis and Steve Lohr in “Tinkering With ChatGPT, Workers Wonder: Will This Take My Job?,” a recent NY Times article. “It is too early to tally the enabled and the endangered, or to gauge the overall impact on labor demand and productivity. But it seems clear that artificial intelligence will impinge on work in different ways than previous waves of technology.” In particular, AI is now “confronting white-collar professionals more directly than ever. It could make them more productive — or obsolete.”

Large Language Models: A Cognitive and Neuroscience Perspective

In the past decade, powerful AI systems have matched or surpassed human levels of performance in a number of tasks such as image and speech recognition, skin cancer classification, breast cancer detection, and highly complex games like Go. These AI breakthroughs have been based on increasingly powerful and inexpensive computing technologies, innovative deep learning (DL) algorithms, and huge amounts of data on almost any subject. More recently, the advent of large language models (LLMs) are taking AI to the next level. And, for many technologists like me, LLMs and their associated chatbots have introduced us to the fascinating world of human language and cognition.

I recently learned the difference between form, communicative intent, meaning, and understanding from “Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data,” a 2020 paper by linguistic professors Emily Bender and Alexander Koller. These linguistic concepts helped me understand the authors’ argument that “in contrast to some current hype, meaning cannot be learned from form alone. This means that even large language models such as BERT do not learn meaning; they learn some reflection of meaning into the linguistic form which is very useful in applications.”

A few weeks ago, I came across another interesting paper, “Dissociating Language and Thought in Large Language Models: a Cognitive Perspective,” published in January, 2023 by principal authors linguist Kyle Mahowald and cognitive neuroscientist Anna Ivanova and four additional co-authors. The paper nicely explains how the study of human language, cognition and neuroscience sheds light on the potential capabilities of LLMs and chatbots. Let me briefly discuss what I learned.

“Today’s large language models (LLMs) routinely generate coherent, grammatical and seemingly meaningful paragraphs of text,” said the paper’s abstract. “This achievement has led to speculation that these networks are — or will soon become — thinking machines, capable of performing tasks that require abstract knowledge and reasoning. “Here, we review the capabilities of LLMs by considering their performance on two different aspects of language use: formal linguistic competence, which includes knowledge of rules and patterns of a given language, and functional linguistic competence, a host of cognitive abilities required for language understanding and use in the real world.

Driving Value with the Metaverse in the Enterprise

A killer app is an IT application whose value to individuals and companies is relatively simple to explain, its use is fairly intuitive, and the application turns out to be so useful that it can singlehandedly propel the success of a new product or service. Word processing and spreadsheets, for example, were major factors in the widespread adoption of personal computers in the 1980s.

I got interested in 3D immersive environments in the mid-2000s when IBM launched an exploratory initiative on the topic. As part of these efforts, we conducted a number of experiments on Second Life, a platform that allowed its users to develop a variety of virtual world applications. We built a number of workplace applications, including virtual meetings, and online events, and even built a virtual replica of Beijing’s Forbidden City in partnership with the Palace Museum that was opened to the public. But, despite our high expectations, we couldn’t find any compelling enterprise killer apps, and after about three years, the project was dissolved. In the ensuing years, 3D immersive technologies and applications continued to advance, but mostly focused on video-games and related industries.

The past few years have seen a surge of interest in the metaverse and extended reality. “After desktop computing, the consumer internet and the smartphone boom, the consumer-computing industry is past due its Next Big Thing,” wrote The Economist in  “A reality check for the metaverse is coming,” an article in its 2022 year-end issue. “[A]n internet which is still largely flat — based on two-dimensional text, images and video — is ripe for replacement with one that is three-dimensional and immersive.”

Are GPT-3 and ChatGPT Truly Intelligent?

Large language models (LLMs) — e.g., BERT, GPT-3, — and chatbots like ChatGPT, are quite impressive. In the coming years, they will likely lead to a number of transformative tools and applications. They may well herald a coming AI revolution. But, how much closer are they getting us to the kind of general intelligence that, for the foreseeable future, only humans have? How can we best sort out their true significance and value from the accompanying hype? Let me discuss how two different articles addressed these questions.

“The success of the large neural language models on many NLP [Natural Language Processing] tasks is exciting” wrote linguistic professors Emiliy Bender and Alexander Koller in their 2020 paper “Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data.” “This has led to claims, in both academic and popular publications, that such models understand or comprehend natural language or learn its meaning. From our perspective, these are overclaims caused by a misunderstanding of the relationship between linguistic form and meaning.”

In their paper, Bender and Koller explain why LLMs like GPT-3 are likely to become innovative language tools, — kind of like highly advanced spell checkers or word processors, — while dismissing claims that they have the ability to reason and understand the meaning of the language they’re generating. Their explanations are couched in linguistic concepts which they carefully define: form, communicative intent, meaning, and understanding.

Can an AI System Exhibit Commonsense Intelligence?

“One of the fundamental limitations of AI can be characterized as its lack of commonsense intelligence: the ability to reason intuitively about everyday situations and events, which requires rich background knowledge about how the physical and social world works,” wrote University of Washington professor Yejin Choi in “The Curious Case of Commonsense Intelligence,” an essay published in the Spring 2022 issue of Dædalus. “Trivial for humans, acquiring commonsense intelligence has been considered a nearly impossible goal in AI, added Choi.”

What is commonsense intelligence? In a 2020 presentation, Choi defined common sense as “the basic level of practical knowledge and reasoning concerning everyday situations and events that are commonly shared among most people.” Common sense is essential for humans “to live and interact with each other in a reasonable and safe way,” and for an AI system, common sense is essential “to understand humans needs and actions better.” She further explained the difference between intuitive commonsense reasoning and analytical rational reasoning by discussing the pioneering research of Daniel Kahneman, — Princeton Professor Emeritus and recipient of the 2002 Nobel Prize in Economics, — and his long time collaborator Amos Tversky, who died in 1996.

In his excellent 2011 bestseller Thinking, Fast and Slow, Kahneman explained that our mind is composed of two very different systems of thinking, System 1 and System 2.  System 1 is the intuitive, unconscious, fast, and effortless part of the mind. Thoughts come automatically and very quickly to System 1 without us doing anything to make them happen.

Is AI (Finally) Coming of Age?

Artificial intelligence first came to light in the mid-1950s as a promising new academic discipline. AI became one of the most exciting areas in computer sciences over the next two decades. But, after years of unfulfilled promises and hype, a couple of so called AI winters of reduced interest and funding set in that nearly killed the field. AI was successfully reborn in the 1990s with a new statistical paradigm based on analyzing large amounts of data with powerful computers and sophisticated algorithms. Now, six decades after the field was founded, AI seems to be finally coming of age.

“2021 saw the globalization and industrialization of AI intensify, while the ethical and regulatory issues of these technologies multiplied,” said the 2022 AI Index report on the progress of AI, which was released in March of 2022 by Stanford’s Institute for Human-Centered Artificial Intelligence (HAI). “2021 was the year that AI went from an emerging technology to a mature technology - we’re no longer dealing with a speculative part of scientific research, but instead something that has real-world impact, both positive and negative,” wrote Jack Clark, co-chair of the AI Index. Multiple factors led to his conclusion, in particular the advent of foundation models like OpenAI’s GPT-3 and Google’s BERT.

In the past decade, increasingly powerful AI systems have matched or surpassed human levels of performance in a number of specific tasks like image and speech recognition. These task-specific deep learning (DL) systems have generally relied on supervised learning, a training method where the data must be carefully labelled, - e.g., cat, not-cat, - thus requiring a big investment in time and money to produce a model that’s narrowly focused on one task and can’t be easily repurposed.

Foundation models have gotten around these DL limitations based on two recent advances, transfer learning and huge scale. Transfer learning takes the knowledge learned from training a specific task and applies it to different but related tasks, - such as using the training in recognizing cars in images and applying it to recognizing trucks and buses. As a result, foundation models can be adapted to many different tasks with relatively small amounts of fine-tuning.

The Current State of Service Science

On October 17, the National Academy of Engineering (NAE) conducted an online forum on Service Systems Engineering in the Era of Human-Centered AI. “With AI advances poised to drive service system productivity and quality - similar to the way previous generations of technology revolutionized agriculture and manufacturing productivity and quality - it is time to take stock for industry-academic-and-government stakeholders on this important topic,” wrote the NAE in its website.

The agenda included an opening keynote by retired IBM executive Nick Donofrio. It was followed by four panels on various aspects of service systems, and concluded with an open discussion of the way forward. I was a member of the panel on Evolving Engineering Education. In my prepared remarks, I reflected on the current state of service science and related sociotechnical systems. Let me share my remarks, slightly edited for clarity.

Service Science, Management and Engineering (SSME) is an initiative launched in IBM’s Almaden Research Lab in the early 2000s in partnership with a number of universities as an interdisciplinary field of study aimed at applying science, technology and innovation to the service sector of the economy.  The service sector is the largest in most economies around the world. At the time, services already accounted for over 70% of GDP and jobs in advanced economies, as well as an increasing portion of the revenues of many companies, including close to 60% of IBM’s revenues.

The Disruptive Economics of AI

In 2017, I attended a seminar by University of Toronto professor Avi Goldfarb on the economic value of AI. Goldfarb explained that the best way to assess the impact of a potentially transformative technology is to look at how the technology reduces the cost of a widely used function. Computers, for example, have dramatically reduced the cost of digital operations like arithmetic by several orders of magnitude. As a result, we’ve learned to define all kinds of tasks in terms of digital operations, e.g., financial transactions, inventory management, word processing, photography. Similarly, the internet has reduced the cost of communications and the Web has reduced the cost of access to information, which has led to a huge increase in applications based on communications and information, like music and video streaming, and digital media.

Viewed through this lens, AI is essentially a prediction technology, and its economic impact is to reduce the cost and expand the number and variety of applications that rely on predictions. A key finding of Stanford’s 2022 AI Index report was that AI is becoming much more affordable and higher performing, leading to the widespread commercial adoption of AI-based applications. “Since 2018, the cost to train an image classification system has decreased by 63.6%, while training times have improved by 94.4%.,” said the report.

Goldfarb and his U of T colleagues Ajay Agrawal, and Joshua Gans published their research in the 2018 book Prediction Machines: The Simple Economics of Artificial Intelligence. And this week, they just published Power and Prediction: The Disruptive Economics of Artificial Intelligence.

“When we published Prediction Machines in 2018, we thought we had said all we needed to on the economics of AI,” wrote the authors in the Preface of their new book. “We were wrong. We laid out a framework for the economics of AI in that book, which remains useful today. However, the Prediction Machines framework only told part of the story, the point solutions part. In the years since, we discovered that another key part of the AI story had yet to be told, the systems part. We tell that story here.”

The AI Maturity Framework

I recently attended a seminar, The Art of AI Maturity, by Accenture executives Philippe Roussiere and Praveen Tanguturi as part of MIT’s Initiative on the Digital Economy (IDE) lunch seminar series. The seminar was based on their recently published article The Art of AI Maturity: Advancing from Practice to Performance.

“Today, so much of what we take for granted in our daily lives stems from machine learning,” wrote the authors in the article’s executive summary. “Every time you use a wayfinding app to get from point A to point B, use dictation to convert speech to text, or unlock your phone using face ID ... you're relying on AI. And companies across industries are also relying on - and investing in - AI to drive logistics, improve customer service, increase efficiency, empower employees and so much more.”

To determine the true state of AI maturity in the marketplace, Accenture conducted a survey in August and September of 2021 of over 1,600 C-suite executives at nearly 2,000 of the world’s largest companies across 16 industries with headquarters in 15 countries. In addition, they interviewed 25 CEOs, Chief Data Officers and Chief Analytics Officers, as well as a number of Accenture AI experts, and developed over 40 case studies on AI transformation.

The Recent Impact of AI on Jobs and Economies

“Recent developments in Artificial Intelligence (AI) have stoked new fears about large-scale job loss, stemming from its ability to automate a rapidly expanding set of tasks (including non-routine cognitive tasks), and its potential to affect every sector of the economy,” said The impact of Artificial Intelligence on the labour market: What do we know so far?, a recently published report by the Organization for Economic Co-operation and Development (OECD). Founded in 1961 to stimulate economic progress and world trade, the OECD now has 38 member countries around the world.

“Furthermore, there are concerns about employee well-being and the broader work environment, linked to the idea that AI may soon become pervasive in the workplace and threaten and undermine humans’ place in it,” noted the report. “However, AI also has the potential to complement and augment human capabilities, leading to higher productivity, greater demand for human labour and improved job quality.”

A number of recent studies have focused on the impact of AI on the future of work. For example, in the spring of 2018, MIT launched a task force on the Work of the Future to understand whether and how our current period of technological disruption differs from prior industrial epochs. The task force final report, The Work of the Future: Building Better Jobs in an Age of Intelligent Machines, released in November of 2020, concluded that:

 “No compelling historical or contemporary evidence suggests that technological advances are driving us toward a jobless future. On the contrary, we anticipate that in the next two decades, industrialized countries will have more job openings than workers to fill them, and that robotics and automation will play an increasingly crucial role in closing these gaps. Nevertheless, the impact of robotics and automation on workers will not be benign. These technologies, in concert with economic incentives, policy choices, and institutional forces, will alter the set of jobs available and the skills they demand.”

The 2022 State of AI in the Enterprise

“Rapidly transforming, but not fully transformed - this is our overarching conclusion on the market, based on the fourth edition of our State of AI in the Enterprise global survey,” said Becoming an AI-fueled organization, the fourth survey conducted by Deloitte since 2017 to assess the adoption of AI across enterprises. “Very few organizations can claim to be completely AI-fueled, but a significant and growing percentage are starting to display the behaviors that can get them there.”

AI is increasingly viewed by workers as a trusted assistant. “Within just the last 18 months, AI capabilities have advanced considerably, maturing from what was often experienced as a bothersome critic - telling workers what to do or pointing out their mistakes - to more frequently serving as a copilot, independently executing on insights and trends surfaced through the power and speed of cloud- based data hosting and computation.”

These conclusions are similar to those of Stanford’s 2022 AI Index report, which found that AI was becoming more affordable and higher performing, with lower training costs and faster training times across a number of AI tasks including recommendation engines, image classification, object detection, and language processing. This has led to the widespread commercial adoption and increased real-world impact of AI systems.

The Open Source AI and Data Ecosystem

“The availability of enterprise-grade open source software (OSS) is changing how organizations develop, maintain, and deliver products,” wrote Ibrahim Haddad in a recent report, Artificial Intelligence and Data in Open Source. Haddad is VP of Strategic Programs at the Linux Foundation (LF) and Executive Director of the LF AI & Data initiative. “Using and adopting OSS can offer many benefits, including reduced development costs, faster product development, higher code quality standards, and more. The open source methodology offers key and unique benefits to the domains of AI and data, specifically in areas of fairness, robustness, explainability, lineage, availability of data, and governance.”

Earlier this year, Stanford released the 2022 AI Index report, its fifth annual study on the impact and progress of AI. The Stanford report noted that “2021 was the year that AI went from an emerging technology to a mature technology - we’re no longer dealing with a speculative part of scientific research, but instead something that has real-world impact, both positive and negative.” 

A few weeks ago I wrote about the impressive scope of the Linux Foundation. The LF supports a large and growing number of open source projects in a wide variety of areas. AI is no different from other technology domains, so it’s not surprising that open source now plays a major role as AI is being increasingly integrated into the economy.

Disruptive Forces Necessitate Bold Decisions

In January of 2021 I attended Predictions 21, an online event organized by Forrester Research.  “Faced with the pandemic, firms did things that once seemed impossible - sometimes overnight,” said Forrester last year, adding that “2021 will be the year that every company - not just the 15% of firms that were already digitally savvy - doubles down on technology-fueled experiences, operations, products, and ecosystems.”

Earlier this year I attended Predictions 22, and was particularly curious to see how things had changed in the intervening year. “Disruptive Forces Necessitate Bold Decisions,” was the overriding message in this year’s event guide.  “The old ways of working no longer work. The future is up for grabs. Leading firms will use the crucibles of 2020 and 2021 to forge a path to an agile, creative, and resilient tomorrow.”

Let me summarize some of Forrester’s key predictions in three areas: technology, customer experience, and industry trends.

Foundation Models: AI’s Exciting New Frontier

Over the past decade, powerful AI systems have matched or surpassed human levels of performance in a number of specific tasks such as image and speech recognition, skin cancer classification and breast cancer detection, and highly complex games like Go. These AI breakthroughs have been based on deep learning (DL), a technique loosely based on the network structure of neurons in the human brain that now dominates the field. DL systems acquire knowledge by being trained with millions to billions of texts, images and other data instead of being explicitly programmed.

These task-specific, DL systems have generally relied on supervised learning, a training method where the data must be carefully labelled, - e.g., cat, not-cat, - thus requiring a big investment of time and money to produce a model that’s narrowly focused on a specific task and can’t be easily repurposed. The rising costs for training ever-larger, narrowly focused DL systems have prompted concerns that the technique was running out of steam.

Foundation models promise to get around these DL concerns by bringing to the world of AI the reusability and extensibility that have been so successful in IT software systems, from operating system like iOS and Android to the growing number and varieties of internet-based platforms.

Can We Trust Our Impressive AI Language Models?

One of the key findings of the 2022 AI Index Report was that large language models (LLMs) are setting records on technical benchmarks thanks to advances in deep neural networks and computational power that allows them to be trained using huge amounts of data. LLMs are now surpassing human baselines in a number of complex language tasks, including English language understanding, text summarization, natural language inference, and machine translation.

A.I. Is Mastering Language. Should We Trust What It Says?, a recent NY Times Magazine article by science writer Steven Johnson, took a close look at one such LLM, the Generative Pre-Trained Transformer 3, generally referred to as GPT-3. GPT-3 was created by the AI research company OpenAI. It’s been trained with over 700 gigabytes of data from across the web, along with a large collection of text from digitized books. “Since GPT-3’s release, the internet has been awash with examples of the software’s eerie facility with language - along with its blind spots and foibles and other more sinister tendencies,” said Johnson.

“So far, the experiments with large language models have been mostly that: experiments probing the model for signs of true intelligence, exploring its creative uses, exposing its biases. But the ultimate commercial potential is enormous. If the existing trajectory continues, software like GPT-3 could revolutionize how we search for information in the next few years.” Instead of typing a few keywords into Google and getting back a long list of links that might have the answer, you’d ask GPT-3 what you’re looking for in English and get back a well-written, accurate answer. “Customer service could be utterly transformed: Any company with a product that currently requires a human tech-support team might be able to train an L.L.M. to replace them.”

A Human-Compatible Safe, Controllable AI

“Even in its present state, the technology of artificial intelligence raises many concerns as it transitions from research into widespread use,” wrote UC Berkeley professor Stuart Russell in a recent essay, If We Succeed. “These concerns include potential misuses such as cybercrime, surveillance, disinformation, and political manipulation; the exacerbation of inequality and of many forms of bias in society; the creation and deployment of lethal autonomous weapons; and the usurpation of human roles in the economy and in social relationships.”

“My concern here, however, is with the potential consequences of success in creating general-purpose AI: that is, systems capable of quickly learning to perform at a high level in any task environment where humans (or collections of humans) can perform well.” Decades ago, Alan Turing expressed a similar concern. While answering a question during a lecture in 1951, Turing said: “It seems probable that once the machine thinking method had started, it would not take long to outstrip our feeble powers. … At some stage therefore we should have to expect the machines to take control.” Over the past decade, a number of public figures like Stephen Hawking and Nick Bostrom have argued that a superintelligent AI could become difficult or impossible for humans to control, and might even pose an existential threat to humanity.

Until recently, we didn’t have to worry about the consequences of such superhuman AIs because AI applications weren’t all that smart and were mostly confined to research labs. But, powerful AI systems have now matched or surpassed human levels of performance in a number of application- specific tasks such as image and speech recognition, skin cancer classification, breast cancer detection, and complex games like Jeopardy and Go. While these application-specific breakthroughs, - generally referred to as soft, or narrow AI, - were once viewed as the exclusive domain of humans, AI still lacks the deeper, general-purpose human intelligence that’s long been measured in IQ tests.

How Can You Trust the Predictions of a Large Machine Learning Model?

Artificial intelligence has emerged as the defining technology of our era, as transformative over time as the steam engine, electricity, computers, and the Internet. AI technologies are approaching or surpassing human levels of performance in vision, speech recognition, language translation, and other human domains. Machine learning (ML) advances, like deep learning, have played a central role in AI’s recent achievements, giving computers the ability to be trained by ingesting and analyzing large amounts of data instead of being explicitly programmed.

Deep learning is a powerful statistical technique for classifying patterns using large training data sets and multi-layer AI neural networks. Each artificial neural unit is connected to many other such units, and the links can be statistically strengthened or decreased based on the data used to train the system. But such statistical methods are not equally suitable for all tasks. Tasks that are particularly suitable for machine learning, exhibit several key criteria, such as the availability of large data sets of well-defined input-output pairs for training ML classifiers, - e.g., carefully labeled cat, not-cat pictures for cat recognition classifiers, and english-french document pairs for machine translation algorithms.

The methods behind a machine learning prediction, - subtle adjustments to the numerical weights that interconnect its huge number of artificial neurons, - are very difficult to explain because they’re so different from the methods used by humans. The bigger the training data set, the more accurate the prediction, but the more difficult it will be to provide a detailed, understandable explanation to a human of how the prediction was made.

A few weeks ago I attended an online seminar, - How Can You Trust Machine Learning? by Stanford professor Carlos Guestrin, - on the difficulty of understanding machine learning predictions. Guestrin’s seminar was based on a 2016 article he co-authored with Marco Tulio Ribeiro and Sameer Singh , “Why Should I Trust You?” Explaining the Predictions of Any Classifier.

The 2022 AI Index Report: AI’s Emerging Real-World Impact

On March 16, Stanford’s Institute for Human-Centered Artificial Intelligence (HAI) released the 2022 AI Index report, its fifth annual study on the impact and progress of AI. The comprehensive study was led by an interdisciplinary group of experts from across academia and industry, and highlights the rapid rate of technical and marketplace AI advances, as well as the growing ethical and regulatory concerns associated with AI.

“2021 was the year that AI went from an emerging technology to a mature technology - we’re no longer dealing with a speculative part of scientific research, but instead something that has real-world impact, both positive and negative,” said Jack Clark, co-chair of the AI Index. “This year’s AI Index tells us that AI is being integrated into the economy and the effects of it are beginning to go global across research, deployment, and even funding.” Two weeks ago I attended an online HAI seminar where Clark gave this excellent presentation on the AI Index.

The report is organized into five chapters: Technical Performance, AI Ethics, R&D, Economy and Education, and Policy and Governance. Let me summarize the key findings of each chapter.

The Tech Trends that Will Most Reshape Companies Over the Next Decade

“In the next decade, we’ll experience more progress than in the past 100 years combined, as technology reshapes health and materials sciences, energy, transportation, and a wide range of other industries and domains,” notes a recent McKinsey study on the The top trends in tech. “The implications for corporations are broad.”

“These trends may not represent the coolest, most bleeding-edge technologies. But they’re the ones drawing the most venture money, producing the most patent filings, and generating the biggest implications for how and where to compete and the capabilities you need to accelerate performance.”

“Unifying and underlying them all is the combinatorial effect of massively faster computation propelling new convergences between technologies; startling breakthroughs in health and materials sciences; an array of new product and service functionalities; and a strong foundation for the reinvention of companies, markets, industries, and sectors.”

How Does AI Improve Human Decision-Making?

“Artificial Intelligence (AI) is the newest general-purpose technology (GPT),” wrote Sukwoong Choia, Namil Kim, Junsik Kim, and Hyo Kang in a recent paper, How Does AI Improve  Human Decision-Making? Evidence from the AI-Powered Go Program. “One remarkable characteristic of AI is its ability to provide humans with high-quality predictions at relatively low cost and to automate a wide range of predictions. AI has already outperformed human professionals in many domains - including strategic gameplay, medical diagnosis, and new drug development.” 

“Furthermore, as a GPT, the domains where AI outperforms humans are expanding at a fast pace. The rapid development and adoption of AI thus raise an interesting yet pressing question about how AI affects human tasks in these various domains. Studies of AI and human capital, for instance, have examined the performance gap between humans and AI, highlighting AI’s potential for replacing jobs.” However, we should keep in mind that new technologies have been replacing workers and transforming economies for over two centuries. But, over time, these same technologies led to the creation of whole new industries and new jobs, increasing productivity and economic output, raising earnings, and augmenting the demand for labor.

Workers can increase their productivity and performance when assisted in their jobs by AI-based tools. But beyond its assistant role, AI can play a major instructional role by training human professionals to make better decisions. Recent technology advances have significantly improved the quality and reduced the cost of AI-based predictions.

The paper raises a number of important questions about the use of AI to improve human decisions making. Can we show quantitative evidence that AI-based predictions actually improve the quality of human decisions? If so, by what mechanisms are AI-based human decisions improved? And, do the improvements vary between depending on ones attitude toward AI-based decisions?

The Promise & Peril of Human-Like Artificial Intelligence

In his 1950 seminal paper, Computing Machinery and Intelligence, Alan Turing proposed what’s famously known as the Turing test, - a test of a machine’s ability to exhibit intelligent behavior indistinguishable from that of a human. If a human at a keyboard couldn’t tell whether they were interacting with a machine or a human, the machine is considered to have passed the Turing test. “Ever since, creating intelligence that matches human intelligence has implicitly or explicitly been the goal of thousands of researchers, engineers and entrepreneurs,” wrote Erik Brynjolfsson, - Stanford professor and Director of the Stanford Digital Economy Lab, - in a recent article, The Turing Trap: The Promise & Peril of Human-Like Artificial Intelligence.

“The benefits of human-like artificial intelligence (HLAI) include soaring productivity, increased leisure, and perhaps most profoundly, a better understanding of our own minds. But not all types of AI are human-like - in fact, many of the most powerful systems are very different from humans - and an excessive focus on developing and deploying HLAI can lead us into a trap. … On the one hand, it is a path to unprecedented wealth, increased leisure, robust intelligence, and even a better understanding of ourselves. On the other hand, if HLAI leads machines to automate rather than augment human labor, it creates the risk of concentrating wealth and power. And with that concentration comes the peril of being trapped in an equilibrium where those without power have no way to improve their outcomes, a situation I call the Turing Trap.”

Over the past decade, powerful AI systems have matched or surpassed human levels of performance in a number of tasks such as image and speech recognition, applications like skin cancer classification and breast cancer detection, and complex games like Jeopardy and Go. These AI breakthroughs are generally referred to as soft, narrow or specialized AI, inspired by, but not aiming to mimic the human brain. They’ve been generally based on machine learning, that is, on the analysis of vast amounts of information using powerful computers and sophisticated algorithms, whose results exhibit qualities that we associate with human intelligence.

The Work of the Future Is Ours to Invent

The Work of the Future: Building Better Jobs in an Age of Intelligent Machines, by David Autor, David Mindell and Elisabeth Reynolds was published last week. The new book is based on the multiyear MIT Task Force on The Work of the Future that they jointly co-chaired.

The Task Force was commissioned by MIT President Rafael Reif in the spring of 2018 to address one of the most critical questions of the digital economy, - an economy that technology universities like MIT have played a major role in bringing about: as emerging technologies raise aggregate economic output and the wealth of nations, will they also enable people to attain higher living standards, better working conditions, greater economic security, and improved health and longevity? Led by Autor, Mindell and Reynolds, the task force involved over 20 faculty members from 12 departments at MIT as well as more than 20 graduate students. Its final report was released in November of 2020. In addition, the Task Force published a number of working papers and research briefs.

“Amidst a technological ecosystem delivering rising productivity, and an economy generating plenty of jobs (at least until the COVID-19 crisis), we found a labor market in which the fruits are so unequally distributed, so skewed towards the top, that the majority of workers have tasted only a tiny morsel of a vast harvest.” was the Task Force overarching conclusion. But, it argued that with better policies in place, more people could enjoy good careers even as new technologies transform the very nature of work.

Let me summarize the Task Force key findings which the book builds on:

Realizing the Economic Promise of Predictive Analytics

Wikipedia defines predictive analytics as a set of statistical techniques, - such as data mining, business analytics, and machine learning, - “that analyze current and historical facts to make predictions about future or otherwise unknown events.” Increasingly powerful and inexpensive computing technologies, new algorithms and models, and huge amounts of data on almost any subject have led to major advances in predictive analytics over the past two decades.

“Worldwide revenue for ‘big data’ and business analytics solutions is forecasted to reach $274.3 billion by 2022,” wrote economists Erik Brynjolfsson, Wang Jin, and Kristina McElheran in The Power of Prediction, a research article published earlier this year. In principle, such widespread use of predictive analytics should have a positive impact of the performance of firms. “However, these investments have yet to yield productivity gains in the aggregate,” said the authors. “At the firm level, managers struggle to close the gap between the promise of predictive analytics and its performance. These concerns have been difficult to tackle empirically due to the rate of technological change and, ironically, a dearth of data.”

To address these concerns, the authors launched a research study in collaboration with the US Census Bureau to collect information on the use of predictive analytics in a representative sample of the US manufacturing industry, - an industry that’s historically been an early adopter of leading-edge technologies.