Irving Wladawsky-Berger

Productivity Gains of Generative AI for Software Developers

In June of 2023, McKinsey published “The Economic Potential of Generative AI: The Next Productivity Frontier.” Based on its analysis of 63 business use cases, and estimates of its effect on the productivity of the global workforce, McKinsey’s study projected that generative AI could add an additional $6.1 trillion to $7.9 trillion annually, boosting the overall impact of artificial intelligence on the global economy by an annual $17.1 trillion to $25.6 trillion.

The McKinsey study found that while generative AI use cases could have an impact on most enterprise business functions, four in particularly could account for approximately 75% of the total annual value from generative AI use cases: customer operations, marketing and sales, research and development, and software engineering.

Digital technologies, including advanced analytics and machine learning algorithms, have been applied to the first three functions for the past few decades, so it’s easier to understand how GenAI would build on previous work as well as give rise to innovative new advances. But that hasn’t quite been the case with software engineering.

In a recent article, “Quantifying the Productivity Gains of Generative AI for Developers,” IBM Fellow Jerry Cuomo nicely explained the role of generative AI in enhancing the productivity of software developers by examining the potential benefits for developers of varying experience levels, from novice to experts. Over the years, Cuomo has played a major role in a number of major software advances, from IBM’s WebSphere Application Server to open source Hyperledger blockchain technologies. He’s currently CTO in IBM's Consulting business unit where he’s responsible for AI-based automation technologies and strategy.

“The productivity gains enabled by the introduction of generative AI allow for a shift to value and create opportunities for increased innovation, improved quality, enhanced customer experience, and faster time-to-market,” wrote Cuomo. “Specifically, we estimate that generative AI can lead to a 15–20% increase in the number of new products or features developed, a 10–15% reduction in the number of bugs found in production, a 5–10% increase in customer retention and loyalty, and a 10–15% reduction in time-to-market for new products or features.”

What Small Children Can Do that AI Doesn’t Know How to Do (Yet)

“It’s natural to ask whether large language models like LaMDA (short for Language Model Dialogue Application) or GPT-3 are really smart — or just double-talk artists in the tradition of the great old comedian Prof. Irwin Corey, ‘The World’s Greatest Authority,’ wrote UC Berkeley cognitive and developmental psychologist Alison Gopnik in ‘What AI Still Doesn’t Know How to Do,’ a July, 2022 WSJ essay. “But I think that’s the wrong question. These models are neither truly intelligent agents nor deceptively dumb. Intelligence and agency are the wrong categories for understanding them.”

“Instead, these AI systems are what we might call cultural technologies, like writing, print, libraries, internet search engines or even language itself. They are new techniques for passing on information from one group of people to another. Asking whether GPT-3 or LaMDA is intelligent or knows about the world is like asking whether the University of California’s library is intelligent or whether a Google search knows the answer to your questions. But cultural technologies can be extremely powerful — for good or ill.”

A recent article in The Economist, “How AI could change computing, culture and the course of history,” made a similar point. That article aimed to give us a sense of the world-changing transformations we might expect from generative AI by discussion two of of the most transformative cultural technologies of the past few hundred years: the printing press and the web browser.

Why Transformative AI Is So Hard to Achieve

“Humans have a good track record of innovation,” wrote Arjun Ramani and Zhengdong Wang in an essay which systematically explains “Why transformative artificial intelligence is really, really hard to achieve.” “The mechanization of agriculture, steam engines, electricity, modern medicine, computers, and the internet — these technologies radically changed the world. Still, the trend growth rate of GDP per capita in the world's frontier economy has never exceeded three percent per year.”

“Yet many people are optimistic that artificial intelligence is up to the job. AI is different from prior technologies, they say, because it is generally capable — able to perform a much wider range of tasks than previous technologies, including the process of innovation itself. Some think it could lead to a Moore’s Law for everything, or even risks on a par with those of pandemics and nuclear war.”

The essay consists of three sections: the transformational potential of AI is constrained by its hardest problems; major technical hurdles remain; and social and economic hurdles may limit AI’s impact. Let me summarize a few of the key arguments in each section.

What’s the Likely Long Term Evolution of AI?

Since the advent of the Industrial Revolution, general purpose technologies (GPTs) have been the defining technologies of their times. Their ability to support a large variety of applications can, over time, radically transform economies, and social institutions. GPTs have great potential from the outset, but realizing their potential takes large tangible and intangible investments and a fundamental rethinking of firms and industries, including new processes, management structures, business models, and worker training. As a result, realizing the potential of a GPT takes considerable time, often decades. Electricity, the internal combustion engine, computers, and the internet are all examples of historically transformative GPTs.

For example, after the introduction of electric power in the early 1880s, it took companies about four decades to figure out how to restructure their factories to harness electric power with manufacturing innovations like the assembly line. It took even longer to develop highly popular electric household products like refrigerators, toasters, washing machines, and air conditioners.

The internet’s precursor, ARPANET,  was initially developed by the US Department of Defense in the late 1960s for military and research applications. It wasn’t until the 1990s that the internet was opened up to companies and consumers in the wider commercial marketplace, becoming the most significant platform for innovation the world has ever since, and ushering a new 21st century digital economy.

Similarly, artificial intelligence first came to light in the mid-1950s as a promising new academic discipline that aimed to develop intelligent machines capable of handling human-like tasks like natural language and playing chess. AI became one of the most exciting areas in computer sciences in the 1960s, ’70s, and ’80s, but after years of unfulfilled promises and hype, a so called AI winter of reduced interest and funding that nearly killed the field set in everywhere.

How to Improve Enterprise Project Management As We Enter the Age of AI?

“Quietly but powerfully, projects have displaced operations as the economic engine of our times,” wrote Antonio Nieto-Rodriguez in “The Project Economy Has Arrived,” a December, 2021 article in the Harvard Business Review (HBR). Nieto-Rodriguez is a Fellow and past global chair of the Project Management Institute and the author of the Harvard Business Review Project Management Handbook: How to Launch, Lead, and Sponsor Successful Projects, on which the HBR article is based.

While that shift has been a long time coming, we still have a ways to go, he added. “According to the research firm the Standish Group, around 35% of the projects undertaken worldwide are successful. Given that we’re talking about tens of trillions of dollars, and the labor of millions of employees, that’s a mind-blowing number. It tells us that we’re not only wasting 65% of the time and money that we’ve invested in our projects but also forfeiting trillions of dollars of new value for organizations and society at large.”

A recent McKinsey global survey on the state of AI in the enterprise found that AI adoption has more than doubled over the past five years. Given that the survey was conducted in 2022, before the November 30 release of ChatGPT propelled AI to a whole new level of interest, we can expect a major increase in enterprise AI projects in the coming years. Nieto-Rodriguez's HBR article is thus particularly relevant as we enter the age of AI. 

“What exactly is a project?,” he asked in the article. While project is a word we all commonly use, the article points out that the word means different things to different people. “That’s a problem. As projects drive more and more of the value that organizations create, everyone needs to have a common understanding of what projects and project management are”:

“Projects involve a series of planned activities designed to generate a deliverable (a product, a service, an event). These activities — which can be anything from a grand strategic initiative to a small program of change — are limited in time. They have a clear start and end; they require an investment, in the form of capital and human resources; and they are designed to create predetermined forms of value, impact, and benefits. Every project has elements that are unique. That’s key: Each contains something that has not been done before.”