“It’s natural to ask whether large language models like LaMDA (short for Language Model Dialogue Application) or GPT-3 are really smart — or just double-talk artists in the tradition of the great old comedian Prof. Irwin Corey, ‘The World’s Greatest Authority,’ wrote UC Berkeley cognitive and developmental psychologist Alison Gopnik in ‘What AI Still Doesn’t Know How to Do,’ a July, 2022 WSJ essay. “But I think that’s the wrong question. These models are neither truly intelligent agents nor deceptively dumb. Intelligence and agency are the wrong categories for understanding them.”
“Instead, these AI systems are what we might call cultural technologies, like writing, print, libraries, internet search engines or even language itself. They are new techniques for passing on information from one group of people to another. Asking whether GPT-3 or LaMDA is intelligent or knows about the world is like asking whether the University of California’s library is intelligent or whether a Google search knows the answer to your questions. But cultural technologies can be extremely powerful — for good or ill.”
A recent article in The Economist, “How AI could change computing, culture and the course of history,” made a similar point. That article aimed to give us a sense of the world-changing transformations we might expect from generative AI by discussion two of of the most transformative cultural technologies of the past few hundred years: the printing press and the web browser.
The printing press, invented by Johannes Gutenberg around 1440, accelerated the spread of knowledge and literacy in Renaissance Europe. Gutenberg’s printing revolution influenced almost every facet of life in the centuries that followed, starting with the Protestant Reformation which used the recently developed printing press to undermine the Catholic Church monopoly in information dissemination.
Then over five centuries later, the web browser, introduced in the early 1990s as a gateway to the fast growing World Wide Web, revolutionized the way digital information is accessed and organized, thus transforming the whole IT industry. The browser ushered our 21st century digital economy by providing an easy way to access the huge variety of information and applications in the Web to anyone with a personal computer, a laptop, or a mobile device and an internet connection.
The very breadth of the printing press and the browser-based World Wide Web makes comparison with generative AI and large language models (LLMs) unavoidable. Printed books and digital information have significantly expanded the knowledge we’ve all had access to, helping us generate lots of new knowledge in all kinds of disciplines.
“Cultural technologies aren’t like intelligent humans, but they are essential for human intelligence,” wrote professor Gopnik. “To paraphrase Isaac Newton, every new human can see so far because they rest on the shoulders of those who came before them. New technologies that make cultural transmission easier and more effective have been among the greatest engines of human progress. … The printing press helped enable both the industrial revolution and the rise of liberal democracy. Libraries, and their indexes and catalogs, were essential for the development of science and scholarship. Internet search engines have made it even easier to find information.”
In his 1950 seminal paper, Computing Machinery and Intelligence, British mathematician and computer scientist Alan Turing proposed what he called the imitation game, — now known as the Turing test, — a test of a machine’s ability to exhibit intelligent behavior indistinguishable from that of a human. If humans at a keyboard couldn’t tell whether they were interacting with a machine or a human, the machine was considered to have passed the Turing test.
While the Turing test has been well known for decades, fewer people know that in the very same paper, Turing presciently suggested that the key to human intelligence is our ability to learn. We should therefore design a learning machine that simulates how children’s minds learn, including their initial state at birth, the education they receive over the years, and any other experiences that have shaped their mind into that of an adult.
“Cultural transmission has two sides — imitation and innovation,” explained Gopnik. “Each generation can use imitation to take advantage of the discoveries of previous ones, and large language models are terrific imitators. But there would be no point to imitation if each generation didn’t also innovate. We go beyond the words of others and the wisdom of the past to observe the world anew and make new discoveries about it. … For true intelligence, a computer should not only be able to talk about the world like a human adult — it should be able to learn about the world like a human child.”
To explore the ability of large AI models to solve problems that require some degree of innovation, Gopnik and her UC Berkeley research team created an online experiment. The experiment enabled them to compare the performance of the AI models to those of typical 4-year olds in solving the kind of new, simple problems that comes naturally to small children.
“We showed 4-year-olds on-screen machines that would light up when you put some combinations of virtual blocks on them but not others; different machines worked in different ways. The children had to figure out how the machines worked and say what to do to make them light up. The 4-year-olds experimented, and after a few trials they got the right answer.”
“Then we gave state-of-the-art AI systems, including GPT-3 and other large language models, the same problem. The language models got a script that described each event the children saw and then we asked them to answer the same questions we asked the kids. We thought the AI systems might be able to extract the right answer to this simple problem from all those billions of earlier words. But nobody in those giant text databases had seen our virtual colored-block machines before. In fact, GPT-3 bombed.” For all their articulate speech, LLMs can’t seem to solve cause-and-effect problems.
The experiment is described in great detail in a May, 2023 research paper co-authored by Gopnik and two of her graduate students, “Imitation versus Innovation: What children can do that large language and language-and-vision models cannot (yet)?”
The 4-year olds had no trouble solving the task by exploring, through trial and error, which blocks made the machine light up. But the AI systems struggled to figure out how to solve the task because the information needed to solve the task was not part of their training data. It was a new, albeit very simple task, that required the kind of trial-and-error, real-world experimentation and innovative thinking that comes naturally to small children. To solve a new problem, reading a book or searching the Web may be a good first step, but ultimately you have to experiment the way the children did.
“One of the secrets of children’s learning is that they construct models or theories of the world,” noted Gopnik in,“The Ultimate Learning Machines,” a 2019 WSJ essay. “Toddlers may not learn how to play chess, but they develop common-sense ideas about physics.” One of the grand challenges in AI is to design a system that understands how the world works as well as an 18-month old.
Generative AI and large language models like ChatGPT are very important cultural technologies. Over time, they may well have a transformative impact on the 21st century similar to the printing press, the browser and other historical information dissemination technologies. “However, cultural evolution depends on a fine balance between imitation and innovation,” wrote Gopnik and her co-authors in their May, 2023 paper. “There would be no progress without innovation, the ability to expand, create, change, abandon, evaluate and improve on existing knowledge and skills. Whether this means recasting existing knowledge in new ways or creating something entirely original, innovation challenges the status quo and questions the conventional wisdom that is the training corpus for artificially intelligent systems.”
“Large language models can help us acquire information that is already known more efficiently, even though they are not innovators themselves. Moreover, accessing existing knowledge much more effectively can stimulate more innovation among humans and perhaps the development of more advanced AI. But ultimately, machines may need more than large scale language and images to match the achievements of every human child.”
Who was it that recognized that the emperor had no clothes :) ?
It seems like we have thrown out the semantic model with the bath water.
Posted by: W. David Maimone | August 28, 2023 at 04:07 PM