Artificial intelligence first came to light in the mid-1950s as a promising new academic discipline. AI became one of the most exciting areas in computer sciences over the next two decades. But, after years of unfulfilled promises and hype, a couple of so called AI winters of reduced interest and funding set in that nearly killed the field. AI was successfully reborn in the 1990s with a new statistical paradigm based on analyzing large amounts of data with powerful computers and sophisticated algorithms. Now, six decades after the field was founded, AI seems to be finally coming of age.
“2021 saw the globalization and industrialization of AI intensify, while the ethical and regulatory issues of these technologies multiplied,” said the 2022 AI Index report on the progress of AI, which was released in March of 2022 by Stanford’s Institute for Human-Centered Artificial Intelligence (HAI). “2021 was the year that AI went from an emerging technology to a mature technology - we’re no longer dealing with a speculative part of scientific research, but instead something that has real-world impact, both positive and negative,” wrote Jack Clark, co-chair of the AI Index. Multiple factors led to his conclusion, in particular the advent of foundation models like OpenAI’s GPT-3 and Google’s BERT.
In the past decade, increasingly powerful AI systems have matched or surpassed human levels of performance in a number of specific tasks like image and speech recognition. These task-specific deep learning (DL) systems have generally relied on supervised learning, a training method where the data must be carefully labelled, - e.g., cat, not-cat, - thus requiring a big investment in time and money to produce a model that’s narrowly focused on one task and can’t be easily repurposed.
Foundation models have gotten around these DL limitations based on two recent advances, transfer learning and huge scale. Transfer learning takes the knowledge learned from training a specific task and applies it to different but related tasks, - such as using the training in recognizing cars in images and applying it to recognizing trucks and buses. As a result, foundation models can be adapted to many different tasks with relatively small amounts of fine-tuning.
The huge scale of these models is based on three recent advances: major improvements in the performance and costs of computer hardware; new transformer architectures that enable these large models to take advantage of highly parallel computers systems; and over 10X more training data than previous DL models, including big chunks of the information on the internet as well as books, articles, reports, and other digital media.
On November 30, OpenAI released ChatGPT, a version of GPT-3 which interacts with users in a conversational format. “The dialogue format makes it possible for ChatGPT to answer followup questions, admit its mistakes, challenge incorrect premises, and reject inappropriate requests.” OpenAI encourages users to try ChatGPT, and many millions have already done, frequently overloading the system with very high traffic.
“How many people have tried ChatGPT?,” I asked ChatGPT before posting this blog in late January, to which it replied: “I do not have an exact number of how many people have tried ChatGPT, but I know that it is widely used by researchers, developers, and companies all over the world to build natural language processing (NLP) applications such as chatbots, language translation, and text summarization. It is also used by many individuals for personal projects and experimentation.”
Over the past few weeks, I’ve read a number of the many articles written about ChatGPT. They span the gamut from great admiration to deep skepticism. Let me discuss a few of these articles.
“ChatGPT is, quite simply, the best artificial intelligence chatbot ever released to the general public,” wrote NY Times technology columnist Kevin Roose in “The Brilliance and Weirdness of ChatGPT.” ChatGPT “landed with a splash … and many of its early fans speak of it in astonished, grandiose terms, as if it were some mix of software and sorcery.” Previous AI tools have gotten good at narrow, well defined tasks, he added “but they still tend to flail when taken outside their comfort zones,” and AI chatbots have been terrible.
“But ChatGPT feels different. Smarter. Weirder. More flexible.” It can write jokes (some of which are actually funny), working computer code and college-level essays. It can also guess at medical diagnoses, create text-based Harry Potter games and explain scientific concepts at multiple levels of difficulty.”
These new generation of chatbots are inspiring both awe and fear. Their economic impact could be similar to that of electricity a century ago. On the other hand, this could be “the beginning of the end of all white-collar knowledge work, and a precursor to mass unemployment.”
“Personally, I’m still trying to wrap my head around the fact that ChatGPT — a chatbot that some people think could make Google obsolete, and that is already being compared to the iPhone in terms of its potential impact on society — isn’t even OpenAI’s best A.I. model,” wrote Roose in his concluding paragraph. “That would be GPT-4, the next incarnation of the company’s large language model.” GPT-4 is rumored to be coming out sometime in 2023 and is expected to be significantly more powerful than GPT-3.
Let me now turn to a more skeptical article. “The articulate new chatbot has won over the internet and shown how engaging conversational AI can be — even when it makes stuff up,” wrote Wired senior writer Will Knight in “ChatGPT’s Most Charming Trick Is Also Its Biggest Flaw.” “ChatGPT stands out because it can take a naturally phrased question and answer it using a new variant of GPT-3, called GPT-3.5. This tweak has unlocked a new capacity to respond to all kinds of questions, giving the powerful AI model a compelling new interface just about anyone can use.”
But, “putting a slick new interface on a technology can also be a recipe for hype,” added Knight. “Despite its potential, ChatGPT also shows flaws known to plague text-generation tools. Over the past couple of years, OpenAI and others have shown that AI algorithms trained on huge amounts of images or text can be capable of impressive feats. But because they mimic human-made images and text in a purely statistical way, rather than actually learning how the world works, such programs are also prone to making up facts and regurgitating hateful statements and biases — problems still present in ChatGPT. Early users of the system have found that the service will happily fabricate convincing-looking nonsense on a given subject.”
Other articles have focused on the business potential of these powerful AI technologies. In “The Tech That Will Invade Our Lives in 2023,” NY Times consumer technology writer Brian Chen predicts that 2023 will bring us a variety of new-and-improved AI assistants.
“Early adopters who have been wowed by the linguistic competence of ChatGPT have just as quickly been stunned by how wrong it can be, particularly with simple arithmetic,” he wrote. Flaws aside, we can realistically expect A.I. companies to improve on the strengths of these chatbots with tools that streamline how we write and read text, A.I. experts say. “For one, it’s very likely that next year you could have a chatbot that acts as a research assistant. … That doesn’t mean that we’ll see a flood of stand-alone A.I. apps in 2023. It may be more the case that many tools we already use for work will begin building automatic language generation into their apps.”
Similarly, in “Artificial intelligence is permeating business at last,” The Economist wrote that “Powerful new foundation models are fast moving from the lab to the real world. Chatgpt, a new ai tool that has recently been released for public testing, is making waves for its ability to craft clever jokes and explain scientific concepts. But excitement is also palpable among corporate users of ai, its developers and those developers’ venture-capital backers.”
2022 may well be remembered as the year when AI finally came of age. As this most recent NY Times article noted,“there are many ways these bots are superior to you and me. They do not get tired. They do not let emotion cloud what they are trying to do. They can instantly draw on far larger amounts of information. And they can generate text, images and other media at speeds and volumes we humans never could.” But, at the same time, we need to learn “what these systems do well and what they cannot, how they will replace human labor in the near term and how they will not.” Realizing the potential of a major new transformation technologies takes considerable time. There’s clearly much work to be done.
Comments