Irving Wladawsky-Berger

Regulation by Design: Embedding Regulatory Objectives into the Design of AI Systems

In its early decades, IT helped companies become significantly more productive by automating many of their processes, reengineering their overall operations, and reaching out beyond the boundaries of the enterprise. But, their actual products were largely unaffected. This has all been changing in the 21st century with the rise of the Internet of Things (IoT), Big Data, and Artificial Intelligence, whose combined impact is driving a new era of smart connected products.

As the world’s digital and physical infrastructures converge, digital technologies are designed right into complex products, — e.g., jet engines, power generators, medical equipment, energy pipelines. Massive amounts of usage data can now be gathered over the internet, stored and analyzed by sophisticated applications to help monitor the product and anticipate potential problems. This is particularly important in industries where safety by design can have a significant economic impact and in some cases, literally save lives. Safety by design features are now in widespread use in all kinds of mechanical, electronic, and other complex physical systems.

What if we could similarly embed regulatory objectives directly into the technical design of AI systems?, asked Robert Mahari in  his keynote on Regulation by Design at the recent 2024 MIT IDE Annual Conference. Mahari is pursuing a joint JD-PhD degree. He received a JD at the Harvard Law School in 2022 and is now a PhD candidate in the MIT Media Lab research group led by professor Alex ‘Sandy’ Pentland.

“Compliance and regulation by design represent a risk-management paradigm that’s uniquely suited for AI,” Mahari said. “Intelligent technology design can proactively prevent failures and risks.”

AI systems are the result of a very complex supply chain, including the choice of the large amounts of data needed to train AI models, and the difficulty of testing, predicting, and explaining the models’ expected behavior. As a result, compliance with regulatory and ethical objectives is very hard for the users of AI-based applications who have little control or understanding of how the AI system works. Instead, as is the case with advanced engineering systems, can an AI system monitor how it’s used, identify high risk sessions, alert its developers and overseers, and thus ensure compliance?

Language is a Tool for Communication, Not for Thinking

I often look to biology and evolution for inspiration when thinking about highly complex systems. So I was very intrigued when several weeks ago I learned about the work being done by Elemental Cognition, a company founded in 2015 by my former IBM colleague David Ferrucci. Ferrucci led the team that developed the Watson computer system, which in 2011 won the television quiz show Jeopardy! against two of the world’s top Jeopardy champions. Let me explain.

Large language models (LLMs), chatbots, and generative AI have introduced technologists like me to the fascinating world of human language and cognition. Last year, for example, I learned the difference between four important linguistic concepts in a 2020 research paper by linguistic professors Emily Bender and Alexander Koller.

• Form is any observable expression of language, whether written, spoken, or signed.
• Communicative intent is the purpose a speaker intends to achieve through language, such as to convey some information or just to socialize.
• Meaning is the relation between the form in which the language is expressed and the communicative intent it’s being used to evoke in the listener or reader.
• Understanding is the listener’s ability to capture the meaning that the speaker intends to convey.

The authors wrote that while the success of LLMs and chatbots on many natural language tasks is very exciting, “these successes sometimes lead to hype in which these models are being described as understanding language or capturing meaning.” However, “a system trained only on form has apriori no way to learn meaning.” Research on language acquisition has found that “human language learning is not only grounded in the physical world around us, but also in interaction with other people in that world. … Human children do not learn meaning from form alone and we should not expect machines to do so either.”

Embracing Beneficial Friction for Human-Centered AI

Psychological studies have shown that our judgement is affected by the many decisions we make every day and the friction-based paint points of making these decisions. So, not surprisingly, we look to technology and algorithms to help us make our decisions as frictionless as possible, said Renee Gosline in her keynote at the 2024 MIT IDE Annual Conference. Gosline is a Research Scientist and Senior Lecturer at the MIT Sloan School of Management, and the head of the Human-First AI group at MIT's Initiative on The Digital Economy. “But when AI is involved, sometime we need to add beneficial friction. Sometimes it’s useful for us to engage System 2,” she added, referencing the pioneering behavioral research on human judgment and decision-making by Daniel Kahneman, for which he was awarded the 2002 Nobel Prize in Economics.

In his 2011 bestseller Thinking, Fast and Slow, Kahneman wrote that our mind is composed of two very different systems for reasoning and making decisions, which he called System 1 and System 2.  System 1 is the intuitive, fast and emotional part of our mind. Thoughts come automatically and very quickly to System 1, without us doing anything to make them happen. System 2 is the slower, logical, more deliberate part of the mind. It’s where we make complex decisions by evaluating and choosing among multiple options.

Gosline’s human-first AI research group aims to “add beneficial friction in AI-mediated systems to ensure that humans are in the loop,” as well as to reduce bad friction. When a choice is being made, “we might want to remove friction so that we can make the best choice with AI’s help. But there may be circumstances when we want to add friction to reduce harm.”

How Does the Linux Foundation Work and What Does It Do?

I have long been involved with the Linux Foundation (LF). In 2000, a consortium of companies, including IBM, HP, and Intel, founded the Open Source Development Labs (OSDL), a non-profit organization to support the continued development of Linux and promote its use in enterprise computing. A few years later, in 2007, the OSDL merged with the Free Standards Group to become the Linux Foundation. At the time, I was leading the Linux initiative that IBM had launched in January of 2000 to embrace Linux across all the company’s products and services. In 2011, when the LF marked the 20th anniversary of Linux at its annual Linux conference, I had the privilege of giving a keynote presentation where I recounted my personal involvement with Linux and open source.

Over the past decade, the LF has undergone  a major expansion, as open source became widely accepted as a paradigm for innovation across a full suite of technologies, proving its utility beyond the collaborative development of the Linux operating system. Its impact has been felt across just about all industry sectors in our fast growing digital economy. Consequently, its annual conferences were rebranded Open Source Summits to be more representative of the LF’s expanded open source mission beyond Linux. In April of 2021, the LF announced the formation of Linux Foundation Research to broaden the understanding of open source projects and to explore new opportunities for open source initiatives. Later that year, Hilary Carter, LF’s then VP of Research, invited me to become a member of the LF Research Advisory Board.

I am truly impressed by the current scope of the Linux Foundation. The LF now includes over 1,400 company members, supports over 1000 open source projects in a variety of vertical and horizontal industry sectors, holds over 70 events around the world every year, and provides training and certification programs to equip developers with essential skills, not just for coding, but to support the management and best practices the projects themselves. The mission of the LF was quite easy to explain when it was mostly focused on Linux, but it’s quite a bit more complicated now.

LF’s Executive Director Jim Zemlin recently addressed the difficulty of explaining what the LF does and how it works in his June 13 weekly newsletter to the LF board and team members. Zemlin wrote that beyond the technical community, many don’t understand much about how the LF operates, such as the important role of the LF’s company members in sponsoring open source projects, and that generally, the key maintainers of LF’s largest projects are not volunteers but full-time employees of member companies.