Irving Wladawsky-Berger

Creating More Intelligent Organizations

I recently read a fascinating online conversation on Collective Intelligence with Tom Malone.  Malone is Professor of Management at MIT’s Sloan School and the founding director of the MIT Center for Collective Intelligence (CCI).  The CCI was founded to study how new communications technologies are changing the way people work together.  In particular, it is studying the various ways that groups of people are now collaborating via the Internet, in order to understand how to best organize these collaborations and enhance them with innovative IT-based tools and platforms.  

Collective intelligence is not a new topic.  From time immemorial, groups of people working together have acted in ways that seem intelligent.  But, as Malone points out, “It’s also possible for groups of people to work together in ways that seem pretty stupid, and I think collective stupidity is just as possible as collective intelligence.  Part of what I want to understand and part of what the people I’m working with want to understand is what are the conditions that lead to collective intelligence rather than collective stupidity.” 

The CCI is conducting research on a number of topics related to collective intelligence.  One of the most intriguing is their attempt to try to ascertain if something like collective intelligence indeed exists, and if so, to try to measure a group’s intelligence using methodologies and statistical techniques similar to those that have been applied to individual intelligence for the past hundred years.

The Future of Jobs in the Digital Economy

I recently participated in a roundtable discussion on Work and Value in the Digital Economy, convened by MIT’s Center for Digital Business.  The roundtable addressed some of the most critical questions we face in our emerging digital economy, including: what is the future of jobs; where will jobs come from in the coming years?; and will the nature of work be significantly different in the digital economy? 

Since the advent of the financial crisis around five years ago, we have been searching for answers to these very important questions.  The US economy has gone through a number of ups and downs over the past seventy years.  Unemployment went up during the down periods, but it did not last long.  The economy started to improve within a relatively short time and unemployment went down once more. 

But our current downturn seems to be fundamentally different from the recessions of the past decades.  While US unemployment rates have now edged down below 8 percent, about 12 million people remain unemployed, with 40 percent classified as long-term unemployed, that is, jobless for 27 weeks or more.  An additional 8 million are involuntary part-time workers because they have been unable to find a full-time job.

It is generally agreed that our persistent high unemployment rate, - the longest such period since the Great Depression, - is primarily caused by a fundamental reshaping of the economy.  The US and other advanced nations are going through structural changes driven to a large extent, by advances in information technologies.  Companies are able to do their present work with fewer people, as a result of the major improvements in IT-based labor productivity in the past two decades.  At the same time, sophisticated IT infrastructures have enabled them to optimize their supply chains and shift work around the world to cut costs.

The Science of Information-based Predictions

For months, just like a good portion of the country, I followed closely our 2012 presidential campaign.  Up to the very end, many pollsters and political pundits were saying that the election was too close to call.  But, whenever friends and colleagues asked my opinion, I invariably told them to go look at FiveThirtyEight.com, the political polling website and blog created by Nate Silver. 

Silver launched the FiveThirtyEight in March of 2008.  He gained national attention when he correctly predicted the results of the 2008 Democratic Party presidential primaries.  His final forecasts for the 2008 presidential elections predicted the winner in 49 of the 50 states, as well as the winner of every race in the Senate.  The FiveThirtyEight has been affiliated with the NY Times since August of 2010.   

Political forecasting attracts a variety of people.  Many pollsters seem to still be using methodologies that worked well a decade or two ago, but look woefully behind the times in the emerging world of big data and advanced analytics. Most political pundits seem to inhabit a kind-of magical realism world of their own: if you say what you want to be true often enough and loud enough, it will eventually come to pass. 

Silver, on the another hand, views information-based predictions, including political forecasting, as a scientific discipline.  You use all available information; you analyze and extract insights out of all that information using sophisticated models and algorithms; you apply human judgement to make predictions based on those insights; and you keep evaluating and adjusting your models and predictions based on how well they perform in the real world.

Big Data, Complex Systems and Quantum Mechanics

Big Data is one of the hottest topics out there.  Big data is a foundational element in IT’s quartet of Next Big Things: Social, Mobile, Analytics and Cloud.  But, as the real world keeps reminding us, it is possible to make bad predictions and decisions even if you use tons of big data to make them.  The 9/11 attacks showed how even highly sophisticated intelligence agencies can fail to pick out highly relevant signals amidst the mountains of data being analyzed.  Our recent financial crisis showed how even the best and brightest can fail to detect an approaching catastrophic storm.  The failure of so many professional forecasters to accurately predict the 2012 presidential election shows that you can find almost any answer you want in all that big data.

Big data is indeed incredibly useful in all kinds of endeavors, but only in the hands of talented professionals who know what they are doing and are aware of its pitfalls and limitations.  What are some of these limitations?  In thinking about this question over the last few years, I started to notice that a number of subtle, non-intuitive concepts that I learned many years ago as a physics student seem to apply to the world of big data and information-based predictions in highly complex systems.  Let me explain. 

Over 300 years ago, Isaac Newton laid down the foundations of classical mechanics with the publication of his Laws of Motion.  The elegant mathematical models of Newtonian physics depict a world in which objects exhibit deterministic behaviors, that is, the same objects, subject to the same forces, will always yield the same results.  These models make perfect predictions within the accuracy of their human-scale measurements.  Classical mechanics works exceptionally well for describing the behavior of objects that are more or less observable to the naked eye.  It accurately predicts the motion of planets as well as the flight of a baseball. 

But, the idea of scientific determinism, which would in principle enable us to predict the future behavior of any object in the universe, began to fall apart in the early 20th century.  Classical mechanics could not explain the counter-intuitive and seemingly absurd behavior of energy and matter at atomic as well as cosmological scales.  Once you start dealing with atoms, molecules, exotic subatomic particles, black holes and the Big Bang, you find yourself in a whole different world, with somewhat bizarre behaviors like the tunneling effect which are governed by the laws of quantum mechanics and relativity.  The orderly, deterministic world of classical physics gives way to a world of wave functions, probability distributions, uncertainty principles, and wave-particle dualities.

Hurricane Sandy: a 21st Century Wake-Up Call?

As I was personally experiencing the fury of Hurricane Sandy a few weeks ago, I thought of the Interstate Highway System and President Dwight D. Eisenhower.  Sandy brought into sharp relief how dependent so many of us have become on digital infrastructures, and the Internet in particular, for almost every aspect of our life and work.  Sandy also demonstrated how inadequate and fragile our digital infrastructures are given our increasing dependence on them.  If President Eisenhower was around today, I reflected in moments of frustration, I was convinced that he would be a strong advocate for building a world class national broadband infrastructure, just like he championed the Internet Highway System over half a century ago.

The National Interstate and Defense Highways Act was signed into law by President Eisenhower in June of 1956.  Most everyone would agree that it has contributed greatly to the economic success of the US over the past fifty years.  But, in addition to economic development, one of the main reasons President Eisenhower championed the Interstate Highway System was national defense.  While serving as Supreme Commander of the Allied forces in Europe during World War II, he saw first hand the role played by the German Autobahn network in its national defense, and felt that the US needed such a modern highway system to provide key ground transport routes for military supplies and troop deployments in case of an emergency or external threat. 

Getting ready for Sandy, and then feeling its power was very different from watching a far away disaster on television.  I got off fairly easily compared with so many in the New York Metropolitan Area,  including our town in Connecticut, whose homes were flooded or were seriously damaged by falling trees.  We lost a few large trees, one of which damaged part of a fence.  The power in our area was out for almost a week, but, like an increasing number of my neighbors in our woodsy area, we have adjusted to the frequent power losses by installing a standby generator, so our house had electricity and heat during the power outage.  We also lost cable and our cable-based Internet access.  My wireless mobile phone was barely usable other than very early in the morning before most of the town was awake.  For a couple of days, even that did not work. 

Over 22 million people live in the Tri-state New York Metropolitan Area.  The region is arguably the world’s center of commerce and banking, home to almost 20 percent of Fortune 500 companies.  New York City is ranked at the top by just every survey of the world’s most competitive cities, particularly because of its appeal to a wide range of businesses and talented people. 

But, as Sandy proved to us all, our physical infrastructure is antiquated, and our digital infrastructure lags far behind the growing importance of the Internet in the economy and in our personal lives.  With few exceptions, the situation is not much better across the US.  The infrastructures of just about all US metropolitan areas are falling behind the rest of the world.