On May 22 I attended the 2013 MIT Sloan CIO Symposium. This year’s theme was The Transformational CIO: Architecting the Enterprise of the Future. “The enterprise of the future will be very different from the one we know,” wrote the organizers in the event’s website. “It will be complex, develop hitherto unforeseen forms, and will have to respond to unforeseen challenges. All while the windows of opportunity and response will continue to shrink. There will not only be the need to be agile and adapt to changes as they occur, but to be proactive in shaping the next generation enterprise to be ready for the future.”
The Symposium included a number of talks and panels on the key issues facing CIOs in our post-digital world, that is, in an era when digital technologies permeate just about every nook and cranny of the business, and where every business is a digital business, the title of one of the talks at the event. While a number of transformative technologies were discussed, big data was the most prominent.
MIT professor Erik Brynjolfsson led an all-MIT academic panel of experts on The Reality of Big Data. In a brief introductory talk, Brynjolfsson pointed out that throughout history new tools beget revolutions. Scientific revolutions are launched when new tools make possible all kinds of new measurements and observations. In 1676, for example, Antonie van Leeuwenhoek used a microscope, a relatively recent and rare tool, to discover the existence of microorganisms in a drop of water. Thus was microbiology born, leading to major discoveries in biology, medicine, public health and food production in subsequent decades and centuries.
Big data is such a measurement revolution made possible by the new digital tools all around us, including location data transmitted by our mobile phones; searches, web links and social media interactions; payments and transactions; the myriads of smart sensors keeping track of the physical world; and so on. As one of the panelists, Media Lab professor Sandy Pentland recently put it in this online conversation:
“This is the first time in human history that we have the ability to see enough about ourselves that we can hope to actually build social systems that work qualitatively better than the systems we’ve always had. . . . That’s a remarkable change. It’s like the phase transition that happened when writing was developed or when education became ubiquitous, or perhaps when people began being tied together via the Internet.”
Continuing with his talk, Brynjolfsson added that beyond being a technology and scientific revolution, big data should be viewed as a management revolution. We have mostly been managing by gut, intuition and hunches because we’ve lacked the appropriately analyzed data to do otherwise. Key business and strategic decisions are frequently made by what he called the HiPPO or Highest Paid Person with an Opinion. “We need to change from opinions and hunches and go with facts and data,” he said. By feeding data to the HiPPOs we can turn them into geeks.
“[The] big data of this revolution is far more powerful than the analytics that were used in the past,” wrote Brynjolfsson and Andy McAfee in a recent Harvard Business Review article: Big Data: The Management Revolution. “We can measure and therefore manage more precisely than ever before. We can make better predictions and smarter decisions. We can target more-effective interventions, and can do so in areas that so far have been dominated by gut and intuition rather than by data and rigor. As the tools and philosophies of big data spread, they will change long-standing ideas about the value of experience, the nature of expertise, and the practice of management. Smart leaders across industries will see using big data for what it is: a management revolution.”
Brynjolfsson cited insurance underwriting, housing sales, and even wine ratings as some of the industries where big data is already having an impact. He referenced a 2011 research paper which he co-authored that found that those companies that have embraced data-driven decision making enjoyed 5 percent higher productivity and 6 percent higher profits that what would be expected given their other investments and IT usage. In addition, data-driven decision making also improved other performance measures in these companies, including asset utilization, return on equity and market value.
Professor Dimitris Bertsimas, another member of the panel, talked about his research analyzing decades of cancer treatment data in the hope of improving the life expectancy and quality of life of cancer patients at reasonable costs. Along with three of his students, he developed models for predicting survival and toxicity using patients’ demographic data as well as data on the chemotherapy drugs and dosages they were given. Their results show that it’s possible to predict future clinical trial outcomes based on past data, even if the exact combination of drugs being predicted has never been tested in a clinical trial before.
Bertsimas told us that his research was motivated by the cancer treatment his own father went through. In 2012, he published a paper with his students, An Analytics Approach to Designing Clinical Trials for Cancer, which he dedicated to the memory of his father.
“We believe that our approach to apply analytics to the design of clinical trials has the potential to signiﬁcantly advance the state of the art in meta-analysis of cancer chemotherapy trials and fundamentally changing the design process for new chemotherapy clinical trials,” they wrote in the concluding remarks. “This approach can help medical researchers identify the most promising drug combinations for treating different forms of cancer by drawing on previously published clinical trials. This would save researchers’ time and effort by identifying proposed clinical trials that are unlikely to succeed and, most importantly, save and improve the quality of patients’ lives by improving the quality of available chemotherapy regimens.”
The third panelist, Professor of Finance Andrew Lo, is looking at data from banks, hedge funds, insurance companies, sovereign funds and other financial institutions to see if he can detect patterns that could alerts us to the potential for another financial crisis. This is a particularly difficult task because financial service companies have largely kept their data segregated, hampering our ability to analyze the financial system as the highly interconnected, interdependent, holistic system it actually is. In fact, in the five years since the crisis, our global financial system has become even more interconnected than it was in the years preceding the crisis.
“We’re only now at the beginning of understanding how to map the financial system,” Lo said. “I’m optimistic that over the next five or ten years we’re going to be building a much more advanced financial system. But over the shorter term, in the next one to five years, I think it’s going to be Moore’s Law versus Murphy’s Law.” His use of Moore’s Law versus Murphy’s Law is explained in this recently published paper of the same title.
The panel strongly believed that privacy is a major area of concern for big data applications, especially in regulated industries like healthcare and financial services. But, there are a number of things we can do to ameliorate these concerns. According to Lo, part of the answer lies in the use of secure multi-party computation, a sub-field of cryptography that enables statistics and other functions to be calculated and shared while keeping the individual inputs used in the calculations totally anonymous.
Professor Pentland advocates that individuals should have the final say about the use of the data collected about them, including the ability to put the data in circulation and turn it into a personal asset by giving permission to share it for value in return. In 2011, he founded the Institute for Data Driven Design - ID3, a research and educational nonprofit to help define the kind of principles, contracts and rules needed to empower individuals to assert greater control over their data and digital identities and authentication. ID3 is developing software mechanisms and an open software platform to implement and enforce these principles.
Our new big data tools have the potential to usher an information-based scientific revolution in healthcare, finance, management and a number of other human endeavors. We need to learn how to best leverage our tools, - their benefits as well as their limits, - and how to surmount major obstacles, including the serious privacy concerns discussed in the panel. Like all scientific revolutions, this will take time, as we learn how to architect not only the enterprise of the future but our future information-based society.