Irving Wladawsky-Berger

Learning to Apply Data Science to Business Problems

This past semester I was involved in an interesting course at MIT’s Sloan School of Management, - Analytics Labs (A-Lab).  A-Lab’s objective is to teach students how to use data sets and analytics to address real-world business problems.  Companies submit project proposals prior to the start of the class, including the business problem to be addressed and the data on which the project will be based. Students are then matched with the project they’re most interested in and grouped into teams of 3-4 students.

A-Lab received over 20 project proposals from different companies, of which 13 were selected by the students.  Each project team was assigned a research mentor to provide guidance as appropriate.  I mentored a 3-student team that worked on a project sponsored by MasterCard.  The students explored the possibility of improving on predictions of the economic performance of emerging markets by coupling existing economic indicators data with consumer behavior based on MasterCard’s transaction data.  This is a particularly interesting project because economic data in emerging markets is often not as reliable as the data in more advanced markets.

But more important for the students, the various A-Lab projects served as a concrete learning experience on what data science is all about, - how to leverage messy, incomplete, real-world data to shed light on a complex and not-so-well-defined problem.

Trust and the On-Demand Economy

The intrinsic structure of companies has long been a subject of study, most famously by Ronald Coase, the eminent British economist and recipient of the 1991 Nobel Prize in economics.  In 1937, Coase published a seminal paper, The Nature of the Firm, in which he explained that, in principle, a firm should be able to find the cheapest, most productive goods and services by contracting them out in an efficient, open marketplace.

However, markets are not perfectly fluid.  Transaction costs are incurred in obtaining goods and services outside the firm, such as searching for the right people, negotiating a contract, coordinating the work, managing intellectual property and so on.  Firms thus came into being to make it easier to get work done.  A well managed company tries to achieve a good balance between the work that gets done within and outside its boundaries.  

Over the past few decades, the firm has been going through dramatic changes, driven by both advances in information technologies and the heightened competitive pressures brought about by globalization.  Fundamental changes have taken place in the structure of firms and in the overall flow of goods and services in the economy.  As this 2005 IBM study noted:

“The nature of competition - increasingly intense, global and unpredictable - requires strength across the board.  So the objective is to decompose the enterprise into its component parts, understand with great precision what is truly differentiating - where the enterprise has strengths and weaknesses - and then make decisions about how to build, buy or partner for world-class capability.” 

“In this model, companies can focus their energies on their true point of differentiation, instead of trying to master many domains and ultimately squander competitive advantage by dispersing focus and investment.  Rather than existing as static and fixed organizations, more enterprises could essentially become an aggregation of specialized entities with complementary interests - expanding, contracting and reconfiguring themselves in a way that best adapts to or even anticipates market dynamics.”

The Rise of the T-Shaped Organization

The concept of T-shaped skills was first introduced over 20 years ago, but its importance, - to both individuals and organizations, - has continued to rise.  A growing number of articles have been extolling the value of T-shaped professionals, that is, individuals who combine deep cognitive, analytical, and/or technical skills in a specific discipline, with broad multidisciplinary, social skills.

As described by IDEO CEO Tim Brown:  “The vertical stroke of the T is a depth of skill that allows them to contribute to the creative process.  That can be from any number of different fields: an industrial designer, an architect, a social scientist, a business specialist or a mechanical engineer.  The horizontal stroke of the T is the disposition for collaboration across disciplines.”

T-shaped skills are increasingly valued in the marketplace.  For example, a recent paper by Harvard professor David Deming showed that labor markets have been rewarding individuals with strong social skills, that is, with interpersonal skills that facilitate interactions and communications with others.  Deming’s research showed that since 1980, social-skill intensive occupations have enjoyed most of the employment growth across the whole wage spectrum.  Employment and wage growth have been particularly strong in jobs requiring both high cognitive and high social skills.  But since 2000, they have fallen in occupations with high cognitive but low social skill requirements, - “suggesting that cognitive skills are increasingly a necessary but not sufficient condition for obtaining a high-paying job.”

Disruptive Innovation Revisited

Disruptive innovation was first introduced by Harvard Business School professor Clayton Christensen in a 1995 Harvard Business Review (HBR) paper co-written with Joseph Bower.  The concept was further elaborated and popularized in Christensen’s 1997 bestseller The Innovator's Dilemma, and in the articles and books he’s written or co-authored since then.  Few business concepts have been as long and widely used.

But as often happens with serious ideas once they become popular, disruptive innovation has joined the pantheon of trendy, overused business buzzwords, - stretched and applied way beyond its intended meaning.  It’s even led to a rather strong backlash in a 2014 New Yorker article by Harvard history professor Jill Lepore.

20 years after its introduction, Christensen has revisited his original concept in What is Disruptive Innovation?, co-written with Michael Raynor and Rory McDonald, and just published in HBR’s December issue.  The authors aim to define what disruptive innovation is and what it isn’t, as well as to update the theory based on the experiences of the past two decades.

“The theory of disruptive innovation… has proved to be a powerful way of thinking about innovation-driven growth…,” they write.  “Unfortunately, disruption theory is in danger of becoming a victim of its own success.  Despite broad dissemination, the theory’s core concepts have been widely misunderstood and its basic tenets frequently misapplied.  Furthermore, essential refinements in the theory over the past 20 years appear to have been overshadowed by the popularity of the initial formulation.  As a result, the theory is sometimes criticized for shortcomings that have already been addressed.”

The Continuing Evolution of Cloud Computing

I’ve been closely following cloud computing for a number of years, having posted my first cloud blog in March of 2008.  A few months later I gave a presentation at a  cloud-based conference.  The overall sense at that conference was that something big and profound was emerging in the IT industry.  Cloud computing could well be The Next Big Thing - one of those massive changes that the IT industry goes through from time to time that really shake things up, - like the advent of personal computers in the 1980s and the Internet in the 1990s. 

Cloud has fulfilled those early expectations, having made much progress in a relatively short time.  As I look back at its evolution over the past few years, I think of cloud as having gone through three main, overlapping phases.

Early adopters viewed cloud as computing-on-demand.  Next, cloud introduced a much more disciplined approach to the architecture and management of large IT infrastructures.  And by now, cloud has become accepted as a major transformational force in IT, in business, and in the economy in general.

Let me discuss each of these phases.