Irving Wladawsky-Berger

The Rise of the Digital Capital Economy

I just read an interesting article that was published last summer in Foreign Affairs, - New World Order: Labor, Capital, and Ideas in the Power Law Economy.  The article was written by MIT professor Erik Brynjolfsson and MIT Principal Research Scientist Andy McAfee, who are also Director and Co-Director respectively of the Initiative on the Digital Economy; and by Michael Spence, professor in NYU’s Stern School of Business, professor and dean emeritus of the Stanford Graduate School of Business, and recipient of the 2001 Nobel Prize in economics. 

A major transition is now taking place in our digital economy.  Over the past few decades, advances in technology, and the Internet in particular, have helped create an increasingly global marketplace for labor and capital.  A highly connected, global economy has been rising all around us, whose magnitude and implications were brought to light by NY Times columnist Tom Friedman in his 2005 bestseller The World is Flat: A Brief History of the Globalized World in the Twenty-first Century. 

Another bestseller, The Second Machine Age, published last year by Brynjolfsson and McAfee, is now helping to explain our new era of data science, AI and advanced automation.  These second age machines are being increasingly applied to activities requiring intelligence and cognitive capabilities that not long ago were viewed as the exclusive domain of humans, while enabling us to process vast amounts of information and tackle ever more complex problems. 

These advanced machines are ushering a new kind of digital capital economy, quite different from the flat-world economy of only a decade ago.  The winners are no longer those able to compete solely based on cheap labor or ordinary capital, both of which are being squeezed by automation.  “Fortune will instead favor a third group: those who can innovate and create new products, services, and business models,” says the article.  “So in the future, ideas will be the real scarce inputs in the world - scarcer than both labor and capital - and the few who provide good ideas will reap huge rewards.  Assuring an acceptable standard of living for the rest and building inclusive economies and societies will become increasingly important challenges in the years to come.”

How Can We Ensure that Our Complex AI Systems Do What We Want Them to Do?

Software-intensive systems are generally quite flexible, - able to evolve and adapt to changing product and market requirements.  However, their very flexibility makes it difficult to adequately anticipate and test all the interactions between the various components of the system.  Even if all the components are highly reliable, problems can still occur if a rare set of interactions arise that compromise the overall behavior and safety of the system.

Things are even tougher with people-centric sociotechnical systems, which not only have to deal with potential software problems, but with the even more complex issues involved in human behaviors and interactions.  And, at the bleeding-edges of complexity are data-driven AI systems, which are generally based on statistical analysis and machine learning methods. 

What is it that makes these systems so intrinsically complex?  Why aren’t they simpler?  What purpose does this complexity serve?  Several years ago, I came across a reasonable answer to these seemingly Socratic questions in a very interesting paper - Complexity and Robustness - by  professors Jean Carlson and John Doyle from UC Santa Barbara and Cal Tech, respectively.  Their paper draws on evolutionary biology to help explain the complexity of man-made systems. 

According to Carlson and Doyle, you can find very simple biological organisms in nature, and you can design very simple objects.  The key ingredient you give up is not their basic functionality, but their robustness - that is, the ability to survive, for biological organisms, or to perform well, for engineered objects - under lots of different conditions, including the failures of individual components or dealing with new, unanticipated events.  Robustness implies the ability to adapt and keep going in spite of a rapidly changing environment. 

There is a continuing struggle between complexity and robustness in both evolution and human design.  A kind of survival imperative, whether in biology or engineering, requires that simple, fragile systems become more robust.  But the mechanisms to increase robustness will in turn make the system considerably more complex.  Furthermore, that additional complexity brings with it its own unanticipated failure modes, which are corrected over time with additional robust mechanisms, which then further add to the complexity of the system, and on and on and on.  This balancing act between complexity and robustness is never done.

The Evolving Automotive Ecosystem

Much as was once the case with electrification, digitization is now transforming industry after industry.  In media and communications, for example, everything seems to be changing at once, from the way content is produced and delivered, to the sources of revenue and profits.  In financial services, money and payment ecosystems are becoming increasingly digital now that just about everyone in the world can afford a personal mobile device.  Smart, connected products are reshaping industry boundaries and forcing companies to ask themselves a fundamental question: What business am I really in? 

This is also the case in the automobile industry, one of the largest in the world.  “Not since the first automotive revolution has there been such stunning innovation in the industry,” notes an excellent recent KPMG report, - Me, my car, my life… in the ultraconnected age.  “Autonomous vehicles are only part of the story.  The convergence of consumer and automotive technologies and the rise of mobility services are transforming the automotive industry and the way we live our lives.” 

Self-driving cars have commanded our attention in the last few years.  The advent of self-driving vehicles like the Google car is not only a truly dramatic milestone in AI, but concrete evidence that digital technologies are having a huge impact on the future of the automobile.  Opinions vary as to the commercial prospects for such truly autonomous vehicles.  Some feel that they will be all around us within a decade, navigating our present roads right along human-driven cars.  Others are not quite so sure due to the highly complex technical and societal issue that remain to be worked out.  Time will tell. 

In any event, car companies are working hard to try to keep pace with the speed of innovation in consumer technologies, a huge challenge to the auto industry, notes the KPMG report: “Melding the two worlds - consumer electronics, with its rapid new product launch cadences and willingness to accept iterative software releases, and automotive engineering, with its mass customization, millions of product configurations, and critical safety, durability, and reliability requirements - is not an easy prospect.”

The Promise and Challenges of Health Analytics

On February 24 I attended a workshop in MIT on the Future of Health Analytics.  The event was sponsored by MIT Connection Science, a recently organized research initiative aimed at leveraging data science to quantify and analyze human behaviors, and to leverage the new insights thus obtained in key societal applications, including healthcare, transportation and finance.  Connection Science, - with which I’m affiliated as a Fellow, - was founded by Media Lab professor Alex “Sandy” Pentland.  He’s the author of several books, including the recently published Social Physics: How Good Ideas Spread.  

I’ve worked with Pentland for the past several years, and have previously written about his research on Reinventing Society in the Wake of Big Data, as well as his work with the World Economic Forum and others on the creation of trust frameworks for the sharing and protection of personal data.  In his opening remarks at the workshop, Pentland talked about Big Data and Health.  The little data breadcrumbs that we leave behind as we move around in the world can now be reality mined to help us better understand our behaviors and thus improve our lives and health.  He discussed several applications of big data to health, based on research at his MIT Human Dynamics Lab, as well as startups he’s involved in.

Following Pentland, Dr. Dennis Ausiello talked about Quantitative Human Phenotyping and the opportunities it offers to transform the practice of medicine.  Ausiello is professor of medicine at the Harvard Medical School, Chief Emeritus of Medicine at Massachusetts General Hospital (MGH), and director and co-founder of the Center for Assessment Technology and Continuous Health (CATCH), a joint MGH-MIT initiative aimed at finding new ways of measuring the human condition, i.e., phenotypes.

What Will Life Be Like in an AI Future?

People have long argued about the future impact of technology.  But, as AI is now seemingly everywhere, the concerns surrounding its long term impact may well be in a class by themselves.  Like no other technology, AI forces us to explore the boundaries between machines and humans.  What will life be like in such an AI future?

Not surprisingly, considerable speculation surrounds this question.  At one end we find books and articles exploring AI’s impact on jobs and the economy.  Will AI turn out like other major innovations, e.g., steam power, electricity, cars, -  highly disruptive in the near term, but ultimately beneficial to society?  Or, as our smart machines are being increasingly applied to cognitive activities, will we see more radical economic and societal transformations?  We don’t really know.

These concerns are not new.  In a 1930 essay, for example, English economist John Maynard Keynes warned about the coming technological unemployment, a new societal disease whereby automation would outrun our ability to create new jobs.

The Slow, Steady March Towards Digital Money

On January 27, Citi and Imperial College held their third annual Digital Money Symposium in London.  As in previous years, the Symposium convened a group of leaders in their field to explore the state of adoption of digital money and its economic and societal impacts around the world.    

Last year’s symposium introduced the Digital Money Readiness Index, a set of global metrics to help quantify digital money adoption based on various measures of socio-economic development.  The Readiness Index aimed to answer two important questions: 

• What factors affect the adoption of digital money around world, and how do they vary across different countries and regions?, and 
•  Does digital money adoption really make a difference, and if so, can we quantify the benefit to governments, companies and individuals?

What Should We Think of Machines that Think?

“Any sufficiently advanced technology is indistinguishable from magic,” is one of Arthur C. Clarke’s, - author of 2001: A Space Odyssey, -  most memorable quotes.  I still remember a Monday in the summer of 1996, when around 4 am I was in my Tokyo hotel room doing e-mail on my laptop while listening over the Internet to a live baseball game being played in New York, where it was Sunday afternoon.  Today this would be no big deal, but at the time it felt like one of those magical moments Clarke had in mind, perhaps the moment when I truly understood the transformative power of the rapidly growing Internet.

Artificial Intelligence may now be going through such an Internet moment.  “Artificial intelligence is suddenly everywhere.  It’s still what the experts call soft A.I., but it is proliferating like mad.”  So starts an excellent Vanity Fair article I recently wrote about.  “Everything that we formerly electrified we will now cognitize,” observed another great article.  “Experts envision automation and intelligent digital agents permeating vast areas of our work and digital lives by 2025, but they are divided on whether these advances will displace more jobs than they create,” was the overriding finding of a report published by the Pew Research Center this past August.

For the past several years, Edge.org has been asking a rather general, philosophical annual question to a diverse group of thinkers.  Last year the question was What Scientific Idea is Ready for Retirement?, and two years ago it was What Should We Be Worried About?  For 2015, Edge.org chose What Do You Think about Machines that Think? as its annual question.

The Big Data Era Is Here

In mid-January, the Transportation Research Board (TRB) of the National Academies held its 94th Annual Meeting in Washington DC.  The conference attracted around 12,000 transportation researchers and practitioners from around the world.  During the meeting, the TRB Executive Committee held a policy session on the impact of big data on transportation systems.  I was one of three panelists invited to the session.  Each of us made a short presentation which was then followed by an extensive discussion between the panel and the Committee.

Big data and related information-based disciplines, - e.g., data science, artificial intelligence, - are everywhere.  Why are we so excited about them?  Is it mostly hype, or is there something truly profound going on?  In my presentation, I tried to briefly address these questions based on three key observations.

Skills and Jobs in the Digital Economy

Few subjects are as important, - or as challenging to predict, - as the future of jobs in our emerging digital economy. While the US unemployment rate continues to improve, - finishing 2014 at 5.6% or 8.7 million people, -  almost one third of the unemployed have been jobless for over 27 weeks.  The total number unemployed or underemployed, - what economists call the U6 unemployment rate, - stands at 11.2% or 17.4 million people.  And, the employment to population ratio remains at under 60%, the lowest such percentage since the 1970s.  “The economic challenge of the future will not be producing enough.  It will be providing enough good jobs,” wrote Harvard professor and former Treasury Secretary Larry Summers in a recent WSJ article. 

Technological revolutions are highly disruptive to economies and societies.  This was the case for much of the Industrial Revolution, as is the case today.  “The digital revolution has yet to fulfil its promise of higher productivity and better jobs,” said The Economist in a special report on Technology and the World Economy in its October 4th issue.  “The modern digital revolution - with its hallmarks of computer power, connectivity and data ubiquity - has brought iPhones and the internet.”  But, “it is disrupting and dividing the world of work on a scale not seen for more than a century.  Vast wealth is being created without many workers; and for all but an elite few, work no longer guarantees a rising income.” 

Middle class jobs have been in decline for the past few decades in the US and other advanced economies, - particularly since 2000.  And, the livelihood of significant additional workers is potentially threatened, as our increasingly smart machines continue to be applied to activities requiring cognitive capabilities that not long ago were viewed as the exclusive domain of human.  How will these relentless advances in technology and automation affect the balance between humans and machines in the workplace, and the skill composition of future jobs? 

This critical question was addressed in a recent paper, Racing With and Against the Machine: Changes in Occupational Skill Composition in an Era of Rapid Technological Advance, by MIT’s Frank MacCrory, George Westerman and Erik Brynjolfsson along with Yousef Alhammadi from the Masdar Institute in Abu Dhabi.  The paper analyzed the changes between 2006 and 2014 in the skill composition of 674 occupations using the US Government’s O*NET data base, - the most comprehensive data sets of occupational skill requirements.  The period from 2006 to 2014 saw the advent of several major digital innovations, including smart mobile devices, social media, big data and analytics, cloud computing and the Internet of Things.

As Big Data and AI Take Hold, What Will It Take to Be an Effective Executive?

Big data, powerful analytics and AI are everywhere.  After years of promise and hype, technology is now being applied to activities that not long ago were viewed as the exclusive domain of humans.  Our digital revolution had led to amazing applications, but also to considerable pain for many workers who’ve been experiencing declining employment and wages.  Mid-skill jobs have been particularly threatened.  Many of these jobs, - which include blue-collar production activities as well as information-based white-collar ones, - are based on well understood procedures that can be described by a set of rules that machines can then follow. 

But, what will be the impact of our increasingly intelligent machines on senior management positions?  In principle, such jobs deal with non-routine, cognitive tasks requiring high human skills, including expert problem solving, complex decision-making and sophisticated communications for which there are no rule-based solutions.  “As artificial intelligence takes hold, what will it take to be an effective executive?” asks a recent McKinsey article - Manager and Machine: The new leadership equation.  “What would it take for algorithms to take over the C-suite?  And what will be senior leaders’ most important contributions if they do?” 

After asking these questions to senior managers across a broad range of industries, McKinsey  concluded that two key things need to happen for technology to more deeply transform their jobs.  First, much still needs to be done to create the proper data sets that would enable intelligent computers to assist in decision-making.  Garbage in, garbage out applies as much to data analysis today as it has to computing in general since its early years.  Organizations must have a data-analytics strategy that cuts across internal informational silos and properly incorporates external information sources like social media. 

And most important, senior managers must learn to let go, something which is quite difficult because it runs counter to decades of organizational practices.  Given our rapidly rising oceans of data, the command-and-control approach to management, where information flows up the organization and decisions are made at high levels, would sink the senior executive teams.  As data science and AI permeate the organization, it’s important to delegate more autonomy to the business units that hopefully have the proper skills, the advanced tools and the necessary information to make better decisions on their own.

“Soft” AI Is Suddenly Everywhere

“Artificial intelligence is suddenly everywhere.  It’s still what the experts call soft A.I., but it is proliferating like mad.”  So starts an excellent Vanity Fair article, - Enthusiasts and Skeptics Debate Artificial Intelligence, - by author and radio host Kurt Andersen.  Artificial intelligence is indeed everywhere, but these days, the term is used in so many different ways that it’s almost like saying that computers are now everywhere.  It’s true, but so general a statement that we must probe a bit deeper to understand its implications, - starting with what is meant by soft AI, versus its counterpart, strong AI. 

Soft, weak or narrow AI is inspired by, but doesn’t aim to mimic the human brain.  These are generally statistically oriented, computational intelligence methods for addressing complex problems based on the analysis of vast amounts of information using powerful computers and sophisticated algorithms, whose results exhibit qualities we tend to associate with human intelligence.

Soft AI was behind Deep Blue, - IBM’s chess playing supercomputer which in 1997 won a celebrated chess match against then reigning champion Gary Kasparov, - as well as Watson, - IBM’s question-answering system which in 2011 won the Jeopardy! Challenge against the two best human Jeopardy! players.  And, as Andersen notes in his article, it’s why “We’re now accustomed to having conversations with computers: to refill a prescription, make a cable-TV-service appointment, cancel an airline reservation - or, when driving, to silently obey the instructions of the voice from the G.P.S.” 

This engineering-oriented AI is indeed everywhere, and being increasingly applied to activities requiring intelligence and cognitive capabilities that not long ago were viewed as the exclusive domain of humans.  AI-based tools are enhancing our own cognitive powers, helping us process vast amounts of information and make ever more complex decisions.

Leveraging the Passions of Fans

On November 21 I participated in a Think & Do workshop at USC’s Annenberg Innovation Lab (AIL).  AIL was founded in 2010 as part of USC’s Annenberg School of Communications and Journalism to study the transformational impact of technology and cultural changes on the media industry.  I’ve been a member of AIL’s Advisory Board since its founding.

The Lab conducts research projects in a number of areas at the intersection of media and technology.  From time to time it hosts what it calls Think & Do events that bring together students, faculty members, research staff, industry executives, artists, entrepreneurs and policy makers to collaboratively explore new research ideas.  The topic for this particular workshop was Leveraging Engagement.  It was aimed at exploring new frameworks for fan engagement in a variety of media, including entertainment, music and sports.

Like few others, the media industries have been severely disrupted by the digital revolution and the forces of creative destruction.  Everything seems to be changing at once, from the way content is produced and delivered, to the sources of revenue and profits.  One of the major changes is the relationship between the creators and distributors of media content and their audience, especially their most committed audience members or fans. 

What do we mean by fan?  The briefing book prepared for the workshop draws a distinction between an audience of relatively passive listeners/spectators and fans.  Fans are “enthusiastic followers or admirers… have a passion, are emotionally connected to the object of their passion, and experience their passion through their own subjective lens.” 

The Internet, smartphones and related technologies, are enabling fans to play a more central and active role in the evolving media ecosystem.  They are active participants in social networks.  They are critics, co-creators, and brand influences.  They are also potential consumers of all kinds of goods and services related to their passion.

The New Era of Smart, Connected Products

For the past few years, some have justifiably questioned whether innovation has been going through a period of stagnation, especially when compared to major 19th and 20th century innovations like electricity, cars and airplanes.  Have we pretty much stopped solving big problems?  “We wanted flying cars - instead we got 140 characters,” is how PayPal cofounder Peter Thiel succinctly expressed this sentiment in a 2011 New Yorker article. 

Others argue that while the nature of innovation is definitely changing as we evolve and adapt to an information-based digital economy, it’s impact is no less transformative.  Last week, for example, I discussed a recent article in Wired by Kevin Kelly on the future of AI.  Kelly fully expects that AI will transform the global economy and civilization in general, much as electricity did more than a century ago.  “Everything that we formerly electrified we will now cognitize,” he said.  “There is almost nothing we can think of that cannot be made new, different, or interesting by infusing it with some extra IQ…  This is a big deal, and now it’s here.”  I totally agree.

An equally optimistic view was expressed in another excellent article, - How Smart Connected Products are Transforming Competition, - published in the November issue of the Harvard Business Review by Michael Porter and James Heppelmann. 

We generally think of products as physical entities, - e.g., clothes, light bulbs, appliances, cars, - some quite simple and some much more complex, built using sophisticated mechanical and/or electrical components.  The article introduces a whole new class of smart connected products - “complex systems that combine hardware, sensors, data storage, microprocessors, software, and connectivity in myriad ways.”

The Future of AI: a Ubiquitous, Invisible, Smart Utility

Our increasingly smart machines are now being applied to activities that not long ago were viewed as the exclusive domain of humans.  Cognitive computing, robotics, self-driving cars and AI in general are some of the hottest topics in academia and business these days.

What should we expect from this new generation of AI machines and applications?  Are they basically the next generation of sophisticated tools enhancing our human capabilities, as was previously the case with electricity, cars, airplanes, computers and the Internet?  Or are they radically different from our previous tools because they embody something as fundamentally human as intelligence? 

Kevin Kelly, - as am I, - is firmly in the AI-as-a-tool camp, as evident by a recent article in Wired, - The Three Breakthroughs That Have Finally Unleashed AI on the World.  Kelly co-founded Wired in 1993, where he served as executive editor until 1999 and continues his association as Editor-At-Large.  He has many other interests as well as being a prolific writer of books and articles.

According to Kelly, the AI future that’s coming into view is nothing like the “potentially homicidal” HAL 9000 from 2001: A Space Odyssey or “a Singularitan rapture of superintelligence.”  The AI he foresees is more like a kind of “cheap, reliable, industrial-grade digital smartness running behind everything, and almost invisible except when it blinks off.  This common utility will serve you as much IQ as you want but no more than you need.  Like all utilities, AI will be supremely boring, even as it transforms the Internet, the global economy, and civilization.”

When Can We Expect Truly Autonomous Vehicles?

A few weeks ago I attended MIT’s Second Machine Age Conference, where I heard a number of very interesting presentations on the evolution of AI, robotics, and other advanced technologies.  The prospects for truly autonomous vehicles was one of the main topics of discussion. With most other topics, there was considerable audience consensus, but not so with self-driving cars.  While many thought that fully autonomous vehicles will be all around us within a decade, others, myself included, were not quite so sure due to the many technical and societal issues involved. 

What do we really mean by self-driving cars?  There seems to be no precise definition.  Are we talking about a human driver assisted by all kinds of advanced technologies, or is there no driver whatsoever?  Will such vehicles operate amidst regular human-driven ones, or will they be confined to special lanes equipped with sophisticated navigational technologies?  And, is self-driving per se the actual objective, or is it a metaphor for the development of near-crashless cars regardless of whether human drivers are still in the picture. 

These questions are not surprising given the very early stages of such a complex area.  When exciting new initiatives are first launched, we sometimes describe them using an attention-grabbing phrase that, while potentially unattainable in practice, should be taken more as a marketing pointer to a general direction rather than as a realistic near-term objective. 

The Evolution of the Internet of Very Smart Things

I recently saw a very interesting presentation, - From Cloud to Fog: The Internet of Things, - by Cisco executive Michael Enescu.  He tells us that the Internet of Things (IoT) is truly taking off, already comprising about 25 billion smart devices at the edges of the Internet, a number that’s expected to more than double over the next decade, and continue to keep growing for the foreseeable future. 

These devices generate massive amounts of data, a lot of which requires real-time actions.  It’s impractical to move all that data to a central cloud for analysis and actions.  Computing and intelligence thus have to move closer to the edge, to both ameliorate the data transport challenges and enable real-time actions as required.  This all leads to what has come to be known as edge computing, - an architectural bridge between the clouds at the center and the IoT devices all around the edges of the Internet.  Cisco calls it fog computing, an allusion to what happens to clouds as they get closer to the ground. 

Cisco defines fog computing as “a highly virtualized platform that provides compute, storage, and networking services between end devices and traditional Cloud Computing Data Centers, typically, but not exclusively located at the edge of network.”  Fog complements cloud with a number of important capabilities, including edge location awareness, low latency in highly constrained connectivity environments, bandwidth and energy optimizations, and nearly unlimited scale. 

Enescu notes that cloud and fog represent two very different IoT paradigms for dealing with huge amounts of data.  In cloud, the data generated by the smart sensors at the edges is transferred and stored in the center, where it’s analyzed, and the appropriate actions then flow back across the network.  With fog on the other hand, the analysis is done in real time at or near the edge devices, from which actions then flow across the network, with only the data to be stored being transferred to the central cloud.

The MIT Second Machine Age Conference

A couple of weeks ago I attended MIT’s Second Machine Age Conference, an event inspired by the best-selling book of the same title published earlier this year by MIT’s Erik Brynjolfsson and Andy McAfee.  The conference presented some of the leading-edge research that’s ushering the emerging second machine age, and explored its impact on the economy and society.  It was quite an interesting event.  Let me discuss a few of the presentations as well as my overall impressions. 

In his opening keynote, Brynjolfsson explained what the second machine age is all about.  “Like steam power and electricity before it, the explosion of digitally enabled technologies is radically transforming the landscape of human endeavor.  Astonishing progress in robotics, automation, and access to information presents major challenges for institutions from small businesses and communities to large corporations and governments, but it also creates opportunities to rethink how we live and work in profoundly positive ways.” 

The machines of the industrial economy, - the first age, - made up for our physical limitations, - steam engines enhanced our physical power, railroads and cars helped us go faster, and airplanes gave us the ability to fly.  For the most part, they complemented, rather than replaced humans.  The second age machines are now enhancing our cognitive powers, giving us the ability to process vast amounts of information and make ever more complex decisions.  They’re being increasingly applied to activities requiring intelligence and cognitive capabilities that not long ago were viewed as the exclusive domain of humans.  Will these second age machines complement or replace humans?

AI, Robotics, and the Future of Jobs

The Pew Research Center has been marking the 25th anniversary of the World Wide Web with a series of reports on the Internet and our digital lives.  I’ve discussed two of the reports, first The Web at 25 in the US, followed by Digital Life in 2025.  I’d now like to discuss a third Pew report, - AI, Robotics and the Future of Jobs, - which was published in early August.

The impact of technology on jobs is a very important subject.  “The economic challenge of the future will not be producing enough.  It will be providing enough good jobs,” wrote Harvard professor and former Treasury Secretary Larry Summers in a recent WSJ article.  These concerns are not new.  In a 1930 essay, English economist John Maynard Keynes wrote: “We are being afflicted with a new disease of which some readers may not yet have heard the name, but of which they will hear a great deal in the years to come - namely, technological unemployment.  This means unemployment due to our discovery of means of economising the use of labour outrunning the pace at which we can find new uses for labour.”

As it turned out, the 20th century saw the creation of many new jobs and industries.  But, fears of technological unemployment have been rising in the emerging digital economy, as our increasingly smart machines are being applied to activities requiring intelligence and cognitive capabilities that not long ago were viewed as the exclusive domain of humans. 

The Pew Research report explored the impact of AI and robotics on the future of employment based on the responses of nearly 1,900 experts to a few open-ended questions, including: “Will networked, automated, artificial intelligence (AI) applications and robotic devices have displaced more jobs than they have created by 2025?; and “To what degree will AI and robotics be parts of the ordinary landscape of the general population by 2025?”

Lean Principles in the Digital Economy

Lean principles is the name given to a group of production techniques developed by Japanese manufacturing companies around the 1970s-1980s to maximize customer value while reducing wasteful resource.  Lean production methods have also been described as aiming “to combine the flexibility and quality of craftsmanship with the low costs of mass production.”  Such methods include quality control of the processes involved in production; just-in-time production to reduce the costs associated with excess inventory; and continuous improvement involving everyone in the organization in the quest for new, easy to implement ideas. 

More recently, lean principles have been playing a key role in a variety of industries beyond manufacturing which must now adapt to the fast changing technologies and markets of the Internet age.  Lean software development, for example, can be viewed as the agile reaction to the highly structured, heavyweight waterfall-based software methods of the past.  In Lean Software Development: An Agile Toolkit published in 2003, Mary and Tom Poppendiek translated the principles of lean production into 7 key lean software development principles: eliminate waste, amplify learning, decide as late as possible, deliver as fast as possible, empower the team, build integrity in, and see the whole. 

As companies transition from the industrial to the digital economy, lean philosophies are influencing just about every aspect of their operations, strategy, organization and culture.  Lean’s core idea, - maximize customer value while minimizing waste, - feels particularly applicable to our times, as organizations must better understand what customers truly value; organize their work activities to efficiently develop and deliver the appropriate products and services; and continuously improve customer value and efficiency based on real marketplace feedback.  Such a pull approach to business is quite different from the push approaches of the past.

The Complex Nature of Cloud-based Innovation

By now, most will agree that cloud computing is a major transformational force in the world of IT.  But, cloud has not the easiest concept to grasp.  Not that long ago, even people who agreed that something big and profound was going on, were not totally quite sure what it was they were excited about.  “There is a clear consensus that there is no real consensus on what cloud computing is,” was the overall conclusion of a June, 2008 conference on The Promise and Reality of Cloud Computing.  Cloud has continued to evolve and advance over the ensuing years.  People are no longer starting their sessions by saying: “let’s define cloud computing,” said an article on a 2013 cloud conference.

I find it helpful to look at cloud along two key dimensions: as a technology to improve IT productivity, and as a platform for enabling business innovation.  A 2012 survey of business and technology executives found that two thirds of respondents viewed cloud as a leading priority for their IT organizations, while one third said that it was a company-wide business priority.  Only one company in six viewed cloud as a way of fostering business innovation.

An increasing number of companies are now deploying cloud-based solutions.  Most are focused on improving the economics of IT.  They are looking to cloud to help them expand their current offerings without major investments in additional IT infrastructure.  Cloud offers financial flexibility, reducing fixed IT costs by shifting from capital to operational, pay-per-use expenses, as well as the ability to easily and economically scale business operations by provisioning IT resources on an as-needed basis.  In addition, cloud can help business users become more agile and keep up with the fast pace of technological and market changes.  Turning to an external cloud service provider is often a faster and less costly way of prototyping and deploying a new application than relying on the internal IT organization.

The Changing Nature of Globalization in Our Hyperconnected, Knowledge-Intensive Economy

In April, 2005, NY Times columnist Tom Friedman published The World is Flat: A Brief History of the Globalized World in the Twenty-first Century.  Friedman’s bestseller did much to explain to a wide audience the impact of the Internet and related technologies on the connected, global economy that was rising all around us.  

A few years later, Friedman was a keynote speaker at a 2011 conference commemorating IBM’s Centennial.  To illustrate the incredibly fast pace of change, Friedman noted that in the short time-span since World is Flat was published, we were already transitioning from a connected to an increasingly hyperconnected world.  Many of the companies and technologies that are now part of our every day conversations, - Facebook, Twitter, Cloud, Smartphones, Big Data, LinkedIn, 4G or Skype, - were not mentioned in the connected global economy he wrote about only 6 years earlier because they had not been born or were in their infancy.

What do we mean by a hyperconnected economy?  I recently found a really good answer in an excellent study by the McKinsey Global Institute (MGI), Global Flows in the Digital Age.  The study takes an in-depth look at the expansion of cross-borders flows in the economy.  It carefully analyzed these flows in 5 different categories:  goods, services, finance, people and data and communications.  It then developed the MGI Connectedness Index, which measures these 5 flows in 131 countries, examined how the flows had changed over the past 10 - 20 years, and predicted how they are likely to evolve over the next decade.

A Data Science Approach to Organizational Health

A 2013 Deloitte report, Success or Struggle?, observed that despite high profits and stock valuations, the reality for many companies is far from upbeat.  Strong competitors keep appearing from all corners of the world.  Advances in technology are eroding the prevailing business models in industry after industry.  Companies are under intense pressure, as they struggle to defend their vulnerable revenues and market share while pursuing elusive new opportunities and profitable growth.  

“Companies are broken and many don’t know,” reads the provocative opening line of the Deloitte report.  It highlights the paradox that “Many companies are reporting record profits, but longer-term trends suggest they are struggling.”  As an indicator of such struggles the report cites the topple rate, which measures how rapidly companies lose their leadership position.  The topple rate has increased by almost 40 percent since 1965.  The tenure of companies on the S&P 500 was 61 years in 1958; it’s now 18 years.  If these trends continue, 75 percent of the S&P 500 companies will have changed over the next 15 years.

It’s been getting harder for even successful companies to maintain their leadership positions.  I remember reading In Search of Excellence: Lessons from America’s Best Run Companies when it first came out in 1982, - one of the most influential business books of all time.  The book examined 43 of the Fortune 500 top-performing US companies, highlighting what made them great and what management lessons could be learned from them.  Similarly, Built to Last: Successful Habits of Visionary Companies, became an influential business best-seller in 1994 as it took and in-depth look at 18 companies which it identified as visionary.

These two books studied over 50 of the most successful companies of the past few decades.  How did these presumably best-managed companies fare over time?  By 2006, about 20% were out of business, almost half were struggling and a only a third were still high performers, according to Beyond Performance: How Great Organizations Build Ultimate Competitive Advantage, a book published in 2011 by McKinsey’s Scott Keller and Colin Price.

An Old Friend, - the IBM Mainframe, - Turns 50

On April 7, 1964, IBM announced the Systems/360 family of mainframes.  “System/360 represents a sharp departure from concepts of the past in designing and building computers,” said IBM’s then chairman and CEO Thomas J. Watson Jr.  “It is the product of an international effort in IBM’s laboratories and plants and is the first time IBM has redesigned the basic internal architecture of its computers in a decade.  The result will be more computer productivity at lower cost than ever before.  This is the beginning of a new generation - not only of computers - but of their application in business, science and government.” 

In April of 1964 I was a second year college student at the University of Chicago, working part time at the university’s computation center, which used IBM computers.  I still remember attending a presentation on the announcement given by a visiting IBM executive.  Over the next several years I used high-end models of S/360 for the physics calculations I was doing as part of my doctorate studies.  My thesis sponsor, - Chicago professor Clemens Roothaan, one of the early leaders in computational sciences, - was consulting with IBM on the design of future versions of S/360, and I was also involved in some of this work.  This relationship with IBM researchers led to my joining the computer sciences department at IBM’s Watson Research Center once I finished my studies in 1970.   

I was closely associated with mainframes through most of my 37 years in IBM.  I was involved in a number of research initiatives on the future of large systems.  After moving to the large systems products divisions in the mid 1980s, I worked on the evolution of mainframes to CMOS-based microprocessors and parallel architectures.  Later in the 1990s and 2000s I worked closely with the mainframe teams as they supported the Internet, Linux and other emerging initiatives I was involved in.   

Having lived through the near demise of mainframes in the early 1990s, - which would have inevitably led to IBM’s own demise, - their ability to have survived after all these years is truly impressive.  A look at the IT industry over the past several decades will reveal the large number of once great products and companies that are no longer around.  Few computer families could trace their vintage to the 1980s, let alone the 1960s.  There is something pretty unique about the mainframe being not only alive but doing so well after all these years.

Why is the mainframe still around, celebrating its 50th birthday last week?  What enables it to keep reinventing itself while embracing the latest technologies, including the Mobile Internet, Cloud Computing and Big Data?  In a world where product life-cycles are measured in web years, what can we learn from the mainframe’s rather unique longevity?

Data Science: from Half-Baked Ideas to Data-Driven Insights

In a Big Data presentation last year, MIT professor Erik Brynjolfsson pointed out that throughout history new tools beget revolutions.  Scientific revolutions are launched when new tools make possible all kinds of new measurements and observations.

Big data is leading to such a measurement-driven revolution, brought about by the new digital tools all around us, including our mobile phones; searches and web links; social media interactions; payments and transactions; and the myriads of smart sensors keeping track of the physical world.  These digital tools are enabling us to collect massive amounts of information on who we are, what we do and how we interact as individuals, communities and institutions.

The explosive growth of big data is in turn giving rise to data science, one of the most exciting new professions and academic disciplines.  Data science is a mashup of several different fields.  Its data part deals with acquiring, ingesting, transforming, storing and retrieving vast volumes and varieties of information.  Its science part seeks to extract insights from the data by applying tried-and-true scientific methods, that is, empirical and measurable evidence subject to testable explanations and predictions. 

One of the most exciting part of data science is that it can be applied to many domains of knowledge, given our newfound ability to gather valuable data on almost any topic. But, doing so effectively requires domain expertise to identify the important problems to solve in a given area, the kinds of questions we should be asking and the kinds of answers we should be looking for, as well as how to best present whatever insights are discovered so they can be understood by domain practitioners in their own terms.

Innovation Hubs in the Global Digital Economy

Last week I wrote about Cloud, Services and the Transformation of Production, based on Part I of Escape from the Commodity Trap: Will the Production Transformation Sustain Productivity, Growth and Jobs?, a paper by UC Berkeley professor John Zysman.  Part II of the paper deals with the economic and policy implication of this ICT-based production transformation. 

“For the advanced industrial democracies to expand the real income of the citizens and sustain growth in employment and productivity, their economies will have to escape from the commodity trap,” writes Zysman in the paper’s abstract.  “The commodity trap is the price-based competition throughout markets for standard goods and services, which puts pressure on wages and profit margins alike.  Clearly, the way out of this trap is to create distinctive high value added products  - both goods and services.  The emerging transformation of the production of goods and services is dramatically altering what is produced, where, how, and who captures the value.  It creates opportunities and challenges.” 

Early in the Industrial Revolution , the so-called Luddites smashed the new textile machines that were threatening their jobs.  Zysman reminds us that while many of the Luddites lost their jobs, the overall English economy prospered and the average standard of living went up. “It is easier to identify the jobs that are destroyed,. . .  than to imagine the ways new jobs will be created.  Of course, the character and location of those jobs may advantage some communities and disadvantage others.” 

Disruptive innovations, - like the power looms allegedly smashed by protesting Luddites, - have been displacing human labor for the past two centuries.  These periods of creative destruction, - when new technology-based innovations led to painful transitions in once-dominant companies and jobs, - eventually worked themselves out.  Over time, these same disruptive innovations rejuvenated the economy, and led to the creation of new industries and jobs.

Cloud, Services and the Transformation of Production

Ever since cloud computing first appeared in the IT world, people have struggled to define what it is and why it’s so important. “There is a clear consensus that there is no real consensus on what cloud computing is,” said the organizer of a 2008 cloud conference in his closing remarks.  Just about everyone agreed that something big and profound was going on, although we were not totally sure what it was yet. 

Is cloud the 21st century version of time sharing, enabling users to get virtual access to different kinds of IT resources and software capabilities without having to own computers?  Is it a return to centralized computing, driven by the rising complexities, management costs and energy inefficiencies of distributed systems?  Is it the evolution to utility computing, where, as with electricity and water, you get your IT from large service providers and pay for it based on usage? 

Or, as The Economist said in an excellent 2008 article, is cloud the next step in the brief history of computers, as the machines “are evaporating altogether and becoming accessible from anywhere”. . . as computing is becoming “more and more disembodied and will be consumed where and when it is needed.” 

“The rise of the cloud is more than just another platform shift that gets geeks excited,” it presciently added.  “It will undoubtedly transform the information technology (IT) industry, but it will also profoundly change the way people work and companies operate.  It will allow digital technology to penetrate every nook and cranny of the economy and of society, creating some tricky political problems along the way.”

Big-Bang Disruptions: The Innovator’s Dilemma in the Digital Economy

I recently read a very interesting article, - Big-Bang Disruption.  The article was written by Larry Downes and Paul Nunes and published in the March, 2013 issue of the Harvard Business Review.  It was later expanded into a book, Big Bang Theory: Strategy in the Age of Devastating Innovation published this past January. 

The authors’ key premise is that innovation is once more undergoing drastic changes, driven by the exponential advances in digital technologies.  Companies are now able to create products that are simultaneously better, cheaper and more appealing than those of competitors.  As a result, “entire product lines and whole markets are now being created or destroyed overnight.”  Examples include Skype, iTunes, the Kindle, Netflix, Facebook, Twitter, and smartphone apps like Google maps and Angry Birds. 

Big-bang disruptions are often unplanned and unintentional.  They are typically discovered through continuous market experimentation.  They upend the conventional thinking on strategy, marketing and innovation, giving rise to a new set of business rules.  “Nearly everything you think you know about strategy and innovation is wrong,” they note.

Some Puzzling Questions about Innovation in the Digital Economy

This semester I am teaching the innovation half of a course on Entrepreneurship and Innovation at NYU’s new Center for Urban Science and Progress (CUSP).  Teaching forces you to take a fresh look at the subjects you are covering, so I find myself revisiting questions I’ve long been thinking about:  What is the essence of innovation in the digital economy and how does it differ from the industrial age innovation of the past two hundred years?

The bulk of the innovations in the industrial economy involved physical objects, physical systems and the physical world around us.  Our biggest scientific advances were in the natural sciences, e.g., astronomy, physics, chemistry, geosciences and biology.  In engineering, we made huge progress in applying science and technology to develop increasingly complex physical objects and infrastructures, such as railroads, cars, airplanes, chemical plants, refineries, power stations, bridges, skyscrapers, computers, microprocessors, and so on.  Ever since the steam engine gave rise to the Industrial Revolution in the late 18th century, we’ve had major waves of innovation every 50-60 years, including railroads, steel, electricity and heavy engineering in the 19th century; and automobiles, airplanes and telecommunications in the 20th century.

The Industrial Revolution had a huge impact on all aspects of the economy and society.  It totally transformed the composition of labor.  In 1840, the vast majority of US jobs where in agriculture, with services accounting for roughly 20% of jobs.  Those figures have been drastically reversing since then.  According to 2009 statistics from the CIA World Factbook, only 0.7% of the US labor force is now involved in agriculture oriented jobs and roughly 20% work in the industrial sector.  The bulk of jobs, almost 80%, are in services, with 3/4 of those service jobs based on processing information in one way or another, - e.g., managerial, professional, technical, sales and office.  Similar changes in labor composition have been taking place in just about every country.      

The emerging 21st century digital economy seems quite different from the industrial economy of the past two centuries.  Dramatic advances in information and communication technologies are now enabling us to apply science, technology and innovation to improve the productivity of information-based service jobs, as well as that of sociotechnical systems, that is, systems in which people play a central role, such as business organizations, healthcare institutions and cities.  Such systems are particularly hard to understand and control because of the complexities inherent in human and organizational behaviors.

The Citi-Imperial Digital Money Readiness Index

On January 29 I attended the 2014 Digital Money Symposium in London, co-sponsored by Citi and Imperial College.  The Symposium convened a group of leaders in their fields to explore the state of adoption of digital money and its economic and societal impacts around the world.  This is the second such Digital Money Symposium, the first one having taken place in London in January of 2013. 

I was a part of the Citi-Imperial team that organized the event.  For this year’s Symposium, we developed a framework that would enable us to discuss the state of adoption of digital money on a more data-driven, scientific basis.  We looked at digital money as a highly complex sociotechnical ecosystem, that is, a technology-intensive system that has major societal, economic and political implications, like cities, healthcare and education.  To help us begin to quantify and understand this ecosystem, we developed the Digital Money Readiness Index, a set of global metrics that enables us to link digital money adoption to various measures of socio-economic development. 

The Index is a multi-functional tool, encompassing publicly available data from 90 countries around the world.  By analyzing its various interdependent components, the team computed a measure of how ready a country is to adopt digital money.  Quantifying the progress being made by each country along the digital money roadmap should help understand the key obstacles the country faces, as well as the potential actions it could take to overcome them.

Overall, the Index aims to help answer two fundamental sets of questions: 

What Matters?: What factors affect the adoption of digital money around world, and how do they vary across different countries and regions? 

 Why Bother?: Does digital money adoption really make a difference, and if so, can we quantify the benefit to governments, companies and individuals?

Big Data, Science and the Humanities

A few weeks ago I participated in an online debate sponsored by The Economist around the question: Are Smart Cities Empty Hype?.  I argued the case against the motion.  In my opening statement I said that I strongly believe that digital technologies and the many data services they are enabling will make cities smarter and help transform them over time.  Later, in my closing remarks, I pointed out that since the qualifier smart essentially means information-based or data-driven, the promise of smart cities is inexorably linked to the general promise of big data and data science, which some have felt are themselves being over-hyped.

Over the holidays, I learned about an intriguing debate that took place this past year in the New Republic.  Billed as Science vs the Humanities, it raised a number of important questions.  To what extent does the promise of data science apply to different kinds of disciplines, including the humanities?  Can big data tools and related methods help the humanities achieve a more scientific understanding of human nature?  Let me make some general comments before getting to the debate itself.

Scientific revolutions are launched when new tools lead to new measurements and observations.  Early in the 17th century, Galileo made major improvements to the recently invented telescope which enabled him to make discoveries that radically changed our view of the universe. Over the centuries we have seen that new tools, measurements and discoveries precede major scientific breakthroughs in physics, chemistry, biology and other natural sciences disciplines.  

Similar breakthroughs have been much harder to achieve beyond the natural sciences, especially when it comes to the study of human nature, that is, the way we feel and behave.  But, the situation has been rapidly changing in the past few decades.  First of all, advances in behavioral science, neuroscience, genomics, evolutionary biology and related disciplines have shed significant new light on such people-oriented studies. And then there is the advent of big data.

Are Smart Cities Empty Hype?

A couple of weeks ago I participated in an online debate sponsored by The Economist.  The debate was built around the question: Are Smart Cities Empty Hype?  Defending the motion was Anthony Townsend, Research Director at the Institute for the Future and adjunct faculty member at NYU’s Wagner School of Public Service.  I took the opposite side, arguing the case against the motion. 

The debate consisted of three phases spread out over roughly ten days.  We each first stated our respective positions in our opening statements, followed a few days later by our rebuttals, and then finally our closing statements.  It was moderated by Ludwig Siegele, online business and finance editor at The Economist, who offered his own remarks in each phase of the debate.  Throughout the process, people were invited to vote on the motion, as well as to post their own comments. 

The debate was inspired, I believe, by The Multiplexed Metropolis, an article Siegele published in the September 7 issue of The Economist which explored the impact of big data on cities.  He wrote that the vast amounts of data generated by the many social interactions taking place in cities might lead to a kind of second electrification, transforming 21st century cities much as electricity did in the past.  “Enthusiasts think that data services can change cities in this century as much as electricity did in the last one,” he noted.  “They are a long way from proving their case.” 

In my opening statement, I said that I strongly believe that digital technologies and the many data services they are enabling will make cities smarter and help transform them over time.  My position is not surprising, given my affiliations with NYU's Center for Urban Science and Progress (CUSP) and Imperial College’s Digital City Exchange, as well as my past involvements with IBM’s Smarter Cities and with Citigroup’s Citi for Cities initiatives.  But, I totally understand why so many, - almost half of those voting and quite a few who left comments, - feel that smart cities are mostly hype.  The case for smart cities is indeed far from proven.

Understanding the Workings of Our Minds

Advances in technology, big data and analytics hold the promise to significantly augment our judgement and expertise and help us make smarter, more effective decisions.  But, as we contemplate these exciting innovations, it’s good to take a step back and ask ourselves a few basic questions: How do we make decisions in the first place? What goes on in our minds when we are making decisions, from the simplest to the most complex? 

I heard a fascinating talk by Daniel Kahneman that directly addressed these questions at the IBM Cognitive Systems Colloquium which I attended last month.  Kahneman is Professor of Psychology Emeritus at Princeton University and Senior Scholar at the Woodrow Wilson School.  In 2002, he was awarded the Nobel Prize in Economics “for having integrated insights from psychological research into economic science, especially concerning human judgment and decision-making under uncertainty.” 

The talk was based on his 2011 bestseller Thinking, Fast and Slow.  The book explains the major discoveries by psychologists and cognitive scientists over the past several decades that have led to our current understanding of judgement and decision-making.  In particular, it describes the pioneering work of Kahneman and his long time collaborator Amos Tversky, who died in 1996.

Humans, Machines and Collective Intelligence

Tom Malone gave a very interesting talk on collective intelligence at the IBM Cognitive Systems Colloquium which I recently attended and wrote about.  Malone is Professor of Management at MIT’s Sloan School and the founding director of the MIT Center for Collective Intelligence (CCI).  His research is primarily driven by this fundamental question: “How can people and computers be connected so that - collectively - they act more intelligently than any individuals, groups, or computers have ever done before?”  This is a very important question to explore to help us understand the impact of our increasingly smart machines on the very nature of work and organizations.  

Malone and his collaborators are conducting research on a number of topics in this area.  Do groups exhibit characteristic levels of intelligence which can be measured and used to predict the group’s performance across a wide variety of cognitive tasks?  If so, can you devise tests to measure the group’s intelligence using methodologies and statistical techniques similar to those that have been applied to measure the IQs of individuals for the past hundred years?

To answer these questions they conducted a number of studies where they randomly assigned individuals to different groups, which then worked on a variety of tasks.  Their research results were published in the October 2010 issue of Science.

Cognitive Systems and Big-Data-driven Applications

In the first week of October I participated in a Cognitive Systems Colloquium hosted by IBM at its Thomas J. Watson Research Center.  IBM defines cognitive systems as “a category of technologies that uses natural language processing and machine learning to enable people and machines to interact more naturally to extend and magnify human expertise and cognition.  These systems will learn and interact to provide expert assistance to scientists, engineers, lawyers, and other professionals in a fraction of the time it now takes.”  

The need for such systems is a result of the explosive growth of data all around us.  Not only are we now able to collect huge amounts of real-time data about people, places and things, but far greater amounts can be derived from the original data through feature extraction and contextual analysis.  One of the key lessons from Watson, - IBM’s question-answering system which in 2011 won the Jeopardy! Challenge against the two best human Jeopardy! players, -  was that the very process of analyzing data increases the amount of data by orders of magnitude.  

This is challenging our ability to store and analyze all that data.  The new generation of cognitive systems will require innovation breakthroughs at every layer of our IT systems, including  technology components, system architectures, software platforms, programming environment and the interfaces between machines and humans.

Smart Cities: A kind of 21st Century “Urban Electrification?”

In The Multiplexed Metropolis, an article in the September 7 issue of The Economist, online business and finance editor Ludwig Siegele explores the impact of big data on cities.  Cities are centers of social interactions, commerce, entertainment and many other human endeavors.  Will the vast amounts of data generated by these activities, - properly collected, analyzed and acted upon, - lead to a kind of second electrification, transforming 21st century cities much as electricity did in the past?

“The power cables that penetrated cities in the late 19th century transformed their shape (there are no tall buildings without lifts), their transit systems, their nightlife, their sewerage (cities need a lot of pumps),” writes Siegele.  “Ubiquitous data services might have impacts as wide-ranging: they could make cities more liveable, more efficient, more sustainable, perhaps more democratic.”  But, he adds a note of caution: “Enthusiasts think that data services can change cities in this century as much as electricity did in the last one.  They are a long way from proving their case.”

In the last few years, a growing number of initiatives have been launched in business, government and academia to help prove the case for data-driven smart cities.  Last year, for example, New York City Mayor Michael Bloomberg announced the formation of NYU’s Center for Urban Science and Progress (CUSP), a partnership between NY City agencies, NYU and other academic institutions, and a number of companies including IBM, Microsoft, Xerox, Cisco and Siemens.  A few weeks ago, mayor Bloomberg personally welcomed the inaugural class in CUSP’s MS in Applied Urban Science and Informatics program.

Data-Driven Decision Making: Promises and Limits

Decision making, - how individuals, groups and organizations go about selecting a course of action among several alternative scenarios, - has long been a subject of study.  Given the explosive growth of big data over the past decade, it’s not surprising that data-driven decision making is one of the most promising applications in the emerging discipline of data science. 

In a recently published article, Data Science and its Relationship to Big Data and Data-Driven Decision Making, Foster Provost and Tom Fawcett succinctly define data-driven decision making as “the practice of basing decisions on the analysis of data rather than purely on intuition.”  Equally succinctly, they view data science “as the connective tissue between data-processing technologies (including those for big data) and data-driven decision making.”

Looking deeper at data-driven decision making, it’s important to understand not only the promise but also the limits.  When can we embed decisions into well understood, automated processes?  When does automation run into limits, and we should view data-driven decision making as a tool to help people make smarter, more effective decisions?  And what are the prospects for its future, as technology, big data and data science continue to advance?

A Kind of Digital “Perfect Storm”

Every day brings new stories about the impact of digital technologies on every aspect of the world around us, including their transformative impact on companies in just about every industry.  Some of the articles are about the creation of innovative new technologies and applications, while others are about the creative destruction such innovations leave in their wake.  Lately, the latter kind of stories seem to predominate.

For a while now, companies have been racing to try to keep with the relentless advances of digital technologies.  But, the race keeps getting tougher every year.  We seem to be heading toward a kind of digital perfect storm, where four major IT trends are each gathering speed while interacting with and amplifying each other: mobile devices, cloud computing, social networks and big data and analytics.

These widely discussed technologies are still in the early stages of deployment to a greater or lesser degree. Mobile is furthest along while cloud adoption is taking off.  Data science applications are starting to get deployed, although we are just beginning to appreciate their profound implications.  And while consumer social networks have been thriving for years, their use in business has been slower than expected. 

The disruptive impact of this digital perfect storm is already being felt by technology companies as well as by those in industries whose transformation is already underway.  And perhaps most important, we know that even greater disruptions are in store in the not too distant future.  The resulting creative destruction makes for interesting stories about the changing fortunes of companies as well as their leaders.

Parallel Supercomputing: Past, Present and Future

From the early days of the industry, supercomputers have been pushing the boundaries of IT, identifying the key barriers to overcome and experimenting with technologies and architectures that are then incorporated into the overall IT market a few years later.  While we generally focus on their computational capabilities as measured in FLOPS, - Floating-point Operations Per Second, -  supercomputers have been at the leading edge in a number of additional dimensions, including the storage and analysis of massive amounts of data; very high bandwidth networks; and highly realistic visualizations.

Through the 1960s, 1970s and 1980s, the fastest supercomputers were based on highly specialized, powerful technologies.  But, by the late 1980s, these complex and expensive technologies ran out of gas and parallel computing became the only realistic alternative to scaling up performance.  

Instead of building machines with a small number of very fast and expensive processors, the early parallel supercomputers ganged together 10s, 100s, and over time 1000s of much less powerful and inexpensive CMOS microprocessors, similar to the micros used in the rapidly growing personal computer and workstation industry.  A similar evolution to microprocessor components and parallel architectures took place a few years later in the mainframes used in commercial applications.

The transition to parallel supercomputing was seismic in nature.  Everything changed, from the underlying computer architecture, to the operating systems, programming tools, mathematical methods and applications.  It took considerable research and experimentation to learn to effectively use these new kinds of machines.  Moreover, there were widely different parallel architecture designs, some coming from universities and others from industry.  It wasn’t clear at all which designs worked well for different kinds of applications and would thus be commercially viable.

The Department of Energy (DOE) national labs have long been among the world’s leading users of advanced supercomputers and played a leading role in the transition to parallel architectures.  In 1983, the DOE’s Argonne National Lab established the Advanced Computing Research Facility (ACRF), an experimental parallel computing lab which brought together computer scientists, applied mathematicians and supercomputer users and vendors to learn how to best use this new generation of parallel machines. 

This past May, Argonne convened a Symposium to mark the 30th anniversary of the ACRF.  The Symposium looked both at the progress made in parallel computing over the past 30 years and the major trends for the future.  I attended the Symposium and led a panel on The Impact of Parallel Computing on the World.

Smart Homes - Building for America's Bravest

In the Fall of 2007, I taught a graduate course in MIT - Technology-based Business Transformation.  The course focused on the major challenges that a business must deal with to successfully bring a disruptive innovation to market, including technical and market strategies as well as organizational and cultural issues.  The course drew heavily on my concrete experiences at IBM, especially what I learned as general manager of the Internet Division in the second half of the 1990s.

Within the limits of a one-semester course, I wanted to give the students a fairly comprehensive picture of what it takes to bring a disruptive innovation to market, including the key role played by marketing and communications.  I wanted the class to hear from a top professional in the field what brand and strategic marketing are all about, so I invited Chris Wall, - Vice Chairman of Ogilvy and Mather, one of the top marketing and advertising agencies in the world, - to be a guest lecturer in the class.  For a number of years, I worked closely with Chris and his O&M colleagues in the development of marketing campaigns for IBM initiatives I was leading.  These included the award-winning e-business campaign launched in 1997, - which explained what we meant by e-business in a series of really clever 30 second vignettes, - and the Linux campaign a few years later which included this incredible ad. 

In his excellent class seminar, Chris explained the difference between trying to establish a brand through mindless jingles or paternalistic assertions that no one believes, versus engaging your audience in a conversation about the future.  To engage in such a conversation, you need to essentially tell a story about your aspirations for the brand and how these aspirations are driving your strategy.  And, the more powerful and complex the messages you are trying to convey, the more important it is to do so by telling a compelling, emotionally resonant and simple story.

Chris and I became friends, and over the years he has shared with me some of the marketing campaigns he’s worked on that he is particularly proud of.  A couple of weeks ago he sent me an e-mail telling me about his latest project with a small foundation, Building for America’s Bravest.  Chris and a few of his Ogilvy colleagues created the integrated marketing campaign for the foundation, including its website, ourbravest.org which has just gone live, and includes a few short video clips along with pictures and stories.

I was frankly left speechless by ourbravest.org.  Compelling and emotionally resonant does not begin to do justice to its content.  The video clips and pictures truly grab you by the throat, or rather, by the heart.  I cannot possibly describe them in words, so please take a look and judge for yourself.

From the Digital Divide to Inclusive Innovation: the Case of Digital Money

On June 26 I participated in a panel discussion on digital money at the Royal Society of Arts (RSA) in London.  The RSA, - whose full name is the Royal Society for the encouragement of Arts, Manufactures and Commerce, - was founded in 1754 to “embolden enterprise, enlarge science, refine art, improve our manufacturers and extend our commerce”.  Today, it calls itself “an enlightenment organisation committed to finding innovative practical solutions to today’s social challenges”. 

The panel was organized to mark the publication of an RSA pamphlet - From the digital divide to inclusive innovation: the case of digital money.  I was a co-author of the pamphlet, along with Imperial College professors David Gann and Gerald George and University of Queensland professor Mark Dodgson.  The four of us were joined in the panel by Citigroup executive Tomasz Smilowicz.

We discussed the profound transformative power of digital money.  Economic transactions are increasingly moving from the physical to the digital realm, democratizing financial services around the world and helping billions join the global digital economy and improve their standard of living.  As we wrote in the pamphlet: 

“Just as communications and publishing have been transformed by digital technologies, so too will financial services.  The progress of digital money will inevitably surprise us and it will develop in unexpected ways, but we believe it is on the cusp of delivering a remarkable transformation in the global economy.  It will end the divide between those who can and those who cannot participate in formal economic transactions.  It may usher in a new era of more inclusive innovation that involves billions of more people around the world in constructing the services that affect their future.”

The Era of Cognitive Computing

Tools have played a central role in human evolution since our ancestors first developed hand axes and similar such stone tools a few million years ago.  Ever since, we’ve been co-evolving right alongside the tools we create.  “We shape our tools and they in turn shape us,” observed noted author and educator Marshal McLuhan in the 1960s. 

The Industrial Revolution led to dramatic improvements in productivity and standard of living over the past two hundred years.  This is due largely to the machines we invented to make up for our physical limitations - the steam engines that enhanced our physical power, the railroads and cars that made up for our slow speed, and the airplanes that gave us the ability to fly. 

Similarly, for the past several decades computers have been augmenting our intelligence and problem solving capabilities.  And, according to IBM’s John Kelly and Steve Hamm, there is much more to come.  In Smart Machines: IBM’s Watson and the Era of Cognitive Computing, Research director John Kelly and writer and strategist Steve Hamm, note that “We are at the dawn of a major shift in the evolution of technology.  The changes that are coming over the next two decades will transform the way people live and work, just as the computing revolution has transformed the human landscape over the past half century.  We call this the era of cognitive computing.”

The Continuing, Transformative Impact of IT

Last month, the McKinsey Global Institute published Ten IT-enabled business trends for the decade ahead.  As is generally the case with McKinsey, this is a well researched, well written report.  There are few surprises.  Any technology expected to have a transformative impact on business over the next decade has to at least be already in the hands of leading edge users.  Disruptive innovations take time to play out.

“Many trends reflect the growing dominance of the Internet as an enabling technology, as well as a model and metaphor for commercial and social interactions,” says the report in its introduction.  “Twenty years into the Internet revolution, businesses and consumers have come to expect that information is a Google search away, friends and associates are always available on social networking sites, and goods and services (including public goods such as education and government services) can be had instantly from an online vendor anywhere in the world at any time of the day.”

The social matrix, the Internet of all things, big data and advanced analytics, and realizing anything as a services are the report’s top four trends.  SMAC - Social, Mobile, Analytics & Cloud, - has become the new plastics, capturing the future of IT in one word, or rather, one acronym.  Just about everyone agrees that these are foundational technologies, like the Internet twenty years ago, that every business must embrace.

New Tools Beget Revolutions: Big Data and the 21st Century Information-based Society

On May 22 I attended the 2013 MIT Sloan CIO Symposium.  This year’s theme was The Transformational CIO: Architecting the Enterprise of the Future.  “The enterprise of the future will be very different from the one we know,” wrote the organizers in the event’s website.  “It will be complex, develop hitherto unforeseen forms, and will have to respond to unforeseen challenges.  All while the windows of opportunity and response will continue to shrink.  There will not only be the need to be agile and adapt to changes as they occur, but to be proactive in shaping the next generation enterprise to be ready for the future.” 

The Symposium included a number of talks and panels on the key issues facing CIOs in our post-digital world, that is, in an era when digital technologies permeate just about every nook and cranny of the business, and where every business is a digital business, the title of one of the talks at the event.  While a number of transformative technologies were discussed, big data was the most prominent.    

MIT professor Erik Brynjolfsson led an all-MIT academic panel of experts on The Reality of Big Data.  In a brief introductory talk, Brynjolfsson pointed out that throughout history new tools beget revolutions.  Scientific revolutions are launched when new tools make possible all kinds of new measurements and observations.  In 1676, for example, Antonie van Leeuwenhoek used a microscope, a relatively recent and rare tool, to discover the existence of microorganisms in a drop of water.  Thus was microbiology born, leading to major discoveries in biology, medicine, public health and food production in subsequent decades and centuries.

Traditional Data Analysis and Data Science

I recently participated in an industry panel on Redefining BI & Analytics in the Cloud at a conference in New York.  Among other questions, moderator Michael Hickins, - a senior editor at the Wall Street Journal and editor of its online CIO Journal, - asked the panel to discuss the difference between the business intelligence and analytics approaches of the past twenty years and the emerging discipline of data science. 

In my opinion, data science should be viewed as a multidisciplinary evolution from business intelligence and analytics.  In addition to having a solid foundation in statistics, math, data engineering and computer science, data scientists must also have expertise in some particular industry or business domain, so they can properly identify the important problems to solve in a given area and the kinds of answers one should be looking for.  Domain expertise is also needed to be able to draw the proper conclusions from their analysis, and to communicate their findings to business leaders in their own terms. 

The emergence of data science is closely intertwined with the explosive growth of big data over the past several years.  Institutions are now wrestling with information coming at them in volumes and varieties never encountered before.  In addition to their multidisciplinary skills, data scientists bring increased breadth and depth to the analysis of these different sources of information compared to traditional analyst roles, as described in this short article in IBM’s website, So what does a data scientist do?: 

“Whereas a traditional data analyst may look only at data from a single source - a CRM system, for example - a data scientist will most likely explore and examine data from multiple disparate sources.  The data scientist will sift through all incoming data with the goal of discovering a previously hidden insight, which in turn can provide a competitive advantage or address a pressing business problem.  A data scientist does not simply collect and report on data, but also looks at it from many angles, determines what it means, then recommends ways to apply the data.”

Reflections on Storytelling

Storytelling has played a central role in human communications since times immemorial.  Storytelling predates writing.  Oral narratives were used by many ancient cultures as a way of passing along their traditions, beliefs and learning from generation to generation.  Over the centuries, the nature of  storytelling has significantly evolved with the advent of writing and the emergence of new technologies that enabled stories to be embodied in a variety of media, including books, films, TV and the Web.

While contemporary storytelling is mostly associated with entertainment, education and culture, it also plays a major role in business.  In particular, storytelling is widely used in the development of brands for products and services as well as companies. 

Establishing a brand requires much more than advertising, mindless jingles or assertions that no one believes.  It is akin to engaging in a conversation with your intended audience, where you talk about your challenges and aspirations for the product and the company in order to create emotional associations and expectations.  In short, you are creating and telling a story about your brand.  And, the more powerful, important and complex the messages you are trying to convey, the more important it is that you do so by telling a compelling and emotionally resonant story. 

Good storytelling is particularly important when introducing a complex and potentially disruptive offering in the marketplace whose value is not well understood.  It’s a good way of explaining what the new innovation is all about in as simple a way as possible.  As a concrete example, let me share my experiences with the development of IBM’s Internet and e-business strategy in the mid-late 1990s.

The Digitization of the Economy

MIT just launched a new Initiative for the Digital Economy to systematically address the impact of digital technologies on business, the economy and society.  The university-wide initiative is being organized by MIT’s Center for Digital Business (CDB), and is led by Erik Brynjolfsson and Andy McAfee, who are respectively the CDB’s director and principal research scientist.

Since the mid 1990s, we’ve been talking about the emergence of an Internet-based digital economy.  It has often been said that powerful disruptive innovations like the Internet are overhyped in the short term but under-hyped in the long term.  This is certainly the case with digital economy.  The term was widely thrown around in the heady dot-com bubble days as people talked about a new economy with eyeballs instead of revenue, profit and cash as key business metrics. 

But, over the past decade the Internet has transitioned from the connected world of PCs, browsers and web servers, to the hyperconnected world of mobile devices, cloud computing and broadband wireless networks.  The physical and the the digital now blend into each other over porous boundaries, as people, institutions and things are all becoming interconnected.  A 21st century digital economy is indeed now emerging in this hyperconnected, inclusive, and increasingly smart world.

The Evolution toward a Global Digital Money Ecosystem

I recently read an excellent article, 2012: The Year in Payments - A Look Back At The News And The Noise In Payments by Karen Webster, CEO of Market Platform Dynamics and founder of PYMNTS.com.  In the article, Webster reviews some of the major developments in digital payments over the last year, and gives her opinion of what is likely to happen in 2013 and beyond.  Beyond her detailed reviews and predictions, the article is a very well written overview of the evolution toward a digital money economy, including the major advances already underway as well as the huge challenges that still lie ahead. 

For several years now I’ve been convinced that the development of a global digital money ecosystem is among the most exciting and important societal challenges in the coming decades.  It’s right up there, in my opinion, with other technology-based transformational innovations, including electricity, the telephone, radio and TV, and the Internet and World Wide Web.

Where are we in digital payments?  Webster believes that 2012 will be viewed in the payments history books as a very good year.  “Payments innovation fueled by the IP-enablement of devices used by consumers and merchants to interact drove payments innovation into high gear.  Players large and small flooded the market with new applications, new business models, and new approaches to transforming the shopping experience.”

Reflections on Big Data, Data Science and Related Subjects

A couple of months ago I was appointed executive-in-residence at the Center for Urban Science and Progress (CUSP), an interdisciplinary applied science research institute led by NYU and NYU-Poly in partnership with academic institutions, global companies and New York City government agencies.  CUSP’s overriding mission is the study of “the grand technical, intellectual, engineering, academic, and human challenges posed by a rapidly urbanizing world.”   It was formally launched last April by New York City’s mayor Michael Bloomberg. 

CUSP’s research and educational programs are centered on urban informatics, - “the acquisition, integration, and analysis of data to understand and improve urban systems and quality of life.”  Big Cities + Big Data and Bringing Urban Data to Life are prominently displayed in its website. This coming September, it will start offering two new programs in Applied Urban Science and Informatics, a 30-credit Master of Science, and a 12-credit Advanced Certificate. 

Given the central role of big data with CUSP and with other initiatives I’m involved with, I’d like to step back and reflect on what this all means.

Hurricane Sandy: a 21st Century Wake-Up Call?

As I was personally experiencing the fury of Hurricane Sandy a few weeks ago, I thought of the Internet Highway System and President Dwight D. Eisenhower.  Sandy brought into sharp relief how dependent so many of us have become on digital infrastructures, and the Internet in particular, for almost every aspect of our life and work.  Sandy also demonstrated how inadequate and fragile our digital infrastructures are given our increasing dependence on them.  If President Eisenhower was around today, I reflected in moments of frustration, I was convinced that he would be a strong advocate for building a world class national broadband infrastructure, just like he championed the Internet Highway System over half a century ago.

The National Interstate and Defense Highways Act was signed into law by President Eisenhower in June of 1956.  Most everyone would agree that it has contributed greatly to the economic success of the US over the past fifty years.  But, in addition to economic development, one of the main reasons President Eisenhower championed the Interstate Highway System was national defense.  While serving as Supreme Commander of the Allied forces in Europe during World War II, he saw first hand the role played by the German Autobahn network in its national defense, and felt that the US needed such a modern highway system to provide key ground transport routes for military supplies and troop deployments in case of an emergency or external threat. 

Getting ready for Sandy, and then feeling its power was very different from watching a far away disaster on television.  I got off fairly easily compared with so many in the New York Metropolitan Area,  including our town in Connecticut, whose homes were flooded or were seriously damaged by falling trees.  We lost a few large trees, one of which damaged part of a fence.  The power in our area was out for almost a week, but, like an increasing number of my neighbors in our woodsy area, we have adjusted to the frequent power losses by installing a standby generator, so our house had electricity and heat during the power outage.  We also lost cable and our cable-based Internet access.  My wireless mobile phone was barely usable other than very early in the morning before most of the town was awake.  For a couple of days, even that did not work. 

Over 22 million people live in the Tri-state New York Metropolitan Area.  The region is arguably the world’s center of commerce and banking, home to almost 20 percent of Fortune 500 companies.  New York City is ranked at the top by just every survey of the world’s most competitive cities, particularly because of its appeal to a wide range of businesses and talented people. 

But, as Sandy proved to us all, our physical infrastructure is antiquated, and our digital infrastructure lags far behind the growing importance of the Internet in the economy and in our personal lives.  With few exceptions, the situation is not much better across the US.  The infrastructures of just about all US metropolitan areas are falling behind the rest of the world.

Reflections on the Nature of Human Organizations

Urbanization is one of the major forces transforming societies all over the world.  The rapid pace of urbanization is succinctly summarized in these sentences from the 2012 World Urbanization Prospects of the United Nations Population Division:  

“Between 2011 and 2050, the world population is expected to increase by 2.3 billion, passing from 7.0 billion to 9.3 billion (United Nations, 2011). At the same time, the population living in urban areas is projected to gain 2.6 billion, passing from 3.6 billion in 2011 to 6.3 billion 2050.  Thus, the urban areas of the world are expected to absorb all the population growth expected over the next four decades while at the same time drawing in some of the rural population.  As a result, the world rural population is projected to start decreasing in about a decade and there will likely be 0.3 billion fewer rural inhabitants in 2050 than today.”

In 1900, less than 15 percent of world’s population lived in cities.  According to the latest UN estimates, 52 percent of the world’s population now lives in urban areas.  That number is estimated to go up to 60 percent by 2030, and near 70 percent by 2050. 

Why are metropolitan areas continuing to grow at such a rapid pace?  After all, back in the 1990s, quite a few people were predicting that the Internet would cause cities to decline.  The Internet would enable people to work from anywhere, so they would choose to live in less populated areas, where lower costs and more space would let them enjoy a better quality of life.  Needless to say, these predictions have not come true.