Dealing with Increasingly Improbable Events

 For a long time, scientists and science-fiction writers alike have pursued the question whether you can accurately predict the future from the past given sufficiently large groups, historical information and computational power. 

For example, in the Foundation Series, one of science-fiction's classics of all times, Isaac Asimov introduced the fictional scientific concept of Psychohistory.  The essential idea in psychohistory, says the Wikipedia entry, is that “while one cannot foresee the actions of a particular individual, the laws of statistics as applied to large groups of people could predict the general flow of future events.  Asimov used the analogy of a gas: an observer has great difficulty in predicting the motion of a single molecule in a gas, but can predict the mass action of the gas to a high level of accuracy.”

Psychohistory relied on extremely advanced models which enabled a purely mathematic approach to predict future events.  However, it was not able to cope with totally unpredictable events.  Thus, when a mutant was born, the fictional character known as the Mule, his totally unanticipated psychic skills invalidated the assumptions underlying the models, and history then proceeded on a very different course that what had been predicted by the psychohistory computers.

Asimov and his wonderful Foundation story have been on my mind in the last few years as I have been thinking about complex organizational systems, where people and the services they perform for each other are the primary components.  In particular, I have wondered about the limits in our ability to accurately model and predict the behavior of such people-based systems, especially as I continue to reflect on the causes of the global financial crisis - our financial systems being an example of such complex, people-based organizational systems.  Will the mutations, highly irrational behaviors and other unpredictable events inherent in such systems always befuddle mathematical models and computers, no matter how sophisticated and powerful we make them.

I recently came across a very good article on the subject:  Risk Mismanagement by business columnist Joe Nocera in the January 2 issue of the New York Times Magazine.  In it, Nocera explores whether the financial meltdown was primarily caused by errors in the mathematical models used to evaluate trades and risk, or by bankers and regulators who misread or ignored the models.  Is the crisis mostly a failure of risk management or a failure of management? 

The article prominently features Nassim Taleb, author of The Black Swan: The Impact of the Highly Improbable.  Taleb uses the concept of the the highly rare Black Swans as a metaphor for high impact, hard-to-predict events beyond the realm of normal expectations.  Black Swans are events that can totally change the course of history, sort of like the unexpected appearance of a mutant with psychic powers in Asimov’s Foundation, except that they occur far, far more often.  As examples of such Black Swan events he cites the rise of the Internet, the personal computer, World War I, the 9/11 attacks, and our ongoing financial meltdown.

Explaining Taleb’s views, Nocera writes that “the greatest risks are never the ones you can see and measure, but the ones you can’t see and therefore can never measure.  The ones that seem so far outside the boundary of normal probability that you can’t imagine they could happen in your lifetime - even though, of course, they do happen, more often than you care to realize.  Devastating hurricanes happen.  Earthquakes happen.  And once in a great while, huge financial catastrophes happen.  Catastrophes that risk models somehow always manage to miss.”

Models based on analyzing historical data are very good at accurately measuring the risk in a portfolio under normal market conditions, the kinds of markets that explain 99 percent of events and follow the familiar bell curve of a Gaussian or normal distribution.  But, every so often, say one percent of the time, improbable events happen that are way outside a normal distribution.  Such market events are totally unpredictable, that is, the future could not have been predicted based on past behavior, because the improbable event is something that has rarely, if ever, happened before.

How do you deal with such an improbable event?  That’s where human judgment comes in, especially the judgment of experienced managers that have been around for a while and can intuitively feel when something does not quite feel right.  The best thing to do at such times is to get together with other experts whose opinions you trust, and see if they share your misgivings.  It is good to assemble people with a diverse set of skills and opinions.  Where someone may see trouble brewing, someone else may see a casual disturbance that will work itself out.  If needed, you can always bring outside experts and add their views to the mix.

Like meteorologists tracking a disturbance in the North Atlantic during hurricanes season, you want to leverage advanced technologies and real-time information to help figure out what's going on.  You should run a variety of models to see how the disturbance might evolve under different scenarios.  You should keep re-running the models as new information comes in.  It typically takes a while to figure out whether a disturbance is a false positive, just part of the expected ups and downs of a normal market, or whether it is a sign that something much more serious is taking place that calls for extraordinary actions.

Mathematical models do a very good job with normal events that is, those events that fall within the 99 percent probabilities of Gaussian curves.  But those models are pretty useless when it comes to predicting what happens with the other one percent of events at the extreme edge of the curve.  The models are not able to tell you whether that one percent is something fairly mild that happens a few times a year, or whether a true black swan is lurking that will cause losses in the billions and billions and threaten to bring down companies, nations and perhaps the overall global financial system.

The issue is not whether the best decisions are based on quantification and numbers, or whether given an uncertain future, decisions should be based on the intuitive beliefs and experience of human.  You need both.  It all depends.  The real danger comes if we rely too much on the mathematical models that work well most of the time, and put too much trust on technically brilliant quants who lack the judgment that comes from experience, especially from having personally seen what a black swan looks like and what it can do to you and your firm.

“Because we don’t know what a black swan might look like or when it might appear and therefore don’t plan for it, it will always get us in the end,” writes Nocera. “Any system susceptible to a black swan will eventually blow up,” Taleb says. The modern system of world finance, complex and interrelated and opaque, where what happened yesterday can and does affect what happens tomorrow, and where one wrong tug of the thread can cause it all to unravel, is just such a system.”

As our world becomes increasingly integrated, fast changing and unpredictable, we expect large improbably disturbances - black swans - to occur more frequently, not only in finance but across business, government and society in general.  Mathematical models, information analysis and fast computers will continue to be extremely valuable tools, critical to the smooth functioning of our complex systems.  But, when the going gets really rough, no machine or model can ever make up for the wisdom that only comes from human judgment and experience.

Comments

[Chris Ward]

So what do we do, as IBMers ?

I guess the ultimate backstop is the Business Conduct Guidelines http://www.ibm.com/investor/governance/business-conduct-guidelines.wss ; all of us sign them every year, we should know what they say; and it should give us guidance so that we don't have to phone Sam Palmisano every 5 minutes and ask his advice. What with them being on the external web site, clients can see them too, as can entities which have no direct financial relationship with IBM.

He needs the 'space' to concentrate on the longer term, the strategic issues, the things that matter beyond the daily circulation of little green pieces of paper. Maybe "Oil" , "Water", and other fundamentals related to continuity of all sorts of businesses.

If you're going to quote Asimov, then I'll quote HitchHiker's Guide. "It isn't usually the little green pieces of paper that are unhappy."

The dollars are our servant, they are not our master.

There's a significant 'shift' going on, at least in the USA and Europe. It appears to be that private enterprises are falling into public hands; instead of Ford, Chrysler, and GM, we are looking at the possibility of "First American Government Automobile Manufacturers". Is that 'Staring into the abyss' from which we might shrink back ? Or is it the inevitable way of the future ? Do Western governments want to operate automobile manufacturing plants and showrooms, to staff them with what in the UK we would call "Civil Servants" ? Or is the aim to place them back into private profit-seeking hands as and when ?

I'm not sure what's happening in the rest of the world; are China and Russia performing the gyration in reverse, putting things in the hands of private businesses, and asking them to compete ?

Day to day, though, the IBMers are adequately funded; correctly in harness to their managers; willing and able to serve in all their different roles from Research Scientist, through Development Engineer, Marketing Specialist, Sales Person, Service Delivery Specialist, Business Consulting Servant, Global Technology Servant, and a number of other specialised but interlocking fields.

Just don't ask us to develop or market Lotus SmartSuite, OS/2, Personal Computers, Hard Disks, Typewriters, or Card Punches. Some things just cannot be done profitably by IBM any more.

Throttle up ... gently, so as to keep in good control ... and let's see where tomorrow leads.

So long as Microsoft and IBM are separate, so long as there's no grand relaunch as "IBM-o-Soft Corporation" with it flagship product "IBM Loto-Soft Office", so long as potential customers have a choice in how to deploy their technology spend, I think we are OK.

The old rivalries are intact. And the public interest is served.

[Jo Grant]

Chaos, in the mathematical sense, stems from feedback situations. I think many of these "Black Swans" are better explained by the greater number of recursive systems in our society. In many ways, the finance sector was due something like this from the time they went electronic.
As IBM looks more at things like self correcting systems and advanced business logic it would be well to remember this lesson.

[Chris Ward]

Jo,

A sense in which everyone's indicator switched to 'sell' at the same time ... but failed to say "who to", and "at what price would they buy".

IBM's in a lot of businesses. I guess when delivering Business Consulting Services, it could be one of the factors advised about. When delivering Global Financial Services, it is a risk factor to assess. But if you're selling Websphere Software, or Lotus Notes, or Mainframe Computers, or chips for games consoles, I suspect that IBM will have to trust that the customer knows what they will do with the product. IBM can (and does) warrant that the product will function to its specifications; but the question of whether the customer can make a profit out of it is surely a 'user management responsibility'

How to sell to a bank, without becoming a bank ? IBM likes selling a lot of technology to banks; but I think has no desire to become a bank, to run a banking business on its own account.

[Mcihael Rowe]

I really enjoyed the book "The Black Swan", as a matter of fact it was the first business book I reviewed on my new Kindle Podcast (Random Abstractions - http://www.michaelrowe01.com/Random_Thoughts/Random_Abstractions/Entries/2009/2/14_Random_Abstractions_-_Episode_2_-_The_Black_Swan.html ). I think it really does point out that we tend to limit our thoughts and actions based on our understanding of things around us. We don't look to find the those things which are outside of our understanding. As a matter of practice, many people will actively discount things that don't confirm to their understanding.

I think the sales question that is raised is a bigger issue, I think we know the who and the what, but one of the big questions that is ignored is the why. While I am a big supporter of research for the desire of knowledge, but when it comes to many IT offerings, I think we constantly drive customers and consumers to by the next big thing, without answering the fundamental WHY question.

We are constantly upgrading applications and tools, but do we really need them? Most people use less than 20% of the functionality of mature products, and yet, they still upgrade to the next version, which is mostly feature bloat. So while competition is good do we need yet another office clone (to address Chris's point). Why not a simplified word processor?

[Richard Schwartz]

The Nocera article you refer to is excellent, however I have a problem with characterizing events as "highly improbable" just because they "seem so far outside the boundary of normal probability that you can’t imagine they could happen in your lifetime." The key word there is "seem".

Probability may deal with uncertain outcomes, but it is itself a mathematically precise concept. "Seem" is dead giveaway of an imprecise application of the concept. The probability in reality was actually very close to one -- we know this because it happened, and because we can easily see in hindsight that the chances of avoiding it were extremely low. It only seemed like the probability was close to zero to the experts because they were not looking at the right inputs and behaviors, and therefore did not follow them through to their logical conclusion.

I'm not a game theorist, but it seems to me that what the experts did not see was that rational behavior by actors throughout the system could lead to a "tragedy of the commons" outcome, not due to an improbable event, but simply due to playing the game through enough rounds. The game was basically one of increasing risk by spreading it around, and taking short-term profits rather than holding reserves to hedge against the increased systemic risk; but without a mechanism to stop the game the total risk rises to the point that it simply can't be spread around enough, there are insufficient reserves, and the game collapses.

A mathematical model that accurately embodied the rules, accurately implemented rational decision-making according to each type of actor's perceived short-term interests, and played the game on the right scale through enough rounds should have reached that outcome. Of course, nobody had that model. A very interesting question though is this: if someone had had that model, and had raised the alarm, would anybody have gone against their short-term interests and actually listened?

[Chris Ward]

Richard,

Have you ever played Monopoly ? (The kids' board game, (c) Parker Brothers in the USA, I think, apologies if I have incorrectly used someone's trade dress)

It always ends with one player having all the money and everyone else bankrupt.

This seems to be roughly what's happened in real life. Not quite so extreme, but the concept is there.

Can we say that we have had enough of Monopoly, we will put the game back in its box and pick another one ? I'm not entirely sure that the next games on my kids' shelf are any better ... Diplomacy and Risk ... but they are different.

How to avoid these extreme outcomes ? Do we need to empower governments to intervene in markets ? What kind of intervention would you allow, and how would you ensure that the intervention was on the vaguely-Benthamite principle of "The greatest good for the greatest number" ... in particular, you might want to ensure that those chosen to govern didn't use their power to line their own pockets.

I would mention Microsoft Windows and Microsoft Office, but although that has enriched a select group of people beyond the wildest dreams of most of the world (and drained cash from the rest of us to do it), I don't actually think that it's Microsoft's fault in any sense. Should a government intervene to break that special tie, to enforce a standard interface, to force interopearbility and facilitate competition ? Or should it be left, to be eventually obsoleted by cellphones and games consoles ?

If you want a probability of an event which either 'happens' or 'does not happen', you have to run reality (or a model) many many times, and count.

[Steven Adler]

No risk can be managed if it isn't first measured.

Our current regulatory infrastructure is simply outpaced by the scale and speed of financial processing, and in that gap greed, fraud, and incremental errors and omissions will creep until the system overloads and we witness an enormous crash.

The meltdown isn't a symptom of the failure of capitalism or dark forces at the fringe. It is simply a normal consequence of human self-interest unmonitored and unchecked.

Computer models based on historical analysis may indeed help future humans avoid catastrophes - if the information is widely available. First step is to make it available.

[Mike Duncan]

Why are black swans always disasters? Or is a disruptive innovation a black swan? Any thoughts.

Sorry for this delayed response - I started reading this blog after this was posted and have just been browsing some back numbers.