Over the last year, I have become intrigued by the rise of cloud computing. An increasing number of articles are being written on the subject, as a quick online search of the term will attest. More and more companies are taking a serious look at cloud computing.
Last summer, IBM launched a study of cloud computing sponsored by the IBM Academy of Technology and led by our top experts in the area. The study was very positive about the opportunities of cloud computing for the IT industry. But it also highlighted the many challenges to be addressed in deploying cloud computing, especially in existing data centers.
So, what exactly are we talking about?
A recent report by Forrester Research said in its executive summary that: "Cloud computing is a new IT outsourcing model that doesn't yet meet the criteria of enterprise IT and isn't supported by most of the key corporate vendors. It's wildly popular with startups, exactly fits the way small businesses like to buy things, and has the potential to completely upend IT as we know it. And there's a high likelihood developers inside your company are experimenting with it right now. Forrester spoke with more than 30 companies in this market to determine its worthiness for enterprise consideration and found that it provides a very low-cost, no-commitment way for enterprises to quickly get new services and capabilities to market that entirely circumvents the IT department. Infrastructure and operations professionals can try to ignore it as it is just in its infancy, but doing so may be a mistake as cloud computing is looking like a classic disruptive technology."
The Forrester report further said: “Cloud computing looks very much like the instantiation of many vendors' visions of the data center of the future; it's an abstracted, fabric-based infrastructure that enables dynamic movement, growth, and protection of services that is billed like a utility. It also has all the earmarks of a disruptive innovation: It is enterprise technology packaged to best fit the needs of small businesses and start-ups--not the enterprise."
BusinessWeek said something similar, albeit in more layman's terms: "A move towards clouds signals a fundamental shift in how we handle information. At the most basic level, it's the computing equivalent of the evolution in electricity a century ago, when farms and businesses shut down their own generators and bought power instead from efficient industrial utilities."
In the past month, I have given a couple of talks on cloud computing, and as always, presentations force you to think very hard about your views on a subject, as well as how best to communicate your ideas. I think of clouds basically as Internet-based networks made up of a very large number of servers and storage components. They contain vast amounts of information, and provide a variety of services to large numbers of people - to their mobile devices as well as their PCs. The users of clouds only care about the services and information they have access to, not about the underlying details of how the cloud works.
So far, this all sounds like a number of concepts we have been talking about for at least half a dozen years: Grid computing, virtualization and IT-as-a-service, let alone the Internet and the World Wide Web. Fundamentally, we are talking about the evolution of IT and the data center in the 21st century.
But, in my opinion, two key factors take cloud computing into a qualitatively different dimension. One is massive scalability. I believe that the kinds of advances that we have become used to in the world of high-end supercomputing are now coming to the more general purpose computing world. I expect that a number of the new applications that data centers will be asked to support will grow by two to three orders of magnitude over the next decade. A 10X - 100X growth over 10 years means roughly that the applications are growing at between 25% and 60% CGR.
Then there is the much higher quality of experience that cloud applications provide to their users. Cloud applications are very different from classic IT applications, whose intrinsic complexities are barely hidden from their users. You truly want users of cloud applications to just be able to access them in the most natural and simplest way possible. Cloud applications should be able to provide a really high quality of experience to massive numbers of users without missing a beat. They should significantly improve the way people deal with the many tasks and devices that surround them in their everyday life – at work, at home, on-the-go, and wherever they happen to be.
Do we have any such workloads in the horizon that will likely grow at prodigious rates and require a human-like quality of experience? Quite a few, I believe: real-time information access and analysis, such as RFID-based supply chains, transportation management and security systems; myriads of new consumer applications - in entertainment, healthcare, payments and financial services; social networks and virtual worlds involving large numbers of people interacting with each other; support of billions and billions of new mobile devices and sensors; and so on.
Are such massively scalable, high-quality-of-experience workloads important only to companies like Google, Yahoo, Amazon, MSN, IBM and similar companies already developing cloud computing infrastructures? Not at all, any more than the Web proved important only to its early adopters. As with the Web in the mid-‘90s, every enterprise will have to develop its own cloud-like capabilities, or work closely with partners that do.
Will companies follow the pattern of electricity’s evolution a century ago and shut down their data centers, relying on highly efficient, professionally managed service providers instead? I expect that many small businesses will do just that, as well as perhaps a number of mid-size companies. But I suspect that many larger companies will not only continue running their own data centers, but find new growth opportunities by selling business services to the wider marketplace that were developed and only available for use within the company, as is already the case with Fidelity, UPS, IBM and a number of other companies.
How well prepared are most enterprise data centers for a 10X - 100X scalability? By their own admission, most are not ready at all. In fact, many CIOs think of their data centers as something like a dog's breakfast of technologies that have evolved over the years with little architectural discipline or company-wide governance.
Many data centers have grown through mergers and acquisitions with a variety of companies, each having brought with them their own separate equipment, architectures and processes. Different departments in the business have often insisted on getting their own servers for their own applications, rather than using the global, shared data center facilities. Small and mid-size servers have proliferated in many companies, each dedicated only to its particular applications, and thus often running at utilizations of 20% or less - that is, those servers are only doing real work 20% or less of the time, while they consume electricity 100% of the time. The costs of managing such highly distributed installations are typically very high.
As we learned in manufacturing over the last thirty years, you cannot achieve world-class productivity and quality unless you leverage engineering disciplines and real-time information and considerably simplify your processes and architectures. Similarly, you just cannot scale a messy data center, especially given the fast growing workloads that they will be required to support over the next decade. System management and energy costs will prove prohibitive. Quality, security and availability will suffer.
Integrating the capabilities of cloud computing into enterprise data centers will require a much more disciplined approach to architecture and governance. Just about every CIO I have ever met advocates such an approach. But because IT cuts across just about all lines of businesses and processes of the organization, they need the strong support of the CEO and top management. Otherwise the enterprise will just follow human nature and prioritize short-term expediency over longer-term discipline. Given the fierce, global, competitive environment we live in, that can prove to be a very costly mistake.