Last week IBM announced a new kind of software platform, IBM Information Server, designed to help businesses and other organizations integrate and derive more value from the disparate sources of information that are spread across their systems. The announcement was made at a major conference we held in Anaheim, Information on Demand 2006. I participated in the conference and led a discussion on Technology and Innovation with a panel of some of our top technical experts in information management in IBM.
We are so surrounded by information in our everyday lives that we don't notice it. It is like the air we breathe. Information comes at us from all sides. Some of it is well structured, such as our bills, financial statements and pay stubs. Most of it is highly unstructured - TV programs, movies, newspapers, books, pictures, conversations, phone calls and e-mails, for example.
From its inception, the IT industry has been dealing with information. After all, the "I" in IT stands for Information. Before the term IT became widely used, we generally referred to commercial computing as Data Processing, because that is what commercial applications did - they processed data stored in highly structured file and database systems. But, while data and information in general have always been a major part of the IT industry, until relatively recently the key focus of the industry has been on applications and their underlying platforms and development tools, rather than on information and related technologies.
The IT industry started waking up to the growing opportunities around information about ten years ago, with the explosive growth of the World Wide Web and the immense variety of content it provided to enormous numbers of people. Then came the dramatic improvements in both the capacity and cost of storage technologies, which enable us to store in digital form the voice, pictures, video and other types of unstructured information that surround us. Finally, major advances in computer performance have enabled us to analyze, search and generally take advantage of all that information that now resides in our systems.
At present, we are surrounded with so much digital information that we are nearly drowning in it. Yet we face the promise of exponential growth in the amount of information we will continue to get in the future. In the last ten years, a large number of products have emerged to help us deal with different kinds of information and the different ways we want to process it. Generally, these products don't work easily with each other which makes it difficult to leverage all that information for business value, and makes it very complicated and costly to manage.
The Information Server we announced last week provides a comprehensive platform for enterprise information architectures. It is a very useful foundation for Service Oriented Architectures (SOA), providing consistent, reusable information services to applications and business processes. Having a set of open, shared services in a common platform enables Information Server to integrate all the various information products from IBM, as well as those from business partners and any other offerings built on the Information Server platform. Its modular architecture permits Information Server to run on highly parallel architectures and Grids and thus scale to process very large volumes of information.
It was in this context that the Innovation and Technology panel I moderated explored the role of information in systems, applications and business solutions. The panel included Curt Cotner, Jeff Jonas, Chet Kapoor, Nelson Mattos and Dan Wolfson. We talked about a variety of subjects and answered questions from the audience for about one hour. Let me comment on a few of the points we discussed.
We talked about how Web 2.0, search engines and video games that are already highly popular in the consumer world are now becoming integrated into the worlds of business, health care, education and government. This is not unlike what happened ten years ago when the Internet and its technologies and standards became part of IT, leading to what became known as e-business.
Situational applications, mashups, wikis, blogs and similar technologies permit very rapid application development by the end users themselves, and properly used, can make a business very flexible and productive. However, as a member of the audience pointed out, there are many processes in business that require high availability, security, performance and other classic "industrial strengths", which are much better provided by tried and true hierarchical and relational data bases than by "good enough" consumer-oriented offerings.
This is an excellent point. The key is to have very good architecture and engineering at all levels – infrastructure, applications and business. Different parts of the IT infrastructure and of the applications and business solutions have very different requirements. To make the point by analogy, if you are designing a skyscraper, you clearly want to be very careful with the foundations and structure of the building, doing lots of simulations and tests to make sure that it can withstand whatever stresses it may be subjected to such as earthquakes, hurricanes, floods, pollution and so on. You will clearly prioritize safety over flexibility for these parts of the project. On the other hand, if you try to apply similarly rigorous design and engineering principles to the interior of the building, making it very difficult and expensive to customize office space to the requirements of the different tenants, you will not have a very people-friendly building. Good design and engineering combine safety and industrial strength with flexibility and human-oriented qualities, using each as appropriate.
The panel participants talked quite a bit about the role of metadata and semantic technologies to help integrate the various information sources with each other and with applications. I think that one of the key reasons that large commercial applications are so inflexible and difficult to modify is that the data access of the applications is "wired in" - connected to a specific data base. While the applications and databases have been separated since the advent of relational databases more than twenty years ago, modifying the application to access a different data source requires serious application changes and testing.
The answer to this dilemma is to provide a sophisticated layer of metadata between the applications and information sources to act essentially as a shared integration or “wiring” layer. Moreover, the richer you make the semantic model embedded in the metadata layer, the more this shared integration layer becomes a kind of "common understanding" among all the various components being integrated, which makes the overall system more adaptable and dynamic. That is, different integration decisions will be made in real time, depending on the overall environment. This metadata-driven, dynamic integration layer is a major feature of the architecture of Information Server.
The audience asked quite a few questions about privacy. Jeff Jonas, - who is a prominent expert on information and privacy, - handled most of these questions. He talked about sophisticated technologies such as anonymizers that permit information to be shared as needed in areas like national security and medical applications, while guaranteeing the privacy of the individuals, unless there is a specific match that requires looking deeper into a particular item of information. In that case, only that item will be further investigated. We know that the technology by itself is not the answer to safeguarding privacy, but it is good to know that such technologies exist. If properly applied, they can go a long way toward alleviating privacy concerns and abuses.
It is heartening to see information, and its many technologies and applications get the attention they deserve. The volume of information at our disposal will keep growing by leaps and bounds, and all that information promises to make us smarter, more innovative and more secure – but only if it is manageable, accessible, and respects our privacy.