In the waterfall development model, a project is broken down into a set of sequential phases, - e.g., conception, analysis, design, construction, testing, deployment, maintenance, - each dependent on the completion of the previous phase. Such a model is most appropriate for complex engineering projects, like bridges, airplanes and computer hardware. It’s also quite applicable for mission critical software and infrastructure projects where security, availability, scalability and other ilities are paramount.
But, a serial, orderly approach isn’t appropriate for projects where time-to-market, usability, and flexibility are the highest priorities. For such projects, an agile development approach is much more suitable. Key to the agile approach is the involvement of the product’s target end-users early in the process to make sure that the product under development is responsive to their requirements.
Agile development is evolutionary in nature, based on the early availability of experimental prototypes, user feedback, continuous improvements, and rapid responses to technology, market and user requirements. It’s not surprising that agile software development took off in the 1990s, when the explosive rise of the Internet brought millions and millions of users online, lots of new user-oriented applications, and short time-to-market cycles in response to fierce competition.
Our increasingly sophisticated smart machines bring with them a whole new set of development challenges. These AI-based systems are generally software-intensive, user-centric, and data-driven. Their complexity goes way up the smarter we want the systems to be. The behavior of an AI system based on machine learning depends on the information we use to train its algorithms rather than on the precise set of software instructions that tell the computer what to do.
AI systems are designed to sense, respond and adapt to a changing environment. Their development feels sort of like bringing up and sending a teen-ager into the world. Given that we want them to be able to handle unanticipated situations, they must have a fair degree of flexibility and autonomy. But, it’s hard to be sure that the system (or the teen-ager) will always behave the way we’d like them to.