On March 22 we held our third seminar in the series Engineering Systems Solutions to Real World Challenges, which is co-sponsored by IBM and MIT's Engineering Systems Division. The objective of the seminars is to show how industry leaders are using engineering-based approaches – e.g., design, tools, processes, analysis, simulations, etc – to address complex problems in their industry or institution.
The first seminar by IBM executive Linda Sanford focused on the IBM transformation initiatives which she leads, covering not only the key role played by technology but also the critical importance of culture to successfully transform a complex organization. Next we addressed healthcare - and in particular the really innovative work taking place at Vassar Brothers Medical Center.
Our latest seminar showcased the importance of applying systems thinking and innovation in a perhaps unexpected area. The seminar titled Engineering Systems Solutions to Real World Challenges in Media and Entertainment focused specifically on digital animation. It was led by Larry Kasanoff, film producer and CEO of Threshold Animation Studios, a small digital entertainment company - employing fewer than 200 - that does digital visual effects for films, TV, theme parks, commercials, video games, music videos and private industry. I have personally known Larry for several years because Threshold has a close technology alliance with IBM.
Digital animation is very costly, time consuming and competitive. Threshold needs to gain some kind of quantum leap advantage in production speed and quality in order to compete with the giants of the animation business, and that is why they had no choice but to push the creative and technology boundaries in animation. In their Digital Research Labs, they are exploring new processes like real-time animation that essentially allows the director to interact with the cartoon characters as if he were directing a live action movie with live actors. In addition, being a small company with limited budgets, one of their key objectives is to leverage advanced technologies to significantly reduce their production time to market and overall costs.
This coming fall, Threshold is releasing its first full-length animated film, Foodfight!, which takes place in a supermarket at night, when the lights go out and the people leave. In his MIT seminar, Larry discussed the challenges involved in making an animated movie like Foodfight! and the innovations they had to come up with. He showed clips from Foodfight!, both of the final film as well as what went into making it. He also answered lots of questions from students and faculty in the audience. A Webcast of the seminar will shortly be posted in the IBM Innovation Lecture Series website as well as in MIT World.
Larry first gave a brief overview of the animation industry. Traditional animation starts with a storyboard, which meticulously lays out the overall story, characters and scenes with images and words, sort of like a giant comic book. Converting the storyboard into an animated movie is very labor intensive. A theater-quality animated movie requires roughly 24 frames for every second of animation, that is, 1500 frames for every minute or about 90,000 frames per hour. In the original hand-drawn animated movies, each frame had to be carefully drawn by hand. Needless to say, this was very costly and time consuming. Computer animation, in wide use in the last 15 years or so, has introduced major improvements, primarily to the back-end processes where the animated scenes are rendered into viewable images. But for the front-end creative processes, animators still have to design their characters and scenes at a workstation, object by object and scene by scene, each one being carefully modeled in 2D or 3D, depending on the film. Computer software provides lots of help in morphing and other techniques for assisting the animators, but the front-end process continues to take significant time and expense.
Larry explained that animators will often ask human actors to move their bodies and faces in a certain way, and do their best to translate those movements and expressions into their drawings or digital animations. He pointed out that having animators painstakingly try to capture and recreate the nuances of people’s motions frame by frame can get in the way of the creative process because it is so time consuming. He wants to achieve the feeling of a real studio, which involves directing live actors and integrating them directly into the film, while working closely with the animators and the rest of the production crew, each doing what they do best in the overall collaborative, creative process of making the film.
Threshold’s answer is to make extensive use of an animation technique known as motion capture, under which the movements and expressions of live actors are captured via fiber optic sensors and converted directly and in real-time to 3D digital files. Motion capture is not a new technique, and it has already been successfully used in selected parts of a few films. In Happy Feet, which has won many awards including the 2007 Oscar for best animated film, the wonderful tap dancing of the hero, an emperor penguin named Mumble, was done by famous tap-dancer, actor and choreographer Savion Glover, and transferred to Mumble through motion capture. Polar Express also made use of motion capture in its production, which enabled Tom Hanks to play six different characters in the animated film. The cover article in the April 2 issue of Business Week, Beyond Virtual Reality - The Mind-Bending New World of Work, is all about motion capture technologies, and how they are bursting out of Hollywood into industries from aerospace to advertising. The article concludes "Motion tracking has all the marks of a disruptive technology, slinking on to the scene in unexpected ways."
Motion capture is the foundation of Foodfight!. It is far more extensively used than in any other film. Because Threshold wanted to use motion capture not just for the human-like characters, all of which were acted out by live actors, but also for the non-human characters, they had to invent new algorithms and software techniques that would translate the human motions and expressions of the live actors to each specific non-human character. Moreover, Larry wanted to direct his actors, get real time feedback on how the actors’ actions were being translated into their animated characters, and thus be able to focus on the characters and story just like a director of non-animated films. This required the invention of a new, real-time animation system with sophisticated algorithms and vast computing power, which were jointly created between Threshold’s Digital Research Lab and IBM.
Having the actors in effect control or program their own characters results not only in more creative animations, but in a significant reduction in the time and expense required to make the overall film. Threshold may perhaps have created the concept of Lean Animation, as they are applying Lean Production methodologies to significantly improve productivity and quality in process after process in the animation industry, as has been previously done for manufacturing, supply chains and other important industry areas.
Larry talked about his vision for the digital animation systems that Threshold has pioneered. Threshold has in fact created a virtual back-lot both more extensive and more economical than the back-lots of the famous movie studies of Hollywood’s golden age. His back-lot contains a rich array of scenery and characters that he can now use to direct different kinds of films without having to start each from scratch. He can even change camera angles in a scene without having to re-shoot the whole scene, as is the case for any other kind of film, animated or live. Moreover, because all the content has been digitally captured, he can re-purpose the same content to generate different kinds of media products, including video games, theme parks and music videos, as well as creating offerings for different kinds of delivery channels, from IMAX to mobile devices. In principle, he is able to adapt his digital content to media channels that as yet do not exist. Larry really feels that Threshold has gone beyond digital animation to create something entirely new – a kind of 21st Century Digital Animation Studio.
Larry Kasanoff’s seminar was a real treat for me because I truly love films – old ones and new ones, blockbusters and little gems from all over the world, films with live actors as well as animated ones. I can't wait to see Foodfight! when it comes out this Fall. I hope it will be a big success and ushers in a whole new round of innovations in the film industry.