Jan 21, 2014

Pixar Makes Painterly CG: New Research Could Change The Look of Their Films

Last summer at SIGGRAPH, Pixar presented a paper offering some clues about one of the major new directions that CG feature animation is headed. The paper, “Stylizing Animation By Example,” explored how filmmakers could achieve more expressive rendering styles that disregard the perfect boundaries of computer graphics rendering and mimic traditional painting techniques.

While non-photorealistic rendering is not a new concept, the system devised at Pixar contains a level of automation and consistency that would allow a sophisticated painterly style to be scaled for feature film production. According to Michael Kass, a senior scientist at Pixar, there are two major innovations in the project: 1) temporal coherence, which is the fluidness of images from frame to frame, and 2) the control over the final look that is gained by an artist who paints individual keyframes. The full paper, as well as additional documentation, can be downloaded on Pixar's website.

Another Pixar senior scientist Kurt Fleischer, who was the technical director of the project, explained its genesis to Computer Graphics World: “Michael [Kass] was working on image filtering when some people in production became interested in the artistic images…We experimented with different kinds of techniques that looked nice and continuous over time, but we didn't have artist-specific control. We only cracked that in the last couple of years.”

Fleischer doesn't identify the Pixar artists who were interested in the approach, but it bears noting that the director and art director who created the artwork for the research project were Teddy Newton and Don Shank respectively. Veteran Pixar animator Sanjay Patel created the animation of the skater. Other people credited in the paper are character shading TD Jamie Frye, production designer Ralph Eggleston, and animator Angus MacLane.

With this new technique, an artist can paint 2D illustrations over a handful of the CG keyframes (“one every 10 to 20 frames”), and the software will fill in the hundreds of inbetweens while maintaining temporal coherence and the desired painterly style. The move away from photorealism has been a longstanding goal of animation artists. David Gainey directed Fishing at PDI/DreamWorks in 1999, and more recently, John Kahrs won an Oscar for Disney's Paperman, which explored a similar set of ideas as the Pixar paper. All kinds of rumors are being tossed around currently about upcoming Disney features that will use non-photorealistic rendering styles, and my guess is that we're 3-5 years away until a major studio feature is released using such techniques. As for Pixar, they'll most likely create an animated short using these rendering techniques before incorporating the technology into a feature-length project.

The elephant in the room that needs to be addressed at some point is whether changing the style of CGI to mimic traditional media is a step forward or backward for the art of computer animation. As much as some computer artists desire to eliminate the digital origins of their work, a clear and troubling division exists between CG processes devised to replicate traditional media, and a film director like Aleksandr Petrov who uses actual oil paints to create animated films. When Petrov animates in oil, the plasticity of the materials directly informs the movement of his figures. In the Pixar process, the decorative surface style is incongruously layered atop a pre-existing piece of animation. In other words, the use of different media in computer rendering doesn't result in different animation effects because the animation is already created; it's an utterly arbitrary aesthetic embellishment.

This dissonance between visual technique and animation remains central to the aesthetic discord in non-photorealistic rendering. While there is a certain empty attractiveness to rendering in an illustrative style, the surface seductiveness has no influence on either the film or the animation besides reducing the emotional authencity of the artwork by distancing it further from its digital roots. Perhaps that's the goal, but it seems to me an aesthetic dead-end.

Dead-end or not, research continues apace into these rendering techniques and its impact will soon be felt in feature animation. This most recent Pixar research into stylized rendering was conducted by the following individuals, who include both Pixar employees and graduate students/PhD candidates from Stanford University, University of Toronto and University of Washington: Pierre Benard, Forrester Cole, Michael Kass, Igor Mordatch, James Hegarty, Martin Sebastian Senn, Kurt Fleischer, Davide Pesare, and Katherine Breeden.

(Thanks, Jonah Sidhom, via Cartoon Brew's Facebook group)