Aug 17, 2016

Video: Style It presents an exciting new way to style CG animation

What if you could sketch a pattern or design on a piece of paper and have all the hand-drawn intricacies of your work instantly preserved in the texture of a 3D model-

Well, it's already a possibility thanks to StyLit, a new development in non-photorealistic rendering by Czech Technical University, in partnership with Adobe. Cartoon Brew saw the tech at SIGGRAPH this year and immediately wondered whether it could also be translated to animation. So we talked to one of the lead researchers.

First, let's get a handle on what StyLit does. Using a specialized algorithm, the app stylizes the rendering of a 3D model based on an example style drawn by the user. To do that, the tool not only analyzes color information but also information about light propagation in the scene.

That's the killer part of this tech: the analysis of light lets StyLit reproduce-in real-time-specific illumination effects such as specular highlights, shadows, glossy reflections, color bleeding, and indirect illumination, “for which artists typically use different stylization techniques,” according to research co-author Daniel Sýkora, an associate professor in the Department of Computer Graphics and Interaction at Czech Technical University in Prague.

So how does all this work- To get the whole pipeline running, two components are necessary. One is a global illumination render which allows StyLit to separate individual lighting effects (such a render can be done using NVIDIA's Iray renderer, for instance). The other component is what Sýkora describes as “our novel guided image synthesis algorithm that enables us to transfer patches from hand-crafted exemplar to corresponding locations in the target rendering.”

But will artists be able to get their hands on StyLit, and is it useful- Right now, the StyLit method is patent pending. Sýkora says there are plans for his university to release a simple executable demo which would allow others to play with the app for research or other non-commercial purposes.

People in the digital arts community certainly seem keen to see what StyLit can do. Matt Wallin, for example, is a former visual effects artist at Industrial Light & Magic and Weta Digital, and now associate professor at Virginia Commonwealth University in the School of the Arts, and says many of his students are traditional illustrators by trade.

“For them to be able to use their traditional drawing skillsets along with their original styles and aesthetic sensibilities to create one of a kind 3D textures and looks is really powerful,” Wallin said. “It has the potential to bring so many more artistic sensibilities into the world of 3D graphics.”

StyLit could find use in animation, too, and that seems like one of the most exciting possibilities for the tool. The researchers have already produced animation out of StyLit by rendering individual animation frames and letting the algorithm stylize them independently. Although the results exhibit a degree of temporal noise, Sýkora notes this can actually be a desirable feature, citing a similar kind of look in Bill Plympton's hand-drawn animation, for example.

Importantly, StyLit was not just developed inside a research vacuum. Six artists co-operated with the researchers and provided feedback. Of course, Adobe Research researchers are also co-authors on the StyLit project.

So does that mean we might see the app as part of some kind of Adobe product- Sýkora could not say, but the idea of an executable demo sounds promising and something to keep an eye out for in the meantime.

To learn more, read the full text of the StyLit research presented at SIGGRAPH 2016.