May 3, 2016

FMX Report #2: What Does the Future of VR Hold

As quickly as it began, FMX is over for another year. Perhaps more than at previous conferences, speakers were especially confident in discussing the current state of the animation, vfx, and digital content creation industries – including the state of virtual reality. Just as importantly, there was much discussion of the future, especially with the clear arrival of vr and interactive and immersive storytelling. Just what should filmmakers do with this new medium- Luckily, pretty much the world's top vr filmmakers were there to weigh in.

VR capture tools, techniques for producing content, and headsets for displaying that content seem to be innovated upon almost weekly. We still don't know quite yet what many of the virtual reality and augmented reality tech companies have in store for us (think: Magic Leap). But there are, of course, already many ways to experience vr and several studios have dived into vr content creation. The question is, how do you make that content compelling-

At FMX, an expert panel of vr and immersive filmmakers hosted by Google Spotlight Stories' Kim Adams was on hand to discuss their relatively short but intense experiences in the field. One of those panelists was Jan Pinkava of Pixar fame (Geri's Game, Ratatouille). Now creative director at Spotlight Stories (part of Google ATAP), Pinkava created an early interactive film for mobile devices in 2013 with Windy Day, while ATAP was still part of Motorola.

Pinkava's film grew out of a mission at ATAP to create something emotional for the smartphone; the group recruited Pinkava, who brought aboard other Pixar veterans like Doug Sweetland and Mark Oftedal. “We wanted to do something with this power, something emotional,” Pinkava told a packed FMX audience. “Coming from Pixar, those films they make are emotional – so we thought, let's make movies. We sat in a room and said, why don't we make a movie and give the camera to the audience-”

After overseeing other films crafted in Google's ongoing Spotlight series, Pinkava has formed his own views about what works in immersive and interactive films, whether they be 360 degree experiences or vr pieces that can be watched with either simple goggles or hard-wired headsets. Pinkava constantly asks, when considering vr proposals, 'Is this the best way to watch this- Is every way I'm going to watch this a good way- As a director, how can you help that experience for the audience be a good one-'

Two other filmmakers on the panel with experience in Google Spotlight Stories were Tim Ruffle and Jason Fletcher-Bartholomew from Aardman Animations. They collaborated on the 360-degree Special Delivery. Writing Special Delivery required a re-think of the traditional short film script. “In our heads we initially wrote a linear script,” said Fletcher-Bartholomew. “We would write in background gags, but we should have concentrated on the main chase. So getting out of that linear mindset was actually quite hard for a number of weeks. Even our storyboard was very linear, but in end we cut it up on different walls. If anyone looked at it they would have thought we were mad. We went back to cardboard cutouts and moving back to theater-like sets.”

Another challenge faced on this Aardman short, the duo shared, was how to lead the viewer around in the story and not have them get lost in the 360 world. “I found it more like choreographing a play than writing a play,” said Ruffle. “We made sure there were elements of our sets that made particular characters visible. It's a bit like video game language. People just seem to understand to look in certain directions because something is happening over 'there.'”

This requirement to consider new ways of choreographing the action in an immersive film was a view also shared by Nexus' co-founder and executive creative director Chris O'Reilly. Nexus is making a new Google Spotlight Story called Rain or Shine (some clips were shown here at FMX). “Directors come in with filmmaking tools, but they all need re-thinking in vr,” said O'Reilly. “We were thinking about how to control spaces. We talked to games companies. We even talked to architects who have experience in cajoling people through airports.”

Aardman and Nexus' experience also highlights one important aspect of vr right now: it's new. Which means everyone is experimenting. Jacquie Barnbrook, a producer with The VR Company, which is in the process of making the much-anticipated The Martian VR experience, says part of the joy and frustration of vr filmmaking right now is just getting through it. “The director of The Martian VR, Robert Stromberg, says we're building the plane as we're flying it – we may die,” Barnbrook related. “Shooting in vr is the most weird and awkward experience you can imagine. Things like our cameras are not yet able to sustain shooting for more than six minutes. We were putting cups of ice on the cameras to cool them down!”

For Mirada Studios' Andy Cochrane, who has worked on everything from a Google Shop VR Tour to The Strain VR, the result of this new wave of interactive filmmaking actually provides the opportunity to serve in multiple roles. “I'm a technical supervisor, a director, a visual effects supervisor. We settled on 'digital and interactive director' because it seemed to be descriptive of something.”

And Cochrane suggests that this new vr/interactive storytelling world – which is heavily concentrated right now on the west coast of the U.S – just happens to come at a time when the visual effects industry on that side of the country has been partially decimated by subsidies and other benefits being offered elsewhere. That, says Cochrane, represents an opportunity to visual effects artists who once worked in Los Angeles and are highly suited to technical and artistic aspects of virtual reality. Indeed, effects artists have for years been dealing with stitching plates, making 3D scenes and characters, and solving complex lighting issues, in helping to tell compelling stories.

The future does look bright in vr and immersive and interactive entertainment, even if we don't quite know just what it 'is' yet. Certainly, vr content providers are doing everything they can to, as Jan Pinkava observes, “put tools out there for their collaborators - filmmakers - to make the things they know how to do. The whole purpose is then to make all this available to you, the audience.”