Jun 14, 2018

Annecy Google Spotlight Stories Releases “Piggy,” a New Approach to VR Interaction

One of the most common concerns expressed by viewers of VCRs and 360-degree films is that while they are watching one part of the action, they may miss another part of the story that may be happening behind them. The latest Google Spotlight Stories film (which is more of an experiment) aims to address this concern.

Directed by Jan Pinkava and Mark Oftedal and premiered this week at Annecy 2018, Piggy is a VR experience in which viewers have a role in determining parts of the story. As you watch an animated pig consider whether or not to eat a whole cake, your head and eye movements determine (to some extent) the pig's actions.

In other words, the creators of Piggy had to develop real-time techniques to respond to the viewer's head and eye movements while continuing to show what looks like a regular short animation Cartoon Brew interviewed Pinkava at Annecy 2018, to talk about why Piggy was experimental, how it was produced, and where the director wants to take this immersive technology next.

Whereas most of the previous Google Spotlight Stories have been short films, Piggy is intentionally experimental. In this case, Pinkava says, “we wanted to do a show that had the right idea to push the interactivity as far as possible, or take a big step forward.”

The pig character, set against a simple white background, is still a fully realized animated character.

“You have to know whether he is looking at Piggy, or at the cake he wants, or at neither, and to be able to pay close attention to the timing of the scene.”

New technology implemented for Piggy made this possible. This includes what Pinkava describes as “view-driven interaction. This involves placing a target in the scene, and basically, when you look at that target, something else happens. In other words, when you are looking at Piggy, if you turn your head toward the target or move your head or device, different things happen in the film.

“There is an underlying engine that works with the story editor,” Pinkava says. The show's designers can create a graph of all the different animations and parts and transition between them based on triggers to see or not see certain places.” In the movie, it is important to Piggy whether or not he sees her eat the cake.”

A further part of Piggy's interactive element is eye contact with the character. Piggy is designed to look directly into the viewer's eyeballs. Not over your shoulder, not off to the side, but directly at you, as if he recognizes you. To do this, the team created a way for Piggy's eyes to move and face the viewer in real time. In virtual reality with six degrees of freedom to walk around the room, her gaze changes in real time and aligns with yours. You believe, understand, and see that Piggy is watching you and responding to you.”

And then there is Piggy's sound and music, which is also part of the interactive aspect of the VR experience. Most of the time, it comes from one of Schubert's songbooks, sung by a German voice with the words slightly altered so that it is actually singing about cakes and pigs. This was arranged by sound designer Scott Stafford. Scott added this as an extra track. He said, 'You know what? Everyone started laughing. '”

”What makes that particular music track so great is also its rhythmic nature. Within our story editor system, the song has an interruptible structure. There are many interruptions in the show when Piggy discovers he is about to eat the cake, but because the music itself is composed in pieces, it can be effectively interrupted, and the comedic effect is excellent.”

Piggy's sound mix also includes, when necessary, the sound of the characters' small runs, moving around within the listener's hearing range. But the interactive element continued here as well. Piggy is off-screen sometimes, where you don't see him, and you hear his voice, but he's not actually there.” You hear Piggy's voice, but he's not really there. He is trying to lose you and would rather you not see him. There are several places in the show where the sound design tells you that he is where he is not.

Google Spotlight Stories has been exploring new stories and new ways of storytelling with each new project, and “Piggy” is a similarly interactive, perhaps It is intended to be a jumping-off point to more complex films, Pinkava says.

“The idea is that you can build a system where the interaction is clearly defined, and then you can refine it and really fine-tune it. Piggy is a very simple idea, but it depends on the right timing for comedic effect. That's the risk with this kind of film. Comedy is timing, and in this case the timing is sort of up to the audience.”

“Piggy” is available on the Google Spotlight Stories app for iOS and Android, Daydream, Steam, and Viveport.