Interactive Cinema for Oculus Rift

Interactive Cinematic Storytelling

How can you create an interactive story with 360 degree footage for Oculus Rift? In fall 2014 I coached a team of students who worked on this project in collaboration with Lectorate Games&Play’s researcher Mirjam Vosmeer. AVROTROS and their cultural hub VondelCS, and VR pioneers from WeMakeVR.

Using live action 360 degree footage the team created an interactive narrative experience for the Oculus Rift to explore how stories can be told in virtual reality. During the experience, you are not just passively watching a story, but you are a character who can actively influence how the narrative unfolds.

About the film

In A Perfect Party you, as the main character are hosting a get-together for your best friend who wants to propose to his girlfriend. Everything depends on you to make the party a success: potential disaster is everywhere. By looking around and interacting with the environment you may try to prevent things from running out of control. It’s up to you to make this party perfect.

A perfect party was featured at LISFE Leiden International Short Film Experience 2015:


What kind of a design process is this?

After hashing out all of the technical particularities of making VR with 360 degree live action footage, we quickly got to the question of how do you even design for this? Students identified links with storytelling, narrative techniques in film (although with heavy limitations because conventional editing is not possible in this case), theater, perhaps subtlemobs or audiowalks and interactive storytelling (basically hyperlinking?). All of these practices have something to bring to live-action 360 degree VR, but the comparisons are far from being one on one.


Creative obstructions

I think that a few obstructions could be identified that needed to be dealt with creatively:

  • Your user is constantly surrounded by footage consisting of 4 quadrants. If you want to make a cut or change content to activate a storyline, you need to make sure the user isn’t watching the quadrant you want to change
  • Every quadrant has to fit with the next one in order to be ‘stitched’ together seamlessly to 360 degrees. So people and objects can’t leave their quadrant or you will clearly see the stitch, breaking the illusion.
  • Your user cannot bring the body into the experience (just the eyes). How do you deal with the fact that it feels like you are there, but you can’t for example use your hands?
  • How do design the action so it doesn’t get boring but the user also has time to explore on his/her own account?
  • The user is free to look wherever they want, how do you tell a story if you can’t use conventional camera techniques to direct user’s attention (enter theater techniques and sound design)?
  • Keeping track of the storyflow: how does a user go from one storyline to a new branch? Do you allow them to go back? What considerations are involved? How does that influence continuity and intelligibility of the story?
  • Etc.


Some tricks

Nour, one of the students wrote a how-to book on live-action VR dealing with many of these issues, and suggestions for useful approaches. Here’s another few tricks that worked quite well for the team. One was to play out scripts in small handmade miniature sets, to simulate hotspots (areas in the film that are activated when the user is looking at it, the helmet senses head movements), and changes in clips. Movements across the scene, timing of those movements etcetera.


Secondly, we developed a different kind of shotlist. Rather than storyboarding the shots and cameramovements. The team developed a set map, marking the position of the 360 degree camera, the quadrantlines, and room for some notes on the action and movement of actors and vehicles in the scene.


Lastly, flowcharts turned out to be the best way to keep track of keypoints in the storyline and identifying problems with continuity and points-of-no-return.

Students: Nour Tanak, Sammie de Vries, Leon van Oord, Nick Valk, Shenyu Zhang. Researcher: Mirjam Vosmeer, Lectorate Games & Play HvA


amsterdam, Design Coaching