Greetings from Forza d'Agro in Sicily, where I am attending the Terre di Cinema cinematography event.
This post continues my investigation of Virtual Reality Cinema. If you haven't read my first post about Google Cardboard, I encourage you to do so now.
I must stress that I am not a VR expert. I am simply a VR newbie who is trying to understand the cinematic implications of the new VR tools. Please don't hesitate to give me your corrections and suggestions in the comments below.
A frame from Blanca Li 360. Note spatial distortion at top and bottom. (concert.arte.tv)
2. Cardboard app
3. Earth maps
4. Blanca Li 360
5. camera issues
The genius of the Google Cardboard system lies in designing a cheap piece of cardboard with two plastic magnifying lenses that can be used with your existing smartphone to create a Virtual Reality viewing system. In my previous post I mentioned some inexpensive headsets: Google Cardboard version 1 (with left-hand magnets), Google Cardboard version 2 (with right-hand cardboard switch), and plastic goggle glasses (marketed by Fincloud and others).
I recently spoke with Franz Troppenhagen, the product manager for Zeiss' VR One headset. Franz told me that the headset's large lenses are very good quality, with no distortion "from edge to edge" — as we would expect from this renowned lens manufacturer. The headset can also be worn with your glasses on. The VR One sells for $130 and comes with a tray for the Samsung Galaxy S5. There is an option for 3-D printing of trays for some other phones.
Many of the VR practitioners I interviewed spoke very highly of the Samsung Gear VR headsets. There are two models: the Gear VR, designed for the Samsung Note 4, and the more recent Gear VR Innovator, which works with the Galaxy S6 and S6 Edge and costs $200. Samsung's business model is to use their hardware as a gateway to proprietary content, and by all accounts, they offer a rich selection of VR movies.
2. Cardboard app
Once you have a Google Cardboard or other headset, your phone requires software to help it display dual VR images on its screen. The place to start is Google's Cardboard app, which you can load directly on your smartphone via Android's Google Play or the iPhone's App Store.
The Cardboard app's home menu lets you choose a VR experience by turning your head. Below are screen shots from the Urban Hiker selection, which puts you into 3-D 360-degree still frames shot in major cities. You can follow arrows to see other views, or get to other cities by looking down at your feet. Depending on your headset model, the "click" can be done via the left-hand magnet or the right-hand cardboard button, or with a touch of the screen.
In the VR cityscapes and museum interior below, you the viewer are placed in a fixed position at the center of an image sphere. You can pan and tilt to see things around you, but you are stuck in one position. This sphere-center POV is the most common arrangement in VR stills and movies. Of course, not all VR projects are 3-D, and, as we shall see in my next post, 3-D VR videos are difficult to shoot.
In the Cardboard app's mask Exhibit below, the VR arrangement is different: the mask is at the center, and you can move along the surface of a sphere around it to observe it from all angles. This sphere-surface POV is typically reserved for CG images, as it's almost impossible to shoot.
3. Earth maps
So how are spherical images laid out for your smartphone's flat screen?
A similar question was addressed long ago by mapmakers who wanted to image the Earth's curved surface on a flat piece of paper. There are several solutions to this map problem, but they all necessarily involve spatial distortion. VR stills and movies often use the so-called "equirectangular projection," which dates back to 100 AD, according to Ptolemy, the great astronomer.
Map of the world with equirectangular projection.
(Eric Gaba – Wikimedia Commons user: Sting)
The main idea of equirectangular projection is to give each horizontal latitude the same length on the map. Therefore the greatest distortion occurs near the North and South Poles, where very short latitude rings get stretched out to be as long as the equator.
4. Blanca Li 360
Blanca Li 360 is a pioneering 2-D VR dance video. Noted choreographer Blanca Li created a wonderful, zany piece in a huge, open office space, which starts as a tame working environment and transforms into a whirlwind performance with the troupe dancing on tables, stairs and chairs with pens, papers and folders. The viewer is in the center of the space with dancers around him, sometimes quite close. I first saw Blanca Li 360 on an Oculus headset, as part of the traveling Home Cinema art show. It was an exhilarating moment.
Blanca Li 360 also offers a simple example of how to shoot a 2-D 360-degree VR video. The piece was shot by cinematographer Frederic Martial Wetter and post-produced by Studio Saint George, a small Parisian VFX company co-founded by Bastien Harispe and James Sénade, who acted as VFX supervisor on the production. I spoke with Bastien, who shared some details but was naturally reluctant to publicize their entire custom workflow.
Operator Rubens Jan behind VFX supervisor James Sénade on the set of Blanca Lee 360, with a Dragon camera and fisheye lens.
Hence, the workflow I describe here -- based on the footage and a documentary by Elsa Guillet-Chapuis -- illustrates one possible approach to making this VR video, but isn't necessarily the one used by the filmmakers:
- The filmmakers shot with a Red Dragon camera, with an 8mm Nikon fisheye lens covered by a Vantage magnifying attachment that created an effective 6mm, with a 220-degree Field of View.
- The Dragon was in 6K mode, its maximum resolution.
- The 360-degree image sphere was created by shooting and joining 2 separate overlapping 220-degree shots
- There were 2 stitches joining the 2 fisheye images
- Stitches between 2 images are always problematic, but here they were even more so because the dancers moving from one image to the other are from different takes, and their movements can't match perfectly.
A frame from Blanca Li 360 with possible stitches on left edge and at center (near pink tape line on table).
- One stitch was masked by a cabinet on the edge of frame, which hid the dancers' passages behind it.
- The other stitch was masked by two stationary people in the center: a woman standing near the corner of the table, and a man leaning on a balcony upstairs, above her.
- Bastien stated that they used their usual tools in post -- Flame by Autodesk and Nuke by The Furnace -- to stitch the images and prepare them for delivery.
In an email to me, Bastien stated his belief in the future of the immersive experience of VR, stressing that "sound is a very important element in its success." In addition to headsets, he adds, viewing without a headset on a smartphone or tablet will help 360-degree movies reach a larger audience quickly and easily.
5. camera issues
The 2-camera scenario above can be seen as an elementary multi-camera system for creating 360-degree VR.
The 2-take shooting here is a special case, as most filmmakers would want 2 or more cameras to shoot simultaneous events. However, not shooting the 2 angles simultaneously allows the 2 camera sensors to occupy the same sphere-center position. This is physically impossible with a multi-camera systems, which have a problem of parallax -- POVs that don't match because they are in different positions. As we shall see in my next post, parallax is an important problem with VR camera systems.
The huge 6mm optics used here would make a multi-camera system impractical for many locations. It would also be difficult to move multiple cameras gracefully. VR camera systems need to be compact to minimize parallax and facilitate movement.
Finally, the workflow above underlines the importance of stitching different camera elements to create an image sphere.
Now that we've detailed a simple 2-camera VR system, my next post will look at multi-camera tools specifically designed for VR production, and present other notable VR movies.
google.com: get cardboard Google's overview
wikipedia: Google Cardboard
Zeiss VR One headset
Samsung Gear VR headset
Gear VR Innovator for Galaxy S6
Gear VR for Note 4
smartphone VR apps
apps can be loaded directly on your smartphone via google play or apple's app store, or via the following links
iTunes preview: Google Cardboard
in360tube.com: the in360tube app
wikipedia: equirectangular projection
Blanca Li 360
concert.arte.tv: Blanca Li 360 Behind the Scenes
This very instructive 12-minute documentary, shot and directed by Elsa Guillet-Chapuis, ends with an equirectangular version of the entire 3-minute VR dance movie.
credits for Blanca Li 360
Saint George Studio Paris
Blanca Li 360 cinematographer Frederic Martial Wetter wipes the 6mm attachment on the set. (concert.arte.tv)
I thank the following people for their help with this post and the next one:
-- Joergen Geerds
founder & CEO - Freedom360, New York
-- Bastien Harispe
co-founder and executive producer - Studio Saint George, France
-- Aron Hjartarson
executive creative director - Framestore VR Studio, California
-- Moe Shore
director of advanced imaging - AbelCine, New York
-- Franz Troppenhagen
VR One product manager - Zeiss, Germany
VR on thefilmbook
VR Cinema 1: Google Cardboard
VR Cinema 2: Image Spheres
VR Cinema 3: Cameras
VR Cinema 4: Content
VR Cinema 5: Futures
Carne y Arena PART 1 - VR by Alejandro Iñarritu with Emmanuel Lubezki
Carne y Arena PART 2 - Notes on Design of VR Cinema