Here are some notes from Professor Dan Pacheco’s short presentation about the Oculus Rift for journalism at the CUNY’s Entrepreneurial Journalism Summit on July 10, 2014.

I’m Dan Pacheco, chair of journalism innovation at the S.I. Newhouse school. I’m sorry I can’t be there in person.  You can also get some background on this post at my site: Journovation.syr.edu.

Everything I talk about today will also be covered in detail on an ONA 14 panel called The Holodeck is Real in September about gesture and body-responsive interfaces.

One of those is the Oculus Rift, a VR headset that for the first time in history can make you feel like you’re somewhere else. Unlike Google Glass, which augments your reality, VR is more like transferred reality or 3D transportation. I like to say that for journalism, this technology makes it possible to almost literally walk a mile in someone else’s shoes.

The effect is so real that the common response from people who use the Rift for the first time are usually along these lines:

“Oh my God, I feel like I’m somewhere else.”
“This is crazy!”
“This is amazing!”
“That’s weird. Where are my hands?”

And that last part is important because what it says is that the feeling of being physically present somewhere is is so strong that your brain is disturbed by the fact that it doesn’t see hands and legs.

Two Types of Immersive Experiences

There are currently two methods for using this for journalism: 1) 3D modeling in a gaming environment, and 2) 360-degree video.

3D Modeling and Gaming Environments

Let’s talk about 3D modeling. Imagine the most realistic scene you have seen in an Xbox or Playstation game — whether it’s Call of Duty or Grand Theft Auto. Then imagine that the models for the buildings are based on buildings from the real world, from a real street. The script of the game isn’t fictional, but reflects either something that really happened, or is a recreation of something that happened or would typically happen.

Using readily available software like Unity3D, 3ds Max and Photoshop you can create a scene with high fidelity to the actual place in a matter of weeks. You can then create and animate avatars using a site called Mixamo to create somewhat realistic people and map audio recordings to them. This is something that USC’s Nonny de la Peña pioneered a couple years ago. Back then it would take her months to build a single scene, but recent advances make it possible to create such a scene in just a single week. I owe a lot to Nonny for both inspiring me, and sharing her knowledge, around VR and journalism.

I have recently been teaching some of my students how to create VR stories, and one is getting quite good. We hope to debut our first immersive story at the Online News Association conference in Chicago in September. At a workshop we will show journalists how they can get started even without any 3D modeling expertise, as the Unity Asset Store has thousands of ready-made buildings, objects and avatars that cost $5-$40, and even some free objects. But if you want to delve into 3D modeling, be ready to hire an expert — ideally someone who has worked in the gaming industry. And get familiar with finding and applying 3D textures.

360 Degree Video

The second method, 360 degree video, is relatively new and thus crude. But I believe that when the Rift and similar devices have high enough definition and frame rate, 360 video will become its own industry. Digitally downloaded experiences will quickly become a popular and almost natural new form of media.

What makes 360 video so different from 360 photos is, again, this sense of presence and being taken somewhere else.  A Silicon Valley startup called Jaunt (http://www.jauntvr.com/) is already working with companies like Dreamworks to capture 3D video that can be experienced in VR headsets, with video shoots running into the hundreds of thousands. The pricing is not so helpful for journalists, but at the other end of the spectrum there’s a new startup called Giroptic (https://www.kickstarter.com/projects/giroptic/the-worlds-first-full-hd-360-camera) that’s shipping a $350 360 HD waterproof camera. I’m happy to say that I will be getting one of these in November thanks to backing the project.

On the Horizon: 3D Scanning

A few years from now you should expect to see the output from a third category of experience seamlessly appear inside these VR environments. The same technology used to scan objects for 3D printers can be used to create objects with accurate texture-mapping into a VR environment. Matterport (http://matterport.com) and Structure.io (http://structure.io) are two startups to watch in this space.

OK, So What are the Applications for Journalism?

There are so many journalistic applications for this that I almost hasten to list ideas out of fear that it will limit peoples’ imaginations. But here are a few projects I and some collaborators are working on.

1) Recreating events that actually happened with full audio and a cast of avatars representing real people (Nonny de la Peña’s speciality). This is very labor intensive, but also very powerful. It creates a sense of empathy that is hard to get without actually being there. This is the next best thing.

2) Science and nature visualizations. Example: showing the effects of flooding and sea level rise in different coastal areas. Through a Unity 3D plugin, you can create the terrain for any place on earth in 5 minutes, then flood it with a plain of water that animates upward. When you put the first-person camera 6 feet in the air you can even simulate what it’s like to be caught in a flood.

3) “Gamified” explanatory pieces, where the user walks around and interacts with different objects and avatars and, through the game experience, learns the different elements of a story. No two person’s path will be the same, but if you use the right game incentives everyone will end up with the same information — and likely retain it better due to the experiential learning involved.

4) Travel journalism. Going to Machu Picchu, or some other exotic location? Take your waterproof 360 degree camera so that others can go along with you!

5) Sports journalism. I predict that at the next World Cup, you will be able to purchase a “V Ticket” to attend virtually. 360 cameras will be all over the stadium, at the sidelines and even at the goalposts or on the field. You’ll use a game controller to switch camera angles — and you will pay dearly for the best vantage points.

What’s the Revenue Potential?

3D game grids and immersive video have obvious opportunities for revenue, including:

• New types of ad placements. In 360 video there’s a circular area directly above and below you, and some videographers are placing sponsorships there.
• Natural product placement in VR gaming environments. For example: if a McDonald’s happened to be in a scene you’re recreating, McDonalds could be urged to pay a little bit for a higher-quality sign, or to render the inside of the restaurant if people want to go inside. This is tricky because in journalism we can’t allow money to affect what’s covered, but with the right ethical guidelines in place I think it could affect how many resources are put into recreating it.
• Explanatory historic pieces that are tied to companies. For example: a visual 3D history of the personal computer, sponsored by Microsoft.

If you’re interesting in learning more about VR journalism, or are working in this space and want to share ideas and practices, please contact me at drpachec at syr dot edu or pachecod at gmail dot com. Thanks!

Dan Pacheco