How to Make VR with a Ricoh Theta S, Nuke, CaraVR & Lots of Luck

Chapter I: Preludes

So when last I left off, we were talking about work. I’ve been doing a lot of it, working out the details that I’ve talked about before of stitching together a series of Ricoh Theta S 360 camera shots in attempts to pull a “stereoscopic virtual reality” still out of my ass using [The] Foundry’s “Nuke” and “CaraVR” plugin. I know it sounds silly, but I actually bought this software – invested the money in the license for both of them, because this stuff excites me so much.

I mean, come one, who doesn’t want to make VR?

Anyhow, the license was a big investment for me and I’m hoping that I can find a way to keep doing what I’m currently doing, hiking out and finding places to share with the world. I think that there are likely all kinds of weird and interesting hidden places (even in plain sight) that people would love to see and share. My hope is that I can find a way to bring those places to the world and I’d like to do it without charging anything for it.

Money seems to take away from the Quality of doing this for me, and it seems like every time that I’ve moved toward creating with the intention of making money from it, the work itself gets changed by that money-making intention and somehow diminishes both my happiness and the work itself.

With that in mind, I’m trying to make Quality Work and want to share it with the world. It’s a hit and miss prospect, this making of VR, because if you get it wrong, it has the potential of hurting your eyes or making you feel a little nauseated. It’s also a little weird because it matters what device you have.

Watching a VR movie on a phone with a Cardboard viewer is different from watching in a headset connected to a phone, which is equally much, much different from experiencing it in a headset like an HTC Vive or Oculus Rift…which is yet again much different if it’s not only video but if you add in interactivity, be it with menus or controllers…which is yet again different from being in a space that you can move in and experience the sensation of the whole world changing around you – true 3D immersive VR, which is largely the current domain of games, but once the problem of “six degrees of freedom” (6DoF) is beaten (and it looks like FaceBook is well on the way), well, it’ll really be an interesting next couple of years.

I hope that the VR standards remain open enough for “artineers”? “articians”? “enginartists”?? to continue to tinker, ask questions, innovate, and try to find answers.

Chapter II: The Daedalus Workshop

I’ve always wanted to call a company that. I just love the idea of a place that generates ideas. Daedalus, that poster child of reason and problem solving, trapped in the Labryinth of his own creation by the raw power of emotion incarnate, the Minotaur – half human, half animal, all instinct. It’s a great myth and there’s lots of layers to unpack there, long before you even get to Ariadne and her thread or Icarus and his fateful flight.

The idea that I had, way back when I heard about CaraVR via announcement email while walking the Pacific Crest Trail, was to see if I could find a way to work with my Ricoh Theta S. But, I lost my camera on the trail (oh, NumChum, where are you now?) and kind of forgot about the email until I got back and was gifted a new camera by an amazing friend of mine.

(I think she’d rather remain nameless but if you are reading, know that I am truly grateful for your kindness and your generosity. Thank you, a thousand times, thank you.)

With NumChum Mark II in hand and a trial version of the software, I set about trying to figure out how to make VR with the camera using CaraVR. It turns out that it was “easy enough” to use the Theta to stitch together movies from the camera, but not so much for stills. That’s because the stills come out of the camera “prestitched” but the movies do not. That means that I could take the raw fish-eye imagery and plug it into CaraVR and get some stereo VR… with really lousy resolution. For more on stitching, The State of VR has a great article.

For reference sake, the Theta’s full frame movie is 1080p (1920×1080), which means that when stitched as a equirectangular (or latlong), it’s at least half size, if not quarter size, of what’s being seen on mobile devices. A “good resolution” seems to be about a 4K latlong (4096×2048)… and the Theta S is able to put out a 5K (5376×2688) on a still. Having done the video stitch and getting some actual stereo, I was intent on finding a “still stitch” so that I could make even better stereoscopic VR.

My first instinct was trying to solve with the video, then somehow use the metadata that’s in the CaraVR “pipeline” on stills shot from the same angle (HDR even!) to create 4K stereoscopic VR. I met with no success and lots of frustration. Likely, there’s something I was overlooking and I will hopefully discover what it is as I move on. Regardless of that setback, I finally solved the problem and was able to get a stereoscopic still at high resolution.

Chapter III: Yreka

Sorry, Yreka’s a funny name of a place the PCT goes near in Northern California that I visited. Beautiful country. Highly recommended. I substituted it for Eureka (also another town in California that I was near, but did not visit) to be clever. One of the pretensions of my culture, methinks.

Anyhow, I finally figured out the problem by “redistorting” the equirectangular images that I shot and plugging them in to CaraVR as fisheye lenses. The breakthrough likely was a result of a new preset in CaraVR’s solver for a “Back to Back” camera which had gotten me playing with numbers and setting, added a few more shots into the view from my setup and, kind of like magic, I had a good stereoscopic solve.

In retrospect, I also got lucky, since I shot the scene while the sky was very overcast and there was little wind, which meant that I didn’t have to content with sharp shadows, high contrast, or movement…all of which were good things, since I decided that I wanted to test my technique one shots taken in the forest, not just my house.

Granted, there are a few weird spots in the VR, where my leg brushed a plant and moved it or the ground where my feet churned the earth up a little, but it was good. I showed it to Brandie. I showed it to another friend, Dan. The stereo worked. The reaction was “wow.”

Y-freaking-reka.

Chapter IV: Making it, Breaking it

With my new magic numbers and workflow, I went into high gear, shooting stills every time I went hiking…aaaaand putting my backlog of work ever and ever longer. I was convinced that I could make it just 1% better with each new shot.

However, I eventually discovered (eventually being something like a month later) that I was spinning my wheels and coming up against limitations to the workflow that hadn’t been apparent with my first shots, namely that any time there was an appreciable amount of time between shots, or if there was a lot of contrast (i.e. lit by sun), then the stitch that I created was basically shite. And by shite, I mean “makes your eyes go all wonky” which is exactly what I’m trying to avoid. Trust me, I’ve experienced the headaches and nausea. So, shite. Dagnabbit!

So, I’ll stop yammering and get to the good stuff!

Chapter V: Wherein we Lay our Scene

I’ve done a lot of fiddling with this particular scene and all of the scenes that I’ve shot after. A number of them are good, but ironically, this is about as good as I have seen come out of my workflow and I’m damned proud of it.

If you look at it in your browser on a computer, you’re unimpressed, I know it. Another damnable 360 image. Woopee! Indeed, I could have posted one image straight out of the Theta S and you’d know no different. However, if you have a Cardboard Viewer of any kind, you should see the handy little icon as such:

look ma, VR!

Tap that bad boy and toss your phone in your viewer. Stereoscopic VR. It’s a still, but still, I’m happy with the way it turned out and that I can host it on my site. WIthout further ado, here it is:

All that, from a single Ricoh Theta S. Num Chum Mark II, my hero!

Chapter VI: Moar to Come

In the time since I shot this video, just before I started to get discouraged by the sun-dappled beauty of the forest, I had decided that I’d need to step up my technology if I was going to make it all work and I invested in a different camera rig, the Insta360 Pro. This camera is going to be epic. I am super excited to see what it can do for stills and for video. It’s in production and I am hoping to see it soon!

I’m also excited that Ricoh has announced the upcoming release of their new 4K Theta. It won’t be stereoscopic, but I love the portability of the existing camera, so I’m sure to check it out. NumChum Mark III!!!

More VR to come!