VR can be a pain – setting up sensors or emitters, connecting cables and configuring the hardware can take so long, it might not seem worthwhile for a quick gaming session. It’s all too easy for that expensive VR headset to be left gathering dust – even if you’ve invested in a powerful gaming PC to support it.
The release of Oculus Go in 2018 went some way towards changing that, offering a VR experience that was fully self-contained, with no wires or need for additional hardware. Just strap on the headset and get started.
It was impressive stuff, and far more affordable than other VR headsets available at the time, but was much more limited than more powerful, tethered headsets. Most significantly, its movement system was ‘orientation only’, meaning you could turn your head to look around the environment, but couldn’t move through it.
That all changed with the release of Oculus Quest in May this year. The Quest is still fully wireless, but unlike the Go, it offers six degrees of freedom – all without the need for any external hardware dotted around your room.
How is that possible? TechRadar spoke to Anna Kozminski, AR/VR software manager at Facebook Zurich in Switzerland, to learn how Oculus cut the cord on virtual reality.
Inside-out tracking
First, a quick explanation: six degrees of freedom means your body can move in three dimensions along the X, Y and Z axes. Moving up and down along the Y axis is called ‘heaving’, moving forward and back along the X axis is called ‘surging’, and moving left and right along the Z axis is called ‘swaying’. You can also turn to face a different axis. A movement between X and Y is called ‘pitch’, a movement between X and Z is called ‘yaw’, and a movement between Z and Y is called ‘roll’.
"To make it feel fully natural, we needed to track the user’s movements extremely precisely in real time"
Anna Kozminski, Facebook Zurich
If you want to create a truly immersive VR experience, that’s essential. “With a VR experience, ultimately you want it to feel as fluid as real life – you don’t want to be distracted because the display is lagging or jittering,” explains Kozminski.
“To make it feel fully natural, we needed to track the user’s movements extremely precisely in real time, so as you move your head and hands, those movements are perfectly represented in the virtual world.”
Oculus Quest is the first consumer device with full six-degrees of freedom tracking, made possible by the Oculus Insight system. This uses a technology called simultaneous location and mapping (SLAM), which uses input from various sensors to fix an object’s location within a constantly updated virtual map. It uses these objects as reference points (much like you’d use landmarks to orient yourself) and anchors them to points in the virtual world.
These points are observed repeatedly to compensate for drift (where tiny measurement discrepancies add up over time, making the map less accurate). The virtual area encompasses your whole field of view, so the system needs to respond as quickly as you can move and any tracking errors or lag are very noticeable.
“This tracking from six degrees of movement has been done before, but with a lot of external sensors in the room, and that presented a big barrier to entry,” said Kozminski. “You can’t share the device, you can’t bring it over to a friend’s house. [Our goal was to] his that level of immersion, but make it more practical so more people can use it.”
Making VR portable
To achieve that, Oculus had to move the software stack that does all the computation onto the headset itself. That presented a serious challenge; how do you put that kind of complex system onto something that’s lightweight, wearable, and powered by a rechargeable battery?
“There were a lot of challenges developing the architecture to use the least amount of power and leave a lot of space on the device for gaming content,” she said. “Oculus Insight is a gaming platform, but if there’s no content to enjoy, it’s moot. [We had to] break the trend that it can only run on powerful PCs, and enable developers to create cool games and experiences.”
"We had to collect thousands and thousands of data sets, coming up with all the different parameters of lighting and décor"
Anna Kozminski, Facebook Zurich
Of course, a room in your house could be totally different from one in your friend’s, so the system had to be capable of handling a huge range of conditions. “The algorithm is very sensitive to what kind of environment it’s used in,” said Kozminski. “The lighting conditions, the size of the space, the amount of texture on the walls and surfaces – but we can’t predict how people are going to set up their living room. Maybe they want to have dim lighting [for example].
“We had to collect thousands and thousands of data sets, coming up with all the different parameters of lighting and décor, then replay on Facebook’s servers and simulate the real use of the headset so we could compute metrics. It was a years-long effort.”
The work was worthwhile, though, and the hardware developers have been hugely impressed by what creatives have managed to achieve with it. We asked Kozminski about her favorites so far.
“I think there are two,” she said. “One is external uses in general, and one is an internal project. When it comes to external uses, I’m a big fan of 360-degree videos, in addition to gaming. When producers take 360-degree cameras into an environment like a rainforest or refugee camp and tell a message in video content, that’s really compelling. You have to be in someone else’s shoes to experience it. The video content is really compelling VR because it feels like you’re really there, and it gives you that first-person perspective, which is powerful in building empathy.
"VR can be a lonely experience if you use it on your own, and our aim is to create meaningful connections"
Anna Kozminski, Facebook Zurich
“We did a demo at OC5 last year – a proof of concept for future applications of Oculus, with six users in a Western-style shootout game in an arena. We were co-locating multiple players. Today you can do multiplayer experiences, but there’s nothing to indicate is that player is actually right next to you. In that demo, we shared the same map across all the devices in that co-located piece.
“VR can be a lonely experience if you use it on your own, and our aim is to create meaningful connections. That demo was a show of where we could go in the future to area-shared experiences – whether it’s playing or just hanging out.”
Breaking down divides
Kozminski and her colleagues at Facebook aren’t resting on their laurels. “Now that we’ve shipped the Quest, our team is busy writing new features that we’re shipping almost every month,” she said.
It’s not all about gaming, though – Oculus is a Facebook project, so it’s no surprise that the ultimate goal is connecting people. And to do that, the barrier to entry will need to be lowered even further
“I’ve got family who live in Canada,” Kosminski explains, “and with these VR experiences we’ve got multiplayer games, but in the future we want to be able to bring two people who are very far apart into the same experience and make them feel like they are in the same place. I want to get to a point where I can put on a headset and hang out with my brother in Canada like we’re in the same space – whether that’s in VR or AR.
“The Oculus Insight technology is what we will use to power AR glasses in the future. We’ve talked about investing in AR, and ultimately when we get to a lighter form factor – something that is more comfortable to wear – more people will use it and we can get more people into this community to have experiences digitally. Getting to AR glasses and dissolving these geographic divides.”
