How Facebook plans to make VR hangouts more real, less virtual

Its plans for VR are ambitious, to say the least

Facebook

Facebook's purchase of Oculus in 2014 raised some eyebrows, but its presentation on how it plans to make VR a viable platform for social communication proved that it's making that money work.

At the firm's F8 developer conference, the company touched on everything from taking selfies in VR to how Oculus Rift's Touch controllers will help to make social interactions in a virtual environment a little more personable.

But perhaps the biggest takeaway from the keynote was just how heavily Facebook and Oculus are pushing VR to be more than just a technology that's fun to use alone, and something that can be experienced with other people – no matter where they are in the world.

Facebook

Creating an authentic presence in VR is a problem that Oculus' research division is working to solve. Sure, you can put on an Oculus Rift or HTC Vive right now, and feel very much a part of the virtual world that you see.

But what happens when you introduce another living, breathing human into the world?

Oculus tried that out with Toybox, a multiplayer tech demo of sorts that lets you and another person toss objects around and trade punches in a physics-based playground. In it, you can see rough outlines of faces and hands.

Oculus Toybox

There's definitely a human on the other side of that connection, but there may as well not be based on how it's rendered.

We saw more of Facebook's progress to make VR a more interactive, social experience at F8 2016. It was a demo that branched further out from the Toybox playground and allowed two users to visit each other digitally.

Despite being in two different physical locations, they were able to draw clothes onto each other with Oculus Touch controllers, and explore a 360-degree view of London together. It's not too different in concept to holoportation, a feature introduced recently for the Microsoft HoloLens that allows people to join each other in an augmented reality environment.

Both of these advancements show that we're getting there, but it would be putting it lightly to say that Facebook wants to take things even further.

You thought today's VR was futuristic?

There's no substitute for being in the same room as the person you're talking to. But, not only can you touch, you can also pick up on physical and behavioral subtleties in this VR environment that you'd probably miss through a Google Hangouts video chat.

During the F8 keynote, Oculus quoted anthropologist Edward Sapir by stating that these small, but meaningful gestures are "...an elaborate code that is written nowhere, known to no one, and understood by all."

And, advanced as it is in its early stages, VR cannot properly crack that code by putting a believable human avatar in a virtual world – yet.

But by properly capturing, displaying and predicting your every movement, the social network company thinks it can make chatting virtually feel as if you're in the same room – or as close as possible with a head-mounted display strapped to your face. Of course, it's easier said than done.

Oculus Rift
Much more expressive than a blank-faced avatar

First, capture the human face as accurately as possible. To do that, Facebook shared some early results from installing a high-resolution camera onto a Rift to capture subtle mouth movements. It needs work, but it's a heck of an improvement over flat 3D models.

Then, there's the challenge of transferring this data over a stable connection. It's an essential ingredient in creating a believable virtual one-on-one with someone. But, as the methodology of capturing every last movement gets more advanced, the amount of bandwidth required to pipe them through to the other side will likely skyrocket.

Panoptic studio
There's clearly some science going on up in here

Facebook is so interested in capturing these minute, uniquely human movements that it built another off-the-wall experiment to get even better at it. The Panoptic Studio is an igloo-looking dome with cameras capturing its subjects inside from every angle. It's highly sensitive at tracking movement, even when it isn't coming from a person.

The idea here is to intensely study the way we tend to act in various situations, social or otherwise. Ideally, the predictive learnings gathered from the studio will work to both make for more convincing social engagement in future VR apps, as well as lighten the load on the hardware-intensive capturing side of the process.

Panoptic studio
Believe it or not, this is the recorded movement of a volleyball match

Ever since its acquisition of Oculus, the question on everyone's mind has been: "So, when will we see a fully virtual version of Facebook?" The answer, just like the arrival of the Oculus Rift was, has been a long time coming, but is still a long way off.

We saw an early glimpse into the future of Facebook in VR and, thankfully, it won't just be a bunch of menus or Candy Crush invitations.

Which virtual reality headset is right for you? See for yourself:

Article continues below