One of the most surprising use cases for Apple’s new Vision Pro VR/AR headset, which it unveiled at its WWDC 2023 event is that it can take spatial 3D photos and videos using its two forward-facing cameras, for immersive playback on its 23 million-pixel display.
We don’t have the detailed spec to indicate the kind of quality you can expect from those two cameras, but at the push of the shutter button on the top-left of the headset the cameras record photos or videos for an immersive 3D VR experience and 4K playback for each eye, complete with spatial audio for videos.
At $3,499 you’re obviously not buying the Vision Pro just as a camera – recording and reliving moments in 3D is a bonus feature of this dual-chip mixed-reality headset – but will you really want to use the camera in the first place?
There’s a camera on your face
Take a moment to watch the Apple Vision Pro presentation above (starting at 1:36:00). It’s a strange image, isn’t it? And I don’t mean the scene being captured using the Vision Pro of the girls blowing bubbles, but rather the image of the smiling dad wearing the headset, which resembles a pair of ski goggles, to capture the moment.
It’s a terrifying vision of future family life. Using what Apple calls eyesight, you can see the dad’s eyes in the headset lenses, and then a flash of light as he takes the picture. But those aren't actually his eyes – what we're seeing from the outside is actually an OLED display that can display the wearer's eyes in real time using the cameras inside the headset, while the display illuminates to make it clear when you're capturing a photo or video.
The Apple Vision Pro packs 12 cameras in all: six external cameras (two forward facing, two downward and two side cameras), two TrueDepth cameras, and four internal infrared cameras, plus a LiDAR scanner. The external cameras track hand gestures and view what'is happening around you, while the internal cameras perform eye-tracking, and what Apple calls Eyesight.
It’s the two forward-facing cameras that can also capture spatial 3D photos and videos, much like a stereoscopic camera, or the Canon dual fisheye lens that records 3D 180 VR photos to a Canon EOS R5. The left and right eye image is viewed together as a 3D image using the Vision Pro headset.
Now step back and view the whole picture. Can you really enjoy a moment in real time with a headset strapped on? And what about engaging with the people you're with? In the video the presenter says: "Spatial photos and videos bring you closer than ever to the moments and people you care about." That I'm calling out as BS.
It’s one thing to view memories using one of the best VR headsets or mixed-reality headsets, but capturing them? No thanks.
The act of using any camera removes us from naturally engaging in a moment: smartphones coming out to grab a video of birthday candles being blown out, a camera viewfinder pressed up to the eye to compose a photo of the kids playing. But a headset? That’s another level of disengagement.
If the Vision Pro wasn’t essentially a super-expensive computer you might actually consider using it outside the home as an action camera – how about capturing actual skiing or extreme sports, because then you’d fit right in rather that looking slightly creepy. But it’s not the type of low-cost, lightweight and rugged kit you'd want to risk in scenarios like that.
I use my camera in moderation in family life, whether it’s a smartphone or mirrorless camera, because it can be intrusive. But a headset is outright voyeuristic. I can picture the conversation now, “Hey kids, watch this. You remember, when I was watching the entire time with a headset strapped on?” Except I won’t be reliving that moment with them either, but on my own with the headset on.
The immersive future for our memories?
While moving around with a headset strapped on is not the most natural way to capture memories, image playback on the other hand will be transformed when viewed in a Vision Pro – at least it will be when you’re on your own, because you can’t share a headset in the same way that you can huddle around a screen or photo print to collectively enjoy the moment.
In the Apple Photos app you can dim the headset display around photos for clearer viewing, and expand them via a hand gesture to fill your entire vision, which will be game changing for panoramic photos especially. Viewing spatial videos will be transformed too, because videos recorded using the Vision Pro are also accompanied by its six-microphone spatial audio.
There's no doubt that the Apple Vision Pro's camera is powerful, smart and simplifies 3D spatial imaging, and it could well be the future way that we engage with memories – for those that can afford the headset in the first place. But capturing those moments? Maybe we should keep real eye contact alive, and stick to using our smartphones and cameras instead.
