Is streaming better than live TV? This was certainly the case for Episode 4 of series 10 of the show - 'Knock Knock' – became available on the BBC iPlayer with a special, spatial binaural soundtrack.
However, users needed a pair of headphones at the ready because this pioneering 3D sound experiment by the BBC didn't work through their TV.
Whereas stereo's two-channels tend to create a soundstage completely inside the listener’s head, binaural audio is all about creating the illusion of space and distance from the listener. We were given a special preview of 'Knock Knock' to watch on an iPad and listen to through Sennheiser HD 205 headphones, and the results are in; binaural is brilliant.
The episode in question lends itself to spatial sound particularly well. It's a claustrophobic horror story set in a big, spooky house (which is actually Fields House in Newport, South Wales, where we had the demo) that creaks from all angles.
During the showing we could hear doors being opened and closed either side of me, people talking behind me, and thunder clapping outside the house. The dialogue moved as the actors did, and the overall impression was also one of immense clarity. It was like being there.
"It sounds like you're in the room rather than just looking at the room," says Cathy Robinson, Audio Supervisor at BBC Wales in Cardiff, who has also helped create binaural soundtracks for everything from A Midsummer Night's Dream, Planet Earth II in 360º and The Ring. However, aside from one tense scene where there are a lot of knocks on a lot of doors from a lot of directions, there are no novelty sequences that use binaural audio just for the sake of it.
"We chose the episode to do in binaural audio because it's a story about a space, and we were able to use binaural audio to enhance that space," says Robinson. "Nothing about the story was changed to make it work better with binaural."
If you think you've never heard binaural audio, well, of course you have. "We listen to binaural audio all day," says Robinson. Binaural simply means ‘listening with two ears’. When we listen in the real world our ears hear all sounds with two ears, and those sounds reach each ear at slightly different times.
That's why a binaural soundtrack designed for headphones can't be played back through a home cinema system – that would introduce another layer of natural decoding that would confuse our brains.
"We naturally listen binaurally, and you get accustomed to binaural audio very quickly because it's like listening to everyday life," says Robinson. In that sense, it's much like high definition video.
Remember how dazzling it was a decade ago to see a HD image on a 60-inch plasma TV? You probably don’t, because within hours of installing it in your living room, you got used to it. "When I'm crafting it, less is more – if you want the effect to come through there needs to be less going on, and much higher frequency pinpoint directional sounds come through very well," says Robinson. "Having lots of sound effects at the same time masks the effect."
Although massively impressive, it's clear that there are no rules yet for producing in binaural sound. There probably needs to be primarily for the director, because the parts of Doctor Who where the camera cut from shot to shot quickly where disorientating; no sooner had you felt you were inhabiting the shot than the perspective changed. The long-duration shots are easily the most impressive. “It's my hope that the creative people will write scripts knowing what the technology is capable of,” says Robinson.
So how does this all compare to surround sound? After all, high-end home cinema systems – typically comprising five speakers and a subwoofer – have recently embraced what sound engineers call 'object-based audio'.
"What we're doing is a separate mix for headphones that lots of people can consume with no special technology," says Robinson. Binaural audio is exclusively for headphones. The ease of decoding is critical for the BBC's experiments, too; while Dolby Atmos requires compatible equipment to decode the mix and place each sound effect in the correct speaker, the BBC's binaural technology uses nature's own decoder – the human ear.
That binaural audio only works on headphones will cause some to scoff, but there are reasons why the BBC has chosen to go ahead with this technology now. "The reason we did it now is because we know people are watching more on phones and tablets and listening on headphones," says Robinson, who thinks that people are also buying great quality headphones now. "It's not trying to replace 5.1 or a TV mix, it's just for those people who want to wear headphones."
It might be seen as akin to the arrival of HD, but unlike HD, binaural audio does not mean more data. The binaural mix ends-up as a regular stereo mix file that's no bigger than the standard 24-bit 48kHz soundtrack on the BBC iPlayer; only the data wrapped inside the file is different. "With this Doctor Who episode we have a binaural mix that's just two channels of audio like any other, it's just that they're designed specifically for playing on headphones and they go right to your ears," says Chris Pike, Senior Scientist – Audio, Research & Development, BBC Design & Engineering.
Pike's hope is that BBC R&D's new binaural software tools that enabled the experiment can help develop a standard that means that one mix of spatial audio can change according to what equipment the audience is listening on, automatically giving them the right format. "For spatial audio it would adapt to whatever set-up people are using, so if you have a 7.1 surround system home cinema it would give you the spatial mix of that, and if you have headphones, then you plug them in and it gives you binaural," he says. "We're not there yet, but we're working on automated algorithms with richer metadata about your 3D scene, so you ultimately get the correct format."
For now, this is all about designing sound for the average ear, but there are ambitions to customise binaural audio for every individual because the shape of your face and head can drastically affect how each ear hears sound. "You can personalise it for someone's ear shape," says Pike, who explains that one day a head-related transfer function (HRTF) – a 'head-print' profile – could be developed for everyone. "In the future it will get better through personalisation," he adds. Some companies are working on so-called head-related transfer function (HRTF) already, with the OSSIC X headphones calibrated to your head and ear features. Meanwhile, the forthcoming Sennheisier Ambeo Smart Headset will bring basic binaural capture to consumer-grade earphones.
However, sound that allows the listener to inhabit a space surely has one killer app; virtual reality headsets. It's something that the BBC is already working on with The Turning Forest, but with 360 degrees comes a change in pace. “Regular binaural audio is static, but dynamic binaural is when you start to hear sounds when you move your head through 360-degrees, and it requires a sensors to make those sounds move,” says Robinson.
For all its spatial cues that add a dizzying sense of immersion to static drama, binaural audio is simply the next stage in upgrading entertainment. But it adds such a feeling of immersion that binaural audio could prove to be the technology that makes 360-degree video and virtual reality go supernova. Seriously, binaural audio is THAT good.
- Check out Sennheiser's binaural audio headset.
