Depending on where you rest your hat, the words “AR battle” could mean some very different things. You might still be hankering to have your Pikachu rain down lightning on a friends’ Squirtle in Pokemon Go, or, if you’re recovering from the iPhone X launch, be wondering when you’ll be able to send your mini troops into a tabletop augmented reality warzone in Directive Games’ The Machines.
But if you’re among the thousands in attendance at the Defence and Security Equipment International 2017 (DSEI) conference at London’s Excel Centre, those battles and warzones in question become altogether more real. As, while the consumer world waits patiently for the giants of Cupertino to lift the covers off the rumored Apple AR glasses, the military has been dabbling in augmented reality interfaces for years.
The transparent nature of AR glasses lets the wearer maintain situational and environmental awareness.
What was once the reserve of fighter pilots, the advances of Moore’s Law means that impressive heads-up display units will soon be standard issue for regular ground troops. Through the use of wearable glasses and headsets, key data points will be overlaid onto a battlefield – everything from mapping information to mission parameters to markers defining the movements of allied troops and enemy forces.
Topographical data can be relayed to a troop along with video feeds from remote overhead drones or fellow forces elsewhere on the field. All the while, the transparent nature of AR glasses (as opposed to the all-encompassing view of a virtual reality headset) lets the wearer maintain situational and environmental awareness. The potential chaos of what’s going on around a soldier can still be seen and actioned upon.
‘Vehicles of glass’ and ‘threat libraries’
“We can tap into a massive amount of sensor data. We can now do real-time image processing to an extent we once couldn’t even imagine,” says Dan Lindell, Platform Manager for the CV90 battle station at BAE Systems Hägglunds, one of the military contractor’s vehicles most heavily-invested with AR abilities.
The CV90 is already a formidable machine, a combat vehicle large enough to hold eight troops and configurable to include 30, 35 and 40mm autocannons.
But, building on BAE’s sixty years worth of experience in designing heads-up displays, the aim is to make a metaphoric “vehicle of glass” through new augmented reality advances – a tank, plane, boat or other machine that retains all the defensive capabilities necessary for modern warfare, while also equipped with enough sensors and imaging systems to allow the troops travelling inside to view the exterior situation as if they were moving in a transparent box.
Where once a gunner on recon would had to have left themselves exposed on the top of a tank in order to operate a machine gun turret, AR headsets will let them not only get a 360-degree view of the battlefield from the relative comfort inside the machine, but also to actively engage automatically-tracked combatants, using AI image recognition systems with access to extensive “threat libraries” that sieve expansive amounts of information in order to make a call on who is or isn’t a foe.
Reducing cognitive load
As consumers, the constant buzz of notifications, app alerts and data points can make simply owning a connected device a stressful experience – one that many fear will be amplified by the coming augmented reality wearables. Won’t these distracting data feeds be even more invasive when they’re placed unavoidably front and center of our field of view?
In a military context however, Lindell believes the exact opposite is true, with the AR device able to relieve the “cognitive load” of a troop under fire.
“Cognitive load can be traced back to three main factors: too many choices, too much thought required for an action, and lack of clarity,” says Lindell.
“Just adding displays with multiple push-button panels around will distract a crew. And then there are display parameters such as resolution, only providing a two dimensional picture. It’s not using the full functionality of the human brain with the senses – ears, eyes and so on.”
Lindell therefore believes that the hands-off, always-ready nature of information presentation that AR offers focusses the decision making process, and relieves that cognitive load.
“The less vehicle crews must think about what they must do to achieve their goals, the more likely that they’ll actually achieve them.”
Manageable, actionable data
It may be troubling to consider troops offloading too much thought to computer systems when lives are at stake – what if a sensor has failed, and an innocent bystander rather than an enemy has been highlighted as an aggressor? But the key element here is that the troop retains agency as to how to act upon the information presented to them in AR, rather than handing it off to an automated system. All the while, AR equipment will be reducing data that would once have been displayed on multiple screens into one portable, wearable device.
“A commander can filter the information he needs from the battle management system, and he can basically add flags, pinpointing each little pixel representing a key position in the real world,” adds Peder Sjölund, Head of New Technologies, BAE Systems Hägglunds.
“By reducing cognitive load, we’re expecting to see stress levels go down, with automatic target recognition letting crew members filter out 90% or more of uninteresting information, giving them just what’s interesting in that moment.”
Holding out for hardware
While the software solution is already well defined, the wearable AR hardware itself is still a work in progress. BAE was demoing its CV90 battlefield systems using a modified consumer Lumus DK-50 headset.
Two sleeker pairs of concept AR spectacles (somewhere between Microsoft’s HoloLens and a pair of Oakleys) were also available to try, complete with thermal imaging capabilities.
But elements like an onboard power system are still being developed, and display technology still needs refinement – whereas a consumer headset can afford a potential color accuracy drop-off at the edge of a transparent AR screen, a misleading display could have disastrous consequences in a battlefield if colors are used to distinguish between friend and foe. And while the busy environments of a living room or city street give a camera and sensor plenty of data points to compute between, it’s a different situation in a vast barren desert or out on the open sea.
Though the more advanced wearable was available to try in prototype form at the DSEI fair, BAE thinks it’ll take until late in 2019 before technology catches up to its vision enough to allow for a commercial product launch.
Misleading color accuracy could have disastrous consequences on a battlefield if colors are used to distinguish between friend and foe.
Sjölund described the AR market’s lead currently see-sawing between consumer and military use cases, making it one of the rare technology areas where everyday users in the near future will find the hardware they can own likely to be on a par with what’s being used in the armed forces. But if the BAE roadmap is anything to go by, those holding out for AR Apple spectacles may find themselves still waiting towards the end of this decade.
Whether packaged as a peacekeeping tool (BAE also envisions different software systems for commercial industries such as aviation, and for rescue services such as firefighters) or highlighted as the machinery of war that it is, it can be difficult to swallow that the same fundamental technology powering your Snapchat stickers will play a part in life-or-death decision making in conflict zones. But it does highlight the versatility of augmented reality applications – and why perhaps the hype in this case will be justified by a wearable computing revolution to come.
- Apple AR glasses: everything you need to know
