How augmented reality will help us search, socialise and more

The future of augmented reality
Blippar's key campaign so far reveals a recipe book when an iPhone is pointed at a bottle of Heinz Ketchup

We've all seen the advert for Moonpig video cards where a mother points her iPhone at her birthday card, which magically plays a video within the virtual confines of the card itself.

As well as noticing that the utterly banal video has been recorded by the two people sitting right beside her, you might also observe that the phrase that describes what's happening here - augmented reality, or AR for short - isn't mentioned at all.

That's probably because it sounds like something from Red Dwarf, but AR technology has been around a while and is starting to innovate its way into our daily lives.

There are other uses of AR - such as the Layar app that uses your smartphone's exact GPS to flash-up details of the nearest pubs, restaurants and hotels - and most of the other uses are also app-driven. Layar is premised on supplying useful geographic information overlayed on whatever is in the viewfinder of your phone's camera.

"Its uses are increasingly becoming more utility-based, for overlaying relevant information on the world around us, as a guide to where we are and what's going on," says Maani Safa, product director of global mobile marketing company Somo and formerly head of mobile for the Telegraph Media Group (where he launched the Telegraph iPhone app and Kindle edition). "Every major city in the world should have an AR app to welcome their visitors," he says. That's slowly becoming the case (Lonely Planet take note).

Starry eyed

Similarly, the flurry of compass and accelerometer-powered stargazing apps such as Star Walk, Distant Suns, SkyView and Night Sky that overlay constellations, planets and real-time satellite fly-pasts in the palm of your hand are arguably the most addictive use of AR thus far (though don't use them in observatories or near real astronomers - your phone's white light pollution won't be welcome).

These stargazing apps offer a combination of either GPS data and image recognition with a database of data, but on that simple premise a whole range of navigation applications can - and are - being developed. However, it's apps that make use of image recognition to trigger AR features that are set to dominate.

Aurasma, an app for iPhone or for Android phones lets each users leave their own images, audio or video - an 'aura', in tech-speak - which is tied to an image that's recognised by the lens of another user's phone camera.

This kind of simple image recognition is slated to add a layer of interactivity to museums (pioneered by James May at the Science Museum), add videos to comics (already done by Marvel), herald a completely new type of textbook (language books in particular could benefit) and transform advertising. Aurasma even allows auras to be left in specific locations that can be unlocked by a smartphone using GPS data.

Science Museum app

Point your phone at this while at London's Science Museum and you'll see an aura of James May describing some of the exhibits

Expect to be able to see the latest film trailers just from scanning a poster on a bus or in a train station, and printed adverts in magazines and newspapers with a world of content to unlock for those who reach for their smartphones.

"Every poster and piece of packaging can give further information using AR," says Safa. "Bringing billboards to life is surely the dream of any advertiser looking to measure direct response to what until now has been a largely immeasurable form of advertising."

In this particular arena image-recognition phone app Blippar is one of the most active, creating AR-based marketing campaigns for Heinz, Xbox, Samsung, Cadbury, Domino's and LG.

Its challenge is to create extra content - called 'blipps' - that get us to interact more with a brand's product. Point your phone at a bottle of ketchup and a 3D book appears to open 'out' of the label of the bottle, complete with free recipes that can be downloaded.

Passing phase?

Is there a danger that AR becomes merely a passing phase in novelty marketing campaigns? "As with any medium, the success of this as a new marketing tool rests entirely on the value of the content delivered," says Jess Butcher, CMO at Blippar, who thinks that content must be functional or educational, entertaining, or value-rich.

"It is precisely for this reason that we don't see AR as the exciting technology in this space - markerless image recognition is what's empowering this whole trend. Image recognition without the use of any codes or watermarking is what's enabling physical creatives, products and real-world imagery to be converted into instant content or interactive experiences."

Markerless audio recognition is also on the cards, with broadcasters expected to employ 'audio watermarking' to trigger exclusive second-screen content on an iPad or smartphone.

How it's triggered is important for convenience, but whether AR succeeds is more about hardware keeping pace, says Tom Gray, senior creative strategist at marketing firm Imagination, maker of a racing simulation AR app for Ford. "Often networks aren't fast enough and GPS and an accelerometer aren't accurate enough for good mobile AR," he says, predicting that apps that overlay information from Twitter and Facebook are inevitable.

AR specs

The ultimate goal of overlaying information on reality borders on the rather cliché idea of virtual reality glasses, something Google and Oakley have talked about recently.

Google's Project Glass is initially focused on making browsing, navigating and video calling completely hands-free, though the technology could later move to regular glasses or contact lenses.

"When someone like Google reaffirms its commitment to creating an augmented world, you can be sure it's a serious proposition," says Safa. "Search should, and will, be as seamless and easy as thinking. With AR, it is moving much closer to that."

Project Glass

Google's Project Glass creates an augmented world that's ultimately about making web searching completely hands-free

Way before any of that happens, AR is predicted to give a much-needed fillip to the publishing industry. "All the printed formats - press, posters, in-retail, books - will stand to benefit most," says Butcher. "They are, by their nature, 'old' before they have hit the street, and this technology enables them to be 'unlocked' with up to date information and interactive experiences in real-time. It's hugely powerful."

Saving print

Safa goes one step further. "Mobile, including AR, might be the last chance to save this industry," he says. "AR gives publishers the chance to incorporate the rich and successful elements of TV and movies into what has always been a one-way, static medium.

"In newspapers, for example, any story that is reported is an opinion, someone else's take on an event or situation. Using AR, you can overlay their written account with primary footage of the event actually taking place, giving people deeper insight into the story. AR could bring an old, dying medium back to life."

Will AR herald a new age of hands-free visual browsing, or merely allow us to point our phones at each other to reveal 'Keep Calm & Carry On' virtual tattoos on our foreheads?

AR-adorned novelty clothes, toys and games are imminent, and might play a crucial role not only in educating us all in what's possible and practical, but in establishing global standards.

Either way, the race is on to put a navigational layer into our digital lives that will eventually sweep away the archaic process of typing search terms into web pages.

Jamie Carter

Jamie is a freelance tech, travel and space journalist based in the UK. He’s been writing regularly for Techradar since it was launched in 2008 and also writes regularly for Forbes, The Telegraph, the South China Morning Post, Sky & Telescope and the Sky At Night magazine as well as other Future titles T3, Digital Camera World, All About Space and He also edits two of his own websites, and that reflect his obsession with travel gear and solar eclipse travel. He is the author of A Stargazing Program For Beginners (Springer, 2015),