Pokémon Go creators are raising the bar for AR… again

Viral sensations are nothing new. The accessibility of social media platforms means any piece of content has the potential to reach billions of screens around the world at the tap of a button, and it’s a phenomenon we’ve seen take hold of music, images and videos that have since become immortalised in the annals of popular culture.

It’s rare, though, to see an entire app just as quickly find its way into the hands of millions of mobile users in every corner of the globe. It was pretty much unheard of, in fact, until the arrival of Pokémon Go in 2016.

The brainchild of developer Niantic and Nintendo, the free-to-play mobile game introduced a novel gameplay system combining GPS and augmented reality (AR) software to generate hundreds of virtual Pokémon to be caught by players in real-world locations. The rest, as they say, is history.  

• Apple Glasses may offer the most comfortable mixed reality experience yet
• Pokémon Go developer’s AR glasses could make you a real-life trainer 
• 3 things we’ve learned about Facebook’s AR glasses  

It became the fastest mobile game to earn $100 million, the most downloaded in its first month of release and has since been downloaded over a billion times. For comparison, Angry Birds – which first launched in 2011 – boasts an equivalent figure of around 300 million downloads. Pokémon Go wasn’t just a technological novelty, it was a cultural force. 

It comes as no surprise, then, to learn that Niantic is continuing to pave the way for the widespread, mainstream use of AR in our everyday lives. The San Francisco company has announced a planet-scale AR platform for current and future generations of AR hardware – Niantic Lightship – offering developers the opportunity to build a diverse range of applications which could see the technology evolve from novel pastime to real social tool. 

TechRadar sat down with Kjell Bronder, lead product manager for AR at Niantic, to talk about Lightship and how these new developer tools could impact the future of gaming, tourism and other industries.

“AR is the perfect way to make the world more engaging, more fun, more interesting,” Bronder tells us over a Zoom call.

“There's been Pokémon games since and Pokémon games after, but none of them have had that same level of success as Pokémon Go. So how do we take everything from this type of location-based AR game and create a platform where we can start building other things?”

Niantic’s new Lightship technology aims to combine real-time mapping, semantic segmentation and a more robust multiplayer experience to offer developers the “richest selection of AR tools and technologies to date.”

Let's get physical

That first key development area, real-time mapping, is concerned with more accurately fitting virtual objects into the real world using smartphone cameras. Lightship essentially aims to make physics possible for virtual objects. “To better augment reality, objects have to act naturally or behave as you would expect, so real-time mapping allows software to better understand your environment,” says Bronder. 

Similarity, semantic segmentation – which isn’t to do with the meaning of words, apparently – allows virtual objects to look, feel and move in realistic ways. Niantic says Lightship can automatically segment different natural outdoor objects in a scene, enabling AR content to interact with specific surfaces. 

“Semantics identifies physical barriers in the real world,” Bronder says. “A virtual ball would bounce off a real-life wall and roll around on the ground, for example. If it hits water, it would splash into it. These are the types of scenarios this technology allows.”

Bronder is keen to stress that this functionality would be made available to as broad an audience as possible, since Lightship only requires the RGB color sensors found in most cameras (as opposed to the LiDAR scanners only available on high end devices like the iPhone 12 Pro). “We want to make sure that we can reach the largest audience, that's something we've learned from Pokémon Go.”

It’s clear, though, that the potential for greater social interaction is at the heart of Niantic’s mission. 

“How do we incite people to explore the world together?,” Bronder continues. 

“There's nothing that's going to bring a virtual object to life more than if multiple people are seeing the same thing at the same time. Naturally, that's really hard when it comes to AR, because it's a computer vision problem. You have to actually understand the pose of all the different devices that are looking at that same scene. You're overlaying on top of the real world, so it's more important than ever that you have real-time communication to keep all of the different objects in sync.”

Lightship aims to offer developers an “easy-to-use, shared area service” which could allow up to eight players to share the same AR experience. Peer-to-peer messaging features would also be built-in to the software. Given that many of today’s mixed reality experiences suffer from a lack of community – even the most successful VR devices, like the Oculus Quest 2, fail to cultivate much meaningful social engagement – Lightship aims to keep future AR applications from being short-lived novelties.

“I think the gaming side is where there’s huge potential for getting people outside exploring the world, as we see with Pokémon Go, but we want to go beyond just gaming – there are many other opportunities out there,” says Bronder. 

“Our game teams are making these tools available to third party developers in several industries – think about tourism, festivals, cities, entertainment, education. AR can help people visualise things and see the world with new eyes. It's going to be really interesting to see what all of these different companies, brands and developers come up with.”

What’s next for Niantic? 

But what about games from Niantic themselves? “I forget how many games we’re working on internally, but I can say it's more than a dozen. We’ve talked recently about partnerships with Nintendo, Pikmin and [popular board game] Catan – but there’s lots of others. Exciting things are in the works.”

Thanks to a Twitter post from Niantic CEO John Hanke earlier in the year, we know the developer is also working on its own pair of augmented reality (AR) glasses, so one of those “exciting things” could be the launch of a fully-fledged Niantic mixed reality device in the near future.

And, of course, we couldn’t let one of the brains behind the most successful mobile game of all time leave without a nod towards the future of the Pokémon Go franchise.

“Let’s just say that the universe continues to expand,” teases Bronder. 

“We're coming up on five years [since its release] and we’re going to celebrate that with the Pokémon Go fest. We envision the Pokémon universe is just going to get broader – there's so much more to do.”

Given how far along Niantic seems to have moved AR technology since 2016, another billion downloads could be on the way soon. With major players in the tech industry like Apple, Samsung and Facebook all reportedly working on their own AR-focused products, too, the technology seems primed to experience an explosion in growth in the years to come. Could a pair of Apple Glasses be the catalyst for the industry-wide adoption of AR software?

In any case, expect a host of new innovative AR applications to arrive in the near future, ready to augment – as is their purpose – our everyday lives in ways we can’t yet imagine. 

“It's creative development, it’s storytelling, it’s imagination,” says Bronder. It’s also just around the corner.

The private beta for the Niantic Lightship Developer Kit is available to join at Niantic.dev.

• Pokémon Go tips and tricks guide: be the best trainer you can possibly be