I’m standing in line with a gaggle of sweaty journalists in 90ºF heat, waiting for a chance to take a virtual swim through New York City's Times Square using a Microsoft HoloLens.
Nearby, a sea of tourists flows past, gawking at a 24 x 34 x 60 ft (about 7 x 10 x 18 m), skeleton-like shipwreck as they pass. Several stop to pull out their phones and try to figure out how to watch the shipwreck come back to life in augmented reality.
“Mama, was that a real boat?” a little boy shouts over the hubbub.
Mel Chin’s life-size recreation of the wreckage of a Civil War-era ship is not, in fact, real. But in Unmoored, the HoloLens tie-in experience to Chin’s giant art installation on 46th and Broadway – titled Wake – the exhibitors inserted models of real boats currently docked in New York Harbor, sailing in the sky in a procession of unique colors and shapes.
It’s meant to evoke the dangers of climate change and sea level rises to New York City (and the world), imagining how this popular vacation destination might only be accessible to boats in the future. But technical issues and augmented reality’s current limitations somewhat marred the message.
AR can’t beat the heat
The free Unmoored AR experience can be viewed in two ways. You can download the free Unmoored app on iOS or Android, and brave NYC’s sightseeing central anytime from now until September 5.
Or, you can rush to try out the limited-time HoloLens experience, which will shut down tonight (Friday the 13th) at 10pm ET.
In truth, nighttime is probably the best time to check out the experience. While I waited my turn to dive into Chin’s vision, Microsoft's team was frantically trying to reboot four HoloLens headsets, which had shut down due to the heat. The reps ended up plopping ice packs on top of them to cool them down.
While awful, tech-breaking heat was apropos to the installation’s message about climate change, it also exposed that HoloLens doesn’t really handle sunlight that well.
Once my experience started, I was asked me to stare at the Wake shipwreck. A bone-white, holographic version of the historic USS Nightingale superimposed itself over the real thing, then began to soar upward into the sky, past all of Times Square’s infamous giant ad boards.
The Jenny Lind figurehead, a popular fixture of ships in the 19th century, looks sadly upon her surroundings, while the aforementioned NYC Harbor boats begin to sail 26 feet over street level, filling the air in a crowded jumble of traffic.
The juxtaposition of hectic New York City streets and environmental devastation undoubtedly conveys a powerful emotion in the viewer. Yet while the concept of an underwater Times Square excited me, in practice the tech couldn’t fully meet the challenge.
The bubbling sounds from the headset notwithstanding, I just didn’t actually feel like I was underwater. Perhaps because of the oversaturation of light, I could barely see the simulated shimmer of water, which made the USS Nightingale and other boats feel like they were slowly soaring through the air instead of coasting on waves.
Microsoft said it added a “neutral density filter” to combat the natural light, which makes me think the holograms would have been completely invisible to an unmodified HoloLens.
I also sometimes had trouble finding the augmented elements, spinning around in circles until I tracked down what I was supposed to be seeing. Plus, I never spotted the sea creatures that were supposed to appear around you. And, the demo may have ended prematurely, right as a message popped up saying to gaze around at all of the boats.
In optimal conditions, or for those who haven’t tried out AR before, I’m sure that the experience would be far more magical.
I returned that evening out of curiosity, and found that Microsoft had ironed out the kinks. All four HoloLens headsets were working, and a huge line of grinning vacationers waited eagerly for their turn.
Smartphone AR versus the sun
While I wasn’t wowed by HoloLens’ underwater journey myself, it was still the definitive way to try out the experience, which makes it too bad that the majority of visitors will have to use the inferior smartphone AR instead.
The battery-guzzling Unmoored app, once downloaded, asks you to walk to one of six spots around the exhibition. You then point your camera at the ground, where the letters A through F are painted, and tap start to commence the experience.
As with the HoloLens demo, the ghostly skeleton of the USS Nightingale is meant to overlay over the real-life model. But, across multiple attempts from vantage points A, B and C, I couldn’t get the app to align properly. Skewed horizontally and sitting at an angle, the hologram just doesn’t look great.
Once the skeleton has rebuilt itself and taken to the sky, I do think it looked pretty cool, and frankly didn’t look that much worse from what I’d seen in the HoloLens, despite the lesser graphics.
But, I did find it disappointing that the different letters didn’t seem to trigger any noticeable differences between them. They’re all there to make sure that tourists don’t all cluster in one spot, not to add any variety.
If you do plan on checking this out, which we’d recommend if you’re in the area, then make sure to download the app beforehand, and bring your portable charger with you. Simply downloading Unmoored, waiting for the AR to load, and using it for 10 minutes made my battery drain from 80% to 20%.
iPhone 6S and beyond running iOS 9 should be okay to run the app, while any Android phone with augmented reality capabilities can use it, too.
Times Square AR: flashy, but lacking substance
During the unveiling, Mel Chin and an assortment of art curators outlined the historical significance of the piece: the USS Nightingale acted as a slaving vessel before the US Navy commandeered it to carry munitions during the American Civil War.
“Wake invokes both the triumphs of New York and some of its darker histories,” said Manon Slome, co-curator at No Longer Empty, which helped sponsor the project.
As the kind of historical nerd who likes to rent the headphones at art exhibitions to learn more about the paintings, I had high hopes about learning more about the history and meaning behind Chin’s work. And, I fully expected that it would openly discuss the climate change science that fuels the installation’s fears.
Instead, neither the app nor the HoloLens experience provide any information – just ambient noises.
For most people, that will be preferable, and art should typically show rather than tell or lecture. But, it would be nice to have the option to learn more through the app.
Ultimately, I suspect that tourists who didn’t attend the unveiling to hear the themes behind the work won’t think much about the message behind the art. They’ll probably just think it’s cool to see ships floating around overhead, and then move on to the next attraction.
- The new Hololens, codenamed Sydney, could arrive early next year with double the field of view
