An AR Collaboration Between The Creators Of Sleep No More and Pokémon Go

ugmented reality (AR) is at times defined by how much the technology's promise has outpaced its real-world use. AR’s main consumer experiences are still Pokémon Go and face filters on social media sites, despite several years of glossy demos. However, advancing AR will not only impact innovative experiences and games, such as Punchdrunk’s immersive show Sleep No More, but also progress hardware, like Apple’s U1 chip and smart glasses

Punchdrunk and Niantic, the creators of Pokémon Go, have announced a collaboration that will cover several projects. Specific details of the projects such as launch dates, cost and what the debut effort will look like, remain classified, but for the crossover episodes, the characters look encouraging.

Now Punchdrunk is no stranger to immersion, their signature piece Sleep No More, allows the audience to wander through a series of rooms in their own time and in any order. The performers act for hour-long loops; as the audience wanders, they collect the snippets of plot to cobble together their whole story.

The reason behind Sleep No More’s success is that it gives the audience a sense of autonomy and free will while at the same time maintaining an environment that does not respond to their choices. While it is immersive, it is not interactive as no matter what the audience does, the performance is set. 

While discussing Sleep No More in a recent interview, Punchdrunk Founder and Artistic Director Felix Barrett explained:

“Although it feels spontaneous, everything is meticulously choreographed… Our process even at the most transitional beats is completely set. With that sort of precision, and the ability to set and rehearse different scenarios … It’s those sorts of learnings, almost like binary coding in a way, that enable us to think about how one could use that information.”

It’s easy to see how the Sleep No More philosophy could be morphed to incorporate AR. Punchdrunk have already experimented with VR with a mobile game with Silverpoint, an app that directed players around the streets of London as well as a project for the Samsung Gear VR named #believeyoureyes. 

Bartlett went on to say: 

“Quite early on I became fascinated in the potential for game mechanics… We’ve scratched the surface with a few projects, but I always knew that for an audience to have real agency over the landscapes they traverse, if they could actually grant them that, they could become the hero of their own living game, their own movie. It felt like that was the next hurdle to cross.”

Niantic have spent years mapping a digital layer on top of most parts of the world since the start of their earliest game, Ingress. Niantic have created a rich AR infrastructure which has become the engine driving its subsequent hits, Pokémon Go and Wizards Unite, and it's what Punchdrunk will tap into for their collaboration, working with Niantic gives Punchdrunk a considerable advantage entering the AR space.

‍

In a recent interview, Niantic’s CEO John Hanke discussed his thoughts on the AR collaboration with Punchdrunk: 

“There’s so many beautiful and interesting and inspiring things out in the world, and thrilling and mysterious things, that we’re programmed to ignore… The goal of a project like this, in my opinion, is to add those points of interaction and that layer of story and gameplay that really causes you to pay attention all the time. It’s not the very particular interactions that this thing is going to deliver, it’s that state of having your eyes open.”

But is this realistic? Can you make people more in tune with their surroundings by creating them experience it through a smartphone? It’s difficult to really comment given how limited the information currently available on the collaboration is. However Hanke illustrates an experience that does not depend on keeping your eyes locked on your smartphone.

“Imagine that you are in a city with a destination that you’re seeking out and a clue that’s going to get you there,” he says. “You’re looking for that in real life. Your phone is in your pocket, and you have a quest that you’re going on, and you’re totally alive in terms of your senses. When you arrive at a specific location, you may pull out your phone, you may scan something, you may have a very specific interaction with the device, but that’s a point in time.”

Now this does sound ideal, until you take into consideration the current pandemic, it may not be the most optimal time to roam the city streets, but Niantic are already factoring this and have made alterations to Pokémon Go's mechanics to better enable indoor play to safely work around Covid-19. As the collaboration with Punchdrunk began prior to when Covid-19 hit its peak, the two companies have had to make some adjustments as they go along. 

On the topic Barrett had this to say: 

“We’re taking it hugely seriously and trying to work out new ways to navigate it so that we can still tell our stories… Although the planning began before Covid and we’re having to change things, we’re quite adaptive, because we have an audience who is mobile. We’re shifting some things, but we’re not actually changing too much.”

The collaboration with Punchdrunk is only one of the many avenues Niantic has begun to explore. Hanke stated that Niantic currently has around 10 new projects in the works, including a previously announced plan to bring the board game Catan into the AR realm. Niantic are also continuing to invest heavily in the Niantic Real World Platform, letting outside developers tap into its AR Cloud. It’s fair to say that Niantic are expanding their world beyond the Charizards and cauldrons.

AR technology is developing rapidly, but a lot has to happen for AR to completely fulfil its mainstream designs and expand beyond the occasional viral trend. As far as we can see, those steps seem to be falling into place, especially with the availability of appealing AR glasses being a step away. This collaboration between Punchdrunk and Niantic, no matter how secretive, promises a new, innovative and exciting avenue for AR technology, which provides a good foundation for the next chapter of AR.