This article was first published in theSun Buzz edition HERE
SNAPCHAT recently announced a host of new features at the Snap Summit 2021. One new feature will allow Snapchatters to shop and try on clothes and accessories virtually, while standalone app Story Studio lets them edit high-quality videos on the fly, along with an inclusive camera to capture darker skin tones accurately. Find out more about the new features HERE
One of the more exciting features is Connected Lenses. Connected Lenses allow Snapchatters to build or carry out an activity together, no matter where they are.
BUZZ had the opportunity to speak to Qi Pan, the director of Computer Vision Engineering at Snap, about some of the upcoming features.
Can you tell me more about Connected Lenses?
“Most of the Augmented Reality (AR) experiences that exist on Snapchat allow you to experience AR on your own, and the social aspect comes when you share the video with your friends.
“With Connected Lenses, we want to go into the next generation and for the first time, you can actually experience the AR experience together with your friends and family. It’s a shared experience and a real-time interaction between multiple users.
“We have two versions of this. One is the remote Connected Lenses, where you’re in a different place from your friends and family, and you can do something together, like build a Lego set [while] sitting on the table or the living room floor.
“The second version is where users use Connected Lenses in the same room and build the Lego on the same table together.”
Which feature are you most excited about?
“For Lens Studio 4.0 [which was] released today, I’m probably excited about Connected Lenses and Body Mesh reconstruction. This is really an amazing understanding of people and the scene around you.
“You can do hand gestures and things like that, and it can be understood by the Snapchat camera. This is very exciting when combined with Connected Lenses because it can recognise other people in the scene and it tracks with them.”
Other than building Legos together, what are the other future activities users can do using Connected Lenses?
“Connected Lenses really opens up the AR world and the shared experience. We’re also looking into utility and other things we can do and go beyond entertainment.”
Does the feature need constant data connection work? Can it work offline?
“For Connected Lenses, you’ll need to be online to experience [it], because we rely on real-time data stream. For a lot of the other features, it’ll require a connection to get the Lens, but once you get the Lens you won’t need to use data.
“For example, the Body Mesh Reconstruction is running on the phone itself, so it doesn’t need to send or use up data. All the processing is happening locally, and offers a great experience because it’s instant.”
Everyone is using different types of phones. With all these new features, will it slow down our phones?
“Some of the features need quite a lot of processing power, but we generally try to design it so that it can be used by phones with [either] a fast or slow CPU. If the phone has lower computational power, we try to tune it down, so you can still have a great experience. We try to tune it so the feature works well across all phones.”
Since Snapchat focuses a lot on virtual and augmented reality, do you think the public is still hesitant to embrace this technology?
“Snapchat has embraced AR in a different way compared to other companies, in that people are using augmented reality without knowing necessarily that it’s augmented reality. We’re seeing the end-user creativity rather than us telling them it’s augmented reality.
“If you look at Face Lenses, for example, people see these amazing, creative ways to express themselves and we don’t necessarily say this is AR or this is not AR. We’re saying this is a really good way to communicate. I think that’s probably the best way for the public to adopt AR: by not actually thinking they’re specifically adopting AR but through gaining enough value-added experience.”
