For the one in six adults in the UK affected by hearing loss, now there’s a solution using augmented reality (AR) that allows them to ‘see’ conversations in real-time.
The makers of XRAI Glass have created a software that subtitles their life on AR smart glasses plugged into their phones.
Following a successful trial program earlier this year, XRAI’s software is now available to download via the Google Play Store and can be used in conjunction with Nreal’s smart glasses. Today, they have launched their service globally in nine languages.
Initially designed to enhance life for people who are deaf or living with loss of hearing, its additional features like a personal assistant, recalling conversations ‘on demand’ and real-time translation can be used by anyone who wants to subtitle their daily life.
Metro.co.uk got to try out the new technology which we thought was pretty neat.
XRAI is the software that does the subtitling and runs on mixed reality smart glasses like Nreal.
‘We are the software layer that sits on the glasses. We wanted to build this to be portable so that if any other device came to the market, we can instantly run our application on that, be it Google, Apple or Meta,’ said Mitchell Feldman, CMO of XRAI Glass.
In July, XRAI launched a public trial period where they worked on finessing it with a few dozen members from the deaf community.
‘We built a healthy community on Discord and started getting feedback and found things that we hadn’t even considered,’ said Feldman.
The team soon realised that they weren’t just a solution for the deaf community but also for people who were neurodiverse, like those with ADHD, who could consume information much better if they read it rather than heard it.
Feldman tells us that XRAI’s team of experts, which included an ex-Meta employee who worked on their AR division and a CTO with a PhD in audiology, had the software ready to ship in a year.
How does it look?
The glasses consist of two televisions in front of your eyes projected into ‘birdbath lenses’ that take the light and bend it into your eyes at 45 degrees for that 3D effect.
Since all of the processing is done on your mobile phone, the glasses themselves are lightweight and comfortable to wear all day.
While there are devices that provide a similar service on mobile phones, using augmented reality lets you have a conversation without looking at your phone.
‘You’ve got those digital elements overlaid onto your analogue world. You can still see the faces of the people you’re talking to,’ said Feldman.
The glasses can be plugged into your TV to display subtitles even for content that doesn’t have them like live television. One of the members from XRAI’s trial group is a lawyer who uses the glasses, plugged directly into the mic system of the courthouse.
The glasses go beyond displaying subtitles, using a technology called speaker diarisation, essentially being able to identify different people’s voices and label them.
How does it work?
XRAI can capture audio from the glasses, Bluetooth microphones, phones or audio streaming devices. This can either be processed on the phone or by their cloud providers where the data is only processed but not stored.
Once the audio is processed into text, it is displayed in augmented reality. All this happens in real-time.
The latency from the first audio input to the first word back is less than one second and they’re working on making that one-third of a second.
The handy thing about XRAI is it also works as a personal assistant like Alexa or Siri. You can say ‘Hey, XRAI, what’s the weather like today in London?’ and have the answer instantly appear on the glasses.
This is done by using deep learning technology, with data from GPT 3, a large language model used for understanding natural language questions.
Privacy and the Metaverse
As with any technology, privacy is a concern, especially since XRAI processes your everyday conversations.
Leaning into the decentralized architecture of Web 3.0, XRAI Glass puts the ownership of the data back into the hands of the user.
‘When these products are free, you’re the product. We want to flip that and make it decentralized,’ said Feldman.
‘So, the owner of the data is you and it will never live in the cloud,’
Like any futuristic tech company, XRAI Glass has its eye on the metaverse which it believes will be a blended reality where smart glasses will overlay digital elements onto real life.
‘The end state for us is that this becomes ubiquitous to life like a contact lens that runs an app on it and the rest of the world doesn’t know what you’re looking at,’ said Feldman.
According to their research, around 47% of Gen Z are now consuming content with subtitles. So, it’s a natural progression for this technology.
XRAI Glass is aiming to reach 100,000 users globally by the end of 2024 and it already has many takers. Love Island’s Tasha Ghouri, its first deaf contestant, has come on board as their Brand Ambassador.
‘Even with my cochlear implant, I struggle still in certain situations, for example at the cinema, in group conversations, with TV that’s not subtitled,’ said Ghouri.
‘I now can use these glasses and not miss out on text in real-time as well. It’s just insane,’
For now, XRAI is only available for Android but the team are in a hurry for Apple to bring out their mixed reality glasses, so they can use their app on them.
How much does it cost?
‘The glasses themselves are around 400 pounds which you can buy through EE, or Amazon. The only other cost is our software if you want it to go beyond just the free version,’ said Feldman.
We also recommend that people invest in a high-quality Bluetooth microphone which makes a world of difference to the experience.
XRAI Glass is available to download from the Google Play store today and is compatible with the Nreal Air Augmented Reality glasses which are available from Amazon Japan (internationally) or Amazon US (USA and Canada).
The software will be available in three different packages: Essentials, Premium and Ultimate, priced from a free version to 49.99 a month.
Source: Read Full Article