How AI is improving captions and audio

A dilemma many of us face is the incompatibility between watching videos online and enjoying crunchy snacks.

It's a dilemma that forces us to often have to choose between one or the other, or to get grubby fingers all over our keyboards.

Subtitles for noisy snacking

Worry no longer, Lay’s potato chips (Walkers Crisps in the UK) has a solution to this very first-world problem with the release of "Crispy Subtitles from Lay's"; a Chrome browser extension that can pick up the sound of crunching crisps and will switch on YouTube captions for you!

Created by Bliss Interactive and Happiness Saigon (Lay's ad agency) the AI behind the app was trained on 178 hours of the sounds of people from around the world merrily crunching away. 

The extension will obviously need to access the microphone on your computer and is just a bit of fun, but it does highlight how we now interact with our devices. Or rather how our devices are becoming far more able to anticipate what our needs might be as opposed to us having to tell them what we want them to do for us.

Captions increase engagement

Of course the fact that this fun extension only runs on YouTube also highlights how much content is out there that is un-captioned and inaccessible to anyone unable to hear your content.

We know that anywhere from 60-85% of videos on social media are watched without sound, whether that be due to a permanent hearing loss or impairment, a temporary loss such as an ear-infection, or a situational impairment such as a noisy room or the opposite end of this spectrum and a quiet space like a library. 

We also know that including captions increases engagement, means people are more likely to watch a video all the way through and will improve SEO. Although most platforms support the inclusion of a subtitle file, this is so often overlooked by creators. 

Auto captioning audio content

Back in August 2020, we wrote about the live captioning available on some Android smartphones and now Google’s chrome browser is able to caption any audio content it ‘hears’, from podcasts to live shows. You can find the feature under - "Settings" - "Advanced" - "Accessibility" and it appears at the top of the list.

If you switch the feature on, whenever you land on a page with spoken audio (it will attempt songs but the results are a bit 'varied') you’ll see a pop-up with the captions overlaying the bottom part of the screen.

You can expand the window to include more lines or close it to, well, close it (but it’ll pop up again if you refresh the page or go to another page with audio). 

Find out how to enable Live Caption in Google Chrome:

Noise suppression techniques

Sticking with the theme of AI-based audio and crisps, Krisp.ai is an app that uses AI to eliminate background noise.

Noise supression has been available within the audio settings of both Microsoft Teams and Zoom (in the desktop app) for some time now, but the Krisp.ai app takes this to another level.

It runs on your desktop or smartphone and works with a reported 800 communication apps, including Microsoft Teams, Zoom, Slack and Google Hangouts. It will work for both ends of the call regardless if the other person has the software installed.

Get 120 free minutes per week

The Krisp.ai ‘AI’ was reportedly trained on more than 25,000 hours of audio with 50,000 different speakers and 20,000 background noises. You can hear the app in action on the Krisp.ai website as well as try it out for free (120 minutes per week).

It does an incredible job of filtering out noises, but is also able to eliminate the echo of your own voice from another person's speakers, as well as reducing room echo.

OK, so we might be coming out of this into a world of blended learning, but at least home classroom and in-person lectures will be quiet. And if you do fancy a quick snack, you can do so without missing anything!

Further resources

• Blog: New accessibility training courses for higher and further education
• Attend our free webinars
• Sign up to the AbilityNet newsletter 
• Subscribe to the AbilityNet podcast

AbilityNet provides a range of free services to help disabled people and older people. If you can afford it, please donate to help us support older and disabled people through technology