Key points in the keynotes

This week was the week of Microsoft's Build and Google's I/O developer conferences; an opportunity for the tech behemoths to showcase new products and new features to a crowd of enthusiastic developers, excited fans and a typically more sceptical press.

Both companies were keen to place privacy and security high on the agenda with greater control and transparency over your data. Simple, user-friendly, privacy controls were revealed as part of Microsoft’s chromium-based Edge browser announcements and Google revealed, amongst other things, incognito mode for maps and a physical hardware switch to disconnect the microphone and camera in its Nest Hub Max smart display.

AI also continues to be a huge development area, and ethics and responsible deployment very publicly pinned to the mast. One of the more significant developments within AI has been the ability to shrink the footprint required to process data so that processing of data can take place on the device as opposed to ‘in the cloud’ giving us more control over our data as well as making theses services far more universally avalable by reducing the need for expensive hardware or reliable data infrastructure.

Accessibility announcements

Microsoft's ongoing commitment to accessibility by design has meant that for all the best reasons there were relatively few purely ‘accessibility features’ announced and instead a host of features and announcements that, although clearly of benefit to disabled people, are productivity or convenience of interaction features that will benefit everyone. In fact, if you search (on Google) for “Microsoft Build 2019 Accessibility” the top result is likely to be the accessibility information for event attendees.

One announcement that did stand out was the ‘Ideas’ feature for Word. Set to initially debut in Word online, Ideas is an AI-based grammar and style checker that uses natural language processing (NLP) and machine learning to suggest improvements to your documents. No longer simply grammar and spelling, but now clarity and conciseness, estimated reading time, gender-inclusiveness of language (all presented as suggestions rather than auto-correcting - you will still have personal choice). There was also talk of it being able to read larger documents and highlight key points; potentially incredibly powerful tool for anyone who struggles with dense text or maintaining attention and focus. 

Hey Google 

Google CEO Sundar Pichai stated; “we’re moving from a company that helps you find answers, to a company that helps you get things done.”  Pichai’s keynote speech focused on the new Android Q features, the improvements to the Google Assistant, as well as the expected hardware announcements like the release of the mid-range Pixel 3a and the re-branding of Google Home products under the nest umbrella, but there were a few key accessibility highlights;

Project Lookout was was announced last year, but is now available for download (on Pixel phones and only in the US). Lookout is an app that runs on your smartphone and narrate the world around you. Designed for blind and visually impaired users, the instructions are as simple as ‘you take your phone out and start pointing it around’ and the app will identify objects and describe them to you; “There’s a chair at 11 o’clock”. It is also able to read the text from signs.

Google Lens showcased a text-to-speech feature that can scan text and read it back in a user’s native language. Impressive, but the key part of the presentation was that the code required for this feature takes up only 100KB space and can therefore be run on very low-end smartphones.

Live Transcribe is described as ‘captioning for the real world’ and provides subtitles to everyday conversations. It’s available in over 70 languages and also has the ability to deal with the nuances and context-specific uses of certain words (although the live demo of this bit would suggest it can be stubborn). The app will also allow non-verbal communication as tapping the screen will bring up a keyboard and enable the user to type responses. 

Live Caption brings the ability to subtitle any media playing on your phone. The subtitles can be moved around the screen as well as expanded to show more lines of text. This takes place on-device (edge computing) so no data is sent to Google and it will therefore work without a data connection (useful for watching content on the tube for example). 

Live Relay uses similar technology to Live Caption and turns a phone call into a transcribed text-based conversation that not only transcribes speech, but also reads out the text responses. Just like Live Caption, Live Relay runs on your device, so conversations are not sent to Google for processing, and it doesn’t need a data connection in order to work.

Voice interaction may be a convenience for some, but for people with a speech impairment or difficult to understand speech, it can be a frustrating experience. Project Euphonia is seeking to improve voice interaction with the use of machine learning to turn hard-to-understand speech and facial expressions into text. 

Project Euphonia is also asking if people who have slurred or hard to understand speech can help to improve the project by recording and submitting a set of phrases. You can read more about it and volunteer to contribute to the project on the Project Euphonia page.