Microsoft’s Seeing AI app uses artificial intelligence to narrate the world for visually impaired

Microsoft has released Seeing AI, an app targeted at visually impaired that is able to provide information about the world around the user. It uses artificial intelligence to describe people, text and objects through the iPhone camera.

https://www.youtube.com/watch?v=bqeQByqf_f8

“A free app that narrates the world around you. Designed for the low vision community, this research project harnesses the power of AI to describe people, text and objects,” Microsoft writes about the app. With the app the iPhone camera is used to watch what’s around which is then described by a voice.

The app can be used in several different situations. Seeing AI recognizes people it has seen before and can tell more about household products by scanning the barcode. It’s also able to speak out text, to describe the scene around an user and to identify currency bills.

To recognize objects and persons, the app uses neural networks. The most important calculations are performed on the phone, so Seeing AI can also be used without an internet connection. Experimental features, such as describing the scene around the user, requires a connection to the cloud.

Seeing AI is currently only available for American Apple users.