AI for Everyone: Accessibility Meets Bixby Vision

From electric wheelchairs to hearing aids, assistive technology is key to enhancing the social participation of persons with disabilities. As image recognition technology continues to advance, mobile devices have the potential to help users with visual impairments overcome barriers in daily life.

By working with users and listening to their feedback, Samsung has developed accessibility features to empower individuals with visual impairments for Galaxy devices.^1,2 To mark this year’s Global Accessibility Awareness Day, the company is putting a spotlight on the three features which have enabled many Galaxy users to live more freely.

Transforming Lives With Innovations

The idea behind Bixby Vision is a simple one. It’s designed to give people the ability to gain more information about their surroundings by using a smartphone’s camera. The new accessibility features, namely Quick Reader, Scene Describer, and Color Detector, build on this capability to help users with visual impairments navigate the world and enjoy more enriching experiences.³

Understanding labels and signs is a daily challenge for users with visual impairments. By reading out written text in real-time, Quick Reader helps these users gain a better understanding of textual information in daily life.⁴

The feature can also recognize over 1,000 common objects and items. From identifying food and vegetables in the kitchen to locating cleaning products, it can help users carry out daily routines with ease.

Scene Describer provides descriptions of any images, including captured scenes and downloaded pictures. This could help users identify potential obstacles when they are navigating their surroundings.

While it’s possible for people with visual impairments to identify the materials and design of their clothes by touch, it’s difficult to do the same when it comes to distinguishing between different colors. The Color Detector provides a way to address this.⁵ With a simple camera scan, the feature can inform users of the color of the item in the frame.

“Before these features came to Bixby Vision, I had trouble picking my work clothes and identifying objects in the office alone,” says Kwangman Moon, a Galaxy user with visual impairment. “Now, I don’t need to give up on these simple tasks just because I can’t see.”

Collaborating With Users

Receiving users’ feedback is essential for the development of any mobile feature. For the latest Bixby Vision accessibility features, Samsung’s collaborative approach takes on even more significant importance.

“One of the problems we faced when testing the features with visually impaired users is that their way of experiencing the world is different from ours,” says Mikael Fodor, UX researcher at Samsung Research United Kingdom. “Being able to observe how the features work in the hands of our users and in real life situations allows us to understand problems and make improvements scientifically.”

For users who collaborate with Samsung engineers, being able to improve accessibility features and help others in similar situations overcome challenges is an empowering process.

“Many companies ask us to share our opinions regarding accessibility features. But for societal, technological, and institutional reasons, our feedback rarely gets reflected in their products,” says Sangjun Kim, a user who contributes to improving the accessibility of Galaxy smartphones. “By contrast, Samsung takes into account even the smallest feedback to improve app features. It’s been a special experience to see my feedback reflected in the real product.”

An Inclusive Mobile Future

While the latest accessibility features on Bixby Vision, as well as the high-contrast theme and Light Sensing on One UI 2, represent a key milestone in Samsung’s innovation history, the company understands that creating an inclusive environment on mobile devices is an ongoing process. Driven by the commitment to make mobile technology truly accessible to everyone, Samsung will continue to collaborate with users to develop boundary-pushing yet meaningful innovations.

¹ Since 2016, Samsung Electronics has been running the Samsung Supporters program, which works with users with disabilities to improve accessibility features.

² New accessibility features, including Quick Reader, Scene Describer, and Color Detector, are available on Galaxy devices with Bixby Vision 3.5 or above.

³ Availability of Scene Describer may vary by region.

⁴ Quick Reader supports 57 languages.

⁵ Color Detector can discern between 33 different colors.