An enhanced navigational aid for the blind has been engineered by combining augmented reality hardware and computer vision algorithms. The Cognitive Augmented Reality Assistant (CARA) system developed at California Institute of Technology uses new software that enables objects to "talk." The technology integrated into a portable headset translates the optical world into English audio and could in the future be available in grocery stores, banks and other locations to help blind people make their way through unfamiliar spaces.
CARA was designed for use in concert with Microsoft’s HoloLens headset computer that can scan a user's environment and identify individual objects. With CARA, each object in the environment is given a voice and will "say" its name upon the user's command. The technology relies on spatialized sound, which causes 
The Microsoft HoloLens wearable augmented reality device. Arrow points to one of its stereo speakers. Source: California Institute of Technologyobjects to sound different depending on their location within a room. If the object is far to the left of the user, its voice will sound like it is coming from the left. The closer the object, the higher the pitch of its "voice."
To avoid numerous objects speaking at once, CARA is programmed with several different modes. In spotlight mode, an object only says its name when the user is facing it directly. As the user’s head turns, objects each say their name, and the pitch of the object's voice provides an auditory cue about its relative distance from the user. In scan mode, the environment is scanned from left to right with objects saying their names accordingly. A third option is target mode, where the user can select one of the objects to talk exclusively and use that as a guide to navigate.
To assess the system’s practical applications, a test route was designed for blind volunteers through one of the buildings on campus. The route was first followed by researchers while wearing the HoloLens, which scanned the environment and saved it to memory. Blind volunteers then navigated the route using CARA as a guide. As each participant began, a voice seemingly emanating from a location ahead on the route called out "Follow me," while also telling the user about stairs, handrails or corners to turn. The seven subjects completed the task successfully on the first try.
The researchers are now implementing new schemes for real-time identification of objects and pedestrians. The study is published in the journal eLife.
