When people with a sight impairment cannot cope with everyday tasks, it is often not due to a lack of competence, but rather because of impediments existing in their environment. A mobile assistance system, which is currently under development by researchers from the Karlsruhe Institute of Technology (KIT) and the Fraunhofer Institute of Optronics, System Technologies and Image Exploitation (IOSB), is designed to help them achieve greater independence in their daily work and make it easier for qualified professionals to secure jobs. Extended-reality glasses (XR), combined with Artificial Intelligence will adapt their working environment to individual vision requirements.
For many skilled workers with a sight impairment, daily tasks can be full of obstacles: Reading a device label, deciphering a handwritten note, or finding their way in unknown spaces is time-consuming or not possible without help. Conventional aids often hit their limits beyond the computer workstation. This is where the AssistiveXR4Work project comes into play: “We’re developing a system that automatically recognizes or magnifies visual information, displays it with high contrast or reads it out loud, depending on the individual vision requirements,” said Professor Rainer Stiefelhagen from KIT’s Center for Digital Accessibility and Assistive Technologies (ACCESS@KIT). “For example, it can automatically recognize and magnify the contents of a whiteboard in a meeting or convert handwritten notes to plain text and further process it without human intervention.” In engineering working environments, on the other hand, the system can help with orientation and highlight relevant components.
The system uses advanced XR glasses with a high-resolution camera image. They capture the surroundings continuously and digitally adapt the display to the individual vision requirements by enhancing contrast, magnifying contents, recognizing texts, and reading them out loud on demand. In addition, an integrated AI system helps with orientation or recognizes objects present in the surroundings. If needed, a human helper can be connected via a live stream.
How Inclusion Helps Securing Skilled Workforce
The project not only helps visually impaired people participate in the labor market, but also addresses the shortage of skilled workforce. Actually, many highly skilled employees lose their jobs or have to shift their field of activity when their vision deteriorates. The goal behind AssistiveXR4Work is to help them pursue their career even then. According to the Institute for Employment Research (IAB), a research institution operated by the Federal Employment Agency, severe disabilities are widespread in Germany, the majority of which are occurring during the person’s professional life. The probability to still be part of the labor force five years after the disability strikes is reduced by roughly 16 percentage points for these persons.
It is planned to construct an operational demonstrator within the four years to come and then test it in practice. The Federal Ministry of Labour and Social Affairs is funding the project with around EUR 1.5 million. (mex)
In close partnership with society, KIT develops solutions for urgent challenges – from climate change, energy transition and sustainable use of natural resources to artificial intelligence, sovereignty and an aging population. As The University in the Helmholtz Association, KIT unites scientific excellence from insight to application-driven research under one roof – and is thus in a unique position to drive this transformation. As a University of Excellence, KIT offers its more than 10,000 employees and 23,000 students outstanding opportunities to shape a sustainable and resilient future. KIT – Science for Impact.