The new "Avikom" research project is innovating an intelligent, assistive audiovisual system for workers in the fields of manufacturing and logistics. The system is designed to provide assistance via a visual display with spoken instructions, which could also be useful for providing on-the-job training. The smart glasses, with a specialized headset, provide information precisely at the time it is needed. Bielefeld University and the University of Applied Sciences Bielefeld are working on the project with four regional companies: the v. Bodelschwinghsch Bethel Foundation, the Handelskammer Ostwestfalen zu Bielefeld (Chamber of Industry and Commerce Bielefeld, Ostwestfalen) and two trade associations. The Bundesministerium für Bildung und Forschung (Federal Ministry for Education and Research, BMBF) and the European Social Fund (ESF) are funding the new research with a total of 2.4 million Euro through April 2022.
"In modern assembly and logistics processes, most work today is still largely done by hand. The new system is meant to help with these kinds of tasks," says Professor Dr. Thomas Schack, who heads the "Neurocognition and Action - Biomechanics" research group at Bielefeld University. Schack is also the coordinator of Avikom. His research group is part of Bielefeld University's Faculty of Psychology and Sports Science and is also a participating member of the Cluster of Excellence CITEC. "Nowadays, it is frequently the case that individually customized products are ordered and then produced as special custom-made assemblies and small series production runs. People who work assembling one product to the next therefore have to be skilled at many different processes. They usually get printed instructions, leaving just one hand free for the assembly itself. This is cumbersome and unproductive because the worker has to concentrate both on reading the instructions and actually doing the assembly. By contrast, the Avikom glasses provide anticipatory, customized assistance."
A Cognitive and Mobile Assistive System
The Avikom glasses use augmented reality to display supplementary information directly in the user's field of vision. In addition to this, the researchers combine the glasses with a pair of intelligent headphones fitted with an integrated microphone (the so called "Headset for Augmented Auditive Reality, HEA²R), which was developed by a start-up project at the Institute for System Dynamics and Mechatronics (ISyM) at the University of Applied Sciences Bielefeld. Via this device, the Avikom system can speak to the user, similar to how a navigation system does. "Employees in loud production areas can also talk to each other via the device without being disrupted by ambient noise," says Professor Dr. Joachim Waßmuth, of ISyM. "The system is therefore equipped with an intelligent procedure for noise cancellation."
Avikom stands for "Audiovisuelle Unterstützung durch ein kognitives und mobiles Assistenzsystem" (in English, "Audiovisual Support with a Cognitive and Mobile Assistive System"). "What makes our assistive system unique is the fact that it doesn't just give instructions. It recognizes the person using it and understands the situation at hand by recognizing individual objects and the steps to be performed. Based on this, it then provides assistance," says Thomas Schack. In order for the system to be able to adjust to the individual user, the worker's skills are assessed in advance using software-based diagnostics. The software is designed to make a predictive 'diagnosis' of the difficulties the person will likely face when performing different work procedures. Based on this diagnostic assessment, the system can then give customized tips and instructions that provide the worker with a focused and motivating approach. "The Avikom system thus offers an excellent opportunity to adapt technical support to the individual needs of workers," says Professor Dr. Günther Maier, a member of Bielefeld University's Faculty of Psychology and Sports Science. His research group "Work and Organizational Psychology" is also participating in the project.
Application Scenarios
Avikom researchers are investigating various scenarios where the system can be used at the four companies involved in the project, and testing on the system is carried out workers at these companies.
- Avikom could be used to take work off the hands of instructors, enabling apprentices and trainees to familiarize themselves with work procedures on the shop floor using the smart glasses for assistance.
- Another potential application is remote support for repairing machines. A mechanic, who might not have the specialized knowledge to carry out a repair, could get assistance from an expert via the smart glasses. And the expert does not have to travel to the site: instead, they can look through the mechanic's glasses, making comments via the headphones and displaying tips in the visual field. The expert could, for example, use an arrow to point to a potentially defective component.
- In warehouse logistics, the glasses could help to ensure, for instance, that workers no longer receive printed work papers, but are told (via visual or audio input) what materials have to be commissioned. The glasses could then help navigate the worker through the warehouse to the items they need to gather, so that they don't have to keep going back to a central computer.
Professor Nancy Ip: Pioneering New Paths in Neurodegenerative Therapy