Speaker
Description
The disassembly of complex technical systems is still predominantly performed manually, as the required process knowledge—such as action sequences, grasping positions, and handling strategies—is difficult to formalize and implement. Observing human workers during task execution offers a promising approach to automatically extract this knowledge. This is particularly relevant in domains such as the disassembly of patch panels on detector interfaces in the CMS experiment, where each sub-detector contains 20 to 30 panels and around ten sub-detectors must be handled. Learning from a limited number of human demonstrations can therefore enable the replication of such processes by robotic systems.
This work presents part of a Learning from Demonstration (LfD) framework which aims to transfer process knowledge from human workers to robotic systems, enabling non-experts to teach robots disassembly tasks. For this purpose, a markerless Optical Motion Capture (OMC) pipeline is developed to extract skeletal hand tracking data in real-time. This data is processed by a Recurrent Neural Network (RNN) to identify and parameterize the operations performed by the worker. It is shown that a virtual scene representation, including object poses, can be dynamically maintained solely based on hand motion. Furthermore, the results indicate that hand joint angles can be used to infer the type of object being grasped. Future work will leverage the extracted information to generate flexible task plans and enable robotic systems to autonomously replicate demonstrated disassembly processes.