Vision-Based Human-Machine Interface for an Assistive Robotic Exoskeleton Glove

Abstract This paper presents a vision-based Human-Machine Interface (HMI) for an assistive exoskeleton glove, designed to incorporate force planning capabilities. While Electroencephalogram (EEG) and Electromyography (EMG)-based HMIs allow direct grasp force planning via user signals, voice and vision-based HMIs face limitations. In particular, two primary force planning methods encounter issues in these HMIs. First, traditional force optimization struggles with unfamiliar objects due to lack of object information. Second, the slip-grasp method faces a high failure rate due to inadequate initial grasp force. To address these challenges, this paper introduces a vision-based HMI to estimate the initial grasp forces of the target object. The initial grasp force estimation is performed based on the size and surface material of the target object. The experimental results demonstrate a grasp success rate of 87. 5%, marking significant improvements over the slip-grasp method (71.9%).