A collaborative telerobotics network framework with hand gesture interface and conflict prevention

Hand gesture control is an efficient modality for telerobot control because of gesture expressiveness and naturalness. This paper discusses a collaborative cybernetic system, where telerobots are controlled simultaneously by a group of distributed operators over the network to accomplish a task. Computer vision algorithms for hand gesture recognition are introduced to facilitate the human-robot interface. The gestures are converted into commands that are delivered to robots for dexterous task completion. Commands from multiple operators are aggregated by a collaboration protocol into a single control stream. The aggregation is updated according to operators performance. A distributed conflict and error detection-prediction network is designed and applied to a case study of collaborative control for a robotic nuclear decommissioning task. Operators use hand gestures to command telerobots to disassemble facilities in a contaminated area. The hypothesis is tested that collaborative control is more effective and less susceptible to conflicts/errors than the standard single-operator control. During collaboration, operators performed gesture commands simultaneously to control a set of robots. The system can reliably recognise operators hands with a 96% accuracy in cluttered backgrounds. Collaboration between expert and novice operators can reduce the time to complete a multi-step task by 45% on average.