Optimal hand gesture vocabulary design using psycho-physiological and technical factors

A global approach to hand gesture vocabulary design is proposed which includes human as well as technical design factors. The human centered desires of multiple users are implicitly represented through indices obtained from ergonomic studies representing the psychophysiological aspects of users. The main technical aspect considered is that of machine recognition of gestures. We review and classify three approaches to this problem: Ad hoc, Rule based, and Analytical. It is believed that this is the first conceptualization of the optimal hand gesture design problem in analytical form. A mathematical dual objective model is developed, which reflects the psychophysiological and technical performance measures upon which a gesture control system is judged. The mathematical program solves a quadratic assignment problem embedded within a heuristic search tree. The quadratic problem, whose solution is a gesture vocabulary GV, (a command-gesture matching) is solved through simulated annealing. A useful feature, included in the formulation, is the priorities given to the matching of complementary pairs of gestures (say thumb up - thumb down) to complementary pairs of commands (say up down). To validate the procedure an example is solved for the design of a medium size robot command GV.