DESIGNING HAND GESTURE VOCABULARIES for NATURAL INTERACTION by COMBINING PSYCHO-PHYSIOLOGICAL and RECOGNITION FACTORS

Helman I. Stern, Juan P. Wachs, Yael Edan

Research output: Contribution to journalArticlepeer-review

23 Scopus citations

Abstract

A need exists for intuitive hand gesture machine interaction in which the machine not only recognizes gestures, but also the human feels comfortable and natural in their execution. The gesture vocabulary design problem is rigorously formulated as a multi-objective optimization problem. Psycho-physiological measures (intuitiveness, comfort) and gesture recognition accuracy are taken as the multi-objective factors. The hand gestures are static and recognized by a vision based fuzzy c-means classifier. A meta-heuristic approach decomposes the problem into two sub-problems: finding the subsets of gestures that meet a minimal accuracy requirement, and matching gestures to commands to maximize the human factors objective. The result is a set of Pareto optimal solutions in which no objective may be increased without a concomitant decrease in another. Several solutions from the Pareto set are selected by the user using prioritized objectives. Software programs are developed to automate the collection of intuitive and stress indices. The method is tested for a simulated car - maze navigation task. Validation tests were conducted to substantiate the claim that solutions that maximize intuitiveness, comfort, and recognition accuracy performance measures can be used as proxies for the minimization task time objective. Learning and memorability were also tested.

Original languageEnglish
Pages (from-to)137-160
Number of pages24
JournalInternational Journal of Semantic Computing
Volume2
Issue number1
DOIs
StatePublished - 1 Mar 2008

Keywords

  • Hand gesture vocabulary design
  • comfort
  • gesture interfaces
  • hand gesture recognition
  • human-computer interaction
  • intuitiveness
  • learning
  • memory
  • multiobjective optimization
  • psycho-physiological
  • semantic behavior

ASJC Scopus subject areas

  • Software
  • Information Systems
  • Linguistics and Language
  • Computer Science Applications
  • Computer Networks and Communications
  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'DESIGNING HAND GESTURE VOCABULARIES for NATURAL INTERACTION by COMBINING PSYCHO-PHYSIOLOGICAL and RECOGNITION FACTORS'. Together they form a unique fingerprint.

Cite this