Toward realistic hands gesture interface: Keeping it simple for developers and machines

Eyal Krupka, Daniel Freedman, Ido Leichter, Kfir Karmon, Ilya Gurvich, Yoni Smolin, Noam Bloom, Aviv Hurvitz, Yuval Tzairi, Alon Vinnikov, Aharon Bar Hillel

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

18 Scopus citations

Abstract

Development of a rich hand-gesture-based interface is currently a tedious process, requiring expertise in computer vision and/or machine learning. We address this problem by introducing a simple language for pose and gesture description, a set of development tools for using it, and an algorithmic pipeline that recognizes it with high accuracy. The language is based on a small set of basic propositions, obtained by applying four predicate types to the fingers and to palm center: direction, relative location, finger touching and finger folding state. This enables easy development of a gesture-based interface, using coding constructs, gesture definition files or an editing GUI. The language is recognized from 3D camera input with an algorithmic pipeline composed of multiple classification/regression stages, trained on a large annotated dataset. Our experimental results indicate that the pipeline enables successful gesture recognition with a very low computational load, thus enabling a gesture-based interface on low-end processors.

Original languageEnglish
Title of host publicationCHI 2017 - Proceedings of the 2017 ACM SIGCHI Conference on Human Factors in Computing Systems
Subtitle of host publicationExplore, Innovate, Inspire
PublisherAssociation for Computing Machinery
Pages1887-1898
Number of pages12
ISBN (Electronic)9781450346559
DOIs
StatePublished - 2 May 2017
Event2017 ACM SIGCHI Conference on Human Factors in Computing Systems, CHI 2017 - Denver, United States
Duration: 6 May 201711 May 2017

Publication series

NameConference on Human Factors in Computing Systems - Proceedings
Volume2017-May

Conference

Conference2017 ACM SIGCHI Conference on Human Factors in Computing Systems, CHI 2017
Country/TerritoryUnited States
CityDenver
Period6/05/1711/05/17

Keywords

  • Hand gesture NUI development
  • Hand gesture recognition

ASJC Scopus subject areas

  • Software
  • Human-Computer Interaction
  • Computer Graphics and Computer-Aided Design

Fingerprint

Dive into the research topics of 'Toward realistic hands gesture interface: Keeping it simple for developers and machines'. Together they form a unique fingerprint.

Cite this