Hand-Object Interaction: From Human Demonstrations to Robot Manipulation

Alessandro Carfì, Timothy Patten, Yingyi Kuang, Ali Hammoud, Mohamad Alameh, Elisa Maiettini, Abraham Itzhak Weinberg, Diego Faria, Fulvio Mastrogiovanni, Guillem Alenyà, Lorenzo Natale, Véronique Perdereau, Markus Vincze, Aude Billard

Research output: Contribution to journalArticlepeer-review

6 Scopus citations


Human-object interaction is of great relevance for robots to operate in human environments. However, state-of-the-art robotic hands are far from replicating humans skills. It is, therefore, essential to study how humans use their hands to develop similar robotic capabilities. This article presents a deep dive into hand-object interaction and human demonstrations, highlighting the main challenges in this research area and suggesting desirable future developments. To this extent, the article presents a general definition of the hand-object interaction problem together with a concise review for each of the main subproblems involved, namely: sensing, perception, and learning. Furthermore, the article discusses the interplay between these subproblems and describes how their interaction in learning from demonstration contributes to the success of robot manipulation. In this way, the article provides a broad overview of the interdisciplinary approaches necessary for a robotic system to learn new manipulation skills by observing human behavior in the real world.

Original languageEnglish
Article number714023
JournalFrontiers in Robotics and AI
StatePublished - 1 Oct 2021
Externally publishedYes


  • anthropomorphic hands
  • data extraction
  • grasping
  • hand-object interaction
  • imitation learning
  • learning from demonstration
  • manipulation
  • transfer learning

ASJC Scopus subject areas

  • Computer Science Applications
  • Artificial Intelligence


Dive into the research topics of 'Hand-Object Interaction: From Human Demonstrations to Robot Manipulation'. Together they form a unique fingerprint.

Cite this