Skip to main navigation Skip to search Skip to main content

Fingers See Things Differently (FIST-D): An Object Aware Visualization and Manipulation Framework Based on Tactile Observations

  • Chenxi Xiao
  • , Naveen Madapana
  • , Juan Wachs

Research output: Contribution to journal › Article › peer-review

3 Scopus citations

Abstract

Planning object manipulation policies based on tactile observations alone is a challenging task due to the multi-factorial variances in the measured point cloud (e.g. sparsity, missing regions, rotation, etc.) and the limited sensory information available through tactile sensing. Nevertheless, the mainstream grasp planners are designed for well-structured point cloud data, and lack the crucial ability to plan grasps in unexplored regions that are common during tactile sampling. Hence, it is crucial to detect the grasp regions from incomplete and unstructured tactile point cloud data. To address this limitation, we propose a novel framework that utilizes a support set of CAD models to augment the tactile observations, and thereby facilitate object recognition, visualization, and developing manipulation policies solely using tactile samples. To cope with the noise and sparsity of tactile observations, we propose uGPIS, a surface reconstruction method that utilizes the occupancy possibility function and the Gaussian Process Regression to recover the underlying surface from tactile point clouds. Then, we complete the partially observed tactile point cloud using the prior knowledge obtained from the support set of full CAD models. This prior information will provide the enriched geometric information that is crucial to determine the grasp regions. Our experimental results on a physical simulation show that our method can successfully combine the prior knowledge from the database to enhance the grasp success rate.

Original languageEnglish
Article number9372831
Pages (from-to)4249-4256
Number of pages8
JournalIEEE Robotics and Automation Letters
Volume6
Issue number3
DOIs
StatePublished - 1 Jul 2021
Externally publishedYes

Keywords

  • Deep learning in grasping and manipulation
  • force and tactile sensing
  • perception for grasping and manipulation

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Biomedical Engineering
  • Human-Computer Interaction
  • Mechanical Engineering
  • Computer Vision and Pattern Recognition
  • Computer Science Applications
  • Control and Optimization
  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'Fingers See Things Differently (FIST-D): An Object Aware Visualization and Manipulation Framework Based on Tactile Observations'. Together they form a unique fingerprint.

Cite this