Colour-agnostic shape-based 3D fruit detection for crop harvesting robots

Ehud Barnea, Rotem Mairon, Ohad Ben-Shahar

Research output: Contribution to journalArticlepeer-review

86 Scopus citations

Abstract

Most agricultural robots, fruit harvesting systems in particular, use computer vision to detect their fruit targets. Exploiting the uniqueness of fruit colour amidst the foliage, almost all of these computer vision systems rely on colour features to identify the fruit in the image. However, often the colour of fruit cannot be discriminated from its background, especially under unstable illumination conditions, thus rendering the detection and segmentation of the target highly sensitive or unfeasible in colour space. While multispectral signals, especially those outside the visible spectrum, may alleviate this difficulty, simpler, cheaper, and more accessible solutions are desired. Here exploiting both RGB and range data to analyse shape-related features of objects both in the image plane and 3D space is proposed. In particular, 3D surface normal features, 3D plane-reflective symmetry, and image plane highlights from elliptic surface points are combined to provide shape-based detection of fruits in 3D space regardless of their colour. Results are shown using a particularly challenging sweet pepper dataset with a significant degree of occlusions.

Original languageEnglish
Pages (from-to)57-70
Number of pages14
JournalBiosystems Engineering
Volume146
DOIs
StatePublished - 1 Jan 2016

Keywords

  • Agrobotics
  • Green sweet pepper
  • Highlights
  • RGB-D
  • Shape
  • Symmetry

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Food Science
  • Animal Science and Zoology
  • Agronomy and Crop Science
  • Soil Science

Fingerprint

Dive into the research topics of 'Colour-agnostic shape-based 3D fruit detection for crop harvesting robots'. Together they form a unique fingerprint.

Cite this