Segmentation and motion parameter estimation for robotic Medjoul-date thinning

Tal Shoshan, Avital Bechar, Yuval Cohen, Avraham Sadowsky, Sigal Berman

Research output: Contribution to journalArticlepeer-review

1 Scopus citations

Abstract

Laborious fruit thinning is required for attaining high-quality Medjoul dates. Thinning automation can significantly reduce labor and improve efficiency. An image processing apparatus developed for robotic Medjoul thinning is presented. Instance segmentation based on Mask R-CNN was applied to identify the fruit bunch components: spikelets and rachis. Motion planning parameters were extracted using the derived masks: rachis center point (RCP), rachis orientation angle, and spikelets remaining length. RCP and rachis orientation angle were computed geometrically, spikelets remaining length was estimated with a convolutional neural network (CNN) and a deep neural network (DNN). Instance segmentation results were accurate, especially for spikelets, for low intersection over union (IoU) (0.3 IoU, fruit determined for thinning identification, spikelets: 98%, rachises: 73%). However, only 66% of the rachises were correctly matched to spikelets. The segmentation of all spikelets and rachises in the images was of medium quality for low IoU (0.3 IoU, F1, spikelets: 0.67, rachis: 0.77), where both precision and recall dropped for higher IoUs. RCP and rachis orientation angle were accurately estimated (0.3 IoU, error, RCP: 2.2 cm, rachis orientation angle: 5.0°). Spikelets remaining length estimation using CNN resulted in better performance than DNN (0.3 IoU, error, CNN: 19.7%, DNN: 24.6%). Spikelets segmentation results are suitable for thinning automation. However, rachis segmentation and matching the rachis and spikelets may still require human intervention during run-time. RCP and rachis orientation angle estimation errors are acceptable, while spikelets remaining length estimation errors are acceptable only for preliminary motion planning and mandate additional tuning during motion execution.

Original languageEnglish
Pages (from-to)514-537
Number of pages24
JournalPrecision Agriculture
Volume23
Issue number2
DOIs
StatePublished - 1 Apr 2022

Keywords

  • Deep neural networks
  • Image processing
  • Medjoul-dates
  • Thinning automation

ASJC Scopus subject areas

  • Agricultural and Biological Sciences (all)

Fingerprint

Dive into the research topics of 'Segmentation and motion parameter estimation for robotic Medjoul-date thinning'. Together they form a unique fingerprint.

Cite this