Apple detection in natural tree canopies from multimodal images

In this work we develop a real time system that recognizes occluded green apples within a tree canopy using infra-red and color images in order to achieve automated harvesting. Infra-red provides clues regarding the physical structure and location of the apples based on their temperature (leaves accumulate less heat and radiate faster than apples), while color images provide evidence of circular shape. Initially the optimal registration parameters are obtained using maximization of mutual information. Haar features are then applied separately to color and infra-red images through a process called Boosting, to detect apples from the background. A contribution reported in this work, is the voting scheme added to the output of the RGB Haar detector which reduces false alarms without affecting the recognition rate. The resulting classifiers alone can partially recognize the on-trees apples however when combined together the recognition accuracy is increased.