This review article analyzes state-of-the-art and future perspectives for harvesting robots in high-value crops. The objectives were to characterize the crop environment relevant for robotic harvesting, to perform a literature review on the state-of-the-art of harvesting robots using quantitative measures, and to reflect on the crop environment and literature review to formulate challenges and directions for future research and development. Harvesting robots were reviewed regarding the crop harvested in a production environment, performance indicators, design process techniques used, hardware design decisions, and algorithm characteristics. On average, localization success was 85%, detachment success was 75%, harvest success was 66%, fruit damage was 5%, peduncle damage was 45%, and cycle time was 33 s. A kiwi harvesting robot achieved the shortest cycle time of 1 s. Moreover, the performance of harvesting robots did not improve in the past three decades, and none of these 50 robots was commercialized. Four future challenges with R&D directions were identified to realize a positive trend in performance and to successfully implement harvesting robots in practice: (1) simplifying the task, (2) enhancing the robot, (3) defining requirements and measuring performance, and (4) considering additional requirements for successful implementation. This review article may provide new directions for future automation projects in high-value crops.