In the face of rising global agricultural labor costs and an aging workforce, a new review published in Engineering explores how agricultural robots are leveraging advanced hand–eye coordination technologies to tackle complex agronomic tasks traditionally performed by humans. The paper, titled “Advance on Agricultural Robot Hand–Eye Coordination for Agronomic Task: A Review,” examines the latest developments in hand–eye coordination systems for agricultural robots, highlighting their configurations, principles, applications, and the challenges that remain.

The study underscores the urgent need for robotic solutions in agriculture, driven by the projected doubling of global food demand by mid-century, coupled with a decline in available agricultural labor. While mechanization rates for staple crops like corn, wheat, and rice in China have surpassed 80%, tasks such as fruit and vegetable harvesting, weed management, and plant pruning remain heavily reliant on manual labor, with labor costs accounting for over 42% of total production costs. This has spurred the development of agricultural robots capable of autonomous perception, decision-making, and execution.

The review delves into various hand–eye coordination configurations used in agricultural robots, including eye-in-hand, eye-to-hand, and multi-eye/arm systems. Eye-in-hand systems, where the camera is mounted on the robot arm, offer flexibility in viewing angles and are ideal for tasks requiring detailed manipulation, such as strawberry harvesting. In contrast, eye-to-hand systems, with cameras fixed on the robot's frame, provide a broader, more stable view and are suited for tasks like selective weeding and pollination. The integration of multiple cameras and arms further enhances the robot's ability to perceive and manipulate targets in complex environments.

Hand–eye relationship calibration is identified as a critical factor in ensuring the accuracy and efficiency of robotic operations. Offline calibration methods, which establish the spatial relationship between the camera and the robot arm before operation, are widely used but can be disrupted by environmental factors such as vibrations and impacts. Online calibration techniques, which allow real-time adjustment of the hand–eye relationship, are emerging as a solution to maintain accuracy in dynamic conditions.

The review also highlights the application of hand–eye coordination in target perception and manipulation. Passive perception methods, where the camera's position is fixed, are effective for tasks involving evenly distributed targets with minimal occlusion. Active perception, which involves adjusting the camera's position to optimize perception, is essential for tasks requiring detailed inspection of discretely distributed or occluded targets, such as harvesting tomatoes hidden behind leaves.

In terms of target handling, the paper discusses single-target handling strategies, which tailor the robot's operational posture to the specific characteristics of each target, and multi-target handling, which focuses on efficiency by planning the sequence and trajectory for multiple targets simultaneously. Collision-free handle is another key area of research, with active obstacle avoidance techniques using sensors and advanced algorithms to guide the robot arm around obstacles.

Despite significant advancements, challenges remain. Maintaining stable hand–eye relations in complex environments, ensuring collision-free handle, and replicating the finesse of human manual skills are identified as key hurdles. Future trends, however, offer promise. End-to-end control strategies that map visual data directly to robotic actions, integration of machinery and agronomy to simplify robotic tasks, and skill transfer from humans to robots through learning algorithms are highlighted as potential pathways forward.

The review concludes that while agricultural robots equipped with advanced hand–eye coordination systems are making inroads into complex agronomic tasks, continued innovation is needed to overcome existing challenges and fully realize the potential of robotic technology in agriculture.

The paper “Advance on Agricultural Robot Hand–Eye Coordination for Agronomic Task: A Review,” is authored by Liang He, Yuhuan Sun, Liping Chen, Qingchun Feng, Yajun Li, Jiewen Lin, Yicheng Qiao, Chunjiang Zhao. Full text of the open access paper: https://doi.org/10.1016/j.eng.2025.01.022. For more information about Engineering, visit the website at https://www.sciencedirect.com/journal/engineering.