Researchers at Osaka Metropolitan University have developed a method for creating realistic virtual tomato farms that automatically generate data for training agricultural AI systems. Their approach offers a way to overcome one of the most labor-intensive tasks in farming: harvesting the crops.

Currently, farmbots use object detection systems to locate tomatoes and artificial intelligence to decide whether they are ripe. However, the use of these systems in the field has been bottlenecked by difficulty training them.

AI systems require large amounts of labeled images, which must be gathered from real farms. The process is time consuming as each tomato has to be manually labeled by drawing bounding boxes and assigned a ripeness category. This process is further complicated by natural variations in lighting conditions, plant shapes, and growing environments between each farm and season.

To address this challenge, a research team led by Takuya Fujinaga of the Graduate School of Engineering, Osaka Metropolitan University, created a virtual agricultural environment, which automatically generates realistic tomato images and their corresponding AI training labels.

To make a virtual farm that closely resembled real conditions, the environment was reconstructed using images manually obtained from camera data collected by agricultural robots.

The team used advanced reconstruction methods to build detailed 3D models and Unreal Engine 5 software together with an emerging reconstruction technique known as 3D Gaussian Splatting to reproduce lighting, textures, and geometry. Their virtual agricultural environment created a simulation closer to real-world situations where leaves overlap tomatoes, lighting changes constantly, stems and shadows create clutter, and fruit are often partially hidden.

Using positional information, the system automatically generated labels showing where tomatoes appeared in each image and how ripe they were. The framework also automatically exported annotations in YOLO format, a widely used standard for AI object detection training.

The researchers used the synthetic datasets to train AI models and showed that they could effectively detect tomatoes in real-world images.

“By comparing differences in the shape of 3D tomato models, lighting conditions, and the amount of data, we identified conditions that affect AI accuracy,” Dr. Fujinaga said. “Understanding how lighting, tomato shape, and dataset size affect detection performance are important discoveries for improving the model in the future.”

“Although we studied tomatoes, similar factors are important in harvesting a wide range of agricultural products,” he added. “We are definitely excited about the potential of applying this method to crops other than tomatoes.”

The findings were published in Smart Agricultural Technology.

Declaration of competing interest
The researchers declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

###

About OMU
Established in Osaka as one of the largest public universities in Japan, Osaka Metropolitan University is committed to shaping the future of society through “Convergence of Knowledge” and the promotion of world-class research. For more research news, visit https://www.omu.ac.jp/en/ and follow us on social media: X, Instagram, LinkedIn.