Real-Time CNN-Based Detection System for an Autonomous Agricultural Robot in Open-Field Conditions

Abstract

The automation of weed control is critical for precision agriculture, yet its implementation on autonomous field robots faces significant challenges due to limited onboard computational resources and dynamic environmental conditions. This study presents the development and evaluation of a real-time computer vision system for an autonomous agricultural robot (agrobot) designed for precision crop monitoring. The proposed system employs an "inverted" detection strategy, where a neural network identifies target crop plants (tobacco) to allow for the subsequent localization of weeds as undetected green mass. The hardware platform is centered on the energy-efficient NVIDIA Jetson Orin Nano, integrated with a GNSS RTK receiver for centimeter-level geotagging, all orchestrated within a ROS 2 framework. To identify an optimal balance between detection accuracy and inference speed under severe hardware constraints, a comparative analysis of various convolutional neural network (CNN) architectures, including YOLOv5 and YOLOv8 variants, was conducted. Experimental results demonstrated that while larger models like YOLOv8x offered higher theoretical accuracy (mAP@0.5 0.86), their inference time (97 ms) was prohibitive for real-time operation. The YOLOv8m model was identified as the optimal compromise, achieving a high frame rate (18 FPS, 55.6 ms inference) with robust detection quality (mAP@0.5 0.84, Recall 0.79). This performance enables the robot to process a continuous video stream at speeds up to 2 m/s, generating precise, georeferenced maps of crop locations. The trained model's efficacy was validated on field data, confirming the system's practical viability for guiding targeted interventions and contributing to significant reductions in herbicide use. The study demonstrates a scalable and efficient approach for deploying sophisticated AI on embedded hardware for precision agriculture.