An Efficient Camera-LiDAR-Based 3D Human Detection and Tracking Algorithm for Socially Aware Mobile Robot Navigation Systems

Abstract

Reliable real-time perception of surrounding humans is essential for mobile robots operating in dynamic human-populated environments. However, maintaining stable human identities and accurate 3D positions in real time remains challenging, especially under sensor noise, partial occlusion, and onboard computational constraints. This paper presents a camera–LiDAR-based framework for 3D human detection and tracking in socially aware mobile robot navigation. The method combines RGB images from an Azure Kinect camera with 3D point clouds from a Velodyne LiDAR. Humans are first detected and tracked in the image domain using YOLOv8 and BoT-SORT, which provide 2D bounding boxes and persistent identity labels. LiDAR points are then transformed into the camera frame, projected onto the image plane, and associated with the tracked detections. To reduce the influence of background points, Euclidean clustering is applied to the associated point sets, and the selected clusters are used to estimate human positions in 3D space. A Kalman filter is further maintained for each tracked individual to smooth the estimated positions and improve temporal consistency. The resulting 3D human states are integrated into a ROS-based mobile robot navigation framework, allowing the robot to perceive nearby humans and adjust its motion accordingly. Experiments conducted on a real mobile robot platform in several indoor scenarios show that the system provides stable 3D human tracking, supporting its practical use in socially aware navigation tasks.