Single frame targetless lidar-camera calibration using region of interest based edge features

Abstract

With the increasing adoption of autonomous systems in both autonomous driving and robotics, accurate perception and sensor integration have become critical for safe and reliable operation. Among multi-sensor configurations, the extrinsic calibration between LiDAR and camera sensors plays a fundamental role in sensor fusion, object detection, and environment perception. This thesis addresses the LiDAR–camera extrinsic calibration problem and proposes a targetless, single-frame calibration framework based on region-of-interest selection, designed to operate without calibration targets and to support online calibration scenarios. The proposed framework builds upon edge-based calibration principles commonly used in the literature, but extends them through systematic integration of ROI-based data selection and multiple optimization strategies. Both manual and automatic ROI selection schemes are investigated. In the manual approach, a single rectangular ROI is defined by the user, whereas in the automatic approach, multiple ROIs are generated using YOLOv7-X-based object detection. Only ROIs containing reliable projected LiDAR points are retained, and in the multi-ROI case, the overall cost is computed as the average of ROI-level costs. LiDAR point clouds and camera images are processed in a time-synchronized manner. LiDAR edge points are extracted based on depth discontinuities along scan lines, while camera images are processed through a multi-stage pipeline to produce a continuous edginess map that assigns higher values to pixels closer to image edges. A scalar cost function is defined by measuring the alignment between projected LiDAR edge points and image edginess values within the selected ROIs, where each projection contributes proportionally to the product of LiDAR depth discontinuity and image edginess. Two optimization strategies are employed to refine the extrinsic parameters. The first is a greedy grid-search method based on discrete neighborhood evaluation. The second is a deterministic Particle Swarm Optimization algorithm using time-varying inertia and acceleration coefficients. To ensure reproducibility and computational efficiency, deterministic seeding and multi-threaded particle evaluation are applied. In addition, a calibration validity assessment mechanism is introduced to detect incorrect calibrations by analyzing cost distributions under small perturbations around the estimated solution. Experiments conducted on the KITTI dataset demonstrate that ROI-based processing significantly reduces computational cost while improving calibration robustness. Automatic ROI selection reduces the average per-iteration computation time by approximately twenty-five to thirty-two percent across all translation axes, achieving speedup factors of up to one point seven times. In terms of accuracy, ROI-based methods consistently outperform the baseline approach, achieving sub-centimeter translation errors and sub-degree rotation errors under small and moderate perturbations. Automatic ROI selection combined with Particle Swarm Optimization is particularly effective in stabilizing rotational estimates under translation and angular perturbations by exploiting geometric constraints derived from multiple scene objects. Overall, the results confirm that the proposed ROI-supported, optimization-based, single-frame LiDAR–camera extrinsic calibration framework provides a computationally efficient, accurate, and stable solution suitable for both offline calibration and online calibration update scenarios in autonomous driving systems.