Large-scale multimodal sensor fusion and object detection

Abstract

The large-scale multimodal sensor fusion and moving object detection for visualization, assessment and automation of industrial processes were studied in this thesis. The overview and analysis of the commonly-used calibration approaches, the existing registration methods suitable for large-scale 2D-to-2D and 2D-to-3D registration, and the segmentation methods suitable for point cloud data were made. Based on the analysis results the fusion of three imaging modalities of different nature such as a point cloud obtained from LiDAR, a set of RGB images, and a set of thermal images was performed. A large-scale calibration target was developed that is visible in all modalities for calibration purposes. The moving object detection task from sequential point cloud data was solved using the plane-based segmentation, clustering, VFH cluster descriptors, and estimation of the clusters proximal shifts with a distance metric. In the experiments, all moving objects were detected.