Work

Real-time multimodal fusion and semantic mapping for robotic tower crane perception

Year: 2026

Type: article

Abstract: Robotic tower crane operation requires real-time perception of complex and rapidly changing construction environments. Conventional Simultaneous Localization and Mapping (SLAM) methods assume smooth sensor motion and emphasize geometry over semantics, limiting their suitability for crane-mounted sensing affected by vibration, rotation, and intermittent movement. This research proposes a multimodal perception framework that integrates Light Detection and Ranging (LiDAR), camera, and Inertial Measurement Unit (IMU) data within a tightly coupled fusion and semantic reconstruction pipeline. A Mahony-filter-based attitude optimization module stabilizes high-frequency vibrations, while a Fast LiDAR-Inertial Odometry (FAST-LIVO2)-inspired LiDAR–visual–inertial fusion strategy achieves centimeter-level three-dimensional (3D) mapping. To enhance scene understanding, an improved Random Sampled and Lightweight Aggregated Network (RandLA-Net) jointly exploits geometric and visual cues for point-level semantic segmentation, with color-aware spatial encoding. Field deployment on an operational tower crane demonstrates superior performance, yielding the lowest global reconstruction errors and highest semantic accuracy. The framework provides a robust perception foundation for autonomous planning, safety monitoring, and intelligent lifting assistance.

Source: Advanced Engineering Informatics

DOI: 10.1016/j.aei.2026.104373

Authors: Yifan Lu, Xiuzhi Deng, Peter E.D. Love, Wei Zhou, Weili Fang

Institutions: Huazhong University of Science and Technology, Curtin University

Language: English

Topic: Robotics and Sensor-Based Localization

Subfield: Aerospace Engineering

Field: Engineering

Domain: Physical Sciences

SDG: Industry, innovation and infrastructure

Open Access status: hybrid

Estimated APC paid: $3,560