Get started
Work

Convolutional and attention mechanisms for 3D object classification using panoramas and beyond

Year: 2025
Type: dissertation
Abstract: Advancements in 3D data acquisition technologies and CAD software have increased the availability of three-dimensional (3D) data, which provides detailed geometric, shape, and scale information with applications in fields like remote sensing, autonomous driving, robotics, and medicine. However, adapting deep learning models for 3D data is challenging due to the complexity of 3D shapes and high computational demands. A common solution is to convert 3D data into 2D images for compatibility with established 2D convolutional neural networks (CNNs). In this Thesis, we convert 3D objects represented as meshes into omnidirectional (spherical) images projected onto a 2D multi-channel plane using equirectangular projection (ERP). However, ERPs introduce non-uniform sampling, with polar regions being more densely sampled than the equator, leading to distortions. Traditional CNNs, originally designed for undistorted perspective images, are not well suited for ERP images. To address this, we propose convolutional modules based on horizontally dilated convolutions (HDCs) with shared weights, adapting the convolutional kernels to the spherical surface and mitigating distortion issues. The first module combines multiple HDCs to produce multi-scale feature maps, while the second one integrates multiple features using row-wise weights to better handle different distortion levels. Additionally, we designed an attention mechanism that incorporates spatial relationships to better guide networks to capture long-range dependencies uniformly across the spherical surface. These modules, integrated into established backbones, are evaluated on 3D object classification and gravity alignment tasks. Compared to the literature, we obtained state-of-the-art results in both applications.
Language: English
Open Access status: green
OpenAlex ID: W7155656345
Cites:
Cited by:
Related to: