Work

CloudUP—Upsampling Vibrant Color Point Clouds Using Multi-Scale Spatial Attention

Year: 2023

Type: article

Abstract: In recent years, there has been a noticeable increase in the inclination towards digitizing our surroundings, encompassing various domains such as virtual reality, cultural heritage conservation, and architectural representation. The computation of high-resolution three-dimensional (3D) colored point clouds and meshes holds significant importance for such applications. However, traditional structure-from-motion (SfM) techniques may produce sparse 3D point clouds when low-resolution input images are used, resulting in a low-quality mesh generation. Traditional point cloud upsampling techniques that improve the 3D point cloud resolution typically work on LiDAR-generated point clouds devoid of color information. Furthermore, most learned point cloud upsampling techniques compute graph features that capture local information by identifying a local neighborhood in a limited region around a point and hence may result in sub-optimal representation. To address these limitations, we proposeCloudUP, a colored 3D point cloud upsampling approach that utilizes multi-scale spatial attention. Specifically, we design a novel across-scale attention mechanism for computing Multi-scale Point-Cloud Features (MPF) that capture 3D shape information of the point with respect to its neighborhood. We further extract spatial neighborhood-guided color features used to predict the color for the upsampled points. The color prediction is trained with a content-preserving loss function that aims to maintain intricate details and vivid colors. Our color refinement pipeline is guided by a vibrant colored dataset (collected by us) to assist in preserving the 3D contents.

Source: IEEE Access

DOI: 10.1109/access.2023.3332141