Work

Low-Rank Tensor Completion with Spatio-Temporal Consistency

Year: 2014

Type: conference-paper

Abstract: Video completion is a computer vision technique to recover the missing values in video sequences by filling the unknown regions with the known information. In recent research, tensor completion, a generalization of matrix completion for higher order data, emerges as a new solution to estimate the missing information in video with the assumption that the video frames are homogenous and correlated. However, each video clip often stores the heterogeneous episodes and the correlations among all video frames are not high. Thus, the regular tenor completion methods are not suitable to recover the video missing values in practical applications. To solve this problem, we propose a novel spatially-temporally consistent tensor completion method for recovering the video missing data. Instead of minimizing the average of the trace norms of all matrices unfolded along each mode of a tensor data, we introduce a new smoothness regularization along video time direction to utilize the temporal information between consecutive video frames. Meanwhile, we also minimize the trace norm of each individual video frame to employ the spatial correlations among pixels. Different to previous tensor completion approaches, our new method can keep the spatio-temporal consistency in video and do not assume the global correlation in video frames. Thus, the proposed method can be applied to the general and practical video completion applications. Our method shows promising results in all evaluations on both 3D biomedical image sequence and video benchmark data sets.

Source: Proceedings of the AAAI Conference on Artificial Intelligence

DOI: 10.1609/aaai.v28i1.9144

Authors: Hua Wang, Feiping Nie, Heng Huang

Institutions: Colorado School of Mines, The University of Texas at Arlington

Language: English

Topic: Tensor decomposition and applications

Subfield: Computational Mathematics

Field: Mathematics

Domain: Physical Sciences

Open Access status: diamond

Funder: National Science Foundation

Awards: III: Medium: Collaborative Research: Robust Large-Scale Electronic Medical Record Data Mining Framework to Conduct Risk Stratification for Personalized Intervention, III: Small: Collaborative Research: A Large-Scale Data Mining Framework for Genome-Wide Mapping of Multi-Modal Phenotypic Biomarkers and Outcome Prediction, SCH: EXP: Collaborative Research: Privacy-Preserving Framework for Publishing Electronic Healthcare Records