Work

Learning Correspondence From the Cycle-Consistency of Time

Year: 2019

Type: conference-paper

Abstract: We introduce a self-supervised method for learning visual correspondence from unlabeled video. The main idea is to use cycle-consistency in time as free supervisory signal for learning visual representations from scratch. At training time, our model learns a feature map representation to be useful for performing cycle-consistent tracking. At test time, we use the acquired representation to find nearest neighbors across space and time. We demonstrate the generalizability of the representation -- without finetuning -- across a range of visual correspondence tasks, including video object segmentation, keypoint tracking, and optical flow. Our approach outperforms previous self-supervised methods and performs competitively with strongly supervised methods.

Source: IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings

DOI: 10.1109/cvpr.2019.00267

Authors: Xiaolong Wang, Allan Jabri, Alexei A. Efros

Institutions: Carnegie Mellon University, University of California, Berkeley

Language: English

Topic: Video Surveillance and Tracking Methods

Subfield: Computer Vision and Pattern Recognition

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funder: National Science Foundation

Award: BIGDATA: F: Collaborative Research: From Visual Data to Visual Understanding