Work

Personalized Cinemagraphs Using Semantic Understanding and Collaborative Learning

Year: 2017

Type: conference-paper

Abstract: Cinemagraphs are a compelling way to convey dynamic aspects of a scene. In these media, dynamic and still elements are juxtaposed to create an artistic and narrative experience. Creating a high-quality, aesthetically pleasing cinemagraph requires isolating objects in a semantically meaningful way and then selecting good start times and looping periods for those objects to minimize visual artifacts (such a tearing). To achieve this, we present a new technique that uses object recognition and semantic segmentation as part of an optimization method to automatically create cinemagraphs from videos that are both visually appealing and semantically meaningful. Given a scene with multiple objects, there are many cinemagraphs one could create. Our method evaluates these multiple candidates and presents the best one, as determined by a model trained to predict human preferences in a collaborative way. We demonstrate the effectiveness of our approach with multiple results and a user study.

Source: IEEE/CVF International Conference on Computer Vision (ICCV)

DOI: 10.1109/iccv.2017.552

Authors: Tae-Hyun Oh, Kyungdon Joo, Neel Joshi, Baoyuan Wang, In So Kweon, Sing Bing Kang

Institutions: Korea Advanced Institute of Science and Technology, MIT Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Microsoft (United States)

Language: English

Topic: Generative Adversarial Networks and Image Synthesis

Subfield: Computer Vision and Pattern Recognition

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funders: Microsoft Research, Ministry of Trade, Industry and Energy, Korea Advanced Institute of Science and Technology

Award: G1580050455