Work

ImmerScope: Multi-view Video Aggregation at Edge towards Immersive Content Services

Year: 2024

Type: conference-paper

Abstract: The multi-camera capture system is an emerging visual sensing modality. It facilitates the production of various immersive contents ranging from regular to neural videos. Although the delivery of immersive content is popular and promising, it suffers from the bandwidth bottleneck when streaming multi-view videos to the cloud (i.e., multi-view video aggregation). Existing works fail to provide a bandwidth-efficient and content-generic solution. Even the closest effort to ours based on the SOTA multi-view video codecs suffers from issues of underutilized dependency and content distortion. In this paper, we present ImmerScope, a multi-view video aggregation framework at the edge with a neural multi-view video codec. It outperforms existing solutions with highly-utilized dependency via neuron connections and distortion awareness via end-to-end training. Evaluations on diverse multi-camera setups show that ImmerScope outperforms single-view codecs by at least 64% bandwidth savings in peak-signal-to-noise ratio with a frame rate of 50 fps.

Source: ACM Conference on Embedded Networked Sensor Systems (SenSys)

DOI: 10.1145/3666025.3699324

Authors: Bo Chen, Hongpeng Guo, Mingyuan Wu, Zhe Yang, Zhisheng Yan, Klara Nahrstedt

Institutions: University of Illinois Urbana-Champaign, George Mason University

Language: English

Topic: Video Coding and Compression Technologies

Subfield: Signal Processing

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Funder: National Science Foundation