Work

Open-Ended Long-form Video Question Answering via Adaptive Hierarchical Reinforced Networks

Year: 2018
Type: conference-paper
Abstract: Open-ended long-form video question answering is challenging problem in visual information retrieval, which automatically generates the natural language answer from the referenced long-form video content according to the question. However, the existing video question answering works mainly focus on the short-form video question answering, due to the lack of modeling the semantic representation of long-form video contents. In this paper, we consider the problem of long-form video question answering from the viewpoint of adaptive hierarchical reinforced encoder-decoder network learning. We propose the adaptive hierarchical encoder network to learn the joint representation of the long-form video contents according to the question with adaptive video segmentation. we then develop the reinforced decoder network to generate the natural language answer for open-ended video question answering. We construct a large-scale long-form video question answering dataset. The extensive experiments show the effectiveness of our method.
Language: English
Open Access status: bronze
OpenAlex ID: W2808181286
FWCI: 2.388
Cites: 28
Cited by: 70
Related to: 10