AdaSeViLA: Adaptive Dynamic Temporal Window and Object-Aware Frame Selection for Video Question Answering
Year: 2026
Type: article
Abstract: Video question answering remains a challenging task that requires a sophisticated understanding of both visual content and temporal dynamics across video sequences. Current approaches typically rely on fixed temporal processing strategies and uniform frame-selection mechanisms, which fail to adapt to the diverse requirements of different question types and may overlook critical visual information. We propose AdaSeViLA, an adaptive framework that enhances video understanding through two key innovations: Adaptive Temporal Window Selection (ATWS) that dynamically adjusts the number of processed frames (3–12 frames) based on question-type classification, and Object-importance-Aware Frame Selection (OAFS) that combines global relevance with local visual saliency for enhanced frame identification. Our approach intelligently allocates computational resources based on question complexity while maintaining high accuracy through improved frame-selection mechanisms. Extensive experiments on three challenging VideoQA benchmarks demonstrate that AdaSeViLA achieves superior performance: 87.4% accuracy on MM-AU (+2.7% over SeViLA), 73.6% on NExT-QA (+0.4% improvement), and 61.6% on STAR (+0.6% gain), while providing up to 4× computational speedup for short-term tasks. These results validate the effectiveness of adaptive temporal processing and object-aware selection in advancing video question answering capabilities.
Source: Applied Sciences
DOI: 10.3390/app16084017
Authors: Zehua Ji, Chao Zhang, Jian Huang, Siyang Li, Xudong Li, Zehao Li
Institutions: Beihang University, Xiaomi (China)
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $2,896
Award: G1107710830
OpenAlex ID: W7155102441