Cultural Moment Benchmark: Evaluating Video Cultural Reasoning and Grounding in Southeast Asia
Year: 2026
Type: conference-paper
Abstract: Cultural understanding in video means more than recognizing what is visible; it requires grasping the symbolic and temporal significance of cultural concepts.We decompose this into three abilities: naming what a concept symbolizes, visually recognizing it on video, and locating its sub-events in time.Existing videocultural benchmarks tend to test what is seen, collapsing these three abilities into a single score that hides the bottleneck.We introduce the Cultural Moment Benchmark (CMB): 306 expert-curated concepts from seven countries in Southeast Asia across five categories.We evaluate each concept through three stages, one per ability.Given a description, Stage 1 (S1) selects from four candidate concept names, Stage 2 (S2) selects from four candidate video moments, and Stage 3 (S3) predicts the start and end times of the moment in a video.To keep each stage focused on a distinct ability, we use three design choices: semantic-similarity distractors (S1, S2), unlabeled video moments (S2), and free-form localization on a different example video (S3).Across six visionlanguage models, failure modes vary by ability and modality.i) Even the strongest closedsource models score below 30% when all three stages must be correct; ii) The three abilities do not fully cascade: naming a concept correctly helps half the models recognize it on video, but recognizing it has little effect on locating the sub-event in time; iii) Audio is complementary, redundant, or distracting depending on the concept, more often distracting in non-Latinscript countries; removing both audio and subtitles hurts Games and Music the most.Our 14-rater human study shows that even Expert raters score below chance on concepts from a neighboring country, indicating that CMB requires country-specific cultural knowledge.CMB acts as a diagnostic harness, attributing failures to a specific ability or modality.
Institution: Singapore Management University
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Awards: G1482925484, G6854926366
OpenAlex ID: W7204203441