Year: 2020
Type: article
Abstract: Determining the emotional sentiment of a video remains a challenging task that requires multimodal, contextual understanding of a situation.In this paper, we describe our entry into the EmotiW 2020 Audio-Video Group Emotion Recognition Challenge to classify group videos containing large variations in language, people, and environment, into one of three sentiment classes.Our end-toend approach consists of independently training models for different modalities, including full-frame video scenes, human body keypoints, embeddings extracted from audio clips, and image-caption word embeddings.Novel combinations of modalities, such as laughter and image-captioning, and transfer learning are further developed.We use fully-connected (FC) fusion ensembling to aggregate the modalities, achieving a best test accuracy of 63.9% which is 16 percentage points higher than that of the baseline ensemble.
Authors: Jin, B T, Abdelrahman, L, Chen, C K, Khanzada, A
Language: English
Topic: Emotion and Mood Recognition
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W4289213428
