A fine-tuned vision transformer-based on limited dataset for facial expression recognition
Year: 2024
Type: conference-paper
Abstract: With the increase of interest in Facial Expression Recognition (FER) in the past few decades. Several challenges surfaced with the invention of many different FER models which are often based on Convolution Neural Network (CNN) architectures. Recently, an attention-based transformer model has been presented to address FER. One of the major issues with Transformers is the need for a large data quantity for training. Therefore, in this paper, we propose to learn how to fine-tune a vision transformer-based (ViT) model using a limited dataset. We will be using the JAFFE Dataset, which consists of only 213 images containing seven different emotions. The proposed method is evaluated using several fine-tuning methods, such as adding dropout, data augmentation, and layer freezing. We compared the models implemented with 5% dropout regularization, augmented dataset (up to 5000 images), and freezing the initial model's layers, fine-tuning around a fourth of the last layers. The best model was achieved by fine-tuning ViT L-16 with 96.06% accuracy, trained with 5% dropout in the augmented dataset, and freezing the initial 21st layers. We also compared our model to the other previous work model and the results showed that our model reached the state-of-the art for the JAFFE dataset.
Source: Procedia Computer Science
Authors: Rio Febrian, Ronald Richie Huang, Nicholas Setiono, Dimas Ramdhan, Andry Chowanda
Institution: Binus University
Language: English
Topic: Emotion and Mood Recognition
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: diamond
Funder: Binus University
Award: G2572987426
OpenAlex ID: W4404449784