Year: 2025
Type: article
Abstract: ISMED Database INTRODUCTION Micro-expressions (ME) are brief, involuntary, and subtle facial expressions. These expressions typically last between 40 and 500 milliseconds (Ekman, 2009). These fleeting expressions are crucial in revealing genuine emotional states, often surfacing when an individual attempts to mask their true feelings, consciously or unconsciously (Ekman & Friesen, 1969). Unlike Macro-facial expressions, which can be controlled or manipulated, MEs are difficult to fake, indicating a genuine emotional state (Matsumoto & Hwang, 2011). The study of MEs has significant implications across multiple domains, including clinical psychology, security, and human-computer interaction, as they offer insights into individuals' deeper, often hidden psychological states (Ekman, 2003). Understanding and interpreting these expressions can thus enhance communication and empathy, improving interactions in both social and professional settings. Various databases have been developed to facilitate the study of MEs, providing researchers and technologists with essential data for training individuals or algorithms and conducting analyses. For instance, the USF-HD (Shreve et al., 2011) and Polikovsky databases (Polikovsky et al., 2009) focus on posed expressions. Similarly, the York DDT (Warren et al., 2009) and SDD (Pfister et al., 2011) used Lie-generation method, primarily featuring Western populations. ME databases like CASME (Yan et al., 2014), CASME II (Yan et al., 2014), and CAS(ME)2 (Qu et al., 2016) databases capture spontaneous expression with high-resolution recordings of spontaneous MEs from East Asian participants, focusing on the fine details of these brief emotional cues (Yan et al., 2014). Each of these databases contributes valuable insights; however, they also come with notable limitations that influence the scope and generalizability of research in the field. ISMED Database Challenges in Existing ME Databases While these databases have advanced the study of MEs, they exhibit significant limitations, particularly in terms of cultural diversity and representation. Most existing datasets predominantly focus on Western or East Asian populations, leaving other cultural groups, such as Indian populations, underrepresented. This lack of diversity raises concern about the generalizability and cross-cultural validity of the research findings. Research has shown that cultural context heavily influences how facial expressions are displayed and perceived (Matsumoto, 1993; Matsumoto & Hwang, 2012) Cultural context is fundamental in shaping the display and interpretation of emotions, and this variability can lead to misunderstandings in cross-cultural interactions. Western cultures, for instance, tend to emphasize the expression of emotions and may interpret direct facial cues as straightforward reflections of internal states (Matsumoto, 2006). In contrast, Asian cultures often prioritize social harmony, and facial expressions may convey less overt emotional information, relying instead on subtle cues integrated with broader contextual understanding (Jack et al., 2009). Such cultural differences are profound, as they highlight that emotional expressions are not universal but rather influenced by cultural norms and practices. This variability poses challenges for developing emotion recognition systems that can function accurately across diverse populations. Existing databases, which predominantly feature Western or East Asian participants, may not be sufficient for developing systems that cater to other cultures. A brief summary of each dataset is also provided to contextualize their methodology, induction paradigms, and cultural focus, with full references included (Table 1). Table 1. Comparative summary of existing micro-expression (ME) databases, including participant ethnicity, sample size, annotation method, emotion induction paradigm, emotional ISMED Database labels used, and feature selection approach. The table highlights the cultural diversity and methodological variability across datasets, underscoring the relevance of the Indian Spontaneous ME Database (ISMED). ME Database Participa nts (Ethnicity ) Sample Annotati on Emotio n Inducti on method Emotio nal Label Feature selection method Canal-9 (Vinciarelli et al., 2009) N/A 24 Sequen ce - Politica l TV debates No emotio nal label Role-Ibased (in debates) YorkDDT (Warren et al., 2009) 9 (Caucasia ns) 18 FACS Lie generati on 6 basic emotio n 6 basic emotion SDD (Radlak et al., 2015) 101(N/A) 183 - Lie generati on Deceiv er & Truth teller Nonverbal potential cue (facial & eye moveme nt) MMEW (Ben et al., 2021) 36 (N/A) 300 (234 from 6 basic E) FACS N/A 6 basic emotio n 6 basic emotion + others USF-HD (Shreve et al., 2011) N/A 100 ME/non-ME Acting 6 basic emotio n 6 basic emotion Polikovsky (Polikovsky et al., 2009) 10 (Caucasia ns, Asian, Indian) 42 FACS Acting 6 basic emotio n + contem pt 6 basic emotion + contempt CASME I (Yan et al., 2013) 19(Chines e) 195 FACS Neutral ization 6 basic emotio n 6 basic emotion CASME II (Yan et al., 2014) 35 (Chinese) 247 FACS Neutral ization 6 basic emotio n 6 basic emotion CAS(ME)2 (Qu et al., 2017) 22 (Chinese) 57(ME) FACS Neutral ization 6 basic emotio n + 4(P, S,N,O) 6 basic emotion + ISMED Database (P,S,N,O ) SMIC (Pfister et al., 2013) 20 (Caucasia ns, Asian, African) 164 Self-reported Neutral ization P,N,S AND H,SU,D ,F,Sa P, S,N & H,Su,D,F ,Sa SAMM (Davison et al., 2018) 32(13 ethnic group) 159(M E) FACS Neutral ization 6 basic emotio n 6 basic emotion ISMED 120 (Indian) FACS & Self-reported Neutral ization 6 basic emotio n-Fear 6 basic emotion + Neutral emotion Algorithms trained on these datasets may struggle when applied in different cultural contexts, leading to misinterpretations and inaccuracies. For instance, an expression considered a sign of discomfort in one culture might be perceived as an expression of agreement in another. Such discrepancies demonstrate the necessity of culturally diverse databases to enhance the reliability and precision of ME detection, particularly as these technologies are increasingly deployed in multinational, multicultural contexts like global security checkpoints, customer service environments, and cross-cultural therapeutic settings. The current database (Indian Spontaneous Micro-expression Database: ISMED) tries to diversify the available data by providing another culturally diverse resource for ME research. With its vast population and rich cultural diversity, India offers a unique opportunity to study how emotions are expressed and perceived within its context. The objective of this study is to fill this gap by documenting spontaneous MEs from Indian participants, offering a comprehensive dataset that captures the nuanced facial expressions across six basic emotions— ranging from happy, disgust, anger, sadness, and fear to surprise. By addressing existing databases' current limitations and highlighting the significance of cultural factors in emotion research, ISMED represents a critical step forward in nonverbal behaviour studies. By emphasizing the need for culturally sensitive and representative ISMED Database research, ISMED contributes to a more inclusive understanding of human emotions and supports creating more accurate, effective, and culturally responsive emotion recognition systems. METHODOLOGY Participants: One hundred and twenty-seven participants (64 male and 63 female) of Indian ethnicity aged 20-35 years volunteered to participate in this study. The mean age of the participants was 26.57 years (SD 4.32). The mean age of the male and female participants was 26.17 years (SD = 4.17) and 26.97 years (SD = 4.45), respectively. Participants with facial occlusions, such as moustaches, beards, spectacles, or makeup, were excluded to ensure clear facial visibility. The inclusion criteria emphasized individuals with low cheek fat and minimal wrinkles, leading to the selection of young adults aged 20–35. To maintain gender balance, the sample included 64 males and 63 females. Participants were carefully selected to represent a diverse range of socio-economic statuses (from lower to upper class) and cultural contexts from various regions of India. The study protocol was reviewed and approved by the Institute Ethics Committee (IITK/IEC/2021-22/II/26). Experimental Set-up: Camera & Light: The video was recorded by the GoPro7 camera with a 4k sensor. It records the participant's video at 240 frames per second with 1080 pixel resolution while using the HEVC video compression option. In order to maximize storage, this mode shrinks the file size. Lighting is an issue for high-speed camera recording, as many light sources in lab use alternating current that refreshes regularly and create a flicker in the recording. Two ISMED Database rectangular LED screen bar lights were positioned on either side of the monitor to avoid it. It ensured consistent lighting on the face and minimized shadow. Emotion Inducement Stimuli: The Audio-visual video clips were used as stimuli to induce emotion in this study. These video clips were taken from the internet. Most of these video clips were taken from Indian movies, web series, plays, and YouTube shorts. Initially, 210 video clips were chosen by the experimenter for the emotion-labelling and rating study. These 210 video clips contain 30 video clips of each emotion (Anger, Disgust, Fear, Happy, Neutral, Sad, Surprise). 11 participants (Age range= 24-32 years, Mean age = 30, SD = 3.04), among them 5 were male (Mean= 29.4, SD= 2.24) and 6 were female (Mean= 26.83, SD = 1.94), These participant were not otherwise involved in the entire study, were involved in the emotion-labelling and rating study, to ensure inter-rater reliability. In the labelling-rating task 210 video clips were first converted into '.avi' format and then were randomly presented using the Psychopy v2.0 on a 23-inch HPW2371d LED monitor screen. The Neutral-Emotion-Neutral (N-E-N) paradigm was used. In each trial first a fixation cross (1 second) of black colour on white background then audio-visual video clips and lastly 2 questions; one of labelling (What emotion do you feel?) with 7 forced choice option (Anger, Disgust, Fear, Happy, Neutral, Sad, Surprise) and rating (What was the intensity of felt emotion?) on a five-point were shown. All 210 video-clips were presented in three sessions on three different days. In this experiment a 20-inch monitor was used in a dimly lit room. After the labelling and rating task, from 210 video clips, 52 video clips were selected (Angry-7, Disgust-8, Fear-6, Happy-8, Neutral-7, Sad-8, Surprise-8). These video-clips have a ISMED Database labelling consistency of 90% and an average rating of 4 or above. The selected 52 video clips will be further used in the study as emotion-inducing stimuli. It is important to note that the stimulus category was used only to ensure balanced representation of all emotions and to avoid experimental bias. The final labelling of micro-expression instances did not rely on stimulus identity but was determined through subsequent AU-based and FACS-based validation (see below). Equipment & Experimental Set-up: The experiment was set-up in a soundproof room with audio-visual clips running on speaker mode. A separate room was used for the experiment running purpose to avoid any unnecessary surrounding disturbance. The experiment was set-up on a 58 x 58 cm monitor, and the resolution was set at 1920 x 1080 pixels. A GoPro camera was mounted in the middle of the upper edge of the monitor. For the lightning purpose (1.2 x 2.8 cm), an LED light was used in on both sides of the monitor panel to cancel out shadows. All extra lights (like-room lights or lights from other sources) were switched off during the experiment. An experimental set-up is shown in Figure 1. Figure 1. An Experimental set-up shows the participant seated in front of the stimulus display monitor with lighting and a GoPro camera positioned for optimal ME recording. The experimenter monitors the set-up from an adjacent station to ensure minimal intrusion and consistent environmental control. ISMED Database The N-E-N paradigm was used to design the experiment. In each trial, a fixation cross ('+' sign) for 1 second, audio-visual video clips of 2 seconds to 1.5 minutes, and two questions of labelling and rating the video clips (each of 5 seconds) were shown. All the 52 video clips were shown in two sessions, each containing 26 trials. A 30 minute time gap was given between the two sessions. Emotion Inducement Experiment Procedure Participants were first introduced to the experiment-room setting to establish rapport. They were briefed about the experiment, which involved watching video clips while maintaining a neutral facial expression to ensure the camera did not record any emotional expressions. Each participant was informed that the winner of the expression suppression game would receive 50 rupees. However, at the end of the experiment, all participants received 50 rupees as compensation for their time. They were also briefed on their rights, including the option to leave the experiment at any point if they felt uncomfortable. Any questions from participants were addressed before proceeding. ISMED Database Each participant signed a hard copy of the consent form before the experimental session began. Once participants consented and wanted to participate in the study, we checked their eyesight, adjusted the audio level to a comfortable setting, and ensured the chin rest was positioned according to their heights. The side panel light brightness was kept constant to avoid variations in video quality. Next, we attached interoceptive sensors to record their heart rate, respiration rate, and galvanic skin response. Participants were then asked to sit comfortably, while the experimenter the was between their and the screen. The of cm was to ensure a constant with a session with the chin rest and sensors and the high-speed camera adjusted to capture the The session included an and two each of a fixation a video and labelling-rating the we checked with participants to ensure they were comfortable with the experiment. their we the first After a the second minutes, began. both sessions, the experimenter was out of to maintain and experimenter bias. Each participant 30 of with 7 participants data were not included to and lighting data were and used for further ME of video ISMED Database the study, researchers two video clips, to a of 30 The video 52 trials. In each of the 52 participants were shown a fixation by an audio-visual video stimulus between 2 seconds and 1.5 They were asked labelling and rating These video included two of and focus only on the that is the of the participants, we need to any other in the video that is In order to to the facial expressions, we by the video into 52 with the fixation label and rating This that only the stimulus were kept for video was carefully by to the and of the stimulus such as the monitor and other background be in The background was by the video to in on the participant's face For this task, a was and used, which was carefully to focus on the surrounding the face to ensure that in and a resolution of pixels. and were to the for subsequent that only important data was AND Feature selection is on the The FACS facial expressions by them into individual (Ekman & Friesen, & et al., 2011). we are using only which in the of the face is the and ISMED Database FACS is either by trained facial by in a video or or by which in that have to the of that can and from video and consistency & This study a method and to and ME we used 2016) to a method on was used for feature selection and ME These were on time and emotion 1). two human the and the of were from the 120 participants during the experiment, these were not used for labelling participants were to their emotions and mask them with a neutral these would the final the emotion in ISMED are in and FACS-based by balanced stimulus The dataset is in Figure Figure of ISMED The of the Indian Spontaneous Database (ISMED). The emotion induction high-speed video at 240 with and feature selection and using feature and validation of micro-expression by trained using with ISMED Database and and of micro-expression by emotion category and The Indian Spontaneous ME Database addressed a critical gap in existing datasets by a diverse range of spontaneous from Indian participants, with the focus on Western and East Asian in studies. from 120 participants 7 to were using a that and recordings ensured capture of subtle facial while including and MEs for Emotion stimuli at 240 Feature & selection ME expression using FACS + ME by emotion category and ISMED Database A a consistent of six basic sadness, anger, and validation an underscoring the reliability of the to other databases such as CASME and ISMED unique cultural to Indian emphasizing the necessity for culturally sensitive datasets to enhance emotion recognition Table of six basic emotions Disgust, Fear, Happy, Sad, Surprise) across time in the ISMED the of expressions for and Emotion 1 - 5 1 1 6 - 2 - 4 - - 4 - - 4 5 - 100 2 - 2 4 18 - 210 While were used during the rating task, the final ISMED dataset on six basic emotions Sad, Fear, and The Neutral category as a and was not included in Table ISMED Database Table 2 the of MEs across different time and six basic notable emotions, particularly as the and consistent across all time with in and This the and of in micro-expression studies. Similarly, a in as time highlighting its and in brief micro-expression while was primarily in indicating as a emotional response. In contrast, a with at and its or representation in the was across all time indicating its in or potential in the was questions about its or contextual relevance within this study. Figure emotion across time and emotions are in highlighting their and in spontaneous ISMED Database These further emphasize that emotional intensity is in time particularly the of resolution in micro-expression the of emotions like such as or potential contextual or within the These insights highlight the of and cultural in micro-expression research, the for for subtle or The of this study was to the Indian Spontaneous ME Database as a culturally for understanding and MEs among Indian ISMED Database demonstrate that ISMED captures a range of spontaneous MEs, revealing the cultural in the Indian By documenting these expressions, ISMED a crucial gap in the current of emotion databases, which predominantly feature Western or East Asian populations. This cultural is for emotion recognition algorithms and them and across diverse populations. ISMED out by providing unique cultural insights that existing databases like While these are for training on posed or spontaneous expressions from other they of the diversity to create global emotion recognition systems. Indian MEs in ISMED with research (Matsumoto, 1993; et al., emphasizing the need for culturally sensitive to understanding emotional By the emotional and highlighting cultural influences on ISMED contributes to existing of the that emotion expression is not but in social and cultural ISMED also offers through its and to and This that human the reliability of ME Such an integrated method a valuable for database how human and can be to capture emotional ISMED has significant in clinical and cross-cultural By ISMED into can ISMED Database create algorithms sensitive to Indian emotional in In clinical contexts, ISMED offers a resource to and emotional to Indian leading to culturally therapeutic practices. insights into Indian MEs can in communication in multicultural providing a for developing systems of interpreting diverse emotional expressions However, the study is not its The is that the database not capture fear expressions according to the It is that the audio-visual stimuli used for emotion induction were not or that a more is to any culturally that may not with the FACS while the controlled setting for may not the of emotional the ISMED dataset on Indian the diversity to capture and annotation is but also which may indicating the need for further of To these limitations, research ISMED to a broader range of participants from different Indian regions and cultural MEs in more enhance the validity of the of algorithms that cultural insights from ISMED will be crucial in creating that to cultural variations between and cultural will be essential to these that emotion recognition systems are and culturally ISMED Database In ISMED represents a significant step forward in the of ME research, providing a culturally that and applied offering for improving emotion recognition systems and to the global understanding of human emotions in diverse cultural ISMED Database is available a controlled The dataset is into on emotional category and the Emotion are also in the of ISMED on an which will controlled but more for may the to a with participant consent and used for and annotation face and is available from the ISMED Database This is a psychological experiment. It is by Research of and Indian Institute of for research It would only of time. In this study, you will on the screen. These have emotional but you would be to maintain a neutral face the experimental This study on understanding how you maintain a neutral face while watching emotional are to the provided by the for the you at any point in the experiment, you are to All will be kept and used for research identity will not be in any data may be with other researchers for you would like to more about you can on the valuable to the research is you to be a participant in this sign I have been the and of such I consent to participate in this of the of
Source: Frontiers in Psychology
Authors: Riya Mishra, Braj Bhushan, K. S. Venkatesh
Institution: Indian Institute of Technology Kanpur
Language: English
Topic: Emotion and Mood Recognition
Subfield: Experimental and Cognitive Psychology
Field: Psychology
Domain: Social Sciences
Open Access status: gold
Estimated APC paid: $3,801
OpenAlex ID: W4415234698