Semantic and Visual Features Drive the Intrinsic Memorability of Co-Speech Gestures
Year: 2024
Type: article
Abstract: Co-speech gestures that teachers spontaneously produce during explanations have been shown to benefit students’ learning. Further, prior work suggests that information conveyed through teachers’ gestures is less likely to deteriorate than through speech (Church et al., 2007). However, how intrinsic features of gestures affect students’ memory remains unclear. The memorability effect denotes a phenomenon where adults with different backgrounds consistently remember and forget particular visual stimuli (static images, dance moves, etc.), owing to the stimuli's intrinsic semantic and visual features. In this study, we investigate whether certain gestures are consistently remembered and, if so, which semantic and visual features are associated with these remembered gestures. We first created 360 10-second audiovisual stimuli by video recording 20 actors producing unscripted natural speech and gestures as they explained Piagetian conservation problems. Two trained experimenters extracted high-level semantics and low-level visual/acoustic features in speech and gesture for each audiovisual stimulus. We then tested online participants’ memories in three different conditions using a between-subjects study-test paradigm: the audiovisual stimuli (gesture+speech condition), the visual-only version of the same stimuli (gesture condition), and the audio-only version of the stimuli (speech condition). Within each of the two experimental blocks, participants encoded nine random stimuli from an actor and made old/new judgments on all 18 stimuli from the same actor immediately after. We discovered that participants show significant consistencies in their memory for the gesture, gesture+speech, and speech stimuli. Focusing on the visual-only (gesture) condition, we found that (1) more meaningful gestures and speech predicted more memorable gestures; (2) using both hands led to more memorable gestures than using one hand. Our results suggest that both semantic (conveyed through speech and gestures) and visual (conveyed through gesture) features make co-speech gestures memorable.
Source: Journal of Vision
Institution: University of Chicago
Language: English
Subfield: Experimental and Cognitive Psychology
Field: Psychology
Domain: Social Sciences
SDG: Quality education
Open Access status: gold
Estimated APC paid: $1,850
OpenAlex ID: W4402904585