Work

What counts as a relevant gesture in the study of multimodal event expressions?

Year: 2024

Type: article

Abstract: This article explores how methodological decisions about which gestures to include impact the analysis of multimodal event representation. Gestures commonly temporally align with semantically co-expressive speech. However, when we consider not single concepts but whole events the picture gets more complicated, since events are not mapped onto single lexical items but across larger stretches of spoken language. To research gesture-speech integration in multimodal event expressions we must decide which gestures are considered ‘relevant’. Using caused motion events as a case study, we apply different definitions of gesture inclusion relating to temporal and semantic alignment. We investigate the impact of definition choice on the results by exploring corpus data from Australian English and two endangered Oceanic languages, Saliba-Logea and Sudest. The study shows comparable trends across some conditions but also language- and definition-specific differences. This has a bearing on how confidently we can compare studies based on different methodologies and/or languages.

Source: Gesture

DOI: 10.1075/gest.25002.mar

Authors: Anna Margetts, Eleanor Jorgensen, Isabelle Burke, Harriet Sheppard

Institutions: Zhejiang Normal University, Monash University, The University of Melbourne, Jinhua University of Vocational Technology, University of the South Pacific

Language: English

Topic: Language, Discourse, Communication Strategies

Subfield: Language and Linguistics

Field: Arts and Humanities

Domain: Social Sciences

Open Access status: hybrid

Funders: Volkswagen Foundation, Monash University, Australian Research Council

Award: Where Gesture Meets Grammar: Crosslinguistic Multimodal Communication