Hearing gestures : vocalisations as embodied projections of intentionality in designing non-speech sounds for communicative functions
Year: 2011
Type: dissertation
Abstract: Finnish summary Diss.It has been argued that we humans understand each other's emotions and intentions on the basis of attuning to the performed actions of others at an embodied level.This dissertation examines the possibilities for utilising such an embodied nature of nonverbal communication in non-speech user interface sound design for human-computer interaction.The approach of this work emphasises situated interactional experiences and the active role of the body and kinaesthesia as a framework of meaning-creation.The approach reciprocally conceives kinaesthetic experience as a mode for performing actions and as well as a mode for understanding them.The central assumption is that the intentionality in gesturally relevant sound events is kinaesthetically graspable through imitative and empathetic involvement in perception.The focus of this research is on studying the communication of intentionality (i.e., intentional and affective states) through nonverbal gestural projections in vocal expressions.The emphasis is on examining the couplings between intonation structures of vocal expressions and the context-situated communicative intent behind them.As an epistemological background, it is assumed that intonation structures relate to kinaesthetic experience of movement, which in itself is meaningful in terms of the perceiver's intoned knowledge -a tacit form of knowledge based on the experiential background of vocal interactions.The proposed action-oriented approach to sonic communication inseparably concerns affective and functional kinaesthetic characteristics involved in sounds.The empirical studies of this dissertation specifically examined intonation patterns involved in vocal gestures performed for certain interactional communicative functions.In this case, the functions were related to the regulation of physical training activity.Results clearly demonstrated couplings between intonation patterns and communicative functions.The findings also showed that these function-specific intonation patterns communicated the intended meanings when they were implemented in user interface sounds.As a result, this dissertation provides a validated framework for utilising vocal gestures in the design of non-speech sounds for communicative functions of human-computer interaction.According to the experimental results, the proposed design method can be also utilised cross-modally, in the design of vibrotactile user interface elements.Moreover, the underlying design principles, outlined in the dissertation, can be more broadly applied to interaction design and to the development of user interfaces.
Author: Kai Tuuri
Institution: University of Jyväskylä
Language: English
Subfield: Music
Field: Arts and Humanities
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W2207731298
