Work

Reconsidering the Modality Principle in Multimedia Learning

Year: 2007
Type: article
Abstract: Reconsidering the Modality Principle in Multimedia Learning Peter Gerjets ([email protected]) Hypermedia Research Unit, Knowledge Media Research Center Konrad-Adenauer-Strasse 40, 72072 Tuebingen, Germany Ralf Rummer ([email protected]) Department of Psychology, Saarland University, P.O. Box 151150 66041 Saarbruecken, Germany Katharina Scheiter ([email protected]) Department of Applied Cognitive Psychology and Media Psychology, University of Tuebingen Konrad-Adenauer-Strasse 40, 72072 Tuebingen, Germany Judith Schweppe ([email protected]) Department of Psychology, Saarland University, P.O. Box 151150 66041 Saarbruecken, Germany Keywords: Modality effect, working memory, multimedia. Well known theories of multimedia design like the Cognitive Load Theory (CLT, Sweller, 1999) and the Cognitive Theory of Multimedia Learning (CTML, Mayer, 2001) provide sets of guidelines that describe how multimedia instruction should be best designed to be in accordance with the limitations of the human cognitive system. One of the most important design decisions pertains to the question of how to combine pictorial with verbal information. This issue is addressed by design recommendations like the split-attention principle or the modality principle. According to these principles learning outcomes will be optimized when text and pictures are presented in spatial and temporal integration and by using different sensory modalities. In order to justify their principles, both the CTML and the CLT refer to Baddeley’s (1999) multimodal working memory model. In addition to a central processing unit, the central executive, this model includes two slave systems, the phonological loop (PL), which is devoted to short-term storage of verbal material, and the visuo-spatial sketch pad (VSSP) which stores visual and spatial information. Hence, if textual and pictorial information is presented in a split- attention format, this is assumed to increase workload within all three subsystems for maintaining and aligning the two pieces of information. Furthermore, visual texts are assumed to demand the same subsystem of working memory as pictures (namely the VSSP), whereas with auditory text presentation an additional cognitive resource is provided for storage (namely the PL). Thus, whenever a picture is presented together with a written text the VSSP may be overloaded, whereas using spoken text unburdens the VSSP because verbal information will then be processed within the PL. We claim that this explanation of the modality principle is based on a misleading interpretation of Baddeley’s working memory model. According to this model not only auditory verbal but also visual verbal materials are stored and processed within the same language-related working memory subsystem (namely the PL). Therefore, we propose an alternative explanation according to which the modality principle is based on two different mechanisms: 1. Due to the limitations of the perceptual systems pictures and visual text cannot be processed simultaneously thus leading to an unavoidable temporal delay. In contrast, auditory text and pictures can be processes in parallel. 2. From basic memory research, it is well-known that retaining the most recent items of an auditory list is better than retaining those of a visual list. This effect is due to the fact that auditory-sensory representations are assumed to be more durable than visual-sensory representations (e.g., Penney, 1989). In addition, it explains that retention of short auditory texts is better than retention of short visual text even when texts and pictures are presented sequentially (cf. Moreno & Mayer, 1999). Acknowledgments We thank the DFG for supporting this research. References Baddeley, A. D. (1999). Essentials of human memory. Hove, UK: Psychology Press. Mayer, R. E. (2001). Multimedia learning. Cambridge, UK: Cambridge University Press. Moreno, R. & Mayer, R. E. (1999). Cognitive principles of multimedia learning: The role of modality and contiguity. Journal of Educational Psychology, 91, 358-368. Penney, C. G. (1989). Modality effects and the structure of short term verbal memory. Memory & Cognition, 17, 398- Sweller, J. (1999). Instruction design in technical areas. Camberwell, Australia: ACER.
Institution: Saarland University
Language: English
Field: Psychology
Open Access status: green
OpenAlex ID: W2587601132
FWCI: 0.229
Cites: 5
Cited by: 2
Related to: 10