9 - The Modality Principle in Multimedia Learning  pp. 147-158

By Renae Low and John Sweller

Image View Previous Chapter Next Chapter


The capacity limitations of working memory are a major impediment when students are required to learn new material. Furthermore, those limitations are relatively inflexible. Nevertheless, in this chapter we explore one technique that can effectively expand working memory capacity. Under certain, well-defined conditions, presenting some information in visual mode and other information in auditory mode can expand effective working memory capacity and so reduce the effects of an excessive cognitive load. This effect is called the modality effect or modality principle. It is an instructional principle that can substantially increase learning. This chapter discusses the theory and data that underpin the principle and the instructional implications that flow from the principle.


There is evidence to indicate that the manner in which information is presented will affect how well it is learned and remembered (e.g., Mayer, Bove, Bryman, Mars, & Tapangco, 1996). This chapter deals with evidence documenting the importance of presentation modes, specifically the modality effect that occurs when information presented in a mixed mode (partly visual and partly auditory) is more effective than when the same information is presented in a single mode (either visual or auditory alone). The instructional version of the modality effect derives from the split-attention effect (see chapter 8), a phenomenon explicable by cognitive load theory (see chapter 2). It occurs when multiple sources of information that must be mentally integrated before they can be understood have written (and therefore visual) information presented in spoken (and therefore auditory) form.

Allport, D. A. , Antonis, B. , & Reynolds, P. (1972). On the division of attention: A disproof of the single channel hypothesis. Quarterly Journal of Experimental Psychology, 24, 225–235
Baddeley, A. D. (1986). Working memory. Oxford, England: Oxford University Press
Baddeley, A. D. (1992). Working memory. Science, 255, 556–559
Baddeley, A. D. (1999). Human memory. Boston: Allyn & Bacon
Brünken, R. , Plass, J. L. , Leutner, D. (2004). Assessment of cognitive load in multimedia learning with dual task methodology: Auditory load and modality effects. Instructional Science 32, 115–132
Brünken, R. , Steinbacher, S. , Plass, J. L. , & Leutner, D. (2002). Assessment of cognitive load in multimedia learning using dual-task methodology. Experimental Psychology, 49, 109–119
Brooks, L. (1967). The suppression of visualization by reading. Quarterly Journal of Experimental Psychology, 19, 289–299
Craig, S. , Gholson, B. , & Driscoll, D. (2002). Animated pedagogical agents in multimedia educational environments: Effects of agent properties, picture features, and redundancy. Journal of Educational Psychology, 94, 428–434
Dennis, I. (1977). Component problems in dichotic listening. Quarterly Journal of Experimental Psychology, 29, 437–450
Frick, R. (1984). Using both an auditory and a visual short-term store to increase digit span. Memory and Cognition, 12, 507–514
Jeung, H. , Chandler, P. , & Sweller, J. (1997). The role of visual indicators in dual sensory mode instruction. Educational Psychology, 17, 329–343
Kalyuga, S. , Chandler, P. , & Sweller, J. (1999). Managing split-attention and redundancy in multimedia instruction. Applied Cognitive Psychology, 13, 351–371
Kalyuga, S. , Chandler, P. , & Sweller, J. (2000). Incorporating learner experience into the design of multimedia instruction. Journal of Educational Psychology, 92, 126–136
Kalyuga, S. , Chandler, P. , & Sweller, J. (in press). When redundant on-screen text in multimedia technical instruction can interfere with learning. Human Factors
Kolers, P. A. (1979). A pattern-analyzing basis of recognition. In L. S. Cermak & F. I. M. Craiks (Eds.), Levels of processing in human memory. Hillsdale, NJ: Lawrence Erlbaum Associates
Leahy, W. , Chandler, P. , & Sweller, J. (2003). When auditory presentations should and should not be a component of multimedia instruction. Applied Cognitive Psychology, 17, 401–418
Levin, J. , & Divine-Hawkins, P. (1974). Visual imagery as a prose-learning process. Journal of Reading Behaviour, 6, 23–30
Margrain, S. (1967). Short-term memory as a function of input modality. Quarterly Journal of Experimental Psychology, 19, 109–114
Mayer, R. E. , & Moreno, R. (1998). A split-attention effect in multi-media learning: Evidence for dual processing systems in working memory. Journal of Educational Psychology, 90, 312–320
Mayer, R. E. , Bove, W. , Bryman, A. , Mars, R. , & Tapangco, L. (1996). When less is more: Meaningful learning from visual and verbal summaries of science textbook lessons. Journal of Educational Psychology, 88, 64–73
Mayer, R. E. , Heiser, J. , & Lonn, S. (2001). Cognitive contraints on multimedia learning: When presenting more material results in less understanding. Journal of Educational Psychology, 93, 187–198
Miller, G. A. (1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review, 63, 81–97
Moreno, R. , & Mayer, R. E. (1999). Cognitive principles of multimedia learning: The role of modality and contiguity. Journal of Educational Psychology, 91, 358–368
Moreno, R. , & Mayer, R. E. (2002). Learning science in virtual reality multimedia environ‭ments: Role of methods and media. Journal of Educational Psychology, 94, 598–610
Moreno, R. , Mayer, R. E. , Spires, H. A. , & Lester, J. C. (2001). The case for social agency in computer-based multimedia learning: Do students learn more deeply when they interact with animated pedagogical agents? Cognition and Instruction, 19, 177–214
Mousavi, S. , Low, R. , & Sweller, J. (1995). Reducing cognitive load by mixing auditory and visual presentation modes. Journal of Educational Psychology, 87, 319–334
Murdock, B. B., Jr. (1971). Four-channel effects in short-term memory. Psychonomic Science, 24, 197–198
Mwangi, W. , & Sweller, J. (1998). Learning to solve compare word problems: The effect of example format and generating self-explanations. Cognition and Instruction, 16, 173–199
Paas, F. , & Van Merriënboer, J. (1993). The efficiency of instructional conditions: An approach to combine mental-effort and performance measures. Human Factors, 35, 737–743
Paas, F. , Renkl, A. , & Sweller, J. (2003). Cognitive load theory and instructional design: Recent developments. Educational Psychologist, 38, 1–4
Penney, C. (1980). Order of report in bisensory verbal short-term memory. Canadian Journal of Psychology, 34, 190–195
Penney, C. (1989). Modality effects and the structure of short-term verbal memory. Memory and Cognition, 17, 398–422
Penney, C. , & Butt, A. (1986). Within- and between-modality associations in probed recall: A test of the separate streams hypothesis. Canadian Journal of Psychology, 40, 1–11
Rollins, H. A. , & Hendricks, R. (1980). Processing of words presented simultaneously to eye and ear. Journal of Experimental Psychology: Human Perception and Performance, 6, 99–109
Rollins, H. A. , & Thibadeau, R. (1973). The effects of auditory shadowing on recognition of information received visually. Memory and Cognition, 1, 164–168
Schneider, W., & Detweiler, M. (1987). A connectionist/control architecture for working memory. In G. H. Bower (Ed.), The psychology of learning and motivation. (Vol. 21, pp. 53–119). New York: Academic Press
Shaffer, L. H. (1975). Multiple attention in continuous verbal tasks. In P. M. A. Rabbitt & S. Dornic (Eds.), Attention and performance V (pp. 157–167). London: Academic Press
Spelke, E. , Hirst, W. , & Neisser, U. (1976). Skills of divided attention. Cognition, 4, 215–230
Sweller, J. , Chandler, P. , Tierney, P. , & Cooper, M. (1990). Cognitive load as a factor in the structuring of technical material. Journal of Experimental Psychology: General, 119, 176–192
Sweller, J. , van Merriënboer, J. , & Paas, F. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10, 251–296
Tarmizi, R. , & Sweller, J. (1988). Guidance during mathematical problem solving. Journal of Educational Psychology, 80, 424–436
Tindall-Ford, S. , Chandler, P. , & Sweller, J. (1997). When two sensory modes are better than one. Journal of Experimental Psychology: Applied, 3, 257–287
Treisman, A. M., & Davies, A. (1973). Divided attention to ear and eye. In S. Kornblum (Ed.), Attention and performance IV (pp. 101–117). New York: Academic Press
Ward, M. , & Sweller, J. (1990). Structuring effective worked examples. Cognition and Instruction, 7, 1–39