TOTAL VIEWS: 454
This study investigates the impact of multimodal input on cognitive load and interpreting fluency among student interpreters by analyzing F0 parameters and interpreting fluency indicators. Participants were divided into three groups: Audio Group (sound only), Video Group (sound and video), and Video+Subtitle Group (sound, video, and subtitles). F0 parameters, including pitch variability and range, were assessed alongside speech rate, pause frequency, and self-repair frequency. Results indicated differences among groups, with the Video Group showing higher pitch variability compared to the Audio Group, while the Video+Subtitle Group demonstrated intermediate performance. Notably, the Video+Subtitle Group had the highest speech rate and longest speech flow, reflecting superior fluency. These findings suggest that multimodal input, particularly with visual and textual elements, reduces cognitive load and enhances interpreting fluency. This study highlights the importance of incorporating multimodal resources in interpreter training to improve performance and better manage cognitive demands.
Anderson, L. (1994). Simultaneous interpretation: Contextual and translation aspects. In S. Lambert & B. Moser-Mercer (Eds.), Bridging the gap: Empirical research in simultaneous interpretation (pp. 101-120). Amsterdam, Netherlands/Philadelphia, PA: John Benjamins.
Balzani, M. (1990). Le contact visual en interpretation simultanée. In L. Gran & C. Taylor (Eds.), Aspects of applied and experimental research on conference interpretation (pp. 93-100). Udine, Italy: Campanotto Editore.
Blatter, R., & Lopez Conceiro, H. (2015). Visual input in simultaneous interpreting: The role of lexical density (Master's thesis). Geneva, Switzerland: University of Geneva.
Bühler, H. (1986). Linguistic (semantic) and extra-linguistic (pragmatic) criteria for the evaluation of conference inter-pretation and interpreters. Multilingua, 5(4), 231-235.
Chen, S. (2017). Note-taking in consecutive interpreting: New data from pen recording. Translation and Interpreting, 9(1), 4-23.
Chen, S., & Kruger, J.-L. (2023). The effectiveness of computer-assisted interpreting: A preliminary study based on English-Chinese consecutive interpreting. Translation and Interpreting Studies, 18(3), 399-420.
Chmiel, A., & Lijewska, A. (2019). Syntactic processing in sight translation by professional and trainee interpreters: Professionals are more time-efficient while trainees view the source text less. Target, 31(3), 378-397.
Chmiel, A., Janikowski, P., & Cieślewicz, A. (2020). The eye or the ear? Source language interference in sight translation and simultaneous interpreting. Interpreting, 22(2), 187-210.
Dam, H. V. (2001). On the option between form-based and meaning-based interpreting: The effect of source text difficulty on lexical target text form in simultaneous interpreting. The Interpreters' Newsletter, 11, 27-55.
Fantinuoli, C. (2023). Towards AI-enhanced computer-assisted interpreting. In G. Corpas Pastor & B. Defrancq (Eds.), Interpreting technologies-current and future trends (pp. 46-71). Amsterdam, Netherlands: John Benjamins.
Gieshoff, A. C. (2021). The impact of visible lip movements on silent pauses in simultaneous interpreting. Interpreting, 23(2), 168-191.
Hild, A. (2011). Effects of linguistic complexity on expert processing during simultaneous interpreting. In C. Alvstad, A. Hild, & E. Tiselius (Eds.), Methods and strategies of process research: Integrative approaches in Translation Studies (pp. 249-267). Amsterdam, Netherlands: John Benjamins.
Huttunen, K., Keränen, H., Väyrynen, E., Pääkkönen, R., & Leino, T. (2011). Effect of cognitive load on speech prosody in aviation: Evidence from military simulator flights. Applied Ergonomics, 42(2), 348-357.
Hyönä, J., Tommola, J., & Alaja, A. M. (1995). Pupil dilation as a measure of processing load in simultaneous interpreting and other language tasks. The Quarterly Journal of Experimental Psychology, 48(3), 598-612.
Jensen, K. T. (2009). Indicators of text complexity. In S. Göpferich, A. L. Jakobsen, & I. M. Mees (Eds.), Behind the mind: Methods, models and results in translation process research (pp. 61-80). Copenhagen, Denmark: Samfund-slitteratur.
Jesse, A., Vrignaud, N., Cohen, M. M., & Massaro, D. (2000). The processing of information from multiple sources in simultaneous interpreting. Interpreting, 5(2), 95-115.
Johannes, B., Wittels, P., Enne, R., Kirsch, K., & Fischer, C. (2007). Non-linear function model of voice pitch dependency on physical and mental load. European Journal of Applied Physiology, 101(3), 267-276.
Korpal, P. (2016). Interpreting as a stressful activity: Physiological measures of stress in simultaneous interpreting. Poznań Studies in Contemporary Linguistics, 52(2), 297-316.
Korpal, P., & Katarzyna, S. (2018). The whole picture: Processing of numbers and their context in simultaneous interpreting. Poznań Studies in Contemporary Linguistics, 54(3), 335-354.
Lang, Y., Hou, L., & He, Y. (2019). The impact of multimodal input on cognitive processing paths in simultaneous interpreting: A corpus-based study. Foreign Languages (Journal of Shanghai International Studies University), 2, 75-86.
Lin, X., Lei, V. L. C., Li, D., & Yuan, Z. (2018). Which is more costly in Chinese to English simultaneous interpreting, "pairing" or "transphrasing"? Evidence from an fNIRS neuroimaging study. Neurophotonics, 5(2), 025010.
Liu, M., Schallert, D. L., & Carroll, P. J. (2004). Working memory and expertise in simultaneous interpreting. Interpreting, 6(1), 19-42.
Petsche, H., Etlinger, S. C., & Filz, O. (1993). Brain electrical mechanisms of bilingual speech management: An initial investigation. Electroencephalography and Clinical Neurophysiology, 86(6), 385-394.
Pinget, A. F., Bosker, H. R., Quené, H., & de Jong, N. H. (2014). Native speakers' perception of fluency and accent in L2 speech. Language Testing, 31(3), 349-365.
Plevoets, K., & Defrancq, B. (2018). The cognitive load of interpreters in the European Parliament. Interpreting, 20(1), 1-28.
Rennert, S. (2008). Visual input in simultaneous interpreting. Meta, 53(2), 204-217.
Rinne, J. O., Tommola, J., Laine, M., Krause, B. J., Schmidt, D., Kaasinen, V., et al. (2000). The translating brain: Cerebral activation patterns during simultaneous interpreting. Neuroscience Letters, 294(2), 85-88.
Seeber, K. G. (2013). Cognitive load in simultaneous interpreting: Measures and methods. Target, 25(1), 18-32.
Stachowiak-Szymczak, K., & Korpal, P. (2019). Interpreting accuracy and visual processing of numbers in professional and student interpreters: An eye-tracking study. Across Languages and Cultures, 20(2), 235-251.
Szarkowska, A., Dutka, Ł., Szychowska, A., & Pilipczuk, O. (2018). Visual attention distribution in intralingual respeaking: An eye-tracking study. In C. Walker & F. M. Federici (Eds.), Eye tracking and multidisciplinary studies on translation (pp. 185-201). Amsterdam, Netherlands: John Benjamins.
Tommola, J., & Lindholm, J. (1995). Experimental research on interpreting: Which dependent variable? In J. Tommola (Ed.), Topics in interpreting research (pp. 121-133). Turku, Finland: Centre for Translation and Interpreting, University of Turku.
Cognitive Load in Multimodal Interpreting: Evidence from F0 and Interpreting Fluency
How to cite this paper: Shun Fu. (2025) Cognitive Load in Multimodal Interpreting: Evidence from F0 and Interpreting Fluency. Journal of Humanities, Arts and Social Science, 9(3), 529-538.
DOI: http://dx.doi.org/10.26855/jhass.2025.03.018