跨学科视角下的多模态话语分析：理论、方法与实践
Multimodal Discourse Analysis from an Interdisciplinary Perspective: Theory, Methods and Practice
摘要: 随着新媒体技术的迅猛发展，多模态话语交际已成为我们日常生活中不可或缺的一部分。本文旨在深入探究多模态话语分析的理论基础，并通过系统的方法，全面梳理和分析国内在这一领域的主要研究焦点及其方法应用。研究从语言学视角切入，对语篇中的多模态话语分析进行了细致而深入的探讨，以期揭示当前研究的不足之处，并为该领域的未来发展趋势、潜在价值及研究方向提供前瞻性的思考与展望。
Abstract: With the rapid development of the new media, multimodal discourse communication has become an indispensable part of our daily life. The purpose of this paper is to explore the theoretical foundations of multimodal discourse analysis, and systematically review and analyze the primary research focuses and methodological applications in this field in China. Drawing from a linguistic perspective, the study conducts a detailed and in-depth exploration of multimodal discourse analysis in textual contexts, aiming to uncover the current shortcomings in research and provide forward-thinking insights and perspectives on the future development trends, potential values, and research directions of this field.
文章引用：谢红海. 跨学科视角下的多模态话语分析：理论、方法与实践[J]. 现代语言学, 2024, 12(8): 750-758. https://doi.org/10.12677/ml.2024.128747

参考文献

[1] Matthiessen, C., Teruya, K. and Lin, W.J. (2016) Key Terms in Systemic Functional Linguistics. Foreign Language Education Press, 77-78.
[2] Halliday, M.A.K. (1978) Language as Social Semiotic. Arnold.
[3] Jewitt, C. (2009) Different Approaches to Multimodality. In: Jewitt, C., Ed., The Routledge Handbook of Multimodal Analysis, Routledge, 1-39.
[4] Kress, G.R. and van Leeuwen, T.V. (2001) Multimodal Discourse: The Modes and Media of Contemporary Communication. Arnold, 20.
[5] Baldry, A.P. and Thibault, P.J. (2006) Multimodal Transcription and Text Analysis: A Multimedia Toolkit and Coursebook. Equinox, 21.
[6] 朱永生. 多模态话语分析的理论基础与研究方法[J]. 外语学刊, 2007(5): 82-86.
[7] 顾曰国. 多媒体、多模态学习剖析[J]. 外语电化教学, 2007(2): 3-12.
[8] 代树兰. 多模态话语研究电视访谈的多模态话语特征[M]. 上海: 上海外语教育出版社, 2015: 41-42.
[9] 胡壮麟. 社会符号学研究中的多模态化[J]. 语言教学与研究, 2007(1): 1-10.
[10] Kress, G.R. and van Leeuwen, T.V. (1998) Front Pages: The Critical Analysis of Newspaper Layout. In: Bell, A. and Garrett, P., Eds., Approaches to Media Discourse, Blackwell, 186-219.
[11] Iedema, R. (2003) Multimodality, Resemiotization: Extending the Analysis of Discourse as Multi-Semiotic Practice. Visual Communication, 2, 29-57. [Google Scholar] [CrossRef] 
[12] Harris, Z. (1952) Discourse Analysis. Language, 28, 1-30. [Google Scholar] [CrossRef] 
[13] Fairclough, N. (1989) Language and Power. Cambridge University Press, 2-3.
[14] van Dijk, T.A. (1997) Discourse as Structure and Process: Discourse Studies: A Multidisciplinary Introduction. SAGE Publications Ltd., 2.
[15] Martin, J.R. and Rose, D. (2007) Working with Discourse: Meaning beyond the Clause. 2nd Edition, Continuum, 7.
[16] 代树兰. 话语、话语分析与多模态话语分析的界定与发展[J]. 西安外国语大学学报, 2013, 21(3): 26-30.
[17] Lemke, J. (2002) Travels in Hypermodality. Visual Communication, 1, 299-325. [Google Scholar] [CrossRef] 
[18] Barthes, R. (1977) Rhetoric of the Image. In: Barthes, R., Ed., Image-Music-Text, Fontana, 32-51.
[19] Nöth, W. (1990) Handbook of Semiotics. Indiana University Press. [Google Scholar] [CrossRef] 
[20] O’Toole, M. (1994) The Language of Displayed Art. Fairleigh Dickison University Press.
[21] Kress, G.R. and van Leeuwen, T.V. (1996) Reading Images: The Grammar of Visual Design. Routledge, 19.
[22] Norris, S. (2004) Analyzing Multimodal Interaction: A Methodological Framework. Routledge.
[23] Kress, G.R. and Hodge, R.I.V. (1988) Social Semiotics. Polity Press.
[24] 李梦洁, 冯德正, 邓谊. 国际多模态话语分析的进展与前沿[J]. 现代外语, 2024, 47(3): 419-430.
[25] O’Halloran, K.L. (2011) Multimodal Discourse Analysis. In: Hyland, K., Ed., The Continuum Companion to Discourse Analysis. Bloomsbury Publishing, 120-137.
[26] Macken-Horarik, M. (2003) A Telling Symbiosis in the Discourse of Hatred: Multimodal News Texts About the ‘Children Overboard’ Affair. Australian Review of Applied Linguistics, 26, 1-16. [Google Scholar] [CrossRef] 
[27] Cheong, Y.Y. (2004) The Construal of Ideational Meaning in Print Advertisements. In: O’Halloran, K.L., Ed., Multimodal Discourse Analysis: Systemic-Functional Perspectives, Continuum, 173-195.
[28] Refaie, E.E. (2003) Understanding Visual Metaphor: The Example of Newspaper Cartoons. Visual Communication, 2, 75-95. [Google Scholar] [CrossRef] 
[29] Moya Guijarro, J. and Pinar Sanz, M.J. (2008) Compositional, Interpersonal and Representational Meanings in Children’s Narrative—A Multimodal Discourse Analysis. Journal of Pragmatics, 40, 161-99. [Google Scholar] [CrossRef] 
[30] Pennock-Speck, B. and Saz-Rubio, M. (2013) A Multimodal Analysis of Facework Strategies in a Corpus of Charity Ads on British Television. Journal of Pragmatics, 49, 38-56. [Google Scholar] [CrossRef] 
[31] Knox, J. (2007) Visual-Verbal Communication on Online Newspaper Home Pages. Visual Communication, 6, 19-53. [Google Scholar] [CrossRef] 
[32] Lipovsky, C. (2014) The CV as a Multimodal Text. Visual Communication, 13, 429-458. [Google Scholar] [CrossRef] 
[33] Bateman, J.A., Veloso, F.O. and Lau, Y.L. (2019) On the Track of Visual Style: A Diachronic Study of Page Composition in Comics and Its Functional Motivation. Visual Communication, 20, 209-247. [Google Scholar] [CrossRef] 
[34] O’Halloran, K.L., Beezer, R.A. and Farmer, D.W. (2018) A New Generation of Mathematics Textbook Research and Development. ZDM, 50, 863-879. [Google Scholar] [CrossRef] 
[35] Pang, K.M.A. (2004) Making History in From Colony to Nation: A Multimodal Analysis of a Museum Exhibition in Singapore. In O’ Halloran, K.L., Ed., Multimodal Discourse Analysis: Systemic-Functional Perspectives. Continuum, 28-54.
[36] Safeyaton, A. (2004) The Lion City as a Text—A Semiotic Study of Singapore’s Orchard Road and Marriott Hotel. In: O’Halloran, K.L., Ed., Multimodal Discourse Analysis: Systemic Functional Perspectives, Continuum, 55-82.
[37] Ravelli, L.J. and McMurtrie, R.J. (2016) Multimodality in the Built Environment: Spatial Discourse Analysis. Routledge. [Google Scholar] [CrossRef] 
[38] Maiorani, A. (2020) Kinesemiotics: Modelling How Choreographed Movement Means in Space. Routledge. [Google Scholar] [CrossRef] 
[39] Martin, J.R. and Unsworth, L. (2023) Reading Images for Knowledge Building: Analyzing Infographics in School Science. Routledge. [Google Scholar] [CrossRef] 
[40] O’Halloran, K.L. (2004) Visual Semiosis in Film. In: O’ Halloran, K.L., Ed., Multimodal Discourse Analysis: Systemic Functional Perspectives, Continuum, 109-130.
[41] Baldry, A.P. (2004) Phase and Transition, Type and Instance: Patterns in Media Texts as Seen through a Multimodal Concordancer. In O’ Halloran, K.L., Ed., Multimodal Discourse Analysis: Systemic-Functional Perspectives. Continuum, 83-108.
[42] Bateman, J. (2014) Using Multimodal Corpora for Empirical Research. In: Jewitt, C., Ed., The Routledge Handbook of Multimodal Analysis, Routledge, 238.
[43] Wildfeuer, J. (2014) Film Discourse Interpretation: Towards a New Paradigm for Multimodal Film Analysis. Routledge, 10. [Google Scholar] [CrossRef] 
[44] Bateman, J. and Schmidt, K.H. (2011) Multimodal Film Analysis. Routledge, 3-7.
[45] Wang, Y. and Feng, D. (2022) Identity Performance and Self-Branding in Social Commerce: A Multimodal Content Analysis of Chinese Wanghong Women’s Video-Sharing Practice on TikTok. Discourse, Context & Media, 50, Article 100652. [Google Scholar] [CrossRef] 
[46] Bateman, J. (2008) Multimodality and Genre: A Foundation for the Systematic Analysis of Multimodal Documents. Palgrave-Macmillan.
[47] Bateman, J., Wildfeuer, J. and Hiippala, T. (2017) Multimodality: Foundations, Research and Analysis—A Problem-Oriented Introduction. De Gruyter Mouton.
[48] Bateman, J. (2014) Developing a GeM Model. In: Norris, S. and Maier, C.D., Eds., Interactions, Images and Texts: A Reader in Multimodality, De Gruyter Mouton, 25-36. [Google Scholar] [CrossRef] 
[49] 李战子. 多模式话语的社会符号学分析[J]. 外语研究, 2003(5): 1-8.
[50] 张德禄. 多模态话语分析综合理论框架探索[J]. 中国外语, 2009, 6(1): 24-30.
[51] 张德禄, 郭恩华. 多模态话语分析的双重视角——社会符号观与概念隐喻观连接与互补[J]. 外国语, 2013, 36(3): 20-28.
[52] 冯德正. 多模态语篇分析的基本问题探讨[J]. 北京第二外国语学院学报. 2017, 39(3): 1-11.
[53] 田海龙, 潘艳艳. 从意义到意图——多模态话语分析到多模态批评话语分析的新发展[J]. 山东外语教学, 2018, 39(1): 23-33.
[54] 潘艳艳. 多模态话语分析到多模态认知批评分析的发展综述[J]. 外国语文, 2020, 36(1): 35-42.
[55] 史兴松, 徐文娟. 近十五年SSCI期刊网络多模态话语研究现状及发展趋势分析[J]. 外国语, 2020, 43(3): 55-66.
[56] 王正. 动态多模态语篇批评话语分析模式的构建与应用[J]. 西安外国语大学学报, 2022, 30(4): 25-30.
[57] Feng, D.Z. and O’ Halloran, K.L. (2012) Representing Emotion in Visual Images: A Social Semiotic Approach. Journal of Pragmatics, 44, 2067-2084. [Google Scholar] [CrossRef] 
[58] 潘艳艳. 战争影片的多模态转喻批评分析[J]. 外语教学, 2020, 41(2): 13-18.
[59] 罗晓春. 多模态语篇分析研究——以电影《贫民窟的百万富翁》海报为例[J]. 语文学刊(外语教育教学), 2012(5): 46-48.
[60] 宋旭, 梅丽莎∙莎米妮∙佩里. 生态哲学观视域下高校英语教材的多模态话语分析——以《新交际英语综合教程》为例(英文) [J]. 中国应用语言学(英文), 2023, 46(3): 459-484, 487.
[61] Zhang, Y.Q. and O’Halloran, K.L. (2012) The Gate of the Gateway: A Hypermodal Approach to University Homepages. Semiotica, 2012, 87-109. [Google Scholar] [CrossRef] 
[62] Feng, D.Z. (2012) Modeling Appraisal in Film: A Social Semiotic Approach. PhD Thesis, National University of Singapore.
[63] 姚银燕, 陈晓燕. 对视频语篇的多模态话语分析——以一则企业形象电视广告为例[J]. 外国语文, 2013, 29(1): 86-91.
[64] 应洁琼, 谢朝群. 英语教学支架语言与手势、身体姿势、表情的意义协同研究[J]. 中国外语, 2024, 21(2): 58-66.
[65] 李妙晴. 改编电影的多模态话语分析——以《大红灯笼高高挂》为例[J]. 电影文学, 2007(15): 98-100.
[66] Hopperstad, M.H. (2010) Studying Meaning in Children’s Drawings. Journal of Early Childhood Literacy, 10, 430-452. [Google Scholar] [CrossRef] 
[67] Lirola, M.M. and Chovanec, J. (2012) The Dream of a Perfect Body Come True: Multimodality in Cosmetic Surgery Advertising. Discourse & Society, 23, 487-507. [Google Scholar] [CrossRef] 
[68] 国防. 多模态话语分析研究热点及趋势分析——基于文献计量学方法[J]. 外语与外语教学, 2016(3): 58-66.
[69] 韦琴红. 视觉环境下的多模态化与多模态话语研究[M]. 北京: 科学出版社, 2009.
[70] Feng, D. (2023) Multimodal Chinese Discourse: Understanding Communication and Society in Contemporary China. Routledge. [Google Scholar] [CrossRef]