Interpreters as Mediators in Medical Settings: A Multimodal AI-Assisted Behavioural Analysis
DOI:
https://doi.org/10.33806/ijaes1309Keywords:
AI, Body language, GCBA, medical interpreters, mediators, non-verbal communicationAbstract
The critical role of interpreters in facilitating communication in healthcare settings is beyond dispute, especially for patients’ comprehension, interaction, and safety. Interpreters have moved from traditional neutral conduits to mediators whose vital responsibility is to enhance communication and facilitate interactional coordination. In modern interpreting studies, being a mediator means actively participating in both verbal and nonverbal communication, including facial expressions, spatial positioning, and posture. The rapid development of artificial intelligence (AI), exemplified by technologies for computer vision and posture estimation, can address these novel methodological challenges in the systematic analysis of behavioural markers. Using AI-assisted behavioural analysis, this study examines whether the medical interpreters portrayed in 20 photos function as mediators. Qualitative multimodal methods were utilised, including the General Communication Behaviour Analysis (GCBA) framework proposed by AI, body language clusters, and Positioning Theory. The analysis revealed behavioural features that correspond with mediation, such as neutral posture, balanced positioning among participants, and controlled facial expressions, which supported interactional coordination and communicative neutrality. The findings show that interpreters are highly skilled communicators in medical settings and effective mediators; in addition, AI-assisted behavioural analysis offers a systematic, theoretically grounded method for investigating interpreter roles through observable nonverbal behaviour. This paper demonstrates that AI-assisted behavioural analysis, as a methodological instrument, applies to interpreting studies by assessing interpreter mediation and extending multimodal research methodologies to include static photo analysis in healthcare interpreting scenarios.
References
Abdulghafor, Ahmed, Sukhrob Turaev and Mohammed A. Ali. (2022). ‘Body language analysis in healthcare: An overview’. Healthcare, 10(7): 12-51. https://doi.org/10.3390/healthcare10071251
Abed, Ahmed Q. and Sura J. Wadia. (2024). ‘A multimodal translation analysis of online app advertisements: The case of Careem as a taxi-sharing app in Iraq’. Multimodal Communication, 13(2): 199–215.
https://doi.org/10.1515/mc-2023-0037
Andersen, Peter A., Michael L. Hecht, Gregory D. Hoobler and Maya Smallwood. (2003). ‘Nonverbal communication across cultures’. In William B. Gudykunst (ed.), Cross-Cultural and Intercultural Communication, 73–90. California: Thousand Oaks, Sage.
Angelelli, Claudia V. (2004). Medical Interpreting and Cross-Cultural Communication. Cambridge: Cambridge University Press.
Baltrušaitis, Tadas, Chaitanya Ahuja and Louis-Philippe Morency. (2019). ‘Multimodal machine learning: A survey and taxonomy’. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(2): 423–443. https://doi.org/10.1109/TPAMI.2018.2798607
Birdwhistell, Ray L. (1970). Kinesics and Context: Essays on Body Motion Communication. Philadelphia: University of Pennsylvania Press.
Bolden, Galina B. (2000). ‘Toward understanding practices of medical interpreting: Interpreters’ involvement in history taking’. Discourse Studies, 2(4): 387–419. https://doi.org/10.1177/1461445600002004001
Boydell, Katherine M., Paula Goering and Terri L. Morell-Bellai. (2000). ‘Narratives of identity: Re-presentation of self in people who are homeless’. Qualitative Health Research, 10(1): 26–38. https://doi.org/10.1177/104973200129118228
Brashers, Dale E., Daena J. Goldsmith and Elaine Hsieh. (2002). ‘Information seeking and avoiding in health contexts’. Human Communication Research, 28(2): 258–271.
https://doi.org/10.1111/j.1468-2958.2002.tb00807.x
Burgoon, Judee K., Laura K. Guerrero and Kory Floyd. (2016). Nonverbal Communication. New York: Routledge.
https://doi.org/10.4324/9781315663425
Burns, Shannon M., Lily Tsoi, Emily B. Falk, Sebastian P. H. Speer, Laetitia Mwilambwe-Tshilobo and Diana I. Tamir. (2025). ‘Interdependent minds: Quantifying the dynamics of successful social interaction’. Current Directions in Psychological Science, 34(5): 271–277.
https://doi.org/10.1177/09637214251323598
Cao, Zhe, Gines Hidalgo, Tomas Simon, Shih-En Wei and Yaser Sheikh. (2021). ‘OpenPose: Realtime multi-person 2D pose estimation using part affinity fields’. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(1): 172–186.
https://doi.org/10.1109/TPAMI.2019.2929257
Chang, Doris F., Elaine Hsieh, William B. Somerville, Jon Dimond, Monica Thomas, Andel Nicasio, Marit Boiler and Roberto Lewis-Fernández. (2021). ‘Rethinking interpreter functions in mental health services’. Psychiatric Services, 72(3): 353–357.
https://doi.org/10.1176/appi.ps.202000085
Chovil, Nicole. (2024). ‘From the neck up: Facial gestures in dialogue’. In Alan Cienki (ed.), The Cambridge Handbook of Gesture Studies, 112 - 130.
Cambridge: Cambridge University Press.
Connell, John. (2013). ‘Contemporary medical tourism: Conceptualisation, culture and commodification’. Tourism Management, 34: 1–13.
https://doi.org/10.1016/j.tourman.2012.05.009
Corsellis, Ann. (2008). Public Service Interpreting: The First Steps. Basingstoke: Palgrave Macmillan.
Davies, Bronwyn and Rom Harré. (1990). ‘Positioning: The discursive production of selves’. Journal for the Theory of Social Behaviour, 20(1): 43–63. https://doi.org/10.1111/j.1468-5914.1990.tb00174.x
Ekman, Paul. (1992). ‘An argument for basic emotions’. Cognition and Emotion, 6(3-4): 169–200. https://doi.org/10.1080/02699939208411068
Fang, Xia, Disa A. Sauter and Gerben A. van Kleef. (2020). ‘Unmasking smiles: The influence of culture and intensity on interpretations of smiling expressions’. Journal of Cultural Cognitive Science, 4(2): 293–308. https://doi.org/10.1007/s41809-019-00053-1
Flores, Glenn, Milagros Abreu, Carrie P. Barone, Richard Bachur and Hong Lin. (2012). ‘Errors of medical interpretation and their potential clinical consequences’. Annals of Emergency Medicine, 60(5): 545–553.
https://doi.org/10.1016/j.annemergmed.2012.01.025
Fu, Bo. (2024). ‘Predictive gaze analytics: A comparative case study of the foretelling signs of user performance during interaction with visualizations of ontology class hierarchies’. Multimodal Technologies and Interaction, 8(90): 1–23. https://doi.org/10.3390/mti8100090
Gentile, Adolfo, Uldis Ozolins and Mary Vasilakakos. (1996). Liaison Interpreting: A Handbook. Melbourne: Melbourne University Press.
Goffman, Erving. (1959). The Presentation of Self in Everyday Life. New York: Doubleday.
Goodwin, Charles. (2000). ‘Action and embodiment within situated human interaction’. Journal of Pragmatics, 32(10): 1489–1522.
https://doi.org/10.1016/S0378-2166(99)00096-X
Grillo, Helga M. and Marsela Enesi. (2022). ‘The impact, importance, types, and use of non-verbal communication in social relations’. Linguistics and Culture Review, 6(3): 291–307.
https://doi.org/10.21744/lingcure.v6nS3.2161
Hale, Sandra. (2015). ‘Community interpreting’. In Franz Pöchhacker (ed.), Routledge Encyclopedia of Interpreting Studies, 65–69. London: Routledge.
Hall, Edward T. (1983). The Dance of Life: The Other Dimension of Time. New York: Anchor Press/Doubleday.
Hargie, Owen and David Dickson. (2004). Skilled Interpersonal Communication: Research, Theory and Practice. Maidenhead: Open University Press.
Harré, Rom and Luk Van Langenhove (eds.). (1999). Positioning Theory: Moral Contexts of Intentional Action. Oxford: Blackwell.
Harris, Brian. (1997). ‘The importance of natural translation’. Target, 9(1): 25–41. https://doi.org/10.1075/target.9.1.03har
Hirvonen, Päivi. (2016). ‘Positioning theory and small-group interaction in the context of joint decision-making’. SAGE Open, 6(2): 1–15. https://doi.org/10.1177/2158244016655584
Hirvonen, Pasi, Michael Bamberg, Luk van Langenhove and Bo Allesøe Christensen. (2025). ‘A discussion of positioning theory: An interview with Michael Bamberg and Luk Van Langenhove’. Culture and Psychology, 31(3): 1090–1106.
https://doi.org/10.1177/1354067X251340188
Hsieh, Elaine. (2003). ‘Interpreters as co-diagnosticians: Overlapping roles and services between providers and interpreters’. Social Science and Medicine, 56(5): 1053–1064.
https://doi.org/10.1016/S0277-9536(02)00150-9
Hsieh, Elaine. (2016). Bilingual Health Communication: Working with Interpreters in Cross-Cultural Care. London: Routledge.
Ibbotson, Piers. (2020). ‘Interpreting under pressure: Emotional regulation and professional composure in high-stakes settings’. The Interpreter and Translator Trainer, 14(3): 215–231.
Kaufert, Joseph M. and Robert W. Putsch. (1997). ‘Communication through interpreters in healthcare: Ethical dilemmas arising from differences in class, culture, language, and power’. Journal of Clinical Ethics, 8(1): 71–87.
Kayi-Aydar, Hayriye (2019). ‘Positioning theory and discourse analysis: Research design and applications’. Cham: Palgrave Macmillan. https://doi.org/10.1007/978-3-319-97337-1_2
Kendon, Adam. (1967). ‘Some functions of gaze-direction in social interaction’. Acta Psychologica, 26: 22–63.
https://doi.org/10.1016/0001-6918(67)90005-4
Kendon, Adam. (2004). Gesture: Visible Action as Utterance. Cambridge: Cambridge University Press.
Knapp, Mark L., Judith A. Hall and Terrence G. Horgan. (2014). Nonverbal Communication in Human Interaction. Boston: Cengage.
Kress, Gunther and Theo van Leeuwen. (2021). Reading Images: The Grammar of Visual Design. London: Routledge.
https://doi.org/10.4324/9781003099857
Kurien, Daisy. (2010). ‘Body language: Silent communicator at the workplace’. IUP Journal of Soft Skills, 4(1-2): 29–36.
Leanza, Yvan. (2005). ‘Roles of community interpreters in pediatrics as seen by interpreters, physicians and researchers’. Interpreting, 7(2): 167–192. https://doi.org/10.1075/intp.7.2.03lea
Li, Saihong, Yifang Wang and Yubo Z. Rasmussen. (2023). ‘Studying interpreters’ stress in crisis communication: Evidence from multimodal technology of eye-tracking, heart rate and galvanic skin response’. The Translator, 28(4): 468–488.
https://doi.org/10.1080/13556509.2022.2159782
Lipovec Čebron, Uršula and Jelka Škraban. (2022). ‘Intercultural mediation and its conflicting allegiances in Slovenia’. Ethnologia Europaea, 52(2): 1–23. https://doi.org/10.16995/ee.3103
Mahdi, Assal I. and Hala Gh. Mohammed. (2025). ‘Accuracy of medical terminologies translated from English into Arabic’. International Journal of Arabic-English Studies, 25(2): 147–168.’
https://doi.org.10.33806/ijaes.v25i2.506
McNeill, David. (2005). Gesture and Thought. Chicago: University of Chicago Press.
McVee, Mary B. and Luk Van Langenhove. (2025). ‘An introduction to positioning theory past and present’. In Mary B. McVee, Luk Van Langenhove, Cynthia H. Brock and Charlotte H. Christensen (eds.) The Routledge International Handbook of Positioning Theory, 3–43. London: Routledge.
Mehrabian, Albert and Susan R. Ferris. (1967). ‘Inference of attitudes from nonverbal communication’. Journal of Consulting Psychology, 31(3): 248–252. https://doi.org/10.1037/h0024648
Mohammed, Hala Gh. and Balsam Mustafa. (2023). ‘Forced to go online: A case study of learning consecutive and simultaneous interpreting under COVID-19 in Iraq’. Translation and Interpreting, 15(1): 176–199.
https://doi.org/10.12807/ti.115201.2023.a09
Niska, Helge. (2007). ‘From helpers to professionals’. In Cecilia Wadensjö, Birgitta Englund Dimitrova and Anna-Lena Nilsson (eds.), The Critical Link 4, 297–310. Amsterdam: John Benjamins.
Norris, Sigrid. (2004). Analyzing Multimodal Interaction: A Methodological Framework. London: Routledge.
Ono, Naoko and Jinghua Yang. (2024). ‘Communication skills of medical interpreters: A qualitative explanatory study of healthcare professionals’ and medical interpreters’ perceptions ’. Healthcare, 12(20): 2-19.
https://doi.org/10.3390/healthcare12202073
Ozolins, Uldis. (2011). ‘Telephone interpreting: Understanding practice and identifying research needs’. Translation and Interpreting, 3(1): 33–47.
Pöchhacker, Franz. (2016). Introducing Interpreting Studies. London: Routledge.
Poggi, Isabella. (2025). The Language of Gaze: Eyes That Talk. London: Routledge.
Poyatos, Fernando. (1997). La Comunicación No Verbal: Cultura, Lenguaje y Conversación. Madrid: Istmo.
Poyatos, Fernando. (2002). Nonverbal Communication Across Disciplines: Culture, Sensory Interaction, Speech, Conversation. Amsterdam: John Benjamins.
Rennert, Susanne. (2008). ‘Visual input in simultaneous interpreting’. Meta, 53(1): 204–213. https://doi.org/10.7202/017983ar
Samsudin, Aisyah J., Ainul F. Abdul Rahman and Hayati B. Ismail. (2021). ‘Medical interpreter challenges during in-person interpretation session: A systematic review’. Journal of Legal, Ethical and Regulatory Issues, 24(1): 1–12.
Seeber, Kilian G. (2013). ‘Cognitive load in simultaneous interpreting: Measures and methods’. In Maureen Ehrensberger-Dow, Susanne Göpferich and Sharon O’Brien (eds.), Interdisciplinarity in Translation and Interpreting Process Research, 18–32. Amsterdam: John Benjamins.
Shiffrar, Maggie. (2008). ‘The visual perception of dynamic body language’. In Ipke Wachsmuth, Martin Lenzen and Günther Knoblich (eds.), Embodied Communication in Humans and Machines, 95–110. Oxford: Oxford University Press.
Souza, Eliana. (2016). Intercultural Mediation in Health Care: From the Professional Medical Interpreters’ Perspective. Bloomington: Xlibris.
Stenum, Jan, Cristina Rossi and Ryan T. Roemmich. (2021). ‘Two-dimensional video-based pose estimation’. Journal of Biomechanics, 117: 110223. https://doi.org/10.1016/j.jbiomech.2021.110223
Tiselius, Elisabet and Kayle Sneed. (2020). ‘Gaze and eye movement in dialogue interpreting: An eye-tracking study’. Bilingualism: Language and Cognition, 23(4): 780–787.
https://doi.org/10.1017/S1366728920000309
Tsai, Yao-Hung Hubert, Shaojie Bai, Paul Pu Liang, J. Zico Kolter, Louis-Philippe Morency and Ruslan Salakhutdinov. (2019). ‘Multimodal transformer for unaligned multimodal language sequences’. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 6558–6569. Florence, Italy: Association for Computational Linguistics. https://doi.org/10.18653/v1/P19-1656
Wadensjö, Cecilia. (1998). Interpreting as Interaction. London: Longman.
Wadensjö, Cecilia. (2022). ‘Interpreting as interaction: Theoretical and analytical implications’. Interpreting and Translation Studies, 22: 1–16. https://www.jstage.jst.go.jp/article/its/22/0/22_2201/_pdf/-char/en
World Health Organization. (2021). Iraq Health System Profile. Cairo: WHO Regional Office for the Eastern Mediterranean.
Zeng, Zhihong, Maja Pantic, Glenn I. Roisman and Thomas S. Huang. (2009). ‘A survey of affect recognition methods: Audio, visual, and spontaneous expressions’. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(1): 39–58.
