The Negative Transfer Effect on the Neural Machine Translation of Egyptian Arabic Adjuncts into English: The Case of Google Translate
DOI:
https://doi.org/10.33806/ijaes.v24i1.560Keywords:
Arabic dialects, Egyptian Arabic, Google Translate, negative transfer, low-resource dialects, neural machine translation, parallel corporaAbstract
Parallel corpora for low-resource Arabic dialects and English are limited and small-scale, and most neural machine translation models, including Google Translate, rely mainly on parallel corpora of standard Arabic and English to train for dialectal Arabic translation. A model well trained to translate to and from standard Arabic is believed to efficiently translate dialectal Arabic, given their similarities. This study demonstrates the impact of not using large-scale, dialect-specific parallel corpora by quantitatively and qualitatively analyzing the performance of Google Translate in translating Egyptian Arabic adjuncts. Compared to human reference translation, Google Translate achieved a low BLEU score of 14.69. Qualitative analysis showed that reliance on standard Arabic parallel corpora caused a negative transfer problem manifested in the literal translation of idiomatic adjuncts, the misinterpretation of dialectal adjuncts as main clause constituents, the translation of dialectal adjuncts after orthographically similar standard Arabic words, and the use of standard Arabic common lexical meanings to translate dialect-specific adjuncts. This study’s findings will be relevant for researchers interested in dialectal Arabic neural machine translation and has implications for investment in the development of large-scale, dialect-specific corpora to better process the peculiarities of Arabic dialects and reduce the effect of negative transfer from standard Arabic.
References
Alkhawaja, Linda, Hanan Ibrahim, Fida’ Ghnaim and Sirine Awwad. (2019). ‘Neural machine translation: Fine-grained evaluation of Google Translate output for English-to-Arabic translation’. International Journal of English Linguistics, 10 (4): 43–60.
Almahairi, Amjad, Kyunghyun Cho, Nizar Habash and Aaron Courville. (2016). First Result on Arabic Neural Machine Translation. ArXiv Preprints.
Baniata, Laith H., Seyoung Park and Seong-Bae Park. (2018). ‘A neural machine translation model for Arabic dialects that utilizes multitask learning (MTL)’. Computational Intelligence and Neuroscience, 2018 (1): 1-10. https://doi.org/10.1155/2018/7534712
Baniata, Laith, Isaac Kojo Essel Ampomah and Seyoung Park. (2021). ‘A transformer-based neural machine translation model for Arabic dialects that utilizes subword units’. Sensors, 21 (19): 6509.
Baziotis, Christos, Mikel Artetxe, James Cross and Shruti Bhosale. (2022). ‘Multilingual machine translation with hyper-adapters’. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. United Arab Emirates: Association for Computational Linguistics, 1170–1185.
Bouamor, Houda, Nizar Habash and Kemal Oflazer. (2014). ‘A multidialectal parallel corpus of Arabic’. Proceedings of the Ninth Conference on Language Resources and Evaluation (LREC’14). Iceland: European Language Resources Association (ELRA), 1240–1245.
Bouamor, Houda, Nizar Habash, Mohammad Salameh, Wajdi Zaghouani, Owen Rambow, Dana Abdulrahim, Ossama Obeid, Salam Khalifa, Fadhl Eryani, Alexander Erdmann and Kemal Oflazer. (2018). ‘The MADAR Arabic dialect corpus and lexicon’. Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018). Japan: European Language Resources Association (ELRA), 3387–3396.
Chen, Song, Jennifer Tracey, Christopher Walker and Stephanie Strassel. (2019). BOLT Arabic Discussion Forum Parallel Training Data LDC2019T01. Web Download. Philadelphia: Linguistic Data Consortium (LDC).
Dabre, Raj, Chenhui Chu and Anoop Kunchukuttan. (2020). ‘A survey of multilingual machine translation’. ACM Computing Surveys, 53 (5): 1–31.
Diab, Nesma. (2021). Out of the BLEU: ‘An error analysis of statistical and neural machine translation of WikiHow articles from English into Arabic’. CDELT Occasional Papers in the Development of English Education, 75 (1): 181–211.
Farhan, Wael, Bashar Talafha, Analle Abuammar, Ruba Jaikat, Mahmoud Al-Ayyoub, Ahmed Bashir Tarakji and Anas Toma. (2020). ‘Unsupervised dialectal neural machine translation’. Information Processing & Management, 57 (3): 102181.
Gadalla, Hassan, Hanaa Kilany, Howaida Arram, Ashraf Yacoub, Alaa El-Habashi, Amr Shalaby, Krisjanis Karins, Everett Rowson, Robert MacIntyre, Paul Kingsbury, David Graff, Cynthia McLemore. (1997). CALLHOME Egyptian Arabic Transcripts LDC97T19 [Corpus]. Web Download. Philadelphia: Linguistic Data Consortium (LDC).
Halliday, Michael A. K. and Christian M. I. M. Matthiessen. (2014). Halliday’s Introduction to Functional Grammar (4th ed.). London and New York: Routledge.
Hamed, Injy, Nizar Habash, Slim Abdennadher, and Ngoc Thang Vu. (2022). ‘ArzEn-ST: A three-way speech translation corpus for code-switched Egyptian Arabic–English’. Proceedings of the Seventh Arabic Natural Language Processing Workshop (WANLP), 119–130. United Arab Emirates Association for Computational Linguistics.
Harrat, Salima, Karima Meftouh, and Kamel Samili. (2019). ‘Machine translation for Arabic dialects (survey)’. Information Processing & Management, 56 (2):262–273.
Hassan, Hany, Mostafa Elaraby and Ahmed Y. Tawfik. (2017). ‘Synthetic data for neural machine translation of spoken dialects’. Proceedings of the 14th International Workshop on Spoken Language Translation, 82–89. Japan: International Workshop on Spoken Language Translation.
Kchaou, Saméh, Rahma Boujelbane and Lamia Hadrich. (2023). ‘Hybrid pipeline for building Arabic Tunisian dialect-standard Arabic neural machine translation model from scratch’. ACM Transactions on Asian and Low-Resource Language Information Processing, 22 (3):1–21.
Kumar, Gaurav, Yuan Cao, Ryan Cotterell, Chris Callison-Burch, Daniel Povey and Sanjeev Khudanpur. (2014). Translations of the CALLHOME Egyptian Arabic corpus for conversational speech translation. Proceedings of the 11th International Workshop on Spoken Language Translation: Papers. USA: Association for Computational Linguistics, 244–248.
Li, Xuansong, Stephen Grimes, and Stephanie Strassel. (2019). BOLT Egyptian Arabic–English Word Alignment -- SMS/Chat Training LDC2019T18. Web Download. Philadelphia: Linguistic Data Consortium (LDC).
Linguistic Data Consortium. (2002a). CALLHOME Egyptian Arabic Speech Supplement LDC2002S38 [Corpus]. Philadelphia: Linguistic Data Consortium (LDC).
Linguistic Data Consortium. (2002b). 1997 HUB5 Arabic Transcripts LDC2002T39 [Corpus]. Philadelphia: Linguistic Data Consortium (LDC).
Lison, Pierre and Jörg Tiedemann. (2016). ‘OpenSubtitles2016: Extracting large parallel corpora from movie and TV subtitles’. Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’2016), 923–929. Slovenia: European Language Resources Association (ELRA).
Meftouh, Karima, Salima Harrat, Salma Jamoussi, Mourad Abbas, and Kamel Smaili. (2015). ‘Machine translation experiments on PADIC: A parallel Arabic dialect corpus.’ Proceedings of the 29th Pacific Asian Conference on Language, Information and Computation, 26–34. China.
Nagoudi, El Moatez Billah, AbdelRahim Elmadany and Muhammad Abdul-Mageed. (2021). ‘Investigating code-mixed modern standard Arabic–Egyptian to English machine translation’. Proceedings of the Fifth Workshop on Computational Approaches to Linguistics Code-Switching, 56–64. Online: Association for Computational Linguistics.
Nagoudi, El Moatez Billah, AbdelRahim Elmadany and Muhammad Abdul-Mageed. (2022). ‘TURJUMAN: A public toolkit for neural Arabic machine translation’. Proceedings of the Open Source and Arabic Corpora and Processing Tools (OSCAT 2022) Workshop, 1–11. France: European Language Resources Association (ELRA).
Richardson, Ashleigh and Janet Wiles. (2022). ‘A systematic study reveals unexpected interactions in pre-trained neural machine translation’. Proceedings of the Thirteenth Conference on Language Resources and Evaluation, 1437–1443. France: European Language Resources Association (ELRA).
Saleh, Fahimeh, Wray Buntine, and Gholamreza Haffari. (2021). ‘Collective wisdom: Improving low-resource neural machine translation using adaptive knowledge distillation’. Proceedings of the 28th International Conference on Computational Linguistics, 3413–3421. Spain: International Committee on Computational Linguistics.
Slim, Amelm, Ahlem Melouah, Usef Faghihi and Khouloud Sahib. (2022). ‘Improving neural machine translation for low resource Algerian dialect by transductive transfer learning strategy’. Arabian Journal for Science and Engineering, 47: 10411–10418.
Sun, Mengtao, Hao Wang, Mark Pasquine and Ibrahim A. Hameed. (2021). ‘Machine translation in low-resource languages by an adversarial neural network’. Applied Science, 11 (22): 10860.
Takezawa, Toshiyuki, Genichiro Kikui, Masahide Mizushima and Eiichiro Sumita. (2007). ‘Multilingual spoken language corpus development for communication research’. International Journal of Computational Linguistics and Chinese Language Processing: Special Issue on Invited Papers from ISCSLP 2006, 12 (3): 303–324.
Tiedemann, Jörg. (2012). ‘Parallel data, tools and interfaces in OPUS’. Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC’12), 2214–2218. Turkey: European Language Resources Association (ELRA).
Tilde. (2023). Interactive BLEU Score Evaluator [Software].
https://www.tilde.com/products-and-services/machine-translation/features/interactive-bleu
United Nations Population Fund. (2023, April 23). World Population Dashboard. https://www.unfpa.org/data/world-population/EG
Pitman, Jeff. (2021, April 28). Google Translate: One billion installs, one billion stories. Google. https://blog.google/products/translate/one-billion-installs/ (Retrieved on May 13, 2023)
Wang, Zirui, Zihang Dai, Barnabas Poczos and Jaime Carbonell. (2019). ‘Characterizing and avoiding negative transfer’. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 11293–11302 USA: Computer Vision Foundation.
https://doi.org/10.1109/CVPR.2019.01155
Wang, Zirui, Zachary C. Lipton and Yulia Tsvetkov. (2020). ‘On negative interference in multilingual models: Findings and a meta-learning treatment’. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 4438–4450. Online: Association for Computational Linguistics.
Zakraoui, Jezia, Moutaz Saleh, Somaya Al-Maadeed and Jihad Mohamed Alja’am. (2021). ‘Arabic machine translation: A survey with challenges and future directions’. IEEE Access, 9: 161445–161468.
Zbib, Rabih, Erika Malchiodi, Jacob Devlin, David Stallard, Spyros Matsoukas, Richard Schwartz, John Makhoul, Omar F. Zaidan and Chris Callison-Burch. (2012). ‘Machine translation of Arabic dialects’. Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 49–59. Canada: Association for Computational Linguistics.
Zhang, Wen, Lingfei Deng, Lei Zhang and Dongrui Wu. (2023). ‘A survey on negative transfer’. IEEE/CAA Journal of Automatica Sinica, 10 (2): 305–329.
