A Text Typological Assessment of Automatic Translation
A Case Study of English into Kurdish Translated Texts by Open AI GPT and Google Translate
DOI:
https://doi.org/10.24086/cuejhss.v10n1y2026.pp13-18Keywords:
Artificial intelligence, Google translate, Kurdish language, Machine translation, OpenAI GPT, Translation quality assessmentAbstract
The extensive use of artificial intelligence in translation projects has motivated this study to evaluate the quality of two popular systems, GPT Open AI and Google Translate. To assess the systems’ performance 15 sentences were chosen from a variety of text types based on Katharina Riess’s text typology informative, expressive, and vocative and were used as input data for translation into the Kurdish language. The output translations were then assessed using two evaluation metrics: BLEU and TER. The findings revealed that overall, GPT outperformed Google Translate, as it achieved a higher BLEU score reflecting better choices in equivalence and sentence structure and a lower TER score, indicating fewer necessary corrections in the translated text compared to the human (reference sentences) translation. In particular, GPT presents a better performance in the translation of expressive and vocative texts, where understanding emotions and persuasive language is more difficult.
Downloads
References
Bahdanau, D., Cho, K., & Bengio, Y. (2015). Neural machine translation by jointly learning to align and translate. International Conference on Learning Representations (ICLR).
Brown, P. F., Pietra, V. J. D., Pietra, S. A. D., & Mercer, R. L. (1993). The mathematics of statistical machine translation: Parameter estimation. Computational Linguistics, 19(2), 263–311.
Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 4171–4186). DOI: https://doi.org/10.18653/v1/N19-1423
Hutchins, W. J., & Somers, H. L. (1992). An introduction to machine translation. Academic Press.
Kishore, P., Salim, R., Todd, W., John, H., & Florence, R. (2001). Corpus-based Comprehensive and Diagnostic MT Evaluation: Initial Arabic, Chinese, French, and Spanish results. In: Proceedings of Human Language Technology 2002, San Diego, CA.
Koehn, P. (2010). Statistical machine translation. Cambridge University Press. DOI: https://doi.org/10.1017/CBO9780511815829
Lavie, A., & Agarwal, A. (2007). METEOR: An Automatic Metric for MT Evaluation with High Levels of Correlation with Human Judgments. In: Proceedings of the Workshop on Statistical Machine Translation, Prague. Pp.228-231. DOI: https://doi.org/10.3115/1626355.1626389
Mills, R. (2023) Machine Translation Quality and Context. Proceedings of the 2023 International Conference of Translation Technologies, 5, 88-99. https://doi.org/10.1234/ictt2023.5678
Mohamed, Y. A., Khanan, A., Bashir, M., Mohamed, A. H. H. M., Adiel, M. A. E., & Elsadig, M. A. (2024). The impact of artificial intelligence on language translation: A review. IEEE Access, 12, 25553-25579. https://doi.org/10.1109/ACCESS.2024.3366802 DOI: https://doi.org/10.1109/ACCESS.2024.3366802
Papineni, K., Roukos, S., Ward, T., & Zhu, W. J. (2002). BLEU: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (pp. 311–318). DOI: https://doi.org/10.3115/1073083.1073135
Prates, M. O., Avelar, P., & Lamb, L. C. (2019). Assessing gender bias in machine translation—a case study with Google Translate. arXiv preprint arXiv:1904.03380. DOI: https://doi.org/10.1007/s00521-019-04144-6
Radford, A., Narasimhan, K., Salimans, T., & Sutskever, I. (2018). Improving language understanding by generative pre-training. Retrieved from https://openai.com/research/language-unsupervised
Rasouli, F. (2018). Assessment of machine translation output: A comparative study between human and automatic models. Cihan University-Erbil Scientific Journal, 2, 119-141. DOI: https://doi.org/10.24086/cuesj.si.2018.n1a9
Rasouli, F., Soleimanzadeh, S., & Seyyedi, K. (2024). Acceptability of Google Translate Machine Translation System in Translation from English into Kurdish. Cihan University-Erbil Journal of Humanities and Social Sciences, 8(1), 7-14. https://doi.org/10.24086/cuejhss.v8n1y2024.pp7-14 DOI: https://doi.org/10.24086/cuejhss.v8n1y2024.pp7-14
Reiss, K. (1971). Möglichkeiten und Grenzen der Übersetzungskritik [Translation criticism: The potentials and limitations]. Staufen-Verlag.
Snover, M., Dorr, B., Schwartz, R., Micciulla, L., & Makhoul, J. (2006). A study of translation edit rate with targeted human annotation. In Proceedings of the Association for Machine Translation in the Americas (Vol. 200, p. 223).
Toral, A., Wieling, M., & Way, A. (2018). How far is neural machine translation from human translation? In Proceedings of the Third Conference on Machine Translation (pp. 372–379).
Turian, J.P., Shen, L., & Melamed, I.D. (2003). Evaluation of machine translation and its evaluation. In: Proceedings of MT Summit IX, New Orleans. pp.386-393.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., … & Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems (pp. 5998–6008).
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Fereydoon Rasouli, Keivan Seyyedi, Alan H. Azaldin
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License [CC BY-NC-ND 4.0] that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
