Evaluating Google Neural Machine Translation from Chinese to English: technical vs. literary texts
As the global need for translation increases, machine translation (MT) has significantly enhanced the efficiency in facilitating information dissemination and cross-cultural communication. However, its quality remains bound by intrinsic limitations among language pairs and text genres. These d...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | en |
| Published: |
Penerbit Universiti Kebangsaan Malaysia
2025
|
| Online Access: | http://journalarticle.ukm.my/26280/1/Gema_Online_25_3_9.pdf http://journalarticle.ukm.my/26280/ https://ejournal.ukm.my/gema/issue/view/1852 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850997554673614848 |
|---|---|
| author | Zhongming Zhang, Syed Nurulakla Syed Abdullah, Muhammad Alif Redzuan Abdullah, Wenqi Duan, |
| author_facet | Zhongming Zhang, Syed Nurulakla Syed Abdullah, Muhammad Alif Redzuan Abdullah, Wenqi Duan, |
| author_sort | Zhongming Zhang, |
| building | Tun Sri Lanang Library |
| collection | Institutional Repository |
| content_provider | Universiti Kebangsaan Malaysia |
| content_source | UKM Journal Article Repository |
| continent | Asia |
| country | Malaysia |
| description | As the global need for translation increases, machine translation (MT) has significantly enhanced the
efficiency in facilitating information dissemination and cross-cultural communication. However, its
quality remains bound by intrinsic limitations among language pairs and text genres. These
discrepancies lead to distinct MT performance when processing technical and literary texts, forming
the core gap and focus. This study aims to compare the quality of Google Neural Machine Translation
(GNMT) in literary and technical texts, investigating error disparities and establishing the abilities and
limits of MT across diverse linguistic contexts. The research was concerned with the English-Chinese
language pair with the Multidimensional Quality Metrics (MQM) framework for manual annotation.
The COMET automatic evaluation metric was also applied for validation and confirmation of quality
differences observed. This study selected five excerpts from Apple product manuals (33 aligned units)
and the novel, the Old Man and Sea (32 aligned units), respectively. Findings included (1) GNMT
performed well with technical texts, but acted less effective with literary texts and technical texts
exhibited notable terminology errors, whereas literary texts showed more stylistic inconsistencies; (2)
MQM scores demonstrated a remarkable difference, with technical texts outperforming literary texts
by 18.57%; and (3) COMET evaluation validated the above observations, confirming a significant
difference between the two text styles. Although GNMT faced challenges with both texts, the quality
remained acceptable within this study. Results recommend improving GNMT algorithms to enhance
accuracy and remedy error patterns and distributions. |
| format | Article |
| id | my-ukm.journal.26280 |
| institution | Universiti Kebangsaan Malaysia |
| language | en |
| publishDate | 2025 |
| publisher | Penerbit Universiti Kebangsaan Malaysia |
| record_format | eprints |
| spelling | my-ukm.journal.262802025-12-03T07:20:15Z http://journalarticle.ukm.my/26280/ Evaluating Google Neural Machine Translation from Chinese to English: technical vs. literary texts Zhongming Zhang, Syed Nurulakla Syed Abdullah, Muhammad Alif Redzuan Abdullah, Wenqi Duan, As the global need for translation increases, machine translation (MT) has significantly enhanced the efficiency in facilitating information dissemination and cross-cultural communication. However, its quality remains bound by intrinsic limitations among language pairs and text genres. These discrepancies lead to distinct MT performance when processing technical and literary texts, forming the core gap and focus. This study aims to compare the quality of Google Neural Machine Translation (GNMT) in literary and technical texts, investigating error disparities and establishing the abilities and limits of MT across diverse linguistic contexts. The research was concerned with the English-Chinese language pair with the Multidimensional Quality Metrics (MQM) framework for manual annotation. The COMET automatic evaluation metric was also applied for validation and confirmation of quality differences observed. This study selected five excerpts from Apple product manuals (33 aligned units) and the novel, the Old Man and Sea (32 aligned units), respectively. Findings included (1) GNMT performed well with technical texts, but acted less effective with literary texts and technical texts exhibited notable terminology errors, whereas literary texts showed more stylistic inconsistencies; (2) MQM scores demonstrated a remarkable difference, with technical texts outperforming literary texts by 18.57%; and (3) COMET evaluation validated the above observations, confirming a significant difference between the two text styles. Although GNMT faced challenges with both texts, the quality remained acceptable within this study. Results recommend improving GNMT algorithms to enhance accuracy and remedy error patterns and distributions. Penerbit Universiti Kebangsaan Malaysia 2025 Article PeerReviewed application/pdf en http://journalarticle.ukm.my/26280/1/Gema_Online_25_3_9.pdf Zhongming Zhang, and Syed Nurulakla Syed Abdullah, and Muhammad Alif Redzuan Abdullah, and Wenqi Duan, (2025) Evaluating Google Neural Machine Translation from Chinese to English: technical vs. literary texts. GEMA: Online Journal of Language Studies, 25 (3). pp. 732-754. ISSN 1675-8021 https://ejournal.ukm.my/gema/issue/view/1852 |
| spellingShingle | Zhongming Zhang, Syed Nurulakla Syed Abdullah, Muhammad Alif Redzuan Abdullah, Wenqi Duan, Evaluating Google Neural Machine Translation from Chinese to English: technical vs. literary texts |
| title | Evaluating Google Neural Machine Translation from Chinese to English: technical vs. literary texts |
| title_full | Evaluating Google Neural Machine Translation from Chinese to English: technical vs. literary texts |
| title_fullStr | Evaluating Google Neural Machine Translation from Chinese to English: technical vs. literary texts |
| title_full_unstemmed | Evaluating Google Neural Machine Translation from Chinese to English: technical vs. literary texts |
| title_short | Evaluating Google Neural Machine Translation from Chinese to English: technical vs. literary texts |
| title_sort | evaluating google neural machine translation from chinese to english: technical vs. literary texts |
| url | http://journalarticle.ukm.my/26280/1/Gema_Online_25_3_9.pdf http://journalarticle.ukm.my/26280/ https://ejournal.ukm.my/gema/issue/view/1852 |
| url_provider | http://journalarticle.ukm.my/ |