Evaluating Google Neural Machine Translation from Chinese to English: technical vs. literary texts
As the global need for translation increases, machine translation (MT) has significantly enhanced the efficiency in facilitating information dissemination and cross-cultural communication. However, its quality remains bound by intrinsic limitations among language pairs and text genres. These d...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | en |
| Published: |
Penerbit Universiti Kebangsaan Malaysia
2025
|
| Online Access: | http://journalarticle.ukm.my/26280/1/Gema_Online_25_3_9.pdf http://journalarticle.ukm.my/26280/ https://ejournal.ukm.my/gema/issue/view/1852 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | As the global need for translation increases, machine translation (MT) has significantly enhanced the
efficiency in facilitating information dissemination and cross-cultural communication. However, its
quality remains bound by intrinsic limitations among language pairs and text genres. These
discrepancies lead to distinct MT performance when processing technical and literary texts, forming
the core gap and focus. This study aims to compare the quality of Google Neural Machine Translation
(GNMT) in literary and technical texts, investigating error disparities and establishing the abilities and
limits of MT across diverse linguistic contexts. The research was concerned with the English-Chinese
language pair with the Multidimensional Quality Metrics (MQM) framework for manual annotation.
The COMET automatic evaluation metric was also applied for validation and confirmation of quality
differences observed. This study selected five excerpts from Apple product manuals (33 aligned units)
and the novel, the Old Man and Sea (32 aligned units), respectively. Findings included (1) GNMT
performed well with technical texts, but acted less effective with literary texts and technical texts
exhibited notable terminology errors, whereas literary texts showed more stylistic inconsistencies; (2)
MQM scores demonstrated a remarkable difference, with technical texts outperforming literary texts
by 18.57%; and (3) COMET evaluation validated the above observations, confirming a significant
difference between the two text styles. Although GNMT faced challenges with both texts, the quality
remained acceptable within this study. Results recommend improving GNMT algorithms to enhance
accuracy and remedy error patterns and distributions. |
|---|