Semi-supervised learning and bidirectional decoding for effective grammar correction in low-resource scenarios
The correction of grammatical errors in natural language processing is a crucial task as it aims to enhance the accuracy and intelligibility of written language. However, developing a grammatical error correction (GEC) framework for low-resource languages presents significant challenges due to the l...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | en en |
| Published: |
PeerJ, Inc.
2023
|
| Subjects: | |
| Online Access: | https://eprints.ums.edu.my/id/eprint/38438/1/ABSTRACT.pdf https://eprints.ums.edu.my/id/eprint/38438/2/FULL%20TEXT.pdf https://eprints.ums.edu.my/id/eprint/38438/ http://dx.doi.org/10.7717/peerj-cs.1639 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1831796033037795328 |
|---|---|
| author | Zeinab Mahmoud Chunlin Li Marco Zappatore Aiman Solyman Ali Alfatemi Ashraf Osman Ibrahim Elsayed Abdelzahir Abdelmaboud |
| author_facet | Zeinab Mahmoud Chunlin Li Marco Zappatore Aiman Solyman Ali Alfatemi Ashraf Osman Ibrahim Elsayed Abdelzahir Abdelmaboud |
| author_sort | Zeinab Mahmoud |
| building | UMS Library |
| collection | Institutional Repository |
| content_provider | Universiti Malaysia Sabah |
| content_source | UMS Institutional Repository |
| continent | Asia |
| country | Malaysia |
| description | The correction of grammatical errors in natural language processing is a crucial task as it aims to enhance the accuracy and intelligibility of written language. However, developing a grammatical error correction (GEC) framework for low-resource languages presents significant challenges due to the lack of available training data. This article proposes a novel GEC framework for low-resource languages, using Arabic as a case study. To generate more training data, we propose a semi-supervised confusion method called the equal distribution of synthetic errors (EDSE), which generates a wide range of parallel training data. Additionally, this article addresses two limitations of the classical seq2seq GEC model, which are unbalanced outputs due to the unidirectional decoder and exposure bias during inference. To overcome these limitations, we apply a knowledge distillation technique from neural machine translation. This method utilizes two decoders, a forward decoder right-to-left and a backward decoder left-to-right, and measures their agreement using Kullback-Leibler divergence as a regularization term. The experimental results on two benchmarks demonstrate that our proposed framework outperforms the Transformer baseline and two widely used bidirectional decoding techniques, namely asynchronous and synchronous bidirectional decoding. Furthermore, the proposed framework reported the highest F1 score, and generating synthetic data using the equal distribution technique for syntactic errors resulted in a significant improvement in performance. These findings demonstrate the effectiveness of the proposed framework for improving grammatical error correction for low-resource languages, particularly for the Arabic language. |
| format | Article |
| id | my.ums.eprints-38438 |
| institution | Universiti Malaysia Sabah |
| language | en en |
| publishDate | 2023 |
| publisher | PeerJ, Inc. |
| record_format | eprints |
| spelling | my.ums.eprints-384382024-03-05T02:35:15Z https://eprints.ums.edu.my/id/eprint/38438/ Semi-supervised learning and bidirectional decoding for effective grammar correction in low-resource scenarios Zeinab Mahmoud Chunlin Li Marco Zappatore Aiman Solyman Ali Alfatemi Ashraf Osman Ibrahim Elsayed Abdelzahir Abdelmaboud P1-85 General QA75.5-76.95 Electronic computers. Computer science The correction of grammatical errors in natural language processing is a crucial task as it aims to enhance the accuracy and intelligibility of written language. However, developing a grammatical error correction (GEC) framework for low-resource languages presents significant challenges due to the lack of available training data. This article proposes a novel GEC framework for low-resource languages, using Arabic as a case study. To generate more training data, we propose a semi-supervised confusion method called the equal distribution of synthetic errors (EDSE), which generates a wide range of parallel training data. Additionally, this article addresses two limitations of the classical seq2seq GEC model, which are unbalanced outputs due to the unidirectional decoder and exposure bias during inference. To overcome these limitations, we apply a knowledge distillation technique from neural machine translation. This method utilizes two decoders, a forward decoder right-to-left and a backward decoder left-to-right, and measures their agreement using Kullback-Leibler divergence as a regularization term. The experimental results on two benchmarks demonstrate that our proposed framework outperforms the Transformer baseline and two widely used bidirectional decoding techniques, namely asynchronous and synchronous bidirectional decoding. Furthermore, the proposed framework reported the highest F1 score, and generating synthetic data using the equal distribution technique for syntactic errors resulted in a significant improvement in performance. These findings demonstrate the effectiveness of the proposed framework for improving grammatical error correction for low-resource languages, particularly for the Arabic language. PeerJ, Inc. 2023 Article NonPeerReviewed text en https://eprints.ums.edu.my/id/eprint/38438/1/ABSTRACT.pdf text en https://eprints.ums.edu.my/id/eprint/38438/2/FULL%20TEXT.pdf Zeinab Mahmoud and Chunlin Li and Marco Zappatore and Aiman Solyman and Ali Alfatemi and Ashraf Osman Ibrahim Elsayed and Abdelzahir Abdelmaboud (2023) Semi-supervised learning and bidirectional decoding for effective grammar correction in low-resource scenarios. PeerJ Computer Science. pp. 1-25. ISSN 2376-5992 http://dx.doi.org/10.7717/peerj-cs.1639 |
| spellingShingle | P1-85 General QA75.5-76.95 Electronic computers. Computer science Zeinab Mahmoud Chunlin Li Marco Zappatore Aiman Solyman Ali Alfatemi Ashraf Osman Ibrahim Elsayed Abdelzahir Abdelmaboud Semi-supervised learning and bidirectional decoding for effective grammar correction in low-resource scenarios |
| title | Semi-supervised learning and bidirectional decoding for effective grammar correction in low-resource scenarios |
| title_full | Semi-supervised learning and bidirectional decoding for effective grammar correction in low-resource scenarios |
| title_fullStr | Semi-supervised learning and bidirectional decoding for effective grammar correction in low-resource scenarios |
| title_full_unstemmed | Semi-supervised learning and bidirectional decoding for effective grammar correction in low-resource scenarios |
| title_short | Semi-supervised learning and bidirectional decoding for effective grammar correction in low-resource scenarios |
| title_sort | semi-supervised learning and bidirectional decoding for effective grammar correction in low-resource scenarios |
| topic | P1-85 General QA75.5-76.95 Electronic computers. Computer science |
| url | https://eprints.ums.edu.my/id/eprint/38438/1/ABSTRACT.pdf https://eprints.ums.edu.my/id/eprint/38438/2/FULL%20TEXT.pdf https://eprints.ums.edu.my/id/eprint/38438/ http://dx.doi.org/10.7717/peerj-cs.1639 |
| url_provider | http://eprints.ums.edu.my/ |