Boosting language models for real-word error detection
Loading...
Author (Corporation)
Publication date
2025
Type of student thesis
Course of study
Collections
Type
04B - Conference paper
Editor (Corporation)
Supervisor
Parent work
Proceedings of the 14th International Conference on Pattern Recognition Applications and Methods (ICPRM 2025)
Special issue
DOI of the original publication
Link
Related research data
Series
Series number
Volume
Issue / Number
Pages / Duration
318-325
Patent number
Publisher / Publishing institution
SciTePress
Place of publication / Event location
Porto
Edition
Version
Programming language
Assignee
Practice partner / Client
Abstract
With the introduction of transformer-based language models, research in error detection in text documents has significantly advanced. However, some significant research challenges remain. In the present paper, we aim to address the specific challenge of detecting real-word errors, i.e., words that are syntactically correct but semantically incorrect given the sentence context. In particular, we research three categories of frequent real-word errors in German, viz. verb conjugation errors, case errors, and capitalization errors. To address the Real-word errors refer to words in texts that exist in the underlying dictionary but are incorrect in the context of the sentence. One open issue in detecting real-word errors is that there is limited data available for training the models, especially for languages other than the dominant language in research (namely English). To counteract this limitation, we propose to incorporate synthetic data in the training process for three categories of real-word errors frequently encountered in German text, viz. conjugation errors in verbs, wrong case selection, and capitalization errors. The first contribution of this paper is that we generate high-quality synthetic data from a real-world text data set provided by a Swiss proofreading agency that can be used for model training. In addition to the introduction of a novel and large-scale synthetic data set, the second major contribution of this paper is that we propose to incorporate ensemble learning methods for language models. Actually, a few approaches have been proposed that combine ensemble learning methods with language models. One such strategy, known as boosted prompting, is inspired by classical boosting algorithms. This method iteratively augments the prompt set with new prompts that better generalize regions of the target problem space where the previous prompts underperform (Pitis et al., 2023). Another approach is to train multiple models and combine them for the final output. In (Li et al., 2019), CNN-based and transformer-based models were combined to tackle the challenge of grammatical error correction. In the present paper, we propose to employ boosting techniques to enhance the training process of language models and ultimately improve the accuracy of language models for detecting real-word errors. To the best of our knowledge, this is the first time that boosting is used in combination with language models in this specific way and for this particular task.
Keywords
Subject (DDC)
Event
14th International Conference on Pattern Recognition Applications and Methods
Exhibition start date
Exhibition end date
Conference start date
23.02.2025
Conference end date
25.02.2025
Date of the last check
ISBN
978-989-758-730-6
ISSN
Language
English
Created during FHNW affiliation
Yes
Strategic action fields FHNW
Publication status
Published
Review
peer-reviewed
Open access category
Closed
License
Citation
Masanti, C., Witschel, H. F., & Riesen, K. (2025). Boosting language models for real-word error detection. In M. Castrillon-Santana, M. De Marsico, & A. Fred (Eds.), Proceedings of the 14th International Conference on Pattern Recognition Applications and Methods (ICPRM 2025) (pp. 318–325). SciTePress. https://doi.org/10.5220/0013251500003905