Enhancing language models with boosting and targeted fine-tuning for real-word error detection
Loading...
Author (Corporation)
Publication date
2026
Type of student thesis
Course of study
Collections
Type
01A - Journal article
Editors
Editor (Corporation)
Supervisor
Parent work
Natural Language Processing Journal
Special issue
DOI of the original publication
Link
Related research data
Series
Series number
Volume
14
Issue / Number
Pages / Duration
100202-100202
Patent number
Publisher / Publishing institution
Elsevier
Place of publication / Event location
Edition
Version
Programming language
Assignee
Practice partner / Client
Abstract
We propose a boosting-based approach to enhance language models of diverse architectures with the goal of detecting real-word errors in documents. • We thoroughly evaluate the benefits and limitations of our novel framework through experiments on a large real-world data set. Based on a thorough error analysis, we generate additional targeted training data to address identified weaknesses and apply targeted fine-tuning to further improve model performance. Over the past years, extensive research has led to significant advancements in tools for the automatic detection and correction of errors in documents. Despite this progress, several challenges remain unresolved. In particular, the identification of real-word errors – errors involving words that are grammatically valid but contextually inappropriate within a given sentence – continues to pose a considerable difficulty. Addressing such errors requires models with a sophisticated understanding of linguistic context. Transformer-based language models are particularly well-suited for this task due to their contextual modeling capabilities. To further enhance their performance, we propose a boosting-based training approach in conjunction with a synthetically generated data set created via pattern-based noise injection. We evaluate this method across three transformer-based architectures, viz. mBERT, LLaMA 3, and Mistral 7B. Our experimental results show that the boosting-based strategy consistently improves real-word error detection across all models. A subsequent in-depth error analysis reveals limitations in the synthetic training data, prompting the development of a targeted fine-tuning procedure designed to address these shortcomings and further optimize model performance. A comparison with prompt-based inference using a large language model demonstrates that specialized, fine-tuned models yield more reliable performance for this task. Finally, an evaluation under realistic class imbalance highlights practical trade-offs between ranking quality and threshold-based detection, particularly for rare error types.
Keywords
Subject (DDC)
Event
Exhibition start date
Exhibition end date
Conference start date
Conference end date
Date of the last check
ISBN
ISSN
2949-7191
Language
English
Created during FHNW affiliation
Yes
Strategic action fields FHNW
Publication status
Published
Review
peer-reviewed
Open access category
Gold
License
Citation
Masanti, C., Witschel, H. F., & Riesen, K. (2026). Enhancing language models with boosting and targeted fine-tuning for real-word error detection. Natural Language Processing Journal, 14, 100202. https://doi.org/10.1016/j.nlp.2026.100202