Towards a standard evaluation method for grammatical error detection and correction
TL
TL;DR
A novel evaluation method for grammatical error correction that addresses problems with previous approaches and scores systems in terms of improvement on the original text by evaluating corrections at the token level using a globally optimal alignment between the source, a system hypothesis, and a reference.
Abstract
We present a novel evaluation method for grammatical error correction that addresses problems with previous approaches and scores systems in terms of improvement on the original text. Our method evaluates corrections at the token level using a globally optimal alignment between the source, a system hypothesis, and a reference. Unlike the M 2 Scorer, our method provides scores for both detection and correction and is sensitive to different types of edit operations.
References23 items
1
The First QALB Shared Task on Automatic Text Correction for Arabic
2
The CoNLL-2014 Shared Task on Grammatical Error Correction
3
The CoNLL-2013 Shared Task on Grammatical Error Correction
4
Problems in Evaluating Grammatical Error Detection Systems
5