Grammar Error Correction with Transformer (GECToR)

What is GECToR?

GECToR (Grammar Error Correction with Transformer) is an open-sourced transformer-based deep-learning model created and maintained by Grammarly. The system is pre-trained on synthetic errorful data (a corpus containing sentences with spelling or grammar errors, along with corresponding tagged corrections) and then fine-tuned in two stages: first on errorful data, and second on a combination of errorful and error-free parallel corpora.

This mathematical fine-tuning process ensures that GECToR adapts its knowledge to the specific nuances of the target domain, resulting in improved performance for grammar correction in that particular context, and is done similarly to our regular training, utilizing an objective function as well as a custom loss function.

The synthetic data is developed using a methodology called g-transformations. These are task-specific ...