shibing624/pycorrector

pycorrector – Chinese Text‑Correction Toolkit

What it does

• Provides a Python library for automatically fixing errors in Chinese text. It handles the most common mistake types:
  • phonetic (sound‑alike) errors
  • visual (shape‑alike) errors
  • Grammar / word‑order errors
  • Proper‑noun (name) errors
• Offers both a simple rule‑based approach (KenLM language model + confusion sets) and a collection of modern neural models (MacBERT, T5, ERNIE, GPT‑style LLMs, BART, etc.) that can be used out‑of‑the‑box or fine‑tuned on your own data.

Key components

Component	Technology	Typical use	Performance hint
KenlmCorrector	KenLM n‑gram language model + handcrafted confusion sets	Fast, low‑resource correction (good for spelling‑only tasks)	9 QPS on CPU, modest F1 (0.34)
MacBertCorrector	MacBERT backbone with added detection layer	Strong spelling & grammar correction, recommended default	~224 QPS on GPU, F1 ≈ 0.40 (CSC)
T5Corrector	Chinese‑pre‑trained T5 fine‑tuned on correction data	Good for longer, more complex edits	Faster than large LLMs, solid F1
ERNIE_CSC	PaddlePaddle ERNIE model fine‑tuned for CSC	High accuracy on standard benchmarks	F1 ≈ 0.44
GPT‑style Correctors (ChatGLM3, LLaMA2, Qwen2.5, Qwen3)	Large language models fine‑tuned for correction	Best overall quality, can handle multi‑character insert/delete	Highest F1 (up to 0.85) but slower (few QPS)
BART / MuCGECBart / NaSGECBart	Seq2Seq BART variants trained on Chinese correction corpora	Good for general text correction, especially when data is noisy

How you use it

• Install with pip install -U pycorrector (or build from source / Docker).
• Choose a model class, e.g. MacBertCorrector('shibing624/macbert4csc-base-chinese').
• Call correct() for a single sentence or correct_batch() for many.
• The return value is a dictionary:

  {
    "source": "原句子",
    "target": "纠正后句子",
    "errors": [("错误词", "正确词", position), ...]
  }
  

• For the lightweight KenLM approach you can also:
  • Load a custom confusion list to improve recall/precision.
  • Load a custom proper‑name dictionary for name‑specific fixes.
  • Switch to a smaller language model if memory is limited.
• Command‑line mode (python -m pycorrector) lets you batch‑process a UTF‑8 text file.

Evaluation

• The repo ships scripts to evaluate on three public Chinese correction benchmarks: SIGHAN‑2015, EC‑LAW, and MCSC.
• Metrics are sentence‑level precision/recall (F1). The table in the README shows each model’s average F1 and per‑dataset scores, as well as GPU used and queries‑per‑second (QPS).

Demo & Resources

• An online demo is hosted at https://www.mulanai.com/product/corrector/ and a HuggingFace Space at https://huggingface.co/spaces/shibing624/pycorrector.
• Example scripts (examples/*/demo.py) illustrate quick usage for every model type.
• Documentation, model cards, and a Chinese‑language‑correction guide are available via the repo’s Wiki and the linked papers.

Who might need it

• Developers building Chinese input‑method editors, OCR post‑processing, voice‑to‑text pipelines, or search‑query sanitizers.
• Researchers comparing Chinese spelling‑correction methods.
• Anyone needing a plug‑and‑play Chinese grammar/spelling corrector in a Python application.

---

All information above is taken directly from the project's README; no external assumptions have been added.