nieldlr/hanzi

HanziJS is a Chinese character and NLP module for Chinese language processing for Node.js

What it solves

HanziJS is a Node.js module designed for Chinese language processing, specifically aimed at helping Chinese language learners explore the structure and meaning of characters. It provides a programmatic way to decompose characters, look up definitions, and analyze phonetic relationships.

How it works

The library provides a set of functions to interact with Chinese linguistic data. It uses several data sources, including CC-CEDICT for definitions, the Junda corpus for character frequency, and the Leiden University corpus for vocabulary frequency.

Key capabilities include:

  • Character Decomposition: Breaking characters down into radicals, primitives, or graphical strokes across multiple levels of depth.
  • Dictionary Integration: Performing lookups for simplified and traditional characters and searching for compound words.
  • Phonetic Analysis: Parsing pinyin syllables into initials and finals to determine the "phonetic regularity" (how closely a character's sound matches its components).
  • Text Processing: Segmenting Chinese phrases into individual words using a longest-match lookup.

Who it’s for

Developers building educational tools for Chinese language learners, NLP applications for Chinese text, or linguistic research tools that require character-level analysis.

Highlights

  • Multi-level Decomposition: Supports four different types of decomposition (Once, Radical, Graphical, and Primitive).
  • Phonetic Regularity Scoring: A detailed scale (0-6) that identifies exact matches, rhymes, and near-miss initials between characters and their components.
  • Pinyin Parsing: Normalizes pinyin spelling to identify actual rhymes regardless of orthographic conventions.
  • Frequency-based Examples: Categorizes dictionary examples into high, medium, and low frequency based on corpus data.

Related

  • Project
    yanyiwu/nodejieba

    A high-performance Node.js implementation of the Jieba Chinese word segmentation tool, providing efficient text splitting, part-of-speech tagging, and keyword extraction.

  • Project
    NLPchina/nlp-lang

    A Java-based foundational utility package for NLP projects providing tools for text normalization, Chinese character conversion, and text similarity calculations.

  • Project
    dongrixinyu/JioNLP

    A Python library for Chinese NLP preprocessing and parsing, providing tools for data cleaning, entity extraction, and automatic LLM evaluation.

  • Project
    yanyiwu/gojieba

    A high-performance Go implementation of the Jieba Chinese word segmentation tool, providing multiple segmentation modes and POS tagging.

  • Project
    go-ego/gse

    A high-performance Go library for multilingual text segmentation and NLP, providing efficient word cutting for languages like Chinese, Japanese, and English.