bab2min/Kiwi

Kiwi(지능형 한국어 형태소 분석기)

What it solves

Kiwi is a high-performance Korean morphological analyzer designed to break down Korean sentences into their smallest meaningful units (morphemes) with high accuracy and speed. It specifically addresses the problem of ambiguity in the Korean language, where different morpheme combinations can result in the same surface text, and provides a mechanism to correct simple typos automatically.

How it works

Built with a C++ core for speed, Kiwi uses a combination of a statistical language model for local context and a Skip-Bigram model for global context to resolve ambiguities. It is trained on the Sejong and Modu corpora and follows the Sejong part-of-speech (POS) tag system. The library supports multi-threading to enable fast processing of large-scale text datasets.

Who it’s for

It is intended for developers and researchers in natural language processing (NLP), text mining, and the humanities who need a reliable, fast, and cross-platform tool for analyzing Korean text.

Highlights

  • High Speed: Optimized for fast text analysis compared to other Korean morphological analyzers.
  • Ambiguity Resolution: Employs a lightweight language model to accurately distinguish between ambiguous morpheme combinations.
  • Broad Compatibility: Provides wrappers for numerous languages including Python, Java, C#, Go, Rust, Swift, and WebAssembly.
  • Additional Utilities: Includes built-in sentence splitting and automatic typo correction features.

Related

  • Project
    bab2min/kiwipiepy

    A Korean morphological analyzer for Python that provides precise tokenization, part-of-speech tagging, and advanced typo and dialect correction.

  • Project
    kakao/khaiii

    A CNN-based Korean morphological analyzer developed by Kakao that uses machine learning to split text into morphemes for NLP preprocessing.

  • Project
    epoko77-ai/im-not-ai

    A CLI tool for Claude Code, Copilot, and Codex that removes AI-specific patterns and translationese from Korean text to make it sound natural and human-like without changing the original meaning.

  • Project
    ikawaha/kagome

    A pure Go Japanese morphological analyzer that tokenizes text and analyzes parts of speech with embedded dictionaries for easy deployment.

  • Project
    chrisryugj/kordoc

    A high-precision document parsing toolset that converts Korean government documents (HWP, HWPX, PDF) into Markdown and structured data, specifically optimized for RAG and AI agent integration.