ikawaha/kagome

What it solves

Kagome is a Japanese morphological analyzer that allows developers to tokenize Japanese text into words and analyze their parts of speech. It eliminates the common deployment hurdles of Japanese tokenization by providing a pure Go implementation with dictionaries embedded directly into the binary.

How it works

It processes text using supported dictionaries (such as MeCab-IPADIC and UniDic) to perform morphological analysis. It offers three segmentation modes—Normal, Search (optimized for search heuristics), and Extended (which includes uni-grams for unknown words)—to handle different tokenization strategies. The tool can be used as a CLI, a Go library, a RESTful API server, or compiled to WebAssembly for browser-based execution.

Who it’s for

Developers building applications that require Japanese text processing, search engines, and Text-to-Speech systems, as well as those who prefer a self-contained Go-based tool over complex C-based dependencies.

Highlights

• Self-contained: Dictionaries are embedded in the binary for easy deployment.
• Multiple Modes: Supports Normal, Search, and Extended segmentation strategies.
• Cross-platform: Available as a Go library, C shared library (for Python/PHP via FFI), WebAssembly, and Docker images.
• Integrated Tools: Includes a RESTful API server, a sentence splitter, and a lattice viewer for debugging tokenization processes.