ctrlb-hq/ctrlb-decompose

What it solves

ctrlb-decompose is designed to solve the problem of "noisy" logs—millions of repetitive, raw log lines that are impossible for humans or LLMs to skim. It collapses thousands of repetitive lines into a small set of typed patterns, reducing the volume of data while preserving the critical information (like variables, statistics, and anomalies).

How it works

The tool uses a two-stage normalization and clustering pipeline that processes logs in a single streaming pass:

1. CLP Encoding: Normalizes variable tokens (like IPs, integers, and floats) into typed placeholders so that structurally identical lines produce the same "logtype."
2. Drain3 Clustering: Uses a tree-based similarity clustering algorithm to group logtypes into patterns, replacing diverging tokens with wildcards (<*>).
3. Variable Extraction & Typing: Classifies extracted variables into semantic types such as IPv4, UUID, Duration, Integer, and Enum.
4. Analysis: Accumulates statistics (quantiles, cardinality) and detects anomalies like frequency spikes or error cascades.

Who it’s for

• Developers and SREs: For rapid terminal-based investigation of system logs.
• LLM Users: Those who want to feed log data into an LLM for analysis without hitting token limits or wasting tokens on repetitive data.
• Claude Code Users: Users of the Claude Code plugin who want to analyze logs via natural language prompts.

Highlights

• Privacy-first: Runs locally as a CLI, in the browser via WASM, or as a Rust library; logs never leave the machine.
• LLM-optimized output: Provides a compact markdown format specifically designed for token-efficient LLM prompts.
• High efficiency: Uses memory-efficient structures like DDSketch and HyperLogLog++ for statistics and cardinality estimation.
• Multiple output modes: Supports Human-readable (ANSI), LLM-optimized, and JSON formats.