Doorman11991/smallcode

What it solves

SmallCode is a terminal-native coding agent specifically optimized for small local LLMs (8B-35B parameters) running on consumer hardware. It addresses the common failures of small models—such as poor tool calling, context window overflow, and tendency to hallucinate or truncate files during edits—by implementing a specialized architecture that compensates for these limitations.

How it works

SmallCode uses several strategies to make small models reliable:

• Context Management: A budget engine caps tool results and uses semantic compression to summarize history instead of simply dropping it.
• Tool Routing: A two-stage process where the model first picks a category (e.g., read, write, search) and then receives only the relevant tool schemas, reducing context overhead.
• Forgiving Parsing: A multi-format parser that can extract tool calls from JSON, YAML, XML, or plain text, auto-repairing common small-model mistakes.
• Editing Primitives: It prioritizes search-and-replace patches over full-file writes to prevent truncation and drift.
• Planning: Complex tasks are decomposed into a TODO-file based plan, which the model references each turn to maintain focus.
• Hybrid Search: Combines regex/keyword matching with semantic ranking for offline, symbol-aware code search.

Who it’s for

Developers who want to run a fully local, private AI coding assistant using models like Qwen or Llama on their own hardware, rather than relying on expensive or privacy-invasive cloud APIs.

Highlights

• Model Escalation: Optional opt-in fallback to stronger cloud models (Claude, OpenAI, DeepSeek) only when local models hard-fail.
• Persistent Shell: Maintains a long-lived bash session so environment variables and directory changes persist across turns.
• Thinking Budget Control: Caps the "thinking" tokens used by reasoning models to prevent waste on trivial tasks.
• Knowledge Injection: Allows users to drop reference notes into a directory to be injected into the system prompt based on keyword overlap.
• Plugin System: Supports project, user, and global plugins for adding tools, commands, and hooks.
• Read/Write Guards: Prevents the model from writing to files it hasn't read yet and manages large file reads to avoid context overflow.