Replacing Claude and GPT with Local LLMs for Coding: Community Insights

The Verdict: Local LLMs are "Good Enough" for Most Coding Tasks

For many developers, local Large Language Models (LLMs) have reached a tipping point where they can handle the majority of daily coding tasks, though they still lag behind frontier models like Claude 3.5 Sonnet or GPT-4o in complex architectural reasoning. The consensus among practitioners is that while local models may not be "frontier-level," they are often sufficient for 80-90% of a typical developer's workflow, offering benefits in privacy, cost, and the absence of usage quotas.

Top Performing Local Models for Coding

Several models consistently emerged as the preferred choices for local coding setups:

  • Qwen 3.6 (27B and 35B): Widely cited as the current gold standard for local coding. The 27B dense model is praised for its accuracy and reliability, while the 35B-A3B (MoE) version is favored for its extreme speed.
  • Gemma 4 (26B and 31B): Noted as a strong alternative, particularly for document analysis, summarization, and specific agentic coding tasks.
  • DeepSeek V4 Flash: Used in high-performance setups (e.g., dual RTX Pro 6000 Blackwell) to achieve high token-per-second (t/s) rates for automated code writing and review.
  • Nemotron Super (110B/120B): Mentioned as capable for long-context "vibe coding" sessions, though some users found Qwen superior for pure coding tasks.

Hardware Requirements and Performance

Running capable coding models locally requires significant hardware investment, typically centering on VRAM and memory bandwidth.

GPU and VRAM Configurations

Users reported success with various high-end setups:

  • Dual RTX 3090s: Capable of running Qwen 3.6 35B at ~150 t/s using 4-bit quantization (UD-Q4_K_XL), fitting the entire 300k context window in VRAM.
  • RTX Pro 6000 Blackwell: Used to drive reasoning models like DeepSeek V4 Flash at 160 t/s.
  • RTX 4000 (20GB VRAM): Able to run Qwen 3.6 35B at approximately 55 t/s.
  • Multi-GPU (4x RTX 5070): Used to run Qwen 3.6 27B with Multi-Token Prediction (MTP) at 50-60 t/s.

Apple Silicon (Mac Studio/MacBook)

Unified memory makes Macs attractive for larger models:

  • M4/M5 Max (128GB+ RAM): Users are targeting these for larger models, noting that 128GB is often the minimum for frontier-like capabilities.
  • Mac Studio (512GB RAM): Used to run Qwen 3.6 27B dense models via llama.cpp at 25-40 t/s.

The Importance of the "Harness"

A recurring theme is that the model itself is only half the battle; the software "harness" (the agentic framework or IDE integration) is critical for productivity.

Popular Tooling

  • Pi (pi.dev): Frequently mentioned as a highly effective harness for agentic coding.
  • OpenCode: Used as a primary interface for interacting with local models.
  • llama.cpp: The standard backend for inference, often used with forks like llama-cpp-turboquant or ik_llama.cpp for performance optimizations like MTP (Multi-Token Prediction).
  • Ollama: Used for simpler self-hosted setups.

Workflow Strategies

To bridge the gap between local and frontier models, developers are employing hybrid strategies:

  • The "Architect-Builder" Split: Using a frontier model (e.g., Claude Sonnet) to generate high-level architecture and detailed execution plans, then feeding those plans piece-by-piece into a local model (e.g., Qwen 3.6) for implementation.
  • Agent Chaining: Splitting complex tasks into a pipeline of specialized agents (e.g., a schema agent, a coding agent, and an orchestrator) to keep local models grounded and prevent them from looping.
  • Local Fuzzing: Using local models for cost-prohibitive tasks, such as overnight automated UI testing via Chrome DevTools MCP, which would be too expensive on cloud APIs.

Key Limitations and Trade-offs

Despite the progress, local LLMs still face significant hurdles:

  • Reasoning Gap: Local models are often described as "juniors" who require precise guidance and strict TDD (Test-Driven Development), whereas frontier models act as "seniors" who can reason about architecture independently.
  • Context Degradation: Many users noted that quality and speed drop significantly once context windows exceed 100k-150k tokens.
  • Ergonomics: Local harnesses are often described as "clunky" compared to the polished experience of GitHub Copilot or Claude Code, lacking sophisticated queue management and sub-agent coordination.
  • Hardware Cost: For those without existing high-end GPUs, the cost of hardware to achieve acceptable speeds can exceed years of cloud subscriptions.

Sources