Self-hosting Kimi K3 and GLM-5.2: GPU Hardware Costs vs Task Resolution
Executive Summary
Self-hosting frontier-class open-weight models like Kimi K3 and GLM-5.2 for AI coding agents involves a direct trade-off between hardware expenditure, concurrency, and task resolution quality. While Kimi K3 requires approximately 20% more hardware investment than GLM-5.2—necessitating an 8×B300 node rather than an 8×B200 node—it delivers a resolution rate of 86.4% on SWEBench Pro tasks, outperforming both GLM-5.2 and Anthropic Opus 4.8 (both at 62.5%).
Kimi K3 vs. GLM-5.2: Performance and Hardware Requirements
Kimi K3's massive 1.4TB weight footprint prevents it from fitting within the memory budget of an 8×B200 node (1.5TB total HBM), as there is insufficient headroom for the KV cache. Consequently, K3 requires an 8×B300 node, which provides 288GB of HBM per GPU (2.3TB per node), resulting in a hardware cost increase of roughly 20%.
Throughput and Latency Trade-offs
Increasing model quality with Kimi K3 comes at the cost of speed and concurrency:
- Concurrency: K3 supports 16 concurrent sessions, compared to 24 for GLM-5.2.
- Throughput: Aggregate token throughput for K3 is approximately 30% lower than GLM-5.2 (122 vs 170 tok/s at 16 users).
- Task Completion Time: Median task time for K3 is roughly 50% longer than GLM-5.2 (38 vs 26 minutes), making K3 approximately 8 times slower than the Claude Code baseline.
Despite these performance penalties, K3's superior resolution rate (86.4%) makes it a more effective tool for complex engineering tasks, provided the organization can tolerate higher latency.
GPU Infrastructure Options for Coding Agents
Choosing the right hardware depends on the desired balance of model quality and the number of developers the system must support.
Hardware Tiers and Model Fit
| Hardware Setup | Recommended Model | Comfortable Concurrency | Performance Note |
|---|---|---|---|
| DGX Spark | Qwen3.6-35B-A3B | 1 User | Slow; ~3× slower than Claude Code |
| 1× H200 | Qwen3.6-35B-A3B | 32 Users | High throughput; 2× slower than Claude Code at 48 users |
| 4× H200 | DeepSeek-V4-Flash | 32 Users | Good balance of quality and speed |
| 8× B200 | GLM-5.2 | 8 Users | Near-frontier quality; slows significantly beyond 8 users |
| 8× B300 | Kimi K3 | 16 Users | Highest resolution rate; high hardware cost |
The "Collapse" Phenomenon
In high-concurrency tests using vLLM, throughput for models like Qwen3.6 and DeepSeek-V4-Flash often crashes rather than tapering off after 48–64 users. This is attributed to the inference engine's default parameters and the balance between prefill and decode requests, as well as KV-cache size limitations.
Cost Analysis: Buying vs. Renting vs. API
The financial viability of self-hosting is determined by GPU utilization. Because hardware is paid for 24/7 but developer demand is spiky (typically 15–22% average utilization), the "break-even" point varies by model.
Utilization Thresholds for Ownership
- High-End Racks (B200): A B200 rack only needs 15% utilization to be more cost-effective than frontier APIs for near-frontier quality tasks.
- Mid-Range Racks (4× H200): A 4× H200 setup must remain 89% busy to beat the pricing of the DeepSeek-V4-Flash API, largely because DeepSeek's API heavily optimizes for cached input tokens (up to 98% of coding agent tokens).
Renting as a Middle Ground
Renting GPUs can be significantly cheaper than APIs for certain models (e.g., a 35× cost reduction for Qwen3.6), but can be more expensive than APIs for others (e.g., DeepSeek) if the workload is not specifically optimized.
Synthesis of Community Insights
Technical discussions around these findings highlight several non-quantifiable advantages and potential optimizations for self-hosting:
- Privacy and Sovereignty: Self-hosting is critical for organizations with strict data residency requirements (e.g., European or Canadian laws) or those handling sensitive medical data.
- Unrestricted Utility: Local models avoid the "refusals" and account bans associated with frontier APIs when performing red-teaming or information security tasks.
- Quantization: Community members suggest that using quantized versions (e.g., int4) could allow frontier-class models to run on smaller, more affordable hardware like the A6000, though this requires further benchmarking on quality loss.
- Asynchronous Workflows: Some argue that the "utilization gap" can be closed by routing non-urgent, automated tasks (e.g., overnight bug fixes) to fill the idle capacity of self-hosted GPUs.