Cerebras Inference: Qwen 3.8 27B Deployment and Performance

Cerebras has expanded its model catalog to include Qwen 3.8 27B, achieving inference speeds of approximately 1,500 tokens per second. This deployment leverages Cerebras's specialized hardware to provide high-throughput access to the unpruned version of the model, targeting users who require near-instantaneous output for agentic workflows or content generation.

Model Specifications and Availability

Qwen 3.8 27B is now available on Cerebras public endpoints via free trial and pay-as-you-go tiers. The model's technical parameters on the platform include:

Model ID: qwen-3.8-27b
Parameters: 27 billion
Context Window: 64k for free tier users and 128k for paid users.
Performance: Approximately 1,500 tokens per second.

Cerebras also hosts the OpenAI GPT OSS (gpt-oss-120b), which operates at approximately 3,000 tokens per second with a context window of up to 131k tokens.

Hardware-Optimized Model Compression

Cerebras maintains a policy of serving original, unpruned models on its public endpoints to ensure architectural integrity. While the company conducts research into Router-weighted Expert Activation Pruning (REAP), these pruned models are reserved for the research community via Hugging Face and are not available through the production API.

To optimize storage without sacrificing quality, Cerebras employs selective weight-only quantization. Weights are stored in 16-bit, 8-bit, or 4-bit formats depending on the layer's sensitivity; sensitive layers are kept at full precision and dequantized on the fly. All activations, attention mechanisms, and the KV cache remain unquantized and in full precision.

User Performance and Cost Analysis

While the raw token-per-second (t/s) speed is high, community feedback highlights several practical bottlenecks and cost considerations for production use:

Rate Limits and Throughput

Users have reported that the public endpoint's rate limits (e.g., 150k to 450k tokens per minute) can be a significant blocker for coding tasks and agentic workflows. Because cached tokens count toward these limits, users may hit quotas rapidly during long sessions.

Cost vs. Speed Trade-off

Comparative tests indicate that Cerebras is significantly faster but more expensive than alternative providers. One user report noted that a session on Cerebras was 2.8x faster than OpenRouter averages but 5.6x more expensive, primarily because Cerebras does not currently offer a price discount for cached tokens.

Utility in Software Development

Feedback on the model's utility for coding is mixed. While the output speed is described as "awesome," some users argue that the net gain in development speed is marginal because other bottlenecks—such as tool calling reliability, shell command execution, and the human need to read the output—offset the raw inference speed.

"The net effect is that I spend about the same time waiting... at least for coding, it actually reconciles me with the 100-200t/sec you can get on DS4 or the like."

Comparison with Local Inference

Some developers suggest that for a 27B parameter model, local inference is a viable alternative. Using tools like ninfer, users have reported achieving 200-400 tokens per second on hardware like the RTX 5090, which may be sufficient for many use cases without the constraints of API rate limits or recurring costs.