Echo: Achieving Fable-level performance with open-weight models at ~1/3 inference cost

Overview and Core Idea

Echo dynamically selects which models to use and how much computation to allocate for each request, aiming to capture the advantage of an oracle that knows in advance which models will be useful.

The project began with an experiment: a pool of models including GLM-5.2, Kimi K2.7 and others was run on the same evaluations. An hypothetical system that, for each problem, knew beforehand which models would help and how to combine their outputs performed substantially better than any single model in the pool. Echo attempts to approximate that advantage without requiring oracle knowledge, deciding per request which models participate, how much compute they receive, and how their intermediate work is combined.

Performance Claims and Evaluation

On the initial evaluation mix, Echo consistently outperformed the best individual model in its pool and reached roughly the same aggregate result as Fable, a stronger comparison system, at about one third of the inference cost.

The evaluation dashboard (available at echo.tracerml.ai/eval) shows the individual model results, the combined Echo results, and cost measurements. The author notes that these results are preliminary and plans to publish stronger evals, including more difficult coding and agentic benchmarks, to map out differences with state‑of‑the‑art systems.

Architecture and Decision Process

For each incoming request, Echo decides:

• Which models from the pool should participate
• How much computation (e.g., token generation steps) to allocate to each
• How to combine the models’ intermediate outputs into a final answer

The system is exposed through a chat interface at echo.tracerml.ai and an OpenAI‑compatible API (docs at echo.tracerml.ai/docs/api). No credit card is required to try the service; each account receives $10 of free credits for use on both the API and the chat. A short video explaining the workflow is available on YouTube.

Limitations and Ongoing Work

Echo still makes suboptimal allocation or combination decisions. The author is spending time analyzing those failures and testing whether the same approach holds up on coding and agentic tasks, where measuring the quality of each decision is harder.

In response to community feedback, the author has fixed issues in the evaluation dashboard UI and the sign‑up flow, and will continue to expand the public eval dashboard with additional benchmarks and suggestions.

Community Feedback and Responses

Many commenters raised concerns about benchmark transparency, sign‑up friction, and the relevance of cost savings.

"No benchmarks, no info on which models are used, ai generated video, just a signup page with nothing else." – @kamranjon

"The HumanEval+ tests, the sole code benchmark, consist of thirty‑one (31) programming questions... I find none of these particularly challenging for a model of Fable caliber..." – @guessmyname

"This reminds me on OpenRouters report that combining multiple different models gave comparable performance to Fable 5." – @Alifatisk

"I think one of the most interesting takeaways is that selection may matter more than model size..." – @smokeeaasd

The author addressed these points, noting that:

• No credit card is required and free credits are provided.
• The eval dashboard will be strengthened with more difficult coding and agentic benchmarks.
• UI and signup issues have been fixed in production.
• Echo’s idea is broader than simple model routing; it also optimizes compute allocation and how intermediate work is combined, distinguishing it from systems like OpenRouter Fusion or Sakana Fugu.

Comparison to Related Approaches

Commenters noted similarities to mixture‑of‑experts (MoE) architectures and routing systems such as OpenRouter Fusion.

"This sounds a lot like a larger‑scale MoE (Mixture of Experts) type of architecture." – @yonatan8070

"So this is the dogpile.com of the askjeeves, alta vista, and lycos approach?" – @dluan

The author clarified that while there are conceptual similarities, Echo’s architecture and optimization objective differ: it focuses on allocating inference efficiently across open‑weight models, deciding not only which models to use but also how much computation each deserves and how to combine their work, rather than merely routing tokens to experts.

Outlook

Echo demonstrates that intelligently routing requests across a pool of open‑weight models and varying compute allocation can achieve frontier‑level performance at a fraction of the cost. Continued work will focus on improving the allocation/combination decisions, expanding benchmarks to coding and agentic domains, and making the system more accessible for external testing.