stevesolun/ctx

What it solves

CTX Fit helps developers find the most cost-effective AI coding configuration that reliably works for their specific codebase. It removes the guesswork of choosing models and instructions by testing various configurations against real tasks derived from the repository's history and verifying the results using the project's own test suites.

How it works

The tool analyzes a repository to identify its languages and verification commands (like pytest or ruff). It then creates a "campaign" where it tests candidate AI configurations against representative tasks.

Winning configurations are selected based on a strict hierarchy: first, candidates must meet a reliability floor; second, the cheapest option is chosen; and third, ties are broken by favoring simpler configurations. The final result is saved as a .ctx/fit-configuration.json sidecar file, which can be applied directly to the working tree or submitted as a pull request.

Who it’s for

Software engineers and maintainers who use AI coding agents and want to optimize their setup for reliability and cost without manually trial-and-erroring different models or prompts.

Highlights

• Automated Benchmarking: Tests AI configurations against real repository tasks and verifies them using native test runners (Python, JS/TS, Go, Rust, Make).
• Cost Optimization: Specifically seeks the cheapest configuration that meets a reliability threshold.
• Safe Execution: Uses Bubblewrap on Linux to run verification commands in network-disabled, isolated namespaces.
• Non-Destructive: Offers --dry-run for planning and --pr to propose changes via GitHub PRs rather than modifying code directly.