N4darae/shrt

What it solves

shrt is a tool for creating and maintaining regression suites for services using RPC chains. It solves the problem of manually writing test chains that often miss critical edge cases (like boundary conditions or missing tokens) by using "contracts" to automatically compose test chains and verify that service behavior remains consistent over time.

How it works

The tool operates by building a descriptor of available RPCs and using contract overlays to define expected behavior. It follows a specific workflow:

1. Derivation: It identifies available RPCs and uses contracts to plan and compose ordered chains of calls.
2. Execution: It runs these chains and records the results.
3. Verification: It compares current runs against "safe spots" (previously approved successful runs) to detect drift or regressions.
4. Gating: A CI gate (shrt gate) executes all chains, checking for regressions, intermittent failures, and changes in response fields, even those not explicitly asserted.

Who it’s for

It is designed for developers and QA engineers who need to ensure the stability of RPC-based services, particularly those using Protocol Buffers (protos), and who want a rigorous, contract-driven approach to regression testing.

Highlights

• Contract-Driven Planning: Automatically composes test chains based on defined contracts rather than manual scripting.
• Safe Spot Verification: Uses a human-approved "safe spot" as a baseline to detect any change in response data (drift).
• Automated Regression Gating: A CI-ready gate that identifies regressions, intermittent failures, and specific suspect RPCs.
• Chain Slicing: Ability to create minimal sub-chains to reproduce specific failures.
• Strict Linting: Enforces high-quality assertions to prevent tests that only check if a service "did not crash."