alibaba/skill-up

What it solves

skill-up provides a structured way to measure, evaluate, and improve "Agent Skills" (capabilities given to AI agents). It replaces ad hoc testing with a repeatable, declarative evaluation loop, preventing regressions and ensuring that skill improvements are measurable across different agent engines.

How it works

The tool uses a declarative configuration system where evaluation environments, engines, and test cases are defined in YAML files. It supports three primary judging strategies: rule-based, script-based, and agent-based judges.

To close the improvement loop, it includes skill-upper, an AI agent skill that can be installed into agents like Claude Code or Codex. skill-upper can read failure reports, diagnose whether the skill or the evaluation case is wrong, repair the code or the test, and add regression cases automatically through natural conversation.

Who it’s for

It is designed for developers creating AI agent skills who need to validate their performance across multiple agent engines (such as Qoder CLI, Claude Code, and Codex) and integrate these tests into CI/CD pipelines.

Highlights

• Eval-to-Evolution Loop: Uses an AI agent (skill-upper) to automatically repair and expand evaluation suites based on failure reports.
• Multi-Engine Support: Compatible with various agent engines including Claude Code, Codex, and Qoder CLI.
• Declarative Config: Uses YAML for defining cases and environments, making tests repeatable and version-controllable.
• CI-Ready: Includes a dedicated GitHub Action to run evaluations on every pull request.
• Anthropic Compatibility: Can import existing evals.json files into its YAML format.