tripleyak/SkillForge

What it solves

SkillForge addresses the problem of creating low-quality or redundant AI "skills" (instructions/prompts) that don't actually improve agent performance. It moves away from document-based approval to a behavioral approach, ensuring a skill is only created if it demonstrably improves an agent's ability to complete a task compared to a baseline without that skill.

How it works

SkillForge uses an evidence-driven pipeline to manage the lifecycle of AI skills:

1. Triage & Red Gate: It checks if a skill already exists and runs a "Red Gate" test where a fresh agent attempts the task without the skill; if the agent succeeds, no new skill is created.
2. Analysis & Spec: It analyzes failures using specific lenses (like Pareto or Root Cause) and creates a tiered specification.
3. Generation & Green Gate: A subagent generates the skill based on the spec, and a "Green Gate" test ensures the agent now clears the previously failed baseline.
4. Review & Ship: The skill is linted and reviewed by an adversarial agent before being shipped with its own regression suite (evals).

Who it’s for

It is designed for users of AI agents (specifically Claude Code and Codex CLI) who want to build, maintain, and optimize a library of reusable skills for their agents.

Highlights

• Behavioral Validation: Uses Red/Green gates to prove a skill actually improves performance.
• Regression Testing: Every skill ships with its own evals/ folder and a runner to ensure it continues to work.
• Ecosystem Health: Includes a "doctor" tool to find trigger collisions, duplicates, and stale references across all skills.
• Cross-Runtime Support: Allows authors to write a skill once and compile it for different targets like Claude, Codex, or AgentSkills.
• Friction Mining: An opt-in tool that analyzes local session history to identify gaps where new skills are needed.