Hugging Face Upskill: Transferring Expert Capabilities to Smaller Models via Agent Skills

Hugging Face has introduced upskill, a tool designed to transfer complex domain expertise from state-of-the-art (SOTA) models to smaller, cheaper, or open-source models. By generating and validating "agent skills"—structured instructions and code files—upskill allows developers to improve the performance of local models on specialized tasks, such as writing CUDA kernels for diffusers models, while reducing operational costs.

Agent Skills: A Medium for Capability Transfer

Agent skills are a method of defining model context using standardized files, such as Markdown for instructions and scripts for code. This format allows capabilities to be easily generated, shared, and reviewed across different models and tools. They are particularly effective for domain-specific or difficult problems that a model cannot solve reliably on its own.

While simple skills based on existing documentation can improve performance for some models, they can also degrade performance or increase token usage for others. This necessitates a rigorous evaluation process to ensure a skill actually provides a "lift" in capability.

The Upskill Workflow: From Teacher to Student

The upskill process follows a teacher-student architecture to move expertise from a high-end model to a more accessible one.

1. Generating the Skill (The Teacher)

The process begins with a "teacher" model (such as Claude Opus 4.5) performing a complex task interactively. This can be done via Claude Code, where the agent builds a kernel and exports the execution trace. The teacher model then converts this trace into a skill file. There are three primary ways to create these skills:

  • Instructing the agent to create a skill file within the same session.
  • Using the Anthropic "skill creator" skill.
  • Using the upskill tool to generate a skill directly from a trace.

2. Validating the Skill

To ensure the skill is functional, upskill generates test cases based on the task trace. It then compares the performance of the model with and without the skill. A successful skill is one where the teacher model maintains its performance, confirming the skill has accurately captured the necessary task logic.

3. Deploying to Smaller Models (The Student)

Once validated, the skill is transferred to a smaller or open-source model. upskill provides an eval command to run the generated test cases on these "student" models.

In benchmarks for writing CUDA kernels, upskill demonstrated significant improvements:

  • Accuracy Lift: One local model (unsloth/GLM-4.7-Flash-GGUF:Q4_0) saw an accuracy increase from 40% to 85% (+45%) when using the skill.
  • Token Optimization: Some models, such as moonshotai/Kimi-K2-Thinking, showed improvements in both accuracy and a reduction in token usage. Conversely, for Claude Opus 4.5, the skill increased token usage without a performance gain, indicating the skill is redundant for the teacher model.

Case Study: CUDA Kernel Development

upskill was applied to the specialized task of building CUDA kernels using the Hugging Face kernels library. The resulting kernel-builder-cuda-kernels skill encodes deep domain expertise that would otherwise require hours of manual documentation research, including:

  • GPU Architecture: Targeting NVIDIA H100 (compute capability 9.0).
  • Memory Management: Aligning shared memory to 128 bytes.
  • Async Operations: Implementing async memory copies for __CUDA_ARCH__ >= 900.
  • Project Structure: Defining build.toml configurations and PyTorch C++ bindings.

With this skill, a model can generate a complete project structure and CUDA implementation for complex requests, such as a "fused LayerNorm + GELU kernel optimized for H100."

Technical Implementation and Usage

upskill is available via pip and supports various generators, including Claude Opus 4.5, OpenAI, and local models via OpenAI-compatible endpoints.

Key Commands

  • Generation: upskill generate "task description" --from ./trace.md
  • Evaluation: upskill eval ./skills/my-skill/ --model haiku --model sonnet
  • Local Model Generation: upskill generate "task" --model opus --eval-model "local-model-name" --eval-base-url http://localhost:8080/v1

Skill Specification

Skills are saved in a directory following the Agent Skills specification, typically containing:

  • SKILL.md: The main instructions (e.g., ~520 tokens for the CUDA kernel skill).
  • skill_meta.json: Metadata and the generated test cases used for validation.

These skills are portable across tools that support the specification, including Claude Code, Codex, and Cursor.

Sources