microsoft/skill-recorder

Desktop app that records your on-screen work session and uses the GitHub Copilot SDK to reconstruct it as an intent + ordered steps, then builds a reusable Skill or Automation for Microsoft Autopilots like Scout, Microsoft Copilot Cowork, or Copilot Studio.

What it solves

Skill Recorder eliminates the need to manually write complex automation scripts. It allows you to teach an AI agent how to perform a task simply by recording yourself doing it once, turning your actions into a reusable procedure.

How it works

  1. Record: You capture your screen activity, terminal commands, and optional narration locally.
  2. Analyze: Using GitHub Copilot, the tool reconstructs your session into a clear, ordered list of steps and an overall intent.
  3. Create: You review and refine the generated plan, then convert it into a "Skill" (for on-demand use) or an "Automation" (for scheduled tasks). The system prefers using native tools like CLI commands over simple UI clicking, making the resulting automation more robust.

Who it’s for

It is designed for users who want to automate repetitive workflows without writing code from scratch, provided they have a GitHub account with Copilot access.

Highlights

  • Generalization: Learns to perform tasks broadly (e.g., submitting all forms) rather than just replaying your specific clicks.
  • Privacy-First: Recording and transcription happen locally on your machine; data is only sent to the cloud when you explicitly trigger the analysis step.
  • Flexible Input: Supports live screen recording or importing existing MP4/WebM video files.
  • Native Tool Integration: Prioritizes using built-in agent tools like gh CLI or web_fetch for cleaner, more reliable execution.

Related

  • Project
  • Project
  • Project
  • Project
  • Project