microsoft/skill-recorder

Desktop app that records your on-screen work session and uses the GitHub Copilot CLI to reconstruct it as an intent + ordered steps, then builds a reusable Skill or Automation for Microsoft Scout, Microsoft Copilot Cowork, or Copilot Studio.

What it solves

It eliminates the manual effort of writing detailed instructions for AI agents. Instead of documenting a process by hand, users can simply record themselves performing a task once, and the system automatically converts that demonstration into a reusable skill or automation that an agent can execute.

How it works

Skill Recorder captures a user's screen activity, including clicks, window switches, browser URLs, and optional spoken narration. This data is processed locally first, and then sent to the GitHub Copilot CLI, which analyzes the session to reconstruct the user's intent and a sequence of steps. The system then generates either a SKILL.md procedure for on-demand agent use or a scheduled automation. Rather than just replaying UI clicks, the resulting skills prefer the agent's native tools (such as the gh CLI) to ensure the process is generalized and robust.

Who it’s for

Users who want to quickly teach AI agents how to perform specific work tasks without writing complex prompts or manual step-by-step guides.

Highlights

  • Automatic Reconstruction: Uses GitHub Copilot to turn raw screen recordings into structured intents and ordered steps.
  • Native Tool Preference: Generates instructions that leverage an agent's native tools (like web_fetch) rather than fragile UI macros.
  • On-Device Processing: Recording, frame extraction, and narration transcription (via Whisper) happen locally on the user's machine.
  • Generalization: Can turn a single example (e.g., submitting one form) into a general skill capable of handling multiple similar tasks.

Related

  • Project
    nossa-y/activity-frames

    activity‑frames records your screen locally, compiles raw snapshots into deterministic “activity frames”, and exposes them as compact context blocks or executable scripts for AI agents, dramatically reducing token cost and improving reliability of repeated computer‑use tasks.

  • Project
    xingkongliang/skills-manager

    A centralized manager for AI agent skills that allows users to install, organize, and sync capabilities across multiple AI coding tools and devices.

  • Project
    yanhua1010/self-media-content-workflow

    A modular suite of AI agent skills for self-media content production, covering everything from trend research and strategy to platform-native copywriting and publishing.

  • Project
    mikiarlo3/ai-copywriter

    A portable agent skill that combines conversion-focused copywriting with a 33-point humanizing engine to remove AI-generated markers and create high-converting marketing text.