muscriptor/muscriptor

A multi-instrument music transcription model developed by Kyutai and Mirelo.

What it solves

MuScriptor solves the challenge of multi-instrument music transcription, converting audio recordings of music into structured digital formats like MIDI and sheet music. It aims to provide a high-accuracy, open-source alternative for musicians and composers who need to transcribe audio into editable notation.

How it works

The system uses a transformer decoder-only architecture to process audio and generate transcriptions. It is available in three model sizes (small, medium, and large) to balance speed and accuracy based on the available hardware. The tool can be run as a web application, a command-line interface (CLI), or integrated via a Python library. For sheet music generation, it integrates with MuseScore 4 to engrave the transcription into PDF and MusicXML formats, including tablature for fretted instruments.

Who it’s for

This tool is designed for musicians, composers, and researchers who want to automate the transcription of multi-instrument recordings into MIDI or professional sheet music.

Highlights

  • Multi-instrument support: Transcribes multiple instruments from a single recording.
  • Professional notation: Generates PDF scores, MusicXML files, and instrument-specific tablature via MuseScore 4.
  • Flexible deployment: Offers a web UI, CLI, and Python API for different workflows.
  • Scalable model sizes: Provides three model variants (103M to 1.4B parameters) to accommodate CPU-only or GPU-accelerated machines.

Related

  • Dispatch
    MuScriptor: Open-Source Audio-to-MIDI Transcription Model

    MuScriptor is an open-source multi-instrument audio-to-MIDI transcription model developed by Kyutai and Mirelo that converts recordings of any genre—including pop, classical, metal, and jazz—into MIDI notes without requiring prior knowledge of the instruments present.

  • Project
    madderscientist/noteDigger

    noteDigger is a pure‑JavaScript web app for audio‑to‑MIDI transcription. It lets users upload audio/video, visualises a spectrogram (STFT/CQT, GPU‑accelerated), runs on‑browser neural‑network models for pitch and instrument‑separation transcription, and provides extensive manual editing tools with keyboard shortcuts. Users can export quantised MIDI files or save a project for later work, all without external libraries or server‑side processing.

  • Project
    OpenMOSS/MOSS-Transcribe-Diarize

    An end-to-end audio understanding model that jointly performs speech transcription and speaker diarization for long-form, multi-speaker recordings in over 50 languages.

  • Project
    Audiveris/audiveris

    An open-source Optical Music Recognition (OMR) application that transcribes music score images into symbolic digital formats for playback and editing.

  • Project
    Makememo/MemoAI

    An AI-powered tool for transcribing audio and video from local files or YouTube links, providing translation and subtitling capabilities.