MuScriptor: Open-Source Audio-to-MIDI Transcription Model

Overview of MuScriptor

MuScriptor is an open-source multi-instrument audio-to-MIDI transcription model developed in collaboration between Kyutai and Mirelo. The model converts audio recordings—regardless of genre, such as pop, classical, metal, or jazz—into MIDI notes. A key technical advantage of MuScriptor MuScriptor is its ability to perform transcription without requiring the user to specify the instruments present in the recording in advance.

Technical Capabilities and Performance

MuScriptor provides high-fidelity audio-to-MIDI conversion that outperforms traditional spectral analysis tools. Users have reported high accuracy in transcribing complex guitar riffs based on triad shapes and iconic song introductions, such as the intro to "In the End" by Linkin Park, which often challenge traditional transcription software.

"This produced a transcription that sounds 1:1 to my ear. Could totally see myself using this as tools I use to learn music."

Implementation and Public Instances

There are several ways to access the MuScriptor model, including the official Kyutai blog and the official source code. Community-driven implementations, such as pianoify.net, provide simplified web interfaces for users to upload MP3, WAV, M4A, or FLAC files (up to 10 minutes) or provide YouTube links.

Some of these public instances may impose specific constraints, such as cropping audio to ten seconds for transcription, which some users have noted as a limiting factor compared to the official model's capabilities.

Source Code and Availability

MuScriptor is open-source, allowing developers to integrate the model into music theory analysis tools or more advanced audio processing pipelines. The source code for community implementations like pianoify.net is available on GitHub at https://github.com/jerryzhou196/pianoify.

Sources

Related

  • Project
    muscriptor/muscriptor

    A multi-instrument music transcription model that converts audio recordings into MIDI and professional sheet music using a transformer decoder architecture.

  • Project
    madderscientist/noteDigger

    noteDigger is a pure‑JavaScript web app for audio‑to‑MIDI transcription. It lets users upload audio/video, visualises a spectrogram (STFT/CQT, GPU‑accelerated), runs on‑browser neural‑network models for pitch and instrument‑separation transcription, and provides extensive manual editing tools with keyboard shortcuts. Users can export quantised MIDI files or save a project for later work, all without external libraries or server‑side processing.

  • Project
    Natooz/MidiTok

    MidiTok is a Python package that converts MIDI and abc music files into token sequences for machine‑learning models such as Transformers, enabling music generation, transcription, and other MIR tasks.

  • Project
    wendy7756/AI-Video-Transcriber

    An AI-powered tool to transcribe, summarize, and archive videos and podcasts from 30+ platforms or local files using Faster-Whisper and LLMs.

  • Dispatch
    OpenAI MuseNet

    MuseNet is a deep neural network capable of generating 4-minute musical compositions across 10 instruments by learning patterns of harmony, rhythm, and style from hundreds of thousands of MIDI files.