noteDigger – Front‑end audio‑to‑MIDI transcription tool

What it does

• A pure‑JavaScript web app that lets you upload an audio (or video) file, visualise its spectrogram, and manually or automatically extract musical notes.
• The extracted notes can be edited with a rich set of mouse/keyboard shortcuts, then exported as a standard MIDI file or as a custom binary “.nd” project file for later continuation.

Why it matters

• Most audio‑to‑MIDI transcribers are heavyweight desktop programs or cloud services. noteDigger runs entirely in the browser, needs no external libraries or frameworks, and therefore stays tiny (a few MB) while still offering advanced features such as GPU‑accelerated STFT/CQT, neural‑network‑based transcription, and harmonic removal.
• By keeping the whole workflow client‑side, it respects user privacy (audio never leaves the machine) and works offline after the initial page load.

Key features (as described in the README)

Feature	How it works / what you get
Audio import	Drag‑and‑drop or file‑menu upload of MP3, WAV, or even video files (MP4, MOV, etc.).
Spectrogram analysis	Real‑time STFT; optional CQT (Constant‑Q Transform) that can be GPU‑accelerated via WebGPU.
Automatic processing	• Harmonic removal using NNLS (non‑negative least squares) to boost the fundamental frequency. • Beat‑tracking and tempo detection (traditional signal‑processing algorithms). • AI‑assisted transcription using two ONNX models: basicamt (instrument‑agnostic pitch transcription) and septimbre (instrument‑separation transcription).
Manual editing	Draw notes directly on the time‑frequency canvas; move, stretch, copy/paste, delete, and lock tracks. All common shortcuts (Ctrl‑Z/Y, Ctrl‑C/X/V, etc.) are supported.
Quantisation & alignment	Two export modes: (1) raw note positions (seconds) for faithful “listen‑as‑you‑draw” workflow; (2) quantised to musical subdivisions based on user‑set precision x, with optional bar‑level BPM changes.
MIDI import/export	Export to standard MIDI (type‑1, multi‑track) with configurable tempo and quantisation. Import existing MIDI files; notes are mapped to tracks and velocities default to 127.
Project saving	Binary “.nd” files store the spectrogram, notes, tracks, and bar information for later re‑opening.
UI	Modern, framework‑free UI with draggable tracks, right‑click context menus, a piano‑roll view, and a top toolbar for switching tools (e.g., beat‑snap, bar‑snap).
Performance tricks	Reduced redraw frequency, unified refresh on any user action, and a “tinySynth” JavaScript synth for real‑time playback of edited notes.

Typical workflow (summarised from the README)

1. Open the app in a modern browser (Chrome recommended).
2. Upload an audio file and choose the analysis method (STFT or CQT; enable GPU if available).
3. Run optional automatic steps: beat detection → harmonic removal → key detection.
4. Use the AI transcription button (“人工智障扒谱”) to generate an initial note map.
5. Refine the result manually: snap notes to beats, merge bars, lock tracks, etc.
6. Export the final result as a MIDI file (quantised for sheet‑music) or save the project for later editing.

Technology highlights

• WebGPU‑accelerated CQT – speeds up constant‑Q transform even on integrated graphics.
• ONNX Runtime in the browser – runs two pre‑trained neural nets for pitch transcription and instrument separation without any server.
• Non‑negative least squares (NNLS) – a classic optimisation used here to subtract harmonic content and emphasise the fundamental.
• Pure‑JS architecture – no React/Vue/etc.; all UI components are handcrafted JS classes, keeping the bundle size minimal.

Who might use it

• Musicians who want a quick, privacy‑preserving way to pull notes from a recording.
• Educators or students experimenting with audio signal processing in the browser.
• Developers looking for a reference implementation of client‑side spectrogram visualisation, WebGPU‑based CQT, or ONNX inference in JavaScript.

Where to try it

• Live demo: https://maddersscientist.github.io/noteDigger/
• Video tutorial (Chinese): https://www.bilibili.com/video/BV1LFRmBcEEU

---

All information above is taken directly from the repository’s README; no additional features have been inferred.