liebharc/homr

homr – Optical Music Recognition with AI

What it does

• Takes a photo or PDF of sheet music and turns it into a MusicXML file, a digital format that music programs (e.g., MuseScore) can read.
• The conversion runs locally via a command‑line tool (homr <image>). The output file is saved next to the input image.

How it works

1. Segmentation – A UNet‑style neural network (borrowed from the oemer project) scans the image and identifies basic visual elements: staff lines, note heads, stems, rests, clefs, bar lines, etc. It produces bounding boxes for each element.
2. Staff reconstruction – Using the detected elements, the code finds “staff anchors” (clefs, bar lines) and estimates the distance between staff lines (the unit size). This lets it rebuild the five‑line staff even when the picture is taken at an angle or has uneven spacing. Braces/brackets are also detected so that grand staffs (piano) or multiple voices can be merged.
3. Symbol recognition – Each staff is de‑warped (perspective‑corrected) and fed to a transformer model (based on Polyphonic‑TrOMR). The transformer predicts a sequence of musical symbols: note durations, pitches, accidentals, rests, tuplets, and some articulation marks. The model also emits a rough image‑position for each symbol, which is stored as a comment in the MusicXML.
4. MusicXML export – The symbol sequence is converted into a standard MusicXML file that can be opened in MuseScore, Sibelius, or any other music‑notation software.

Getting started

• Python 3.11/3.12 is required.
• Install via uv (the fastest way) or Poetry. Example with uv for CPU inference:

  uvx --from 'homr[cpu]' homr path/to/score.png
  

• For GPU acceleration, replace the extra with cuda (NVIDIA) or rocm (AMD).
• Run on several images at once: homr img1.png img2.png → creates img1.musicxml, img2.musicxml.
• Combine multiple MusicXML files with the external tool relieur.

Demo & resources

• Quick online demo: https://homr.site (or launch the provided Colab notebook).
• Android front‑end: the Andromr app uses this library under the hood.

Current limits

• Focuses on pitch and rhythm in treble/bass clefs.
• Does not handle dynamics, articulation beyond a few marks, double sharps/flats, or many other advanced notation symbols.

Technical highlights

• Two‑stage pipeline (segmentation → transformer) mirrors state‑of‑the‑art OMR research.
• Uses pretrained UNet models from oemer and a transformer model from Polyphonic‑TrOMR.
• Adds positional comments (<!-- imgpos: x, y -->) to each note in the MusicXML for downstream processing.

Who might use it

• Musicians or archivists who want to digitize printed scores quickly.
• Researchers experimenting with music‑related machine‑learning pipelines.
• Developers building apps that need to ingest sheet music (e.g., practice apps, music‑search engines).

Citation If you incorporate homr in academic work, cite the original oemer and Polyphonic‑TrOMR projects as indicated in the README.

---

TL;DR – homr is an open‑source Python tool that applies modern deep‑learning (UNet + transformer) to convert images of sheet music into editable MusicXML files, runnable on CPU or GPU via simple command‑line commands.