NeptuneHub/AudioMuse-AI

AudioMuse‑AI – AI‑powered music‑library explorer

What it is – An open‑source, self‑hosted application that analyses the audio content of the files in your personal music collection (no reliance on tags or external APIs). It builds a “sonic fingerprint” for each track and stores the results in a PostgreSQL database. Using those fingerprints it can cluster similar songs, generate mood‑oriented playlists, map the whole library on a 2‑D visual canvas, and even answer natural‑language queries like “high‑tempo, low‑energy music”.

Why it matters – Most music‑organising tools depend on metadata (artist, genre, etc.) which is often missing or inaccurate. AudioMuse‑AI looks at the actual sound, so it can discover hidden relationships between tracks and create playlists that reflect how the music feels rather than how it’s labelled.

---

Core capabilities (as described in the README)

Feature	What you get
Multiple‑server support (v3.0.0+)	Connect any mix of Navidrome, Jellyfin, LMS, Lyrion, Emby or Plex to a single AudioMuse‑AI instance. Duplicate detection ensures each physical file is analysed only once.
Clustering & Music Map	Automatic grouping of sonically similar tracks and a 2‑D visual map that lets you explore your collection by “sound region”.
Instant playlists	Prompt the AI with a description (e.g., high‑tempo, low‑energy) and receive an on‑the‑fly playlist.
Playlist from similar songs	Pick a favourite track and get a discovery playlist of all songs sharing its sonic signature.
Song Paths	Generate a listening journey that bridges two arbitrary songs with intermediate tracks that smoothly transition in sound.
Sonic fingerprint / listening‑habit playlists	The system tracks what you play and can suggest tracks that match your personal listening profile.
Song Alchemy	Interactive “add / subtract” UI to fine‑tune a vibe, preview the result on the 2‑D map, and push the final list to your media server.
Text & lyrics search	Search by mood, instruments, genre, or by lyric content (supports 72 languages).
Search by recording	Upload or record a 20‑second audio snippet and find the matching track(s) in your library, or locate alternate recordings of a known song.

---

Architecture at a glance

• Flask web UI – serves the browser interface, the Swagger API docs, and handles authentication (JWT cookie).
• Worker containers – perform heavy audio analysis and clustering; they can run on CPU or, optionally, on NVIDIA GPUs (regular -nvidia images or experimental -nvidia-arm).
• PostgreSQL v15 – stores fingerprints, clustering results, user settings, and the job queue (since v2.6 the queue moved from Redis to PostgreSQL).
• Plugins – a simple plugin system lets third‑party extensions add extra functionality; plugins are persisted on a shared volume mounted into both Flask and worker containers.
• Deployments – provided as Docker‑Compose, Helm chart for Kubernetes, and native binary packages for macOS, Linux (deb/rpm) and Windows.

---

Getting started (quick‑start Docker path)

1. Clone the repo and copy the example env file:

   cp deployment/.env.example deployment/.env
   

2. Edit the .env (at least set POSTGRES_PASSWORD).
3. Launch the stack:

   docker compose -f deployment/docker-compose.yaml up -d
   

4. Open http://localhost:8000 → run the Setup Wizard → start an Analysis of your media server.
5. When finished, explore the UI pages for clustering, music map, instant playlists, etc.

---

Native (non‑container) installation

• macOS (Apple‑silicon) – download AudioMuse‑AI-arm64.zip, remove quarantine flag, and run the bundled app.
• Linux – install the .deb or .rpm (x86_64 or aarch64). The service can be started with audiomuse‑ai start or enabled as a user systemd unit.
• Windows – unzip the portable archive and run AudioMuse‑AI.exe start.

All native packages embed an embedded PostgreSQL instance, so no external database is required.

---

Hardware & performance notes

• Minimum: 4‑core CPU with AVX2 (or any recent ARM), 8 GB RAM, NVMe SSD.
• GPU acceleration is optional; the -nvidia images use CUDA for faster analysis and clustering. An experimental -nvidia-arm image targets DGX Spark‑class ARM GPUs.
• Virtual machines must expose the host CPU’s AVX2 instructions; QEMU without AVX2 will cause the service to abort.

---

Extensibility & community

• Plugins – documented in docs/PLUGIN.md; a public catalog lives at github.com/NeptuneHub/AudioMuse-AI-plugins.
• Helm chart – AudioMuse‑AI‑helm repo provides a ready‑to‑use Helm chart for Kubernetes deployments (AMD64 & ARM64).
• Contributing – guidelines are in CONTRIBUTING.md; the project welcomes code, documentation, and plugin contributions.
• Managed cloud option – Elestio offers a hosted version for users who prefer not to self‑host.

---

Release highlights (most recent)

• v3.3.0 – adds experimental -nvidia-arm image for ARM‑based NVIDIA GPUs.
• v3.2.0 – switches the job queue to PostgreSQL, removing the need for Redis.
• v3.0.0 – introduces multi‑server support with duplicate detection.
• v2.6.0 – plugin system made official.
• v2.5.0 – Plex integration added.

---

Bottom line

AudioMuse‑AI is a genuine AI‑driven music‑library tool: it extracts acoustic features, builds embeddings, and uses those embeddings to power clustering, similarity search, natural‑language playlist generation, and visual exploration. The project is fully open‑source, offers multiple deployment models (Docker, Kubernetes, native binaries), and integrates with the major self‑hosted media servers.