jamesondh/chimera-vocal-synth

What it is

CHIMERA is a voice synthesizer that runs entirely in your web browser. Instead of playing back recorded audio samples, it simulates the physics of a human voice — the lungs, vocal folds, airway, tongue, lips, and nose — in real time. The heavy lifting is done by a Rust engine compiled to WebAssembly, which runs inside the browser's audio system. The same Rust code also runs natively for testing and as a command-line tool called chimera.

What you can do with it

Play — Drag the tongue, lips, and jaw around in a live cutaway view of a head. Control the voice with a vowel pad, knobs, your computer keyboard, or a Web MIDI device.
Speak — Type English text (using the CMU pronunciation dictionary, with a letter-to-sound fallback) or IPA, and the engine turns it into mouth gestures and speech rhythm. You can see the gestural score and a consonant chart on screen.
Sing — Use a piano roll where each note has lyrics; consonants are timed to land just before the beat.
Beatbox — Pads and a step sequencer, where every drum sound is actually an articulatory gesture (like a mouth movement), not a sample.
Bestiary — Non-human voices: a songbird (with a two-voice syrinx, the bird equivalent of vocal folds), a seal, an alligator, and an elephant.
Duet — A URL parameter (?show=<id>) plays a pre-rendered song with two 3D heads.

Male, female, and child vocal tracts blend continuously into each other. Everything you see on screen — air pressure in the airway, airflow, the vocal folds, spectra, and formants — animates live from the actual engine state.

How to run it

You need Rust (with the wasm32-unknown-unknown target) and Bun installed. Then:

bun install
bun run dev        # builds the WebAssembly, then starts Vite on 127.0.0.1:5317
bun run test       # builds wasm, runs cargo tests, then Bun tests

The audio system requires a secure context, so open it on localhost or over HTTPS (the README suggests tailscale serve as one option). Vercel builds it via vercel.json.

Offline rendering and measurement

The CLI can render audio without a browser:

cargo run --release -p chimera-cli -- render speech|text|song|notes|beatbox|creature|vowels …
cargo run --release -p chimera-cli -- diagnose       # checks nasal leakage, formants, bandwidths, pitch
cargo run --release -p chimera-cli -- compare csd    # scores the voice against a real singer (Children's Song Dataset)
cargo run --release -p chimera-cli -- compare tsp    # scores speech against 23 real speakers (TSP speech database)
bun tools/spectrogram.ts in.wav out.png              # makes a spectrogram image from an audio file

Documentation

The repo includes docs on the acoustic model (finite-difference time-domain airway simulation, sources, anatomy, gestures), the engine's real-time and determinism contract, the UI design, comparisons against real singers and speakers, and a hill-climbing log used to tune the voice one measured step at a time.

License

MIT. Bundled third-party files keep their own licenses: the CMU Pronouncing Dictionary and the fonts.