lemonade-sdk/lemonade

Lemonade – a fast, local AI server

Lemonade is an open‑source local AI server that lets you run modern large language models, speech‑to‑text, text‑to‑speech, image‑generation and other multimodal models on your own hardware. It mimics the OpenAI/Anthropic/Ollama HTTP APIs, so any existing client that talks to those services can point at http://localhost:13305/v1 and use the locally‑hosted models instead of a cloud endpoint.

Two ways to use it

Flavor	What it is	Typical use
Lemonade Server	A background service you install on Windows, macOS, Linux or Docker.	Run once and let any number of apps (VS Code, browsers, chat‑bots, etc.) connect via the standard API.
Embeddable Lemonade	A single portable binary you ship inside your own application.	Give a third‑party app its own private AI stack without requiring a separate installer.

What you can do out‑of‑the‑box

• Chat & coding with LLMs such as Gemma, Qwen, Claude‑style models.
• Image generation (Stable Diffusion, SDXL‑Turbo, etc.).
• Speech‑to‑text (Whisper, Moonshine) and text‑to‑speech (Kokoro, OpenMOSS).
• Audio & 3D generation via experimental back‑ends.
• Hybrid cloud/off‑load – route some requests to a remote OpenAI‑compatible provider while keeping others local.

Supported hardware & back‑ends

Modality	Engine	Backend examples	Devices
Text generation	llamacpp, flm, ryzenai‑llm, vllm (exp.)	cpu, cuda, rocm, vulkan, metal, hrx (exp.)	x86/ARM CPUs, NVIDIA GPUs, AMD GPUs, Apple Silicon, AMD XDNA2 NPU
Speech‑to‑text	whispercpp, moonshine	cpu, npu, metal, vulkan, rocm	Same as above
Text‑to‑speech	kokoro, openmoss (exp.)	cpu, cuda, vulkan, rocm, metal	Same as above
Image generation	sd‑cpp, thenoise (exp.)	cpu, cuda, vulkan, rocm, metal	Same as above
3‑D generation	trellis (exp.)	cuda, vulkan, rocm	Same as above

The README includes detailed tables of supported NVIDIA compute capabilities (Turing → Blackwell) and AMD ROCm families (gfx1100‑gfx120X), so you can see whether your GPU/CPU/NPU is covered.

Quick start (CLI)

# install – MSI for Windows, .deb/.rpm for Linux, .pkg for macOS, or Docker
# then pull a model
lemonade pull Gemma-4-E2B-it-GGUF

# run a chat model
lemonade run Gemma-4-E2B-it-GGUF

# launch a coding assistant (Claude‑style)
lemonade launch claude

# image generation
lemonade run SDXL-Turbo

# speech transcription
lemonade run Whisper-Large-v3-Turbo

You can list available models (lemonade list), manage aliases for fail‑over (lemonade alias …), and query which back‑ends your machine can use (lemonade backends).

Installation options

• Windows – MSI installer (download from the releases page).
• Linux – Packages for Arch, Debian, Fedora, Ubuntu, Snap, or build from source.
• macOS – .pkg installer.
• Docker – Pull the official container image.
• Source – Follow the docs/dev/getting-started.md guide.

Ecosystem & integration

Lemonade ships a Model Manager UI for browsing and downloading models, and a Marketplace where community members list apps that already work with the server (e.g., Firefox Chatbot, AnythingLLM, Open WebUI, GitHub Copilot, n8n, etc.).

To embed Lemonade in your own software, use the Embeddable Lemonade binary and follow the guide in docs/embeddable/README.md.

Community & licensing

• Apache‑2.0 license, free for commercial and personal use.
• Active Discord community, contribution‑friendly (PRs welcome).
• Roadmap driven by working groups (see docs/dev/working-groups).

---

Bottom line: Lemonade is a genuine, production‑ready platform for running a wide range of AI models locally, with extensive hardware support and drop‑in compatibility with existing OpenAI‑style client libraries.