lemonade-sdk/lemonade

Lemonade helps users discover and run local AI apps by serving optimized LLMs right from their own GPUs and NPUs. Join our discord: https://discord.gg/5xXzkMu8Zk

What it solves

Lemonade provides a way to run powerful AI models locally on a user's own hardware (CPU, GPU, and NPU) instead of relying on expensive, privacy-invasive cloud APIs. It simplifies the deployment of multi-modal AI—including text, image, speech, and 3D generation—by automatically optimizing for the specific hardware of the host PC.

How it works

Lemonade operates in two primary modes:

  1. Lemonade Server: A local service that exposes standard OpenAI, Anthropic, and Ollama APIs, allowing users to connect existing AI applications to their local hardware.
  2. Embeddable Lemonade: A portable binary that developers can package directly into their own software to provide built-in local AI capabilities without requiring the user to install a separate server.

It supports a wide range of model formats (GGUF, FLM, ONNX) and leverages various inference engines (like llama.cpp and whisper.cpp) across multiple backends including NVIDIA CUDA, AMD ROCm, Vulkan, Metal, and XDNA2 NPUs.

Who it’s for

  • End users who want free, private, and local AI for chat, coding, and content generation.
  • Software developers who want to integrate multi-modal AI into their applications as a portable, auto-optimizing binary.
  • AMD hardware owners looking to maximize the performance of Ryzen AI, Radeon, and Strix Halo systems.

Highlights

  • Multi-modal support: Handles text generation, speech-to-text, text-to-speech, image generation, and experimental 3D generation.
  • Broad hardware compatibility: Optimized for NVIDIA GPUs, AMD GPUs/NPUs, Apple Silicon, and standard CPUs.
  • API compatibility: Works with any OpenAI-compatible client library.
  • Model Management: Includes a built-in Model Manager to browse and download models from sources like Hugging Face and ModelScope.