ddalcu/mlx-serve

Native LLM inference server for Apple Silicon. OpenAI + Anthropic API compatible. No Python. Zig backend, Swift frontend macOS app with chat, music, voice, video generation.

What it solves

mlx-serve provides a high-performance, local inference server for Apple Silicon Macs, eliminating the need for Python or cloud dependencies. It solves the problem of fragmented model formats and API compatibility by supporting both MLX and GGUF formats and exposing multiple industry-standard APIs (OpenAI, Anthropic, and Ollama) on a single port.

How it works

Built as a native Zig binary, the server integrates MLX and an embedded llama.cpp to run a wide variety of LLMs and multimodal models. It utilizes custom Metal kernels, continuous batching, and four types of speculative decoding (PLD, draft companions, Gemma 4 drafter, and Qwen MTP) to maximize decode speeds on Apple hardware. The project also includes a macOS menu-bar app (MLX Core) for model management, chat, and agent orchestration.

Who it’s for

Developers and AI enthusiasts using Apple Silicon Macs who want a fast, lightweight local LLM server that integrates seamlessly with existing coding agents (like Claude Code, Cursor, and Zed) and other AI tools.

Highlights

  • Multi-Format Support: Runs any MLX-format model and any GGUF model from HuggingFace.
  • Unified API: Single port supports OpenAI, Anthropic Messages, and Ollama APIs.
  • Multimodal Capabilities: Native generation of images, video, music, speech (with voice cloning), and 3D models.
  • High Performance: Outperforms LM Studio in decode speed using identical MLX weights.
  • Agent-Ready: Built-in agent loop, MCP tool calling, and a sandboxed Linux VM for safe agent shell execution.
  • Zero Python: Ships as a single, small Zig binary or a notarized macOS app.
  • LAN Sharing: Allows using models hosted on another Mac over the network via Bonjour.

Related

  • Project
    lablup/mlxcel

    A high-performance native Rust runtime for LLM, VLM, and audio inference on Apple Silicon, CUDA, and ROCm, removing Python from the production request path.

  • Project
    jjang-ai/vmlx

    A self-hosted inference server for Apple Silicon that provides OpenAI, Anthropic, and Ollama compatible APIs for running LLMs, VLMs, and image generation models locally.

  • Project
    raullenchai/Rapid-MLX

    Rapid-MLX is an open-source LLM inference server for Apple Silicon Macs, built on Apple's MLX framework. It serves open-weight models (like Qwen) via OpenAI- and Anthropic-compatible APIs, tuned for fast, reliable tool calling by coding agents — with claims of up to 4× faster decoding than Apple's reference server, plus continuous batching, persistent prefix caching, a CLI, a desktop app, and image-generation support.

  • Project
    cubist38/mlx-openai-server

    An OpenAI-compatible API server for running local MLX models on Apple Silicon, supporting text, multimodal, image, and audio models.

  • Project
    nicedreamzapp/claude-code-local

    A local server that enables Claude Code to run on Apple Silicon Macs using open-source models via the MLX framework, eliminating the need for cloud API keys and ensuring data privacy.