raullenchai/Rapid-MLX

The fastest local AI engine for Apple Silicon. 4.2x faster than Ollama, 0.08s cached TTFT, 100% tool calling. 17 tool parsers, prompt cache, reasoning separation, cloud routing. Drop-in OpenAI replacement. Works with Claude Code, Cursor, Aider.

What it solves

Rapid-MLX provides a high-performance local AI engine specifically optimized for Apple Silicon (M1 through M4 chips). It eliminates the need for complex setup or third-party shims by offering a native MLX-based environment that is significantly faster than alternatives like Ollama, while remaining compatible with standard OpenAI and Anthropic API formats.

How it works

The engine uses pure MLX kernels to leverage Apple Silicon's hardware bandwidth. It implements several performance optimizations, including continuous batching, a prompt cache (using radix and DeltaNet RNN snapshots), and a quantized live KV cache. It can be deployed as a command-line interface (CLI) for direct chatting or as an HTTP server that mimics the OpenAI/Anthropic APIs, allowing it to act as a backend for various IDEs and agent frameworks.

Who it’s for

It is designed for developers and AI enthusiasts using M-series Macs who want to run LLMs locally with maximum efficiency, as well as those who need a local API backend for AI agents, coding assistants (like Cursor or Aider), and Python frameworks (like LangChain or PydanticAI).

Highlights

  • Apple Silicon Native: Built on pure MLX kernels without llama.cpp fallbacks or Metal shims.
  • API Compatibility: Drop-in replacement for OpenAI and Anthropic APIs, supporting routes for chat completions, messages, embeddings, and audio.
  • Broad Ecosystem Support: Wire-verified compatibility with 11 agent CLIs (including Claude Code and Aider) and 3 major Python frameworks.
  • Hardware-Aware: Includes a RAM detector that recommends specific model sizes and quantizations based on the user's Mac configuration.