incoai/splash

A local inference engine for Apple silicon, built around the model.

What it solves

Splash is a local inference engine specifically optimized for Apple silicon (M3 and newer) that allows users to run large language models (LLMs), coding agents, and vision-capable models on a single Mac with high performance.

How it works

It achieves high speeds by combining DFlash 2 speculative decoding, specialized Metal kernels, and automatic memory planning. The engine reuses cached prefixes and automatically batches concurrent requests to maximize throughput. It supports GGUF and MLX 4-bit formats, converting weights to kernel layouts during loading without requiring local compilation or Xcode.

Who it’s for

Developers and AI researchers using Apple silicon Macs (specifically M3 or newer) who want to run high-performance local LLMs, coding agents, or OpenAI/Anthropic-compatible applications locally.

Highlights

  • High Performance: Significantly faster decode speeds compared to llama.cpp on Apple silicon.
  • Broad Compatibility: Supports OpenAI and Anthropic API standards, including streaming, tool calls, JSON Schema output, and inline PDFs.
  • Hardware Optimized: Built specifically for Apple silicon with specialized Metal kernels and automatic memory/context sizing.
  • Integrated Features: Includes a built-in chat page and support for vision-capable models.

Related

  • Project
    ARahim3/mlx-dspark

    mlx‑d​spak is a Python package / macOS app that runs loss‑less speculative decoding (DSpark & DFlash) on Apple‑silicon Macs, giving 2‑5× faster text generation for many open‑source LLMs and exposing a local OpenAI‑compatible API.

  • Project
    SharpAI/SwiftLM

    A native Swift inference server for Apple Silicon that serves MLX models via an OpenAI-compatible API, featuring SSD expert streaming and KV cache quantization.

  • Project
    lmstudio-ai/mlx-engine

    An Apple MLX-based LLM engine for LM Studio that enables optimized local inference of text and vision models on macOS.

  • Project
    raullenchai/Rapid-MLX

    A high-performance LLM inference server for Apple Silicon that optimizes MLX for faster local inference and reliable tool-calling for coding agents.

  • Dispatch
    Optimizing Local Inference: A Deep Dive into the DeepSeek 4 Flash Metal Engine

    An exploration of a specialized local inference engine for DeepSeek 4 Flash on Apple Metal, highlighting the shift toward hardware-specific optimization and the challenges of large-context processing.