StayLameBro/backburner

Your iPhone helps your Mac run a 27B model: faster prompt reading and more context over a USB-C cable

What it solves

Backburner addresses the hardware limitations of running large language models (LLMs) locally on consumer devices. It allows users to run models that are too large for a single device by distributing the computational load and memory requirements across a Mac and one or more iPhones or iPads connected via high-speed USB-C.

How it works

The project uses a custom fork of llama.cpp to split the model's layers between the Mac and the connected iOS device. During the prefill phase, the Mac processes the initial layers while the iPhone processes the remaining layers on its GPU. For long-context tasks, the iPhone acts as an external memory store, holding older key-value (KV) pages and computing attention over them using its GPU and Neural Engine. The system ensures that the final output remains identical to running the model on a single, sufficiently powerful machine.

Who it’s for

This project is for power users and developers who want to run large, high-context LLMs (like Qwen3.8-27B) locally on Apple hardware that lacks the unified memory or GPU power to handle them alone. It is specifically designed for users with Apple Silicon Macs and newer iPhones (15 Pro or later) or iPads who are comfortable with command-line tools and sideloading apps.

Highlights

  • Increased Context Capacity: Enables running models with up to 128k–140k context length on devices that would otherwise be limited to 64k or less.
  • Faster Prefill: Accelerates prompt processing by pipelining layer computation across the Mac and iPhone, reducing wait times by 20–44%.
  • Neural Engine Integration: Utilizes the iPhone's Neural Engine to optimize attention calculations for cached context, further improving performance.
  • Lossless Inference: Guarantees that model outputs remain token-identical to single-machine inference.
  • Extensible Architecture: Designed to eventually support a distributed network of Macs, PCs, and mobile devices to run even larger models.

Related

  • Dispatch
    Optimizing Local Inference: A Deep Dive into the DeepSeek 4 Flash Metal Engine

    An exploration of a specialized local inference engine for DeepSeek 4 Flash on Apple Metal, highlighting the shift toward hardware-specific optimization and the challenges of large-context processing.

  • Project
    waybarrios/vllm-mlx

    A vLLM-style inference server for Apple Silicon that provides continuous batching and paged KV cache for high-throughput local serving of multimodal models.

  • Project
    youssofal/MTPLX

    A native Mac application and CLI that accelerates local LLM inference on Apple Silicon using Multi-Token Prediction (MTP) for faster decoding speeds.

  • Project
    ztxz16/fastllm

    A high-performance LLM inference engine designed for local and server deployment, supporting mixed-device execution (GPU/CPU/Disk) and advanced optimization techniques like speculative decoding.

  • Project
    bstnxbt/dflash-mlx

    A speculative decoding implementation for Apple Silicon that uses block-diffusion draft models to accelerate LLM inference without losing output quality.