lmstudio-ai/mlx-engine

LM Studio Apple MLX engine

What it solves

It provides a high-performance LLM engine for LM Studio on macOS, allowing users to run large language models and vision models locally with optimized performance on Apple Silicon.

How it works

The engine is built upon the Apple MLX framework, leveraging mlx-lm for text generation, mlx-vlm for vision model inferencing, and Outlines for structured output. It integrates directly into LM Studio (version 0.3.4+) for Mac, though it also supports standalone demo usage via Python.

Who it’s for

Mac users who want to run local LLMs and vision models within the LM Studio ecosystem using Apple's optimized MLX framework.

Highlights

  • Apple Silicon Optimization: Built specifically for macOS 14.0+ and the MLX framework.
  • Multimodal Support: Supports both text generation and vision models (such as Llama-3.2-Vision, Pixtral, and Qwen2-VL).
  • Speculative Decoding: Includes a demo for speeding up inference using a draft model.
  • Structured Output: Integrates Outlines to ensure LLM responses follow specific formats.

Related

  • Project
    jjang-ai/mlxstudio

    A native macOS app for Apple Silicon that provides a user-friendly interface to run LLMs, vision models, and image generation locally using the MLX framework.

  • Project
    jjang-ai/vmlx

    A self-hosted inference server for Apple Silicon that provides OpenAI and Anthropic compatible APIs for running LLMs, VLMs, and image generation models locally.

  • Project
    Blaizzy/mlx-vlm

    A package for inference and fine-tuning of Vision Language Models and Omni Models on Mac using MLX, featuring speculative decoding and multimodal support.

  • Project
    ml-explore/mlx-lm

    A Python package for generating text and fine-tuning large language models on Apple silicon using the MLX framework.

  • Project
    vllm-project/vllm-metal

    A vLLM plugin that enables high-performance LLM inference on Apple Silicon Macs by using MLX as the compute backend.