ml-explore/mlx-swift-lm

LLMs and VLMs with MLX Swift

What it solves

It provides a Swift package for developers to build applications using large language models (LLMs) and vision language models (VLMs) specifically optimized for the MLX Swift framework.

How it works

The project offers a common API (MLXLLMCommon) and specific implementations for LLMs (MLXLLM), VLMs (MLXVLM), and embedding models (MLXEmbedders). It integrates with various tokenizer and model downloading packages through protocol conformance, allowing developers to load models from sources like Hugging Face. It also includes a bridge (MLXFoundationModels) to integrate MLX models with Apple's FoundationModels framework and a tool for grammar-constrained generation (MLXGuidedGeneration) to ensure outputs follow specific schemas like JSON or EBNF.

Who it’s for

Swift developers building AI-powered tools and applications on Apple platforms who want to leverage MLX for efficient model inference and fine-tuning.

Highlights

  • Support for both LLMs and Vision Language Models (VLMs).
  • Capabilities for both low-rank (LoRA) and full model fine-tuning, including support for quantized models.
  • Grammar-constrained generation to force model outputs into specific JSON schemas or EBNF.
  • Integration with Apple's FoundationModels API for use with LanguageModelSession.
  • Flexible integration with various tokenizer and downloader packages.

Related

  • Project
    eastriverlee/LLM.swift

    A Swift library for running local LLMs on Apple platforms, providing high-level abstractions for structured output, tool use, and chain-of-thought reasoning.

  • Project
    ml-explore/mlx-swift-examples

    A collection of example programs and libraries for MLX Swift, demonstrating how to build LLM, VLM, and image generation apps on iOS and macOS.

  • Project
    ml-explore/mlx-swift

    MLX Swift provides Swift bindings for the MLX tensor library, enabling high‑performance machine‑learning (including LLMs, vision‑language models and diffusion) on Apple‑silicon GPUs (Metal) and Linux (CPU/CUDA). It offers core tensor ops, neural‑network layers, optimizers and example apps, and can be added to Xcode or SwiftPM projects.

  • Project
    ml-explore/mlx-lm

    A Python package for generating text and fine-tuning large language models specifically optimized for Apple silicon using the MLX framework.

  • Project
    huggingface/AnyLanguageModel

    A Swift package that provides a unified, drop-in replacement for Apple's Foundation Models framework, enabling support for multiple on-device and cloud LLM providers.