ml-explore/mlx-swift-lm

LLMs and VLMs with MLX Swift

What it solves

MLX Swift LM is a Swift package designed to simplify the creation of applications using large language models (LLMs) and vision language models (VLMs). It provides the necessary infrastructure to load, run, and fine-tune these models on Apple hardware using the MLX Swift framework.

How it works

The package provides a common API for LLMs and VLMs, along with example implementations of various model architectures. It integrates with tokenizer and model downloading packages via protocol conformance, allowing developers to choose between different integration methods. It also includes specialized libraries for grammar-constrained generation (JSON Schema or EBNF) and a bridge to Apple's FoundationModels framework for standardized API usage.

Who it’s for

Swift developers building AI-powered tools and apps for macOS, iOS, and visionOS who want to leverage MLX for efficient local model execution.

Highlights

  • Model Loading: Integrated support for various tokenizer and model downloading packages.
  • Fine-tuning: Supports both Low-rank (LoRA) and full model fine-tuning, including support for quantized models.
  • Architecture Support: Includes implementations for a wide range of LLM and VLM architectures.
  • Guided Generation: Provides grammar-constrained generation to ensure model outputs follow specific schemas like JSON.
  • FoundationModels Bridge: Bridges MLX models into Apple's FoundationModels.LanguageModel for use with LanguageModelSession.

Related

  • Project
    eastriverlee/LLM.swift

    A Swift library for running local LLMs on Apple platforms, providing high-level abstractions for structured output, tool use, and chain-of-thought reasoning.

  • Project
    ml-explore/mlx-swift-examples

    A collection of example programs and libraries for MLX Swift, demonstrating how to build LLM, VLM, and image generation apps on iOS and macOS.

  • Project
    ml-explore/mlx-swift

    A Swift API for the MLX array framework that enables machine learning research and experimentation on Apple silicon.

  • Project
    ml-explore/mlx-lm

    A Python package for generating text and fine-tuning large language models on Apple silicon using the MLX framework.

  • Project
    Blaizzy/mlx-vlm

    A package for inference and fine-tuning of Vision Language Models and Omni Models on Mac using MLX, featuring speculative decoding and multimodal support.