Goekdeniz-Guelmez/mlx-lm-lora

Train Large Language Models on MLX.

What it solves

MLX-LM-LoRA provides a comprehensive toolkit for fine-tuning Large Language Models (LLMs) locally on Apple Silicon. It enables users to customize models like Llama, Mistral, Qwen, and Gemma using a wide variety of efficient training methods and preference optimization algorithms, reducing the need for expensive cloud GPU infrastructure.

How it works

The project leverages the MLX framework to optimize training on Mac hardware. It supports multiple training types, including LoRA (Low-Rank Adaptation), DoRA (Weight-Decomposed Low-Rank Adaptation), full-precision training, and quantized training (QLoRA).

It implements a vast array of training algorithms:

  • Supervised Fine-Tuning (SFT): Standard instruction tuning.
  • Preference Optimization: Includes DPO, CPO, ORPO, and Online DPO, which align models to human preferences without necessarily needing a separate reward model.
  • Reinforcement Learning: Implements GRPO, GSPO, Dr. GRPO, DAPO, RLHF Reinforce, and PPO.
  • Quantization Aware Training (QAT): Simulates quantization effects during training to improve the performance of the final quantized model.

Who it’s for

  • AI Developers: Those wanting to run and fine-tune LLMs on Apple Silicon.
  • ML Engineers: Researchers experimenting with different preference optimization and RLHF techniques locally.
  • Model Customizers: Users looking to integrate Quantization Aware Training to maintain model quality after quantization.

Highlights

  • Apple Silicon Optimized: Built specifically for the MLX framework.
  • Extensive Algorithm Support: Covers everything from basic SFT to advanced GRPO and PPO.
  • QAT Integration: Supports 4-16 bit quantization projection during training for SFT, DPO, and ORPO.
  • Flexible Configuration: Supports both command-line flags and YAML configuration files.
  • Custom Reward Functions: Allows users to define their own Python-based reward functions for GRPO training.

Related

  • Project
    ARahim3/mlx-tune

    A library for fine-tuning LLMs, vision, audio, and OCR models natively on Apple Silicon using an Unsloth-compatible API for easy prototyping and scaling to cloud GPUs.

  • Project
    ml-explore/mlx-lm

    A Python package for generating text and fine-tuning large language models on Apple silicon using the MLX framework.

  • Project
    OptimalScale/LMFlow

    LMFlow is an extensible Python toolbox for fine‑tuning large language models. It supports full‑parameter training, LoRA, the memory‑efficient LISA algorithm, QLoRA quantization, and a suite of speed‑up tricks (FlashAttention‑2, DeepSpeed, vLLM, SGLang). After training, it provides simple scripts for chat‑style inference and optional Gradio/Flask deployment. The library is designed for Linux/Colab environments and offers optional extras for multimodal models, distributed reward‑model inference, and more.

  • Project
    youssofal/MTPLX

    A native Apple Silicon runtime that accelerates local LLMs by utilizing built-in multi-token prediction heads to increase decoding speed without changing output quality.

  • Dispatch
    Fine-tuning Llama 2 with Direct Preference Optimization (DPO) via TRL

    Hugging Face has integrated Direct Preference Optimization (DPO) into the TRL library, enabling the alignment of Llama 2 and other LLMs using a simplified binary cross-entropy loss instead of complex RLHF pipelines.