Goekdeniz-Guelmez/mlx-lm-lora

MLX‑LM‑LoRA

What it is A Python library that lets you fine‑tune large language models (LLMs) on Apple‑silicon Macs using the MLX framework. It adds a suite of efficient training methods—LoRA, DoRA, full‑precision, and quantized training (QLoRA, QAT)—and a large collection of preference‑based optimization algorithms (DPO, ORPO, RLHF, PPO, etc.).

Why it matters Training LLMs usually requires powerful GPUs. MLX‑LM‑LoRA makes it possible to do the same on a MacBook or Mac Studio, leveraging the Apple GPU and the low‑memory tricks built into MLX. It also bundles many of the newest alignment‑training techniques, so researchers and hobbyists can experiment with RL‑from‑human‑feedback, preference optimisation, and quantisation‑aware training without leaving the Apple ecosystem.

Key features (as described in the README)

• Model support: Works with any model supported by the upstream MLX‑LM project (Llama, Mistral, Qwen, Gemma, OLMo, MiniCPM, etc.).
• Training methods:
  • LoRA (low‑rank adapters) and DoRA (weight‑decomposed LoRA)
  • Full‑precision fine‑tuning
  • Quantized training (4‑, 6‑, 8‑bit QLoRA)
  • Quantisation‑aware training (QAT) for SFT, DPO, ORPO, DSLA
• Optimization algorithms:
  • Supervised Fine‑Tuning (SFT)
  • Direct Preference Optimisation (DPO)
  • Odds‑Ratio Preference Optimisation (ORPO)
  • Directional & Similarity‑aware Latent Alignment (DSLA)
  • Contrastive Preference Optimisation (CPO)
  • Group‑relative policies (GRPO, GSPO, Dr. GRPO, DAPO)
  • Reinforcement‑Learning‑from‑Human‑Feedback variants (RLHF‑Reinforce, PPO, KL‑PO)
• Custom preference models: You can train your own preference model for online training loops.
• Memory tricks: Gradient accumulation, checkpointing, recurrent training safeguards, and compiled optimizer updates to keep RAM usage low on Apple silicon.
• Configuration: All options are exposed via command‑line flags or a YAML config file; flags override the config.

Installation

pip install -U mlx-lm-lora

The package is published on PyPI and licensed under Apache 2.0.

Quick start

mlx_lm_lora.train \
  --model <model‑id> \
  --train \
  --data <dataset‑id> \
  --iters 600

Replace <model-id> with a model from the MLX‑LM hub (e.g., Goekdeniz-Guelmez/Josiefied-Qwen2.5-0.5B-Instruct-abliterated-v1) and <dataset-id> with a dataset on the mlx‑community hub.

How to use

1. Pick a training mode – --train-mode sft, dpo, orpo, dsla, grpo, etc.
2. Select an adaptation type – --train-type lora (default), dora, or full.
3. Optional quantisation – add --qat-enable and set --qat-bits (2‑16) for QAT, or use --quantize 4 for QLoRA.
4. Provide a dataset – the library accepts JSONL files with either prompt/completion pairs (SFT) or preference triples (prompt, chosen, rejected).
5. Run – the CLI streams progress and writes checkpoints automatically.

Example: LoRA SFT

mlx_lm_lora.train \
  --model Goekdeniz-Guelmez/Josiefied-Qwen2.5-0.5B-Instruct-abliterated-v1 \
  --train \
  --train-mode sft \
  --train-type lora \
  --data mlx-community/hermes-3 \
  --batch-size 4 \
  --learning-rate 1e-5 \
  --iters 1000

Example: QAT‑enabled DPO

mlx_lm_lora.train \
  --model <model> \
  --train \
  --train-mode dpo \
  --data <pref‑dataset> \
  --qat-enable \
  --qat-bits 4

Where to find more All notebooks, sample YAML configs, and step‑by‑step walkthroughs live in a companion repo: Goekdeniz-Guelmez/mlx-lm-lora-example-notebooks.

Who might use it

• Researchers prototyping new alignment objectives on a Mac.
• Developers who want to fine‑tune a small LLM for a specific task without renting GPU cloud.
• Educators demonstrating LLM training pipelines on commodity hardware.

---

TL;DR: MLX‑LM‑LoRA is a full‑featured, Apple‑silicon‑focused library for fine‑tuning LLMs with LoRA, quantisation, and a wide range of preference‑based training algorithms, all runnable via a simple CLI.