Epistates/pmetal

PMetal: high-performance Apple Silicon framework for local LLM inference, LoRA/QLoRA fine-tuning, serving, quantization, and MLX/Metal acceleration.

What it solves

PMetal is a comprehensive machine learning platform designed specifically for Apple Silicon. It eliminates the need to leave the Apple ecosystem for high-performance ML tasks by providing a unified suite for model training, fine-tuning, and inference, leveraging low-level Metal GPU kernels and the Apple Neural Engine (ANE).

How it works

Written in Rust, PMetal operates as an embeddable SDK and a set of tools. It integrates with the MLX C++ bridge and custom Metal kernels to optimize performance on M-series chips. The platform is accessible through three primary interfaces:

  • Desktop GUI: A Tauri and Svelte application for visual model management and training.
  • Terminal TUI: A 20-tab control center for deep configuration and monitoring.
  • CLI: A command-line interface for rapid execution of training, inference, and quantization tasks.

It also supports distributed training across multiple Macs via a "home cluster" using Thunderbolt, Ethernet, or Wi-Fi, automatically forming a ring for gradient all-reduce operations.

Who it’s for

  • ML Engineers and Researchers who want to train and fine-tune models on Apple hardware.
  • Rust and Python Developers looking to integrate high-performance ML capabilities into their own applications via the provided SDKs.
  • Apple Silicon Users who want a GUI or TUI for managing local LLMs, quantization, and model merging.

Highlights

  • Full-Stack Tooling: Includes a GUI, TUI, and CLI for every stage of the ML lifecycle (pretraining, SFT, LoRA, QLoRA, distillation, and inference).
  • Hardware-Optimized: Automatically detects chip family (M1 through M5) and tunes kernel parameters based on device tier (Base, Pro, Max, Ultra).
  • Distributed Training: Thunderbolt-aware clustering for multi-Mac distributed training with automatic fabric fallback.
  • Extensive Model Support: Supports a wide array of architectures including Llama, Qwen, DeepSeek, Mistral, Gemma, and Phi.
  • Advanced Techniques: Implements GRPO reasoning training, knowledge distillation, speculative decoding (DFlash), and various model merging strategies (SLERP, TIES, DARE).

Related

  • Project
    oxiglade/mlx-rs

    Rust bindings for Apple's MLX library, enabling high-performance machine learning and array operations optimized for Apple Silicon.

  • Project
    ARahim3/mlx-tune

    A library for fine-tuning LLMs, vision, audio, and OCR models natively on Apple Silicon using an Unsloth-compatible API for easy prototyping and scaling to cloud GPUs.

  • Project
    lablup/mlxcel

    A high-performance native runtime and server for LLMs, VLMs, and audio models on Apple Silicon and NVIDIA CUDA, removing Python from the inference path.

  • Dispatch
    Apple M6 and M5 Ultra launch: 2 nm silicon, quad‑die architecture, and massive AI compute

    Apple unveiled the M6 2 nm chip in a new Mac mini and the M5 Ultra quad‑die SoC in a new Mac Studio, delivering up to 2.4× faster CPU performance, 30% more AI GPU compute, and up to 1.2 TB/s memory bandwidth for on‑device large language models.

  • Project
    perplexityai/pplx-garden

    A collection of open-source inference technologies from Perplexity AI, featuring RDMA-based communication for LLMs, a CPU-optimized tokenizer, and a Metal-based inference server for Apple Silicon.