ostris/ai-toolkit

What it solves

AI Toolkit provides a unified, easy-to-use training suite for diffusion models, enabling users to train image, video, and audio models on consumer-grade hardware without needing to manage complex environment setups manually.

How it works

The toolkit supports a wide array of models (including FLUX, SDXL, and Wan 2.1) and provides two primary ways to interact with the training process: a Command Line Interface (CLI) using YAML configuration files and a web-based Graphical User Interface (GUI) for monitoring and managing jobs. It includes an experimental manager that automates the installation of PyTorch, Node.js, and FFmpeg to keep the environment isolated from the system.

Who it’s for

It is designed for creators and developers who want to fine-tune diffusion models using techniques like LoRA and LoKr, particularly those working with limited hardware resources or those who prefer a simplified setup process.

Highlights

• Broad Model Support: Compatible with a vast list of image, video, instruction/edit, and audio diffusion models.
• Automated Setup: An experimental manager handles dependency installation and environment creation.
• Flexible Training: Supports LoRA and LoKr training with the ability to target or exclude specific model layers.
• Simplified Data Handling: Automatically handles image resizing and aspect ratios, requiring only images and matching text caption files.
• Cloud Integration: Provides official templates and guides for training on RunPod and Modal.