KohakuBlueleaf/LyCORIS

Lora beYond Conventional methods, Other Rank adaptation Implementations for Stable diffusion.

What it solves

LyCORIS provides a suite of parameter-efficient fine-tuning (PEFT) algorithms to customize Stable Diffusion models. It allows users to train small, portable adapter models (rather than full model checkpoints) that can be easily shared and combined, reducing the storage and compute requirements for fine-tuning.

How it works

The project implements various low-rank adaptation techniques that go beyond standard LoRA. It provides standalone PyTorch wrappers that can be applied to any module, as well as integration with popular training scripts like sd-scripts and interfaces like sd-webui and ComfyUI.

Key algorithms implemented include:

  • LoRA/LoCon: Standard low-rank adaptation and its extension to convolutional layers.
  • LoHa: Low-rank Hadamard product.
  • LoKr: Kronecker product adaptation.
  • (IA)³: Infused Adapter.
  • DyLoRA: Dynamic LoRA.

Recent updates have introduced experimental fused kernels using Triton and TileLang to significantly reduce device time and VRAM usage during training and inference.

Who it’s for

  • AI Artists and Model Trainers: Users who want to customize Stable Diffusion for specific characters, styles, or concepts without training a full model.
  • ML Researchers: Those exploring different low-rank adaptation methods for image generation.
  • Developers: People building image generation pipelines who need a way to integrate multiple fine-tuned adapters.

Highlights

  • Diverse Algorithm Support: Implements a wide range of PEFT methods including LoHa, LoKr, and (IA)³.
  • High Performance: Experimental fused kernels (Triton/TileLang) offer 1.4x–7.3x speedups over eager mode.
  • Broad Integration: Native support in sd-webui, ComfyUI, and InvokeAI.
  • Flexible API: Can be used as standalone wrappers for any PyTorch module or via high-level training scripts.

Related

  • Dispatch
    Using LoRA for Efficient Stable Diffusion Fine-Tuning

    Hugging Face has integrated Low-Rank Adaptation (LoRA) into the diffusers library, enabling faster Stable Diffusion fine-tuning with significantly lower VRAM requirements and model weights as small as 3 MB.

  • Project
    huggingface/finetrainers

    A library for the accessible training and fine-tuning of diffusion models, focusing on memory-efficient text-to-video and text-to-image generation.

  • Project
    huggingface/peft

    A library providing state-of-the-art parameter-efficient fine-tuning methods to adapt large pretrained models using minimal compute and storage.

  • Dispatch
    SDXL and Stable Diffusion Fast Inference with Latent Consistency LoRAs

    Hugging Face introduces LCM LoRAs, a method to enable high-quality image generation in 4 to 8 steps for SDXL and Stable Diffusion models, significantly reducing inference time.

  • Project
    LykosAI/StabilityMatrix

    A multi-platform package manager and inference interface for Stable Diffusion that provides one-click installation and shared model management for various Web UIs.