autodistill/autodistill

autodistill – From unlabeled images to a tiny edge‑ready model

What it is – autodistill is a Python library that lets you distill a large, slow foundation model (the Base Model) into a small, fast supervised model (the Target Model) without any manual labeling. The workflow is:

1. Feed a folder of raw images to a Base Model.
2. An Ontology tells the Base Model how to turn its predictions into class labels.
3. The Base Model auto‑labels the images, producing a Dataset.
4. A Target Model (e.g., YOLOv8, DETR) is trained on that dataset, yielding a Distilled Model ready for deployment on the cloud or edge devices.

Core concepts (as defined in the README)

Concept	Role
Task	The type of prediction you want (currently detection / instance segmentation; classification coming soon).
Base Model	Large foundation model that can generate labels (e.g., Grounded‑SAM, Grounding‑DINO, OWL‑ViT, LLaVA‑1.5).
Ontology	A mapping that tells the Base Model what prompt to use and how to translate its output into your class names (e.g., {"shipping container": "container"}).
Dataset	Auto‑labeled images produced by the Base Model, stored in a format that the Target Model can consume.
Target Model	Small, fast supervised model that is fine‑tuned on the auto‑labeled dataset (YOLOv8, YOLO‑NAS, YOLOv5, DETR, etc.).
Distilled Model	The final, lightweight model you ship to production.

Main features (from the README)

• Pluggable interface – Base and Target models are separate plugins, so you can mix‑and‑match any supported foundation model with any SOTA detector.
• Automatic labeling – No human annotation step; the Base Model creates the training set.
• Fast supervised training – Uses lightweight detectors that train in minutes on a single GPU.
• Model ownership – The distilled model is yours to host wherever you like.
• Edge‑ready deployment – Exported models can run on CPUs, micro‑controllers, or any edge runtime.

Quick‑start example (exactly as shown in the README)

# Install core + two plugins (Grounded‑SAM as base, YOLOv8 as target)
pip install autodistill autodistill-grounded-sam autodistill-yolov8

# One‑liner that labels and trains
autodistill images \
  --base="grounding_dino" \
  --target="yolov8" \
  --ontology '{"prompt": "label"}' \
  --output="./dataset"

The command reads every image in ./images, asks Grounding‑DINO to label them according to the supplied prompt, writes a YOLO‑compatible dataset to ./dataset, and then trains a YOLOv8 model.

A more explicit Python snippet (also from the README):

from autodistill_grounded_sam import GroundedSAM
from autodistill.detection import CaptionOntology
from autodistill_yolov8 import YOLOv8

# 1️⃣ Base model + ontology
base = GroundedSAM(ontology=CaptionOntology({"shipping container": "container"}))
base.label(input_folder="./images", output_folder="./dataset")

# 2️⃣ Target model training
target = YOLOv8("yolov8n.pt")
target.train("./dataset/data.yaml", epochs=200)

# 3️⃣ Inference with the distilled model
pred = target.predict("./dataset/valid/your-image.jpg", confidence=0.5)
print(pred)

The README also shows how to upload the resulting model to Roboflow for hosted inference.

Available model plugins (status as of the README)

Task	Base models (✓ = ready)	Target models (✓ = ready)
Object detection	Grounded‑SAM 2, DETIC, Grounded‑SAM, Grounding‑DINO, OWL‑ViT, SAM‑CLIP, LLaVA‑1.5, Kosmos‑2, OWLv2, Roboflow‑Universe, CoDet, Azure Custom Vision, AWS Rekognition, Google Vision	YOLOv8, YOLO‑NAS, YOLOv5, DETR (others in progress)
Instance segmentation	Grounded‑SAM, SAM‑CLIP, SegGPT, FastSAM (partial)	YOLOv8 (others in progress)
Classification	(planned)	ViT, YOLOv8, YOLOv5 (planned)

Installation notes

• The core package (autodistill) only defines the abstract pipeline; you must also install the plugins for the specific Base/Target models you want.
• Plugins are published as separate PyPI packages (e.g., autodistill-grounded-sam, autodistill-yolov8). This keeps heavy dependencies isolated.
• For development you can clone the repo and pip install -e ..

When to use it

• Rapid prototyping: You have a collection of images but no annotations and need a detector quickly.
• Edge deployment: You want a model that runs in real‑time on limited hardware.
• Cost‑sensitive pipelines: Avoid paying for large‑model API calls for every inference; use the large model only once for labeling.

Known limitations (as described in the README)

• Currently limited to vision tasks; language‑only tasks are not yet supported.
• The quality of the distilled model depends heavily on the Base Model’s labeling accuracy; imperfect base predictions will propagate.
• Some Base/Target combinations are still “work in progress” (e.g., YOLO‑6, YOLO‑7, many segmentation targets).
• Large foundation models may be gated behind private APIs or require substantial GPU resources for labeling.

Where to learn more

• Tutorial: https://blog.roboflow.com/autodistill
• Docs: https://docs.autodistill.com
• Demo notebook: https://colab.research.google.com/github/roboflow-ai/notebooks/.../how-to-auto-train-yolov8-model-with-autodistill.ipynb
• YouTube walkthrough: https://www.youtube.com/watch?v=gKTYMfwPo4M

---

All information above is taken directly from the project's README; no additional features have been inferred.