Intellindust-AI-Lab/EdgeCrafter

[TMLR 26] EdgeCrafter: Compact ViTs for Edge Dense Prediction via Task-Specialized Distillation

What it solves

EdgeCrafter addresses the challenge of deploying high-performance Vision Transformers (ViTs) on edge devices. It provides compact models that maintain high accuracy for dense prediction tasks—such as object detection, instance segmentation, and pose estimation—while remaining small and fast enough for real-time edge deployment.

How it works

The project utilizes task-specialized distillation to create a series of compact ViTs. It offers a variety of model sizes (Small, Medium, Large, and Extra-Large) tailored for different performance-latency trade-offs. These models are designed to be efficient across various hardware backends, including NVIDIA GPUs (via TensorRT), Intel iGPUs/dGPUs (via torch.xpu), and CPUs.

Who it’s for

This tool is for AI researchers and developers building computer vision applications for edge hardware who need a balance between high precision (AP) and low latency.

Highlights

  • Multi-Task Support: Specialized models for object detection (ECDet), instance segmentation (ECSeg), and pose estimation (ECPose).
  • Hardware Flexibility: Native support for CUDA, Intel XPU (iGPU/dGPU), and CPU.
  • Edge Optimized: Designed for low latency, with the smallest models achieving inference speeds as low as 5.41ms on an NVIDIA T4.
  • Broad Integration: Integrated into tools like Intel® Geti™, LightlyTrain, and LibreYOLO.

Related

  • Project
    lucidrains/vit-pytorch

    A comprehensive PyTorch library implementing the original Vision Transformer (ViT) and dozens of its architectural variants for image classification.

  • Dispatch
    Fine-Tuning Vision Transformer (ViT) for Image Classification with Hugging Face Transformers

    Hugging Face provides a guide on fine-tuning the Vision Transformer (ViT) model for image classification using the datasets and transformers libraries, demonstrating the process with the beans dataset.

  • Project
    open-edge-platform/geti_v2

    An end-to-end computer vision platform that uses active learning and smart annotations to build and deploy optimized AI models with minimal data.

  • Project
    lightly-ai/lightly-train

    A computer vision framework for pretraining, fine-tuning, and distilling SOTA models like DINOv3 and LTDETRv2 for tasks such as object detection and segmentation.

  • Project
    NVIDIA-AI-IOT/Lidar_AI_Solution

    A collection of highly optimized CUDA and TensorRT implementations for 3D Lidar perception, speeding up sparse convolutions and sensor fusion for self-driving systems.