erfanzar/EasyDeL

What it solves

EasyDeL is a high-performance framework for training, fine-tuning, and serving large-scale machine learning models. It bridges the gap between the ease of use found in libraries like HuggingFace Transformers and the extreme performance of JAX-based systems, allowing developers to deploy LLMs, multimodal models, and vision models on TPU and GPU clusters without sacrificing hackability.

How it works

Built on Flax NNX and JAX, EasyDeL provides a modular architecture that supports a wide array of distributed training strategies (DP, FSDP, TP, EP, SP) and optimized kernels via Triton (GPU) and Pallas (TPU). It includes a unified API for 16 specialized trainers (covering SFT, RLHF, and distillation) and a dedicated inference engine called eSurge, which implements continuous batching and paged KV caching for high-throughput serving.

Who it’s for

It is designed for ML engineers and researchers who need to train or deploy massive models at scale on TPU/GPU clusters but want a transparent, readable codebase that is easy to modify and customize.

Highlights

• Broad Model Support: Includes 70+ architectures such as Llama, Qwen, Mistral, DeepSeek, and Gemma.
• Enterprise Inference: The eSurge engine provides an OpenAI-compatible API, streaming, and multimodal serving for text, image, and video.
• Diverse Training Suite: Specialized trainers for DPO, ORPO, GRPO, and knowledge distillation.
• Advanced Attention: Implements over 10 mechanisms, including Flash Attention 2 and Ring Attention.
• Multimodal Capabilities: Supports vision-language models (LLaVA, Qwen2-VL), speech recognition (Whisper), and diffusion models.
• Unified Management: The eLargeModel class simplifies configuration, building, and orchestration of large models.