OpenVGLab/OmniLottie

What it solves

OmniLottie addresses the difficulty of creating complex vector animations (Lottie files) by providing an end-to-end multimodal generator. It allows users to create these animations using various inputs—text, images, or videos—rather than manually designing them in specialized software.

How it works

The project utilizes pre-trained Vision-Language Models (VLMs) to translate multi-modal instructions into parameterized Lottie tokens. It can process three primary workflows:

• Text-to-Lottie: Generating animations from descriptive text prompts.
• Image-to-Lottie: Creating animations based on a provided image and optional text guidance.
• Video-to-Lottie: Converting existing video files into vector-based Lottie animations.

Who it’s for

This tool is designed for designers, developers, and researchers who need high-quality, scalable vector animations for web or app interfaces without requiring manual frame-by-frame animation skills.

Highlights

• Multimodal Input: Supports text, image, and video as starting points for animation generation.
• Large-scale Dataset: Includes MMLottie-2M, a dataset of two million richly annotated Lottie animations.
• Standardized Benchmarking: Provides MMLottieBench to evaluate the quality and capabilities of vector animation generators.
• Flexible Deployment: Offers both original PyTorch weights and HuggingFace-compatible formats, including a Gradio demo and a community-contributed ComfyUI plugin.