Deploying DeepFloyd IF with BentoML

TL;DR

Hugging Face has detailed a workflow for deploying DeepFloyd IF using BentoML, an open-source model serving framework. This integration allows developers to manage the complex, multi-stage architecture of DeepFloyd IF by scaling inference runners independently across different GPU resources to optimize production performance.

DeepFloyd IF Architecture and Capabilities

DeepFloyd IF is an open-source text-to-image model that operates directly in pixel space, distinguishing it from latent diffusion models like Stable Diffusion. It utilizes a modular structure consisting of a frozen text encoder and three cascaded pixel diffusion modules:

  • Stage 1: Generates a base image of 64x64 pixels.
  • Stage 2 and 3: Progressively upscale the image to a final resolution of 1024x1024 pixels.

To enhance language understanding for complex prompts, DeepFloyd IF integrates the T5-XXL-1.1 Large Language Model (LLM) as its text encoder.

Production Deployment with BentoML

BentoML provides a unified framework to transition models from the Hugging Face Hub to production-ready AI applications. The deployment process follows a five-step lifecycle:

  1. Define a model: Utilize models trained in libraries like PyTorch or TensorFlow.
  2. Save the model: Store models in the BentoML local Model Store for management and serving access.
  3. Create a BentoML Service: Use a service.py file to wrap the model and define serving logic via Runners, which handle model inference at scale and expose APIs for input/output processing.
  4. Build a Bento: Package the service and models into a "Bento"—a deployable artifact containing all necessary code and dependencies via a YAML configuration file.
  5. Deploy the Bento: Containerize the Bento into a Docker image for Kubernetes deployment or use Yatai for automated scaling on Kubernetes.

Technical Implementation and Resource Management

Deploying DeepFloyd IF requires significant computational resources. The recommended hardware includes at least 2x16GB VRAM GPUs or 1x40GB VRAM GPU. For production-grade serving, a single Tesla T4 is not recommended.

GPU Allocation Strategies

BentoML enables the independent scaling of Runners for each stage of the DeepFloyd IF pipeline, allowing developers to allocate resources based on the specific needs of each stage:

  • High VRAM (40GB+): All models can run on a single GPU.
  • Dual Tesla T4 (15GB each): Stage 1 can be assigned to the first GPU, while Stages 2 and 3 are assigned to the second.
  • Mixed GPU Setup: Stage 1 can be assigned to a primary GPU (e.g., T4), with Stages 2 and 3 distributed across two additional GPUs with smaller VRAM.

Deployment Workflow

To implement this setup, developers use import_models.py to download the required stages to the BentoML Model Store and service.py to define the API that accepts JSON prompts and negative prompts to return a generated image. The final artifact is packaged using bentoml build and can be served locally via bentoml serve or deployed to the cloud using bentoml containerize.

Sources