1038lab/ComfyUI-QwenVL

ComfyUI-QwenVL custom node: Integrates the Qwen-VL series, including Qwen2.5-VL, Qwen3-VL, Qwen3.5-VL, Qwen3.6-VL (MoE), and Qwen3.8-VL, with GGUF support for advanced multimodal AI in text generation, image understanding, and video analysis.

What it solves

This project integrates Alibaba Cloud's Qwen-VL series of vision-language models into ComfyUI, allowing users to perform high-performance image and video perception and reasoning within a node-based workflow. It addresses the challenge of running large multimodal models on varying hardware by providing multiple quantization options and optimized attention mechanisms.

How it works

The suite uses a dual-engine architecture supporting both PyTorch (via Transformers) and GGUF (via llama.cpp). It features an intelligent video auto-scaling system that calculates a safe token budget per frame to prevent CUDA Out-of-Memory (OOM) errors and context overflow. To maximize speed, it implements a smart attention selector that prioritizes SageAttention, then Flash Attention 2, and finally SDPA based on the user's GPU architecture.

Who it’s for

It is designed for ComfyUI users who want to incorporate advanced vision-language capabilities—such as detailed image description, video analysis, and prompt enhancement—into their generative AI pipelines.

Highlights

  • Broad Model Support: Native support for Qwen3-VL (2B to 32B), Qwen3.5-VL, Qwen3.6-VL (MoE), Qwen3.8-VL, and Qwen2.5-VL.
  • Dual-Engine Backend: Flexibility to use high-performance Transformers or memory-efficient GGUF models.
  • Intelligent Video Handling: Dynamic token budget estimation to preserve resolution for small clips while preventing crashes on high-res videos.
  • Detailed Control: Offers both simple and advanced nodes for fine-tuning sampling parameters (temperature, top_p, beam search).
  • Hardware Optimization: Integrated SageAttention support and on-the-fly quantization (4-bit, 8-bit, FP16) to balance VRAM usage and speed.

Related