Fannovel16/comfyui_controlnet_aux

What it solves

This project provides a comprehensive set of plug-and-play nodes for ComfyUI that generate "hint images" (preprocessors) for ControlNet. These hint images are used to guide the structure, pose, or depth of an AI-generated image based on a source image.

How it works

It integrates various preprocessor models from the original ControlNet annotators, connecting them to the Hugging Face Hub for model weights. It offers a wide range of specialized nodes for different extraction tasks, including line extraction, depth estimation, and pose detection. For performance, it supports GPU acceleration via TorchScript and ONNXRuntime for specific estimators like DWPose.

Who it’s for

Users of ComfyUI who want to precisely control the layout and structure of their AI image generation workflows using ControlNet.

Highlights

• Extensive Preprocessor Library: Includes line extractors (Canny, HED, Lineart), depth/normal estimators (MiDaS, Zoe, Depth Anything), and pose estimators (OpenPose, DWPose).
• AIO Aux Preprocessor: A single node that allows quick access to most preprocessors without needing to switch nodes manually.
• Pose Data Export: Ability to save pose keypoints as images or export them as OpenPose-format JSON for developers and API users.
• Hardware Acceleration: Support for ONNXRuntime and TorchScript to speed up pose estimation on GPUs.