comfyui_LLM_party: a comprehensive suite of ComfyUI nodes for building complex LLM and agentic workflows

What it solves

ComfyUI LLM Party provides a comprehensive set of nodes for building LLM workflows within the ComfyUI ecosystem. It bridges the gap between large language models and image generation workflows, allowing users to integrate AI assistants, RAG systems, and complex agentic behaviors directly into their visual node-based environment.

How it works

The project implements a variety of custom nodes for ComfyUI that support multiple model loading methods:

• API-based: Supports OpenAI-compatible APIs, Azure OpenAI, Grok, and various others via oneapi or aisuite.
• Local Loading: Supports loading models directly from the transformer library, GGUF format via llama-cpp-python, and local hosting through Ollama.
• VLM Integration: Includes support for Vision-Language Models (VLMs) like Llama-3.2-Vision and Qwen2.5-VL for image-to-text tasks.

It enables the construction of everything from simple prompt generation for Stable Diffusion to complex agent-agent interaction modes (radial and ring) and integration with social apps like Discord and Feishu.

Who it’s for

• ComfyUI users who want to integrate LLMs into their image generation pipelines.
• AI researchers and students needing a visual interface for parameter debugging and model adaptation.
• Streaming media workers requiring a one-stop LLM + TTS + ComfyUI workflow.
• Developers building localized industry knowledge bases using RAG and GraphRAG.

Highlights

• Broad Model Support: Compatible with a vast array of API and local models, including GGUF and transformer-based LLMs/VLMs.
• Agentic Workflows: Supports multi-tool calling, role setting, and complex agent interaction patterns.
• Advanced RAG: Implements industry-specific word vector RAG and GraphRAG for knowledge base management.
• MCP Tooling: Integration with the Model Context Protocol (MCP) to connect to external MCP servers for expanded tool capabilities.
• Streaming Output: Real-time text display in the console for API calls.