tong-io/tongflow

TongFlow : An Open-Source Multi-Modal GenAI Workflow Studio

What it solves

TongFlow provides a visual, low-barrier way to orchestrate complex generative AI workflows across multiple modalities. It eliminates the need for manual node connection or complex parameter tuning by allowing users to simply add, transform, and combine different AI capabilities to create advanced content like lip-synced videos, music videos, or 3D models.

How it works

The platform operates on a core concept where every AI model is treated as a modality transformation (e.g., text-to-image or text-to-audio). Users interact with a canvas using three primary operations:

  • Add: Import text, images, audio, video, documents, 3D models, or even URLs.
  • Transform: Apply specific AI capabilities like image generation, video upscaling, speech synthesis, or music rearrangement.
  • Combine: Fuse outputs together, such as performing lip-syncing, character swapping, or merging audio and video tracks.

The system uses a plugin-based architecture where capabilities are packaged as independent plugins. These can run via cloud APIs (like OpenAI or Gemini) or on local/cloud GPU compute (via Modal).

Who it’s for

  • Content Creators: Users looking to automate complex creative tasks like generating music videos or digital humans.
  • AI Developers: Those wanting to build and deploy custom modality transformation plugins.
  • Prototypers: Individuals needing a lightweight desktop or cloud-based studio to experiment with multi-modal AI workflows.

Highlights

  • Multi-modal support: Handles text, image, video, audio, 3D models, and documents.
  • Extensible plugin system: Allows any platform to package and publish independent capability nodes.
  • Flexible deployment: Available as a cloud studio, a lightweight desktop app, or via Docker for self-hosting.
  • Diverse capabilities: Includes advanced features like motion capture, music stem separation, and video watermark removal.