QwenLM/Qwen-MM-Plugins

Make any agent harness multimodal-native.

What it solves

It provides a set of native multimodal plugins that allow AI agents to interact with and process diverse media types—such as images, videos, audio, 3D models, and CAD files—directly rather than relying on separate APIs or basic shell commands.

How it works

Capabilities are installed as "Skills" and optional MCP (Model Context Protocol) servers. These plugins integrate with various agent harnesses (like Claude Code or Qwen Code) to give the underlying model the ability to read local files, call specialized model services for OCR and transcription, or even drive external software like Blender and FreeCAD.

Who it’s for

Developers building AI agents who want to add multimodal capabilities to Qwen models, as well as users of agent harnesses who need their AI to handle complex media tasks like video analysis, document visualization, or 3D modeling.

Highlights

  • Core Media Handling: Native reading of local images, video frames, and documents with support for cropping and bounding-box annotation.
  • Specialized Tooling: Direct control over 3D software including Blender (modeling/rendering) and FreeCAD (parametric CAD).
  • Advanced Video Memory: Hierarchical memory for long videos to allow querying without re-watching the entire file.
  • Omni-Capabilities: Audio-visual memory and the ability to convert tutorial videos into illustrated PDFs.
  • Broad Integration: Supports multiple agent harnesses and uses uv for on-demand Python dependency management.

Related

  • Project
    SamurAIGPT/Generative-Media-Skills

    A multimodal toolset and MCP server that enables AI agents to generate and edit professional images, videos, and audio using a schema-driven architecture.

  • Project
    1038lab/ComfyUI-QwenVL

    A ComfyUI extension that integrates Qwen-VL vision-language models for image and video understanding, featuring dual HF and GGUF backends and GPU-optimized inference.

  • Project
    ParisNeo/lollms-webui

    A unified local web interface for accessing hundreds of LLMs and multimodal AI models for text, image, video, and music generation.

  • Project
    simonw/llm-gemini

    A plugin for the LLM CLI tool that provides API access to Google's Gemini models, supporting multimodal inputs and server-side tools like Google Search.

  • Project
    inference-sh/skills

    A collection of AI agent skills and SDKs that enable models to generate images, videos, search the web, and automate social media via the inference.sh CLI.