Tencent/WeMM-Embedding

WeMM-Embedding is a family of universal multimodal embedding models by the WeChat Vision Team at Tencent, supporting multimodal understanding and retrieval.

What it solves

WeMM-Embedding addresses the need for a unified way to represent different types of data—text, images, videos, and visual documents—as mathematical vectors (embeddings). This allows diverse multimodal inputs to be compared or searched within a single shared space, enabling high-performance retrieval and analysis across different media types.

How it works

It is a family of multimodal embedding models (available in 2B, 4B, and 9B parameter sizes) that process various inputs and generate a representation from the last-layer hidden state at a specific <embedding> token. The models utilize Matryoshka embeddings, which allow users to truncate the embedding vector to smaller dimensions (e.g., from 2048 down to 256) while retaining most of the original performance, reducing storage and computation costs.

Who it’s for

This project is designed for developers and researchers building multimodal search engines, retrieval-augmented generation (RAG) systems, or any application that requires comparing text, images, and videos in a unified vector space.

Highlights

  • Universal Modality Support: Handles text, images, videos, visual documents, and interleaved multimodal inputs.
  • Flexible Dimensions: Supports Matryoshka embeddings for adjustable vector sizes without significant loss in accuracy.
  • High Performance: Achieves state-of-the-art results on MMEB-v2 and MMEB-v3 benchmarks across image, video, and visual document tasks.
  • Deployment Ready: Compatible with popular inference frameworks like vLLM, SGLang, and Sentence Transformers.

Related

  • Project
    TIGER-AI-Lab/VLM2Vec

    A unified framework for training and evaluating omni-modality embedding models that produce fixed-dimensional vectors for text, images, videos, audio, and visual documents.

  • Dispatch
    Qwen3-VL-Embedding and Qwen3-VL-Reranker Release

    Qwen has released Qwen3-VL-Embedding and Qwen3-VL-Reranker, a series of multimodal models designed for high-precision cross-modal retrieval across text, images, screenshots, and video.

  • Project
    FlagOpen/FlagEmbedding

    A comprehensive retrieval toolkit for search and RAG that provides state-of-the-art embedding models and rerankers for text and multimodal data.

  • Project
    vespa-engine/sample-apps

    A collection of reference applications and tutorials for Vespa, demonstrating how to implement hybrid search, RAG, and multimodal vector retrieval at scale.

  • Project
    embeddings-benchmark/mteb

    A multimodal toolbox for evaluating embeddings and retrieval systems, providing a standardized benchmark and leaderboard for comparing model performance.