Tencent/WeMM-Embedding

WeMM-Embedding is a family of universal multimodal embedding models by the WeChat Vision Team at Tencent, supporting multimodal understanding and retrieval.

What it solves

WeMM-Embedding provides a unified way to represent different types of data—text, images, videos, and visual documents—as mathematical vectors (embeddings). This allows for efficient searching and retrieval across different modalities, enabling a system to find a video clip or an image based on a text description or vice versa.

How it works

The project provides a family of models (2B, 4B, and 9B parameters) that process multimodal inputs. It generates embeddings by extracting the last-layer hidden state at a specific <embedding> token position and applying L2 normalization.

To optimize storage and computation, the models use Matryoshka Embeddings, which allow users to truncate the full embedding to a smaller dimension (e.g., from 2048 down to 256) while retaining a high percentage of the original performance.

Who it’s for

Developers and researchers building multimodal retrieval systems, search engines, or AI agents that need to understand and compare text, images, and videos in a single shared space.

Highlights

  • Universal Multimodal Support: Handles text, images, videos, and visual documents, including interleaved inputs.
  • Flexible Dimensions: Supports Matryoshka embeddings for adjustable vector sizes without significant loss in accuracy.
  • High Performance: Achieves state-of-the-art results on MMEB-v2 and MMEB-v3 benchmarks.
  • Easy Integration: Compatible with transformers, SentenceTransformers, vLLM, and SGLang for serving.

Related