netease-youdao/BCEmbedding

What it solves

BCEmbedding addresses the performance gap in bilingual and crosslingual retrieval tasks, specifically between Chinese and English. It provides a two-stage retrieval system (embedding and reranking) optimized for Retrieval Augmented Generation (RAG) across various professional domains like medicine, law, and finance, without requiring task-specific fine-tuning or instructions.

How it works

The project implements a two-stage retrieval pipeline:

1. Embedding Model: A dual-encoder that generates semantic vectors for efficient first-stage retrieval in English and Chinese.
2. Reranker Model: A cross-encoder that performs deeper semantic analysis to refine and rank search results. It supports English, Chinese, Japanese, and Korean, and can handle long passages (up to 32k tokens).

Who it’s for

It is designed for developers building RAG applications who need high-precision, crosslingual search capabilities, particularly those integrating with frameworks like LangChain or LlamaIndex.

Highlights

• Bilingual/Crosslingual Proficiency: Specialized in bridging the linguistic gap between Chinese and English.
• RAG-Optimized: Tailored for translation, summarization, and question-answering tasks.
• Instruction-Free: The embedding model works across multiple tasks without needing specific query instructions.
• Long-Context Reranking: The reranker supports passages up to 32k tokens and provides meaningful relevance scores to filter low-quality results.
• Easy Integration: Includes built-in tools for seamless use with LangChain and LlamaIndex.