Ettin Reranker Family v1 release notes

Ettin Reranker Family v1 release notes

TL;DR

Hugging Face released six new Sentence Transformers CrossEncoder rerankers (ettin-reranker-17m-v1 through ettin-reranker-1b-v1) built on Ettin ModernBERT encoders and trained via pointwise MSE distillation from mixedbread-ai/mxbai-rerank-large-v2. The models are state-of-the-art at their respective sizes, offering improved accuracy and speed for retrieve‑then‑rerank pipelines.

Model Family and Release

The Ettin Reranker family consists of six CrossEncoder models ranging from 17 million to 1 billion parameters, all released under the Apache 2.0 license. Each model is based on a corresponding Ettin encoder from Johns Hopkins University’s Ettin suite and was distilled from the teacher mxbai-rerank-large-v2 using a single‑epoch pointwise MSE loss on a dataset of ~143M (query, document, score) pairs.

Architecture Details

All six rerankers share the same modular architecture, differing only in the Ettin encoder backbone size. The architecture consists of a Transformer (loaded as AutoModel to enable unpadded inputs), CLS pooling, a dense‑GELU block, LayerNorm, and a final dense layer that outputs a single relevance score. Hidden sizes, layers, and total parameters (including the head) are:

  • 17M: hidden 256, layers 7, 17.6M params
  • 32M: hidden 384, layers 10, 32.8M params
  • 68M: hidden 512, layers 19, 68.6M params
  • 150M: hidden 768, layers 22, 150.9M params
  • 400M: hidden 1024, layers 28, 401.6M params
  • 1B: hidden 1792, layers 28, 1.00B params All models support up to 8192 tokens of context thanks to the ModernBERT‑style pre‑training of the Ettin encoders.

Usage

The released models are standard Sentence Transformers CrossEncoders and can be loaded with three lines of code. For maximum throughput, enable bfloat16 and Flash Attention 2 via model_kwargs.

from sentence_transformers import CrossEncoder

model = CrossEncoder(
    "cross-encoder/ettin-reranker-32m-v1",
    model_kwargs={"dtype": "bfloat16", "attn_implementation": "flash_attention_2"},
)

The rank method returns sorted indices and scores for a query and a list of candidate documents. An end‑to‑end retrieve‑then‑rerank pipeline uses a fast embedder (e.g., sentence-transformers/static-retrieval-mrl-en-v1) to obtain top‑K candidates, then the reranker to reorder them.

Retrieval Results (MTEB)

On the MTEB(eng, v2) Retrieval benchmark (NDCG@10 averaged over six embedder pairings), the Ettin rerankers achieve the following scores:

  • ettin-reranker-1b-v1: 0.6114
  • ettin-reranker-400m-v1: 0.6091
  • ettin-reranker-150m-v1: 0.5994
  • ettin-reranker-68m-v1: 0.5915
  • ettin-reranker-32m-v1: 0.5779
  • ettin-reranker-17m-v1: 0.5576 For comparison, the teacher mxbai-rerank-large-v2 scores 0.6115, and the 1B Ettin model matches it within 0.0001. The smallest 17M model outperforms the 33M ms-marco-MiniLM-L12-v2 (0.5066) by +0.051 NDCG@10.

Speed Benchmarks

Throughput was measured on a single NVIDIA H100 80GB using bfloat16 and the best supported attention implementation. Results (pairs per second) are:

  • ettin-reranker-17m-v1: 7517
  • ettin-reranker-32m-v1: 6602
  • ettin-reranker-68m-v1: 4913
  • ettin-reranker-150m-v1: 3237
  • ettin-reranker-400m-v1: 1738
  • ettin-reranker-1b-v1: 928 The teacher mxbai-rerank-large-v2 achieves 387 pairs/second, so the 1B Ettin model delivers 2.4× the throughput while preserving quality. On CPU (Intel Core i7-13700K) the 17M model reaches 267.4 pairs/second, considerably faster than ms-marco-MiniLM-L6-v2 (143.9 pairs/second).

Training Recipe

The models were trained with a single‑epoch pointwise MSE distillation recipe. The teacher is mixedbread-ai/mxbai-rerank-large-v2; the loss is MSELoss on raw teacher logits (range approx. [−12, 22]). The training data is the released dataset cross-encoder/ettin-reranker-v1-data, containing ~143M (query, document, score) triples from lightonai/embeddings-pre-training and a rescored subset of lightonai/embeddings-fine-tuning. Hyperparameters that vary by size are learning rate and global batch size; all other settings are constant (num_train_epochs=1, warmup_ratio=0.03, bf16=True, evaluation every 5% of steps using NanoBEIR mean NDCG@10).

  • 17M: lr 2.4e-4, global batch 1024
  • 32M: lr 1.2e-4, global batch 512
  • 68M: lr 3e-5, global batch 256
  • 150M: lr 1.5e-5, global batch 192
  • 400M: lr 7e-6, global batch 256
  • 1B: lr 3e-6, global batch 512 The training script uses a modular Transformer to enable unpadded inputs for Flash Attention 2, which contributes to the observed speedups.

Conclusion

The Ettin Reranker family provides a strong, efficient drop‑in replacement for legacy MiniLM rerankers, offering better accuracy and speed across the entire size spectrum. The 17M and 32M models are particularly attractive for latency‑critical applications, while the 1B model matches the quality of a 1.5B teacher at a quarter of the parameters and 2.4× the throughput.

Sources