Sentence Transformers Integration in the Hugging Face Hub

Hugging Face has integrated the Sentence Transformers framework into the Hugging Face Hub, enabling users to access, share, and test over 90 pretrained models across more than 100 languages. This integration simplifies the creation of semantically meaningful embeddings for applications such as semantic search and multi-lingual zero-shot classification.

Interactive Model Testing with New Hub Widgets

To facilitate model exploration, Hugging Face introduced two new interactive widgets specifically for Sentence Transformers models:

  • Feature Extraction Widget: This widget allows users to visualize the sentence embeddings (the numerical vectors) generated by a model.
  • Sentence Similarity Widget: This widget enables users to compute and compare the semantic similarity between different sentences directly within the Hub interface.

Programmatic Access via Inference API

Sentence Transformers models hosted on the Hub are accessible via the Inference API, allowing developers to programmatically call models without managing infrastructure. Users can send a request containing a source sentence and a list of candidate sentences to receive similarity scores.

Streamlined Model Sharing and Discovery

The integration allows researchers and developers to share trained Sentence Transformers models with the community in minutes using the model.save_to_hub("my_new_model") method.

When a model is uploaded, the Hub automatically generates a model card that lists the architecture layers and provides instructions for using the model with both the Sentence Transformers and transformers libraries. To ensure discoverability, users can filter the Hugging Face Hub for all models tagged with sentence-transformers.

Future Roadmap and Integration

Hugging Face plans to further enhance this integration by including training and evaluation data directly in the automatically created model cards, similar to the functionality found in transformers version v4.8.

Existing Sentence Transformers repositories in the Hub can enable the new widgets and Inference API by updating their model card metadata with the following tags:

---
tags:
  - sentence-transformers
  - sentence-similarity # Or feature-extraction!
---

This integration was made possible through the huggingface_hub library, which provides the API and widget infrastructure for supported libraries.

Sources