fastText Integration with Hugging Face Hub

Hugging Face has integrated official mirrors of fastText models into the Hugging Face Hub, providing streamlined access to word vectors for 157 languages and a specialized language identification model. This integration allows developers to download and deploy these scalable text representation and classification tools using the huggingface_hub library.

fastText Technical Architecture

fastText, originally open-sourced by Meta AI in 2016, is designed for efficient text representation and classification. It employs several key NLP techniques to achieve scalability and performance:

  • Subword Information: The library utilizes subword information to improve text representation.
  • N-gram Representation: Sentences are represented using bags of words and bags of n-grams.
  • Hidden Representations: It uses a hidden representation to share information across different classes.
  • Hierarchical Softmax: To optimize computation speed, fastText implements hierarchical softmax, which leverages the imbalanced distribution of classes.

Hub Integration and Model Availability

The official mirrors of these models are hosted within the Meta AI organization on the Hugging Face Hub. This integration provides two primary categories of models:

  1. Word Vectors: Pre-trained vectors for 157 different languages.
  2. Language Identification: A dedicated model for detecting the language of a given text.

To facilitate testing, Hugging Face has added support for text classification and feature extraction widgets directly on the model pages, allowing users to interact with the language identification and word vector models in the browser.

Implementation and Usage

Users can load fastText models from the Hub using the hf_hub_download function from the huggingface_hub library.

Language Identification

To detect the language of a text string, the model can be loaded and used as follows:

import fasttext
from huggingface_hub import hf_hub_download

model_path = hf_hub_download(repo_id="facebook/fasttext-language-identification", filename="model.bin")
model = fasttext.load_model(model_path)
model.predict("Hello, world!")

Word Vector Retrieval and Nearest Neighbors

For feature extraction, users can retrieve the vector for a specific word or find the nearest neighbors to a word vector to identify semantically similar terms:

import fasttext
from huggingface_hub import hf_hub_download

# Loading word vectors
model_path = hf_hub_download(repo_id="facebook/fasttext-en-vectors", filename="model.bin")
model = fasttext.load_model(model_path)
vector = model['bread']

# Querying nearest neighbors
model_path_nn = hf_hub_download(repo_id="facebook/fasttext-en-nearest-neighbors", filename="model.bin")
model_nn = fasttext.load_model(model_path_nn)
model_nn.get_nearest_neighbors("bread", k=5)

Sources