The Hugging Face Hub for Galleries, Libraries, Archives and Museums

What is the Hugging Face Hub

The Hugging Face Hub is a central repository where people can share and access machine learning models, datasets, and demos. It hosts over 190,000 machine learning models, 33,000 datasets, and over 100,000 machine learning applications and demos. These resources cover a wide range of tasks and domains, including pre-trained language models, text, image and audio classification, object detection, and generative models.

What can you find on the Hugging Face Hub

The Hub provides access to machine learning models, datasets, and Spaces that support various modalities and tasks.

Models

The Hub provides access to machine learning models covering various tasks and domains. Many machine learning libraries have integrations with the Hub, allowing users to directly use or share models via those libraries.

Datasets

The Hub hosts over 30,000 datasets. These datasets cover a range of domains and modalities, including text, image, audio, and multi-modal datasets, and are valuable for training and evaluating machine learning models.

Spaces

Hugging Face Spaces is a platform that allows users to host machine learning demos and applications. Spaces range from simple demos that explore model predictions to more involved applications. Users can host Gradio and Streamlit applications, or use custom Docker images, and can quickly create hosted versions of popular tools such as Argilla and Label Studio using Docker templates.

How to find relevant models on the Hub

Users can find openly shared models that match their goals by filtering the Hub by task and language.

For example, when working with digitized Norwegian documents with minimal metadata, a Named Entity Recognition (NER) model can help identify locations mentioned in text. Users can filter models by the token-classification task, which includes NER models, and further filter by language Norwegian to narrow the set. Many of these models include a model widget that lets users test performance on their data. After selecting a model, users can click the "use in Transformers" button to get code for loading the model with the Transformers library, or click the "deploy" button to explore options for hosting the model behind an API.

Walkthrough: adding a GLAM dataset to the Hub

Adding a GLAM dataset to the Hub can be done through the browser interface without writing code.

Create a new dataset repository

The first step is to click the "New Dataset" button in the dropdown menu in the top right corner of the Hub, choose a name for the repository, optionally select an owner organization, and specify a license.

Upload files

After creating the repository, users click "Add file" under the "Files" tab, select the data files to upload, and can upload single or multiple files. Once files are selected, users commit the changes to finalize the upload.

Adding metadata

It is important to add metadata to make the dataset discoverable. Users can edit metadata using the Metadata UI editor to specify license, language, tags, and other fields. Additionally, users should outline the dataset’s construction, strengths, and weaknesses in the README.md file, which serves as a dataset card. The Hub provides a template dataset card that offers prompts for useful information.

Datasets preview

After uploading, the Hub provides a preview of the dataset, which helps users better understand its contents.

Other ways of sharing datasets

Users can consult the datasets documentation for alternative approaches that may suit specific use cases.

Why GLAM might want to use the Hub

Galleries, libraries, archives, and museums have several motivations for contributing to the Hub.

Exposure to a new audience: the Hub is a central destination for people working in machine learning and AI, so sharing collections exposes them to this audience and opens collaboration opportunities.
Community: the Hub’s community features let users ask questions and engage with shared materials, allowing people to build on each other’s work and lower barriers to using machine learning in the sector.
Diversity of training data: sharing domain-specific datasets helps address the lack of relevant training data for GLAM data, improving model performance on real-world collections.
Climate change: sharing trained models reduces duplication of effort, which can collectively lower the carbon footprint associated with training machine learning models.

Example uses of the Hugging Face Hub

Several GLAM-related projects already use the Hub to share models, datasets, and demos.

BigLAM

BigLAM, an initiative from the BigScience project, has made over 30 GLAM-related datasets available via the Hub to increase accessibility.

Nasjonalbiblioteket AI Lab

The AI lab at the National Library of Norway actively uses the Hub, sharing approximately 120 models, 23 datasets, and six machine learning demos. These include language models trained on Norwegian texts from the National Library and Whisper speech-to-text models trained on Sámi languages.

Smithsonian Institution

The Smithsonian shared an application hosted on Hugging Face Spaces that demonstrates two machine learning models trained to identify Amazon fish species. The project aims to empower communities with tools for more accurate measurement of fish species numbers in the Amazon, and the Spaces demo lowers the barrier for people wanting to use these tools.

Hub features for Galleries, Libraries, Archives and Museums

The Hub includes features that are particularly helpful for GLAM institutions.

Organizations: users can create an organization on the Hub to share their institution’s artefacts in a dedicated space.
Minting DOIs: the Hub supports issuing Digital Object Identifiers (DOIs) for models, datasets, and demos, providing persistent identifiers often required by journals, conferences, and funders.
Usage tracking: users can view monthly download statistics for datasets and models, or total lifetime downloads, to demonstrate impact.
Script-based dataset sharing: if a dataset is already hosted elsewhere, users can provide access via the Hub using a dataset loading script.
Model and dataset gating: the Hub supports gating, allowing owners to add access controls to models and datasets when needed.

How can I get help using the Hub

Users can find detailed information in the Hub documentation, the datasets sharing guide, and the Transformers model sharing guide. For assistance, the Hub offers a discussion forum and a Discord community where users can ask questions and receive support.