Stable-Baselines3 Integration with Hugging Face Hub

Hugging Face has integrated Stable-Baselines3 into the Hugging Face Hub, enabling researchers and enthusiasts to host and load PyTorch Deep Reinforcement Learning (DRL) models. This integration streamlines the process of sharing pre-trained agents trained in environments such as Gym, Atari, MuJoco, and Procgen.

Model Hosting and Distribution

The integration allows users to discover and share saved Stable-Baselines3 models via the Hub. Users can find community-contributed models by filtering for stable-baselines3 on the Hugging Face Hub.

Downloading Models from the Hub

To load a saved model from the Hub into Stable-Baselines3, users must install the huggingface_hub and huggingface_sb3 libraries. The process requires the repository ID (repo-id) and the specific filename of the model zip file within that repository.

Example workflow for loading a model:

  1. Install dependencies: pip install huggingface_hub huggingface_sb3.
  2. Use load_from_hub from the huggingface_sb3 library to retrieve the checkpoint.
  3. Load the checkpoint into a Stable-Baselines3 agent (e.g., using PPO.load(checkpoint)).

Sharing Models to the Hub

Users can upload their trained agents to the Hub by first authenticating via huggingface-cli login or notebook_login() for Jupyter/Colab environments. Once authenticated, the push_to_hub function from the huggingface_sb3 library is used to upload a saved model zip file to a specified repository ID.

Future Roadmap

Hugging Face plans to expand the Deep Reinforcement Learning ecosystem on the Hub through the following initiatives:

  • Library Integrations: Integrating RL-baselines3-zoo and other Deep Reinforcement Learning libraries.
  • Model Collections: Uploading pre-trained agents from the rl-trained-agents collection.
  • Algorithm Implementation: Implementing Decision Transformers.

Technical Implementation

This integration was made possible through the huggingface_hub library, which provides the necessary API and widgets for library support. Hugging Face provides a guide for other library maintainers who wish to integrate their tools with the Hub.

Sources