Hugging Face Trackio Release

Hugging Face has introduced Trackio, an open-source Python library designed for lightweight experiment tracking. It provides a local dashboard for visualizing metrics and parameters, with the ability to sync these results to Hugging Face Spaces for free, public or private collaboration.

Key Technical Advantages

Trackio is designed to solve common pain points associated with proprietary or complex experiment tracking tools. Its primary advantages include:

  • Seamless Sharing and Embedding: Users can embed training plots directly into documentation or blog posts via iframes, allowing stakeholders to view metrics without needing to create accounts.
  • Data Accessibility and Transparency: Unlike tools with proprietary APIs, Trackio makes it straightforward to extract and analyze recorded data. It specifically facilitates the tracking of GPU energy usage by pulling data directly from the nvidia-smi command, which can then be added to model cards to quantify environmental impact.
  • Performance Optimization: The library's lightweight design allows researchers to control when tensors are moved from GPU to CPU during logging, reducing the impact on training throughput.
  • Local-First Architecture: Logs and dashboards persist locally by default, ensuring data remains under the user's control before optional syncing to the cloud.

API Compatibility and Usage

Trackio is built as a drop-in replacement for wandb. The API is compatible with wandb.init, wandb.log, and wandb.finish, allowing users to migrate by simply changing their import statement to import trackio as wandb.

Installation

Trackio can be installed via pip or uv:

pip install trackio

Visualization and Sharing

Users can launch a local dashboard using the command trackio show or the Python function trackio.show(). To sync a dashboard to Hugging Face Spaces, users provide a space_id during initialization:

trackio.init(project="fake-training", space_id="org_name/space_name")

To prevent data loss from the ephemeral nature of Spaces' Sqlite databases, Trackio automatically converts the database to a Parquet dataset and backs it up to a Hugging Face Dataset every five minutes. Users can optionally specify a custom dataset_id during trackio.init().

Ecosystem Integration

Trackio integrates natively with the Hugging Face ecosystem, requiring minimal setup for existing users of these libraries:

  • Transformers Trainer: By setting report_to="trackio" in TrainingArguments, the Trainer API automatically logs metrics to Trackio.
  • Accelerate: By initializing the Accelerator with log_with="trackio", users can log training steps and metrics using accelerator.log().

Design Principles and Current Limitations

Trackio is built on the following core principles:

  • API Compatibility: Ensuring seamless migration between tracking libraries.
  • Lightweight Core: The codebase is under 1,000 lines of Python, making it easy for the community to modify.
  • Open Source and Free: All features, including hosting on Hugging Face, are provided at no cost.
  • Infrastructure: Built on top of Hugging Face Datasets and Spaces for visualization and data handling.

As the library is currently in beta, it does not yet support artifact management or complex visualizations. Hugging Face encourages the community to contribute feature requests via GitHub issues.

Sources