Hugging Face and Renumics Spotlight Integration for Scalable Data Inspection

Hugging Face and Renumics Spotlight have integrated to allow machine learning practitioners to interactively explore and visualize datasets with a single line of code. This integration simplifies the process of identifying critical data clusters and failure modes by bridging the gap between the Hugging Face datasets library and interactive visualization tools.

Seamless Integration with Hugging Face Datasets

Spotlight is designed to work directly on top of the Hugging Face datasets library, eliminating the need for data copying or pre-processing. It leverages the library's use of Apache Arrow tables to store tabular metadata and unstructured data (such as images and audio) in a unified format.

Key technical characteristics of the integration include:

  • Lazy Loading: To maintain performance, Spotlight loads tabular data into memory for efficient client-side analytics while loading memory-intensive unstructured samples (video, audio, images) lazily on demand.
  • Automatic Type Inference: In most instances, data types and label mappings are inferred automatically from the dataset features. For ambiguous cases, the Spotlight API allows for manual assignment of data types, such as spotlight.Image or spotlight.dtypes.CategoryDType.
  • Minimal Setup: Users can launch a visualization by loading a dataset and calling spotlight.show(ds).

Leveraging Model Results for Data Inspection

Raw unstructured data often provides limited insight. To uncover critical data segments and model failure modes, Spotlight allows users to integrate model outputs—such as predictions and embeddings—directly into the visualization workflow.

Model Enrichment Workflow

Practitioners can use the transformers library to compute embeddings and predictions, then store these results back into a Hugging Face dataset using the .map() function. This "enriched" dataset can then be visualized using specific Spotlight layouts, such as the debug_classification layout, which provides tools like similarity maps and confusion matrices to analyze model performance.

Data Storage Recommendations

Hugging Face recommends storing prediction results directly within a Hugging Face dataset to maintain label mappings and utilize the library's batch processing capabilities.

Customizing Data Inspection Workflows

Spotlight provides both a GUI and a Python API to customize how data is inspected:

  • GUI Customization: Users can interactively change, save, and load visualization layouts within the interface.
  • Inspector Widget: A specialized widget capable of representing multimodal data, including time series, audio, video, text, and images.
  • Python API: The API allows for the creation of custom curation workflows for exploratory data analysis (EDA), model monitoring, and model debugging. It can also integrate the results of external data quality check scripts via the data issues widget.

Deployment on Hugging Face Hub

Beyond local use, Spotlight visualizations can be hosted on the Hugging Face Hub using Hugging Face Spaces. This allows developers to showcase their datasets or model results to the other users. Users can duplicate existing Spotlight example spaces and configure them by specifying the HF_DATASET environment variable to point to their own dataset, including specific splits, subsets, or revisions.

Sources