Hugging Face Hub Search API Updates

Hugging Face has updated the huggingface_hub library to provide a more intuitive, programmatic interface for searching models, datasets, and Spaces. These updates eliminate the need for users to manually navigate the web interface or guess API query parameters, allowing for complex filtering directly within Python or Jupyter environments.

Simplified Search with ModelSearchArguments

The ModelSearchArguments and DatasetSearchArguments classes act as human-readable "cheat sheets" that map user-friendly terms to the specific formatted strings required by the Hugging Face API. This removes the trial-and-error process of determining how parameters should be written.

Available Search Attributes

The ModelSearchArguments class provides a namespace for the following searchable attributes:

  • author: Filter by the creator of the model.
  • dataset: Filter by the dataset used for training (e.g., model_args.dataset.glue returns 'dataset:glue').
  • language: Filter by the language of the model.
  • library: Filter by the framework, such as PyTorch (e.g., model_args.library.PyTorch returns 'pytorch').
  • license: Filter by the model license.
  • model_name: Filter by the specific model name.
  • pipeline_tag: Filter by the task, such as Text Classification (e.g., model_args.pipeline_tag.TextClassification returns 'text-classification').

Users can pass these arguments into the api.list_models() method using the filter parameter to retrieve matching models.

Advanced Filtering with ModelFilter

For complex queries involving multiple criteria or multiple values per criterion, the ModelFilter class provides a coordinated approach to searching. Instead of simple strings, ModelFilter allows users to define lists of requirements for tasks, datasets, and libraries.

For example, a user can search for models that meet all of the following criteria simultaneously:

  • Tasks: Both text-classification and zero-shot-classification.
  • Datasets: Both Multi NLI and GLUE.
  • Libraries: Both PyTorch and TensorFlow.

Technical Implementation: AttributeDictionary

The improved search experience is powered by a utility class called AttributeDictionary. Inspired by the AttrDict class from the fastcore library, AttributeDictionary enhances standard Python dictionaries by enabling tab-completion for keys, which is critical for exploratory programming.

Key Characteristics of AttributeDictionary

  • Attribute Access: Keys can be accessed as attributes (e.g., ad.key) rather than just as dictionary keys (e.g., ad['key']).
  • Nested Exploration: It supports nested dictionaries, allowing for deep exploration of API responses.
  • Deletion: Keys can be deleted using either del ad.key or del ad[key].
  • Constraint: If a key contains a number or special character, it must be accessed using standard dictionary indexing (ad['key']) because Python syntax does not allow attributes to start with numbers or contain special characters.

Programmatic Model Information

Beyond searching, the huggingface_hub library allows users to retrieve detailed information about a specific model using its model ID via the api.model_info('model_id') method.

Sources