Hugging Face Ethics and Society Newsletter #2: Addressing Bias in Machine Learning

Hugging Face has released a comprehensive guide on addressing bias in machine learning, emphasizing that bias is a complex, sociotechnical issue that cannot be solved by a single technical intervention. The core takeaway is that machine biases should be viewed as risk factors for discrimination-based harms, and mitigation requires a shared responsibility across every stage of the ML development cycle.

Machine Bias as a Risk Factor for Social Harm

Machine bias—the mechanism by which systems encode negative stereotypes or exhibit disparate performance across population groups—can amplify existing social inequities. Hugging Face identifies four primary ways automation and scale exacerbate these issues:

  1. Locking in behaviors: Technology can freeze social biases in time, hindering social progress.
  2. Spreading harmful behaviors: Biases can be exported beyond the original context of the training data.
  3. Amplifying inequities: Systems may overfocus on stereotypical associations when making predictions.
  4. Removing recourse: Biases can be hidden within "black-box" systems, making it difficult for affected individuals to seek justice.

The Role of Context

Bias is not a static property of a model but is dependent on the deployment context. A machine bias at the model level may be irrelevant in one application but catastrophic in another. For example, a text-to-image model that associates professional settings with light skin tones is a risk factor that:

  • Is irrelevant if the model is used for backgrounds where people are explicitly disabled.
  • Amplifies stereotypes if used for stock images on Wikipedia.
  • Causes direct discrimination if used by police departments to generate suspect sketches.

Mitigating Bias Across the ML Development Cycle

Because bias is ubiquitous and complex, Hugging Face recommends a multipronged approach targeting three key stages of development.

1. Task Definition

The impact of bias often begins with how a task is framed and what objective is optimized. For instance, optimizing a recommendation system solely for engagement can reduce media diversity and lock in past stereotypes.

Recommendations for Task Definition:

  • Investigate reports of bias in the field prior to the introduction of ML.
  • Identify at-risk demographic categories specific to the use case.
  • Examine if the optimization objective reinforces biases and explore alternatives that favor diversity and long-term positive impact.

2. Dataset Curation

Datasets often encode human stereotypes and lack representation, leading to disparate performance. Hugging Face suggests that developers reflect on the origin of the data, the people represented, and the curation process.

Key Tools for Dataset Analysis:

  • Dataset Cards: A documentation framework to share insights on social impact and bias.
  • Disaggregators Library: A tool to quantify dataset composition to minimize representation harms.
  • Data Measurements Tool: Uses normalized Pointwise Mutual Information (nPMI) to surface stereotypical associations between terms (e.g., gendered pronouns).

3. Model Training and Selection

Documenting and measuring bias in models allows developers to choose the safest model for a specific application or guide the training of new ones.

Evaluation and Visualization Strategies:

  • Model Cards: A reporting framework for ethical considerations and disaggregated evaluation.
  • Output Visualization: For generative models, comparing outputs across adjectives and professions helps identify reproduced stereotypes. For classification models, tools like the SEAL app group similar mistakes to identify failure modes that lead to disparate performance.
  • Quantitative Benchmarks: Utilizing benchmarks like BOLD, HONEST, and WinoBias to evaluate targeted behaviors in language models.
  • Evaluation on the Hub: A tool to run evaluations on large models that would otherwise be computationally prohibitively expensive for individual developers.

Summary of Hugging Face Bias Mitigation Tools

Stage Recommended Tools
Task Definition ML Tasks directory, Task Exploration tool
Dataset Curation Dataset Cards, Disaggregators library, Data Measurements Tool
Model Development Model Cards, SEAL app, DiffusionBiasExplorer, Evaluation on the Hub, Bias Score Cards

Sources