mozilla/bugbug

Platform for Machine Learning projects on Software Engineering

What is bugbug?

bugbug is an open‑source Python toolkit that applies machine‑learning models to help Mozilla engineers manage bugs and code changes. It trains classifiers that can automatically:

  • suggest who should be assigned to a new bug,
  • decide whether a patch is likely to cause a regression or test failure,
  • label bugs as crashes, performance issues, security problems, spam, etc.,
  • predict whether a bug needs QA, documentation, or uplift, and many other triage tasks.

The project ships a small data‑mining pipeline that extracts features from Bugzilla tickets, GitHub issues, and the Firefox source‑code history, then wraps the models (including Keras‑based neural nets) in a scikit‑learn‑compatible API.

Why it matters

Bug triage is a huge manual effort for large codebases like Firefox. By automatically classifying bugs and patches, bugbug can:

  • Reduce the time developers spend routing bugs to the right component or person.
  • Surface risky patches early, helping reviewers focus on code that may break builds or tests.
  • Provide data‑driven insights (e.g., which bugs are likely regressions) that improve release quality.

In short, it turns the wealth of historical bug and commit data into actionable predictions, making the development workflow faster and more reliable.

Who is it for?

  • Mozilla engineers who work with Bugzilla, the Firefox source tree, or GitHub issues and want ML‑assisted triage.
  • Researchers or developers interested in software‑engineering ML who need a real‑world dataset and baseline models for tasks like defect prediction, assignee recommendation, or test selection.
  • Other open‑source projects (with some adaptation) that have similar bug‑tracking and version‑control data.

How to get started

  1. Install dependencies – the README recommends the modern uv tool:
    uv sync               # core deps
uv sync --group test  # test deps
uv sync --extra nlp   # optional NLP extras
    Python 3.14+ is required.
  2. Run the trainer – pick a model name (e.g., defect) and train it:
    python -m scripts.trainer defect
    Training takes ~30 minutes on a typical machine.
  3. Classify a bug – after training (or automatically downloading a pre‑trained model), classify a Bugzilla bug by ID:
    python -m scripts.bug_classifier defect --bug-id 123456
  4. Optional: use Taskcluster – add Train on Taskcluster: <model> to a PR description to have Mozilla’s CI train the model in the cloud.
  5. Data mining – if you need custom data, follow the “Running the repository mining script” steps to pull the Firefox repo and generate commit‑level features.

What’s inside the repo?

Directory / file Purpose
bugbug/labels Manually curated label sets used for training.
bugbug/db.py Simple JSON‑based storage for cached data.
bugbug/bugzilla.py API helpers to fetch bugs from Mozilla’s Bugzilla.
bugbug/repository.py Scripts that walk the mozilla‑central Mercurial repo to extract commit information.
bugbug/bug_features.py Feature‑engineering utilities that turn bug/commit metadata into vectors.
bugbug/model.py Abstract base class for all classifiers.
bugbug/models/ Concrete implementations (e.g., defect, component, regression).
bugbug/nn.py Glue code to embed Keras neural‑net models into scikit‑learn pipelines.
bugbug/nlp/ Text‑processing helpers for bug descriptions and comments.
bugbug/github.py Helpers for pulling issues from GitHub repositories.

Limitations & open questions

  • Mozilla‑centric – the data pipelines are tuned for Bugzilla and the Firefox repo; using it elsewhere requires writing new adapters.
  • Training cost – full model training can be time‑ and compute‑intensive (30 min+ on a CPU). GPU support isn’t mentioned.
  • Model freshness – the README notes that pre‑trained models are downloaded automatically, but it doesn’t describe how often they are refreshed.
  • Evaluation metrics – only a few classifiers list accuracy/precision/recall; comprehensive benchmarking isn’t provided.

TL;DR

bugbug = a Python library that mines Mozilla’s bug and source‑code data, builds machine‑learning classifiers for triage tasks (assignee suggestion, regression detection, spam filtering, etc.), and offers scripts to train or use those models. It’s a practical example of ML‑assisted software‑engineering tooling, primarily aimed at Firefox developers but adaptable to other projects with effort.

Related

  • Project
    evidentlyai/evidently

    An open-source Python framework to evaluate, test, and monitor ML and LLM-powered systems, helping developers detect data drift and ensure output quality.

  • Project
    dotnet/machinelearning

    ML.NET is an open‑source, cross‑platform machine‑learning framework for .NET that provides data loading, transformation, a catalog of algorithms, and support for TensorFlow/ONNX models, all consumable via the `Microsoft.ML` NuGet package.

  • Project
    Dicklesworthstone/ultimate_bug_scanner

    A multi-language bug scanner designed to act as a quality guardrail for AI coding agents by detecting common security vulnerabilities and resource management errors.

  • Project
    huggingface/evaluate

    A library for standardized evaluation and comparison of machine learning models, providing a wide range of metrics across NLP and Computer Vision.

  • Project
    shankarpandala/lazypredict

    Lazy Predict is a Python library that automatically runs dozens of machine-learning models (classification, regression, time-series) with default settings and returns a ranked comparison table, so you can quickly see which model families are worth pursuing before doing detailed tuning. It supports GPU acceleration, categorical encoding, timeouts, cross-validation, and custom metrics.