mozilla/translations

The code, training pipeline, and models that power Firefox Translations

What it solves

This project provides the infrastructure to train and deploy machine translation models that run locally on the user's device within the Firefox browser, rather than relying on a cloud-based translation service.

How it works

It uses the Marian NMT engine for fast translation and provides an end-to-end training pipeline. The process is managed by orchestrators like Taskcluster (for hybrid cloud scalability) or Snakemake (for local or Slurm cluster execution). Training metrics are tracked via a custom module that parses logs and publishes them to Weights & Biases.

Who it’s for

Developers and researchers interested in building efficient, client-side machine translation models for web browsers.

Highlights

  • Client-side focus: Specifically designed for the Bergamot project to enable translation directly in the browser.
  • End-to-end pipeline: Handles everything from dataset selection and cleaning to final model training.
  • Scalable orchestration: Supports hybrid cloud workers via Taskcluster.
  • Open models: Trained models are hosted in a public Google Cloud Storage bucket under the MPL 2.0 license.

Related

  • Project
    tensorflow/tflite-support

    A toolkit for developing and deploying TFLite models to mobile devices, providing tools for metadata management, automatic API generation, and optimized task libraries.

  • Project
    alibaba/pipcook

    A JavaScript application framework that allows Web engineers to train, serve, and deploy machine learning models by bridging the Node.js runtime with the Python ML ecosystem.

  • Project
    ZHZisZZ/dllm

    A library that unifies the training, inference, and evaluation of diffusion language models, providing scalable pipelines and recipes for converting AR models into diffusion models.

  • Project
    google-ai-edge/litert-samples

    A collection of official and community samples, model recipes, and deployment tools for LiteRT and LiteRT-LM to enable high-performance on-device ML execution.

  • Project
    InternLM/lmdeploy

    A toolkit for compressing, deploying, and serving LLMs and VLMs, offering high-performance inference engines and quantization to maximize throughput.