EthicalML/awesome-production-machine-learning

A curated list of awesome open source libraries to deploy, monitor, version and scale your machine learning

What it is: A curated, community-maintained list of open-source libraries for running machine learning (ML) in production — not for building models, but for everything that happens after a model is trained: deploying it, monitoring it, tracking versions, scaling it, and keeping it secure.

What it contains: The list is organized into roughly 25 categories, each with links to specific tools. Categories include AutoML, data pipelines, deployment and serving, evaluation and monitoring, explainability and fairness, feature stores, model training and orchestration, privacy and safety, plus several "industry-strength" sections for specialized areas like computer vision, natural language processing, recommender systems, and reinforcement learning.

How it stays current: The project publishes monthly release summaries of newly added libraries, so you can watch the repository to get updates. It also provides a separate search toolkit (hosted on Hugging Face) to quickly navigate the toolchain.

How to contribute: There's a CONTRIBUTING.md file with requirements for submitting pull requests, and the README emphasizes community support for the list's steady growth.

Extra resources: The README links to a 10-minute video overview of ML operations (plus a newer 2024 update), a newsletter for machine learning engineers with over 70,000 subscribers, and a companion list called "Awesome Production GenAI" for generative AI applications.

Bottom line: If you're a practitioner who needs to find the right open-source tool for a production ML task — whether that's serving a model, tracking experiments, or monitoring performance — this is a well-organized, actively updated directory to browse.

Related

  • Project
    confident-ai/deepeval

    DeepEval is an open‑source Python framework that lets you write pytest‑style tests for any LLM‑based app—chatbots, agents, RAG pipelines, or multimodal generators. It ships with dozens of research‑backed metrics (e.g., G‑Eval, JevEval, task‑completion, hallucination, bias) that can run on any model locally, and you can add custom metrics. The library integrates with popular toolkits (LangChain, OpenAI, Anthropic, LlamaIndex, etc.) to automatically trace model calls, enabling both end‑to‑end and step‑level evaluation. Tests run from the CLI, fit into CI/CD, and can be linked to the commercial Confident AI platform for production‑time monitoring. Ideal for developers and teams who need quantitative quality gates for LLM applications.

  • Project
    DataTalksClub/mlops-zoomcamp

    A free, open-source, 9-week course that teaches MLOps skills for putting machine learning models into production. For data scientists and engineers with some programming experience. Self-paced now; live cohorts pause but certificates were offered.

  • Project
    openobserve/openobserve

    OpenObserve is an open-source, cloud-native observability platform that combines logs, metrics, traces, RUM, and AI monitoring in a single tool, claiming up to 140x lower storage costs than Elasticsearch via Parquet columnar storage and S3-native design. It offers a cost-effective, self-hostable alternative to Datadog, Splunk, and Elasticsearch with a single-binary deployment and OpenTelemetry native support.

  • Project
    comet-ml/opik

    An open-source LLM observability and evaluation platform for AI agent tracing, prompt management, and production monitoring.