mlcommons/ck

What it solves

Collective Knowledge (CK) provides a way to make AI and ML research more reproducible and efficient. It addresses the difficulty of running complex ML workloads across different hardware, software, and datasets, which often makes it hard to experimental setups to reuse or reproduce.

How it works

The project consists of several components that turn software projects into file-based repositories of portable artifacts (such as code, data, and models) with associated metadata. These artifacts are chained together into technology-agnostic automation workflows.

Key components include:

• Collective Mind (CM/CMX): An automation framework that provides a unified command-line interface and Python API to manage these artifacts and workflows.
• MLOps and MLPerf automations: A collection of portable automation recipes used to compose, run, and optimize ML applications, specifically helping to modularize and automate MLPerf benchmarks.
• Collective Knowledge Playground: A platform to index automations, compare MLPerf benchmarking results, and organize optimization challenges.

Who it’s for

Researchers, developers, and the community members who need to run AI/ML workloads efficiently across diverse platforms and support open science and reproducible research.

Highlights

• Technology-Agnostic: Works across diverse models, datasets, software, and hardware.
• C-Make style automation: Extends the concept of cmake with Python automations and JSON/YAML metadata.
• FAIR Principles: Designed to support FAIR (Findable, Accessible, Interoperable, and Reusable) data principles.
• Broad OS Support: Runs natively on Ubuntu, MacOS, Windows, RHEL, Debian, and Amazon Linux.