replicate/cog

Cog is an open-source tool that packages machine learning models into ready-to-run containers. It removes the pain of writing Dockerfiles and dealing with CUDA versions, letting you define your model's environment and inputs/outputs in simple Python and YAML files.

Key features

• No Dockerfile needed – You describe the environment in a cog.yaml file (e.g., GPU, Python version, system packages). Cog builds a Docker image with best practices automatically.
• CUDA handled for you – Cog knows which CUDA/cuDNN/PyTorch/TensorFlow/Python combos work together and sets them up correctly.
• Simple model definition – You write a Runner class in Python with setup() (load model) and run() (process input, return output). Inputs and outputs are typed, and Cog generates an OpenAPI schema and validates them.
• Automatic HTTP API – Cog creates a RESTful API (using a Rust/Axum server) from your model's types, so you can call it via curl or any HTTP client.
• Production-ready – The built Docker image can be deployed anywhere Docker runs, including your own infrastructure or Replicate.

How it works

1. Define the environment in cog.yaml (e.g., gpu: true, python_version: "3.13", python_requirements: requirements.txt).
2. Define the model in run.py with a Runner class that loads the model in setup() and runs inference in run().
3. Use commands:
   • cog run -i image=@input.jpg – build and run locally, passing inputs.
   • cog build -t my-model – build a Docker image for deployment.
   • cog serve -p 8080 – start a local HTTP server for testing.

Installation

• macOS: via Homebrew (brew install replicate/tap/cog) or install script.
• Linux: via install script or manual download.
• Windows 11: use WSL 2, then follow Linux instructions.
• Docker: can install Cog inside a Docker image.

Prerequisites – Docker (with Buildx if using Docker Engine) on macOS, Linux, or Windows 11 with WSL 2.

Why it exists – Researchers often struggle to deploy ML models to production. Cog was created by Andreas Jansson (ex-Spotify) and Ben Firshman (ex-Docker, creator of Docker Compose) to make this process simple and open-source.

Next steps – The README links to guides for getting started, using notebooks, deploying, and detailed references for cog.yaml and the Runner interface.