aws/deep-learning-containers

AWS Deep Learning Containers (DLCs)

What it is – A collection of ready‑made Docker images that bundle popular AI/ML frameworks (PyTorch, TensorFlow, vLLM, Ray, SGLang, …) together with the exact CUDA, driver and library versions needed to run them on AWS compute services such as EC2, Amazon SageMaker, and Amazon EKS. The images are built, tested and continuously patched for security.

Why it matters – Setting up a deep‑learning environment on the cloud normally involves choosing the right framework version, matching CUDA/cuDNN, installing system libraries and keeping everything up‑to‑date. DLCs remove that friction: you pull a single container tag (e.g. 2.14-cu133-amzn2023-sagemaker) and you have a fully‑compatible, production‑ready runtime that works out‑of‑the‑box on AWS hardware.

Key features (as described in the README)

• Broad framework coverage – PyTorch, TensorFlow, vLLM (LLM serving), Ray Serve, SGLang, and more, each with GPU‑enabled and CPU‑only variants.
• Up‑to‑date releases – Automated CI pipelines continuously publish new versions (e.g., PyTorch 2.14, vLLM 0.31.0) and include the latest CUDA (13.3), NVIDIA drivers, DeepSpeed, FlashInfer, etc.
• Security‑first – Every image is scanned and patched for known vulnerabilities; a support policy defines how long each version receives updates.
• Multi‑service support – Tags are provided for plain EC2, SageMaker training/inference, and EKS‑based serving, so the same container can be used across the whole AWS AI stack.
• Specialised LLM serving images – vLLM and vLLM‑Omni images include weight‑cache daemons, FlashMLA, and EFA‑optimized networking for low‑latency inference at scale.
• Documentation & tutorials – Links to a docs site, a catalog of available images, step‑by‑step tutorials, blog posts, and a hands‑on workshop for deploying LLMs on EKS.

Typical workflow

1. Choose a tag that matches the desired framework and hardware (e.g., vllm-0.31.0-gpu-py312-ec2).
2. Pull the image from the public Amazon ECR gallery.
3. Run it on an AWS instance or as a SageMaker job – no manual CUDA install needed.
4. For LLM serving, start the container with the provided vllm preload or Ray Serve entry‑points and expose the OpenAI‑compatible API.

Who should use it – Data scientists, ML engineers, and DevOps teams that want a reliable, AWS‑optimised runtime for training, fine‑tuning, or serving models, especially large language models.

License – Apache‑2.0 (permissive, allows commercial use).