duoan/TorchCode

🔥 LeetCode for PyTorch — practice implementing softmax, attention, GPT-2 and more from scratch with instant auto-grading. Jupyter-based, self-hosted or try online.

What it solves

TorchCode provides a structured environment for ML engineers to practice implementing core PyTorch operators and architectures from scratch. It targets the specific technical skills required for ML coding interviews at top AI companies, where candidates are often asked to implement components like softmax or Transformer blocks without using high-level libraries.

How it works

The platform offers 41 curated problems ranging from basic activation functions to complex attention mechanisms. It uses a Jupyter-based system (or a standalone Next.js/FastAPI web UI) that integrates an automated judge to provide instant feedback on correctness, gradient verification, and timing. Users can run the environment via Docker, locally, or instantly through Hugging Face Spaces and Google Colab.

Who it’s for

ML engineers and candidates preparing for technical interviews at companies like Meta, Google DeepMind, and OpenAI who need to move beyond reading papers to writing production-ready tensor code.

Highlights

  • Curated Problem Set: 41 problems covering fundamentals (ReLU, LayerNorm), attention mechanisms (Multi-Head, GQA, Flash Attention), and architectures (LoRA, ViT).
  • Automated Judge: Provides immediate pass/fail feedback, gradient checks, and timing metrics.
  • Flexible Deployment: Supports one-click Colab access, Docker containers, and a full-stack web interface.
  • Learning Aids: Includes hints for stuck users and reference solutions for studying optimal implementations.

Related

  • Project
    FareedKhan-dev/train-llm-from-scratch

    A complete from-scratch PyTorch implementation of an LLM pipeline, covering everything from raw text tokenization and Transformer architecture to SFT and advanced RLHF alignment techniques like PPO and GRPO.

  • Project
    pytorch/ao

    A PyTorch-native library for model optimization that provides quantization and sparsity tools to accelerate training and inference while reducing memory usage.

  • Project
    kengz/SLM-Lab

    A modular PyTorch framework for Deep Reinforcement Learning that allows users to configure and run RL experiments via JSON specs across various Gymnasium environments.

  • Project
    meta-pytorch/torchcodec

    A PyTorch-native library for decoding and encoding videos, audio, and images directly into tensors for ML training and inference.

  • Project
    facebookresearch/multimodal

    A PyTorch library for training state-of-the-art multimodal models at scale, providing modular building blocks and pretrained weights for models like CLIP and BLIP-2.