Exorust/TorchLeet

TorchLeet — What It Is

TorchLeet is a free, open-source collection of 68 PyTorch coding problems drawn from real machine-learning/AI engineering interviews at companies like Google, Meta, Anthropic, OpenAI, and DeepMind. The author compiled the list from first-person reports of actual interview questions.

How It Works

• Each problem comes as a Jupyter notebook with a question file and a separate _SOLN solution file.
• You open the question notebook, fill in the ... and #TODO blocks, then compare your work against the solution.
• The README explicitly warns not to use GPT or other AI to solve the problems — the point is to struggle through them yourself to deeply understand PyTorch. (The author notes they used GPT to draft some initial code but solved and tested every problem personally.)

Three Tracks

Track	Focus	Questions
Basics	Core PyTorch, classical ML, fundamentals	24
LLM Learning Path	Build an LLM from scratch in order	23
Advanced	Systems, kernels, modern architectures, alignment	48

Questions overlap between tracks. Many problems are tagged with the companies that reportedly ask them.

LLM Learning Path

A guided sequence to build a full LLM from scratch, one problem at a time:

1. Foundations — Byte Pair Encoding, sinusoidal embeddings, ROPE embeddings, RMS norm, attention from scratch
2. Core Transformer — multi-head attention, grouped query attention, KV cache, sliding window attention
3. Full Model — implement SmolLM from scratch
4. Alignment & Fine-Tuning — KL divergence loss, LoRA, SFT on SmolLM, DPO loss, PPO for RLHF, GRPO (DeepSeek-R1)
5. Decoding & Inference — temperature sampling, top-k, top-p, speculative decoding, continuous batching, a complete inference engine
6. Systems — mixture of experts layer

Advanced Topics

• Modern architectures: CLIP contrastive loss, 2D positional embeddings, knowledge distillation, DDPM diffusion, DDIM + classifier-free guidance, Mamba, Vision Transformer + MAE, Swin Transformer
• Alignment: LoRA, DPO, PPO, gradient checkpointing, GRPO
• Inference systems: KV cache, speculative decoding, continuous batching, GPTQ quantization, RAG embedding search, full inference engine
• GPU kernels: fused softmax in Triton, FlashAttention-2 in Triton, FSDP, ring attention
• Hard foundations: custom autograd, transformer from scratch, GAN, seq2seq with attention, explainable AI

AI Tutor (MCP Server)

The project includes an MCP server (torchleet-mcp) that connects AI assistants (Claude Code, Codex, Claude Desktop, Cursor, VS Code) to the problem set. It provides four learning guides:

• torchleet-tutor — progressive hints through problems
• torchleet-interview-prep — timed mock interviews for specific companies
• torchleet-review — a senior ML engineer persona reviews your code
• torchleet-explain — deep dives from intuition to math to code

The AI Tutor enforces a no-spoilers teaching style — it coaches rather than gives answers.

Company Quick-Reference

The README includes a table mapping companies to priority questions (e.g., Anthropic's list includes contrastive loss, 2D positional embeddings, top-p sampling, KV cache, DPO, PPO, speculative decoding, Mamba, FSDP, GRPO, ring attention).

Getting Started

# Install PyTorch first (pytorch.org)
# Clone the repo, then run a notebook:
jupyter notebook torch/basic/lin-regression/lin-regression.ipynb

There's also a companion website at torch-leet.vercel.app with a terminal interface and AI Tutor setup instructions.