rohitg00/ai-engineering-from-scratch

What is this project?

ai-engineering-from-scratch is a free, open-source (MIT-licensed) learning curriculum for AI engineering. It's built around a specific philosophy: you implement the internals of models, retrieval pipelines, and agent runtimes yourself — not just read about them. Then you test your code, inspect failures, and keep the artifacts you produced as evidence of what you've learned.

The README advertises 542 lessons across 20 phases, in Python, TypeScript, Rust, and Julia. You can learn three ways:

On the website (aiengineeringfromscratch.com) — open any lesson, no setup needed.
With a coding-agent tutor — if you have Node.js and npx, you can install an "AI tutor" skill that walks you through lessons interactively.
By running local code — clone the repo and run the lesson scripts yourself.

The learning method

The core loop is the same for every lesson (from the README):

Read the lesson's docs/en.md and explain the core idea in your own words.
Type and build the important code yourself — don't just copy.
Run the lesson command from the repository root.
Keep evidence: the command, working directory, exit code, meaningful output, and the artifact you changed or produced.
Continue only when you can explain the output and make one small change without guessing.

This "keep evidence" principle is what makes it different from watching tutorials — you end up with a folder of your own runnable code and results.

The three hands-on projects

The README highlights three full projects, each with staged starters, reference implementations, and local graders. You start with a starter that intentionally fails until you implement each stage.

01 · Retrieval Evaluation Lab (Python)

A candidate system improves mean NDCG (a ranking metric) but ranks one query's most relevant result lower. Your job: build a query-by-query comparison that catches the regression and can fail a release check. You implement ranking validation, precision/recall, rank-sensitive metrics, and system comparison.

What you keep: a reproducible comparison with per-query deltas and the judgments used to score them. The README notes a discipline: "Metrics describe those judgments; they do not establish answer correctness."

02 · Agent Trace Debugger (TypeScript)

A supplied execution trace still takes 100 ms, but total token usage rises by 200 and one span starts failing. Your job: separate overlapping child work from parent execution time, and produce a report that exposes the change. You implement JSONL parsing, parent validation, interval arithmetic, and an inspectable timeline.

What you keep: the input trace, an HTML timeline, and a JSON regression report. The README stresses preserving exclusive per-span token counts so parent and child usage aren't counted twice.

03 · Tool Call Firewall (Rust)

A write changes after review, or an approval is replayed. Your job: validate the call envelope, check the caller's role and path, and consume an approval bound to the exact request and content. The caller application supplies identity; the model proposes an operation.

What you keep: an audit receipt showing the requested operation and the policy decision. The README is explicit about limits: "Approvals are single-use within one invocation; this project does not provide persistent authorization or an operating-system sandbox."

All three projects are graded by a script (python3 scripts/project_test.py ...) and expect certain language versions (Python 3.10+ for the first and third; Node.js 22.18+ for the second).

Learning paths and specializations

You don't need to scan 542 lessons before starting. The README suggests picking one goal:

Goal	Starting point
Model foundations (new to everything)	Setup and tooling, Phase 0
LLM systems (know Python, want math + ML foundations)	Math Foundations, Phase 1
Agents and delivery	The agent loop, Phase 14

There's also a 10-question placement quiz that maps your knowledge to a starting phase and saves a personalized study plan.

Beyond the general path, there are focused study routes you can invoke through the AI tutor:

Model Context Protocol (MCP) — a 17-lesson route covering stateless requests, transports, bidirectional work, security, reliability, registry governance, and conformance evidence.
Agent Skills — a 5-lesson route: contract, discovery, invocation, sandbox boundaries, then release evals and portability.
Certification tracks — specific routes for Anthropic certifications (CCAO-F, CCDV-F, CCAR-F, CCAR-P) and MCPA (a 34-lesson route on the 2026-07-28 protocol).

Each of these has a dedicated skill (e.g., learn-mcp, learn-agent-skills, claude-certification, mcpa-certification) that creates its own learning-progress file and records evidence as you go.

Local setup

The README shows a quick start:

git clone https://github.com/rohitg00/ai-engineering-from-scratch.git
cd ai-engineering-from-scratch
python3 phases/00-setup-and-tooling/01-dev-environment/code/verify.py --route beginner
python3 phases/01-math-foundations/01-linear-algebra-intuition/code/vectors.py

The verify.py preflight separates requirements needed now from tools needed later, and every failure includes the detected reason and a corrective command. The vectors.py lesson is dependency-free and ends by showing that a matrix times a vector is the operation inside a neural network layer — the README suggests saving that terminal output as your first evidence.

Practical details

License: MIT, free, open source.
Languages: 30+ translated versions of the README are listed (Chinese, Japanese, Korean, Spanish, Hindi, etc.).
Sponsor: SerpApi (a web search API company) supports the project.
Website: aiengineeringfromscratch.com — links throughout the README point to specific lessons there.
Interactive lab shown: a gradient-descent visualization where twenty starting points descend a quadratic loss surface, with adjustable learning rate (you can compare GD, momentum, and Adam in the code).

What's intentionally NOT promised

A few explicit boundaries in the README are worth noting because they show maturity:

The Tool Call Firewall project stops at "single-use approval within one invocation" — no persistent auth, no OS sandbox.
The Retrieval Evaluation Lab's metrics describe judgments about relevance — they don't claim to establish "answer correctness."
The AI tutor requires a compatible host (Codex, Claude Code, or similar) and Node.js; if you don't have those, the website path still works but can't run the interactive labs.