The Little Book of Generative AI Foundations v2: A 195-Page Mathematical Primer

The Little Book of Generative AI Foundations is a 195-page arXiv preprint by Tianhua Chen that derives major generative-model families from shared mathematical foundations rather than surveying every recent architecture. Its revised v2, last updated on 8 September 2026, adds visual explanations throughout.

What the book is

The Little Book of Generative AI Foundations: An Intuitive Mathematical Primer is a compact, derivation-oriented introduction to the mathematics behind modern generative AI. Tianhua Chen wrote it for mathematically curious researchers, practitioners, and students who want to understand how generative models are derived and how they relate to one another.

The book is listed under Machine Learning (cs.LG) and Artificial Intelligence (cs.AI) on arXiv as arXiv:2605.29713. The preprint is 195 pages in its revised form. The author describes it as a foundation-building primer rather than a survey of every recent architecture or implementation detail.

The book develops a connected route through several major generative modelling families:

PCA and probabilistic PCA
Variational autoencoders
Diffusion models
Normalising flows
Autoregressive factorisations
GANs and Wasserstein GANs
Energy-based models

The stated aim is to make the structure of generative modelling more accessible without removing the mathematical substance needed to understand how these models are derived and related.

What changed in v2

Version 2 of the preprint is a 195-page revision that adds visual explanations throughout. The first version was submitted on 28 May 2026. The revised version was submitted on 8 September 2026.

The arXiv submission history lists v1 at 125 KB and v2 at 1,168 KB. The v2 entry is the current version and is cited as arXiv:2605.29713v2. The author's comments state that comments and corrections are welcome.

The paper is available in several formats: PDF, experimental HTML, and TeX source. arXiv also lists an audio summary.

The mathematical route through generative models

The book connects generative model families by deriving them from shared foundations rather than treating them as separate tricks. The abstract names PCA, probabilistic PCA, variational autoencoders, diffusion models, normalising flows, autoregressive factorisations, GANs, Wasserstein GANs, and energy-based models as the families connected by the book's route.

A Hacker News commenter highlighted the book's accessible starting point. @p1esk wrote:

Wow, this book is surprisingly readable! I like how he starts from basics of linear algebra.

That comment points to one of the book's intended strengths: it builds from mathematical basics before moving into generative modelling. The abstract frames the book as derivation-oriented, meaning it aims to show how models are derived and related, not merely to list architectures.

Reception on Hacker News

The Hacker News submission drew 280 points and 48 comments, with readers broadly positive about readability but flagging notation and wording issues. The discussion did not converge on a single verdict. Several commenters praised the book's approachable structure, while others pointed to specific passages that they said needed another editorial pass.

One of the sharpest criticisms concerned the first formula. @StilesCrisis wrote:

The very first formula in the text, explicitly described as "as simple as possible," names its variable ê1 with no explanation given. I think this paper needs another pass.

Another commenter objected to the wording used to explain coordinate changes. @nh23423fefe quoted the passage and responded:

I don't like: > Changing coordinates. Given a vector x expressed in the standard basis, we change coordinates by applying x_eig = V^−1 x. > This operation does not change the vector itself; it only changes how the vector is represented, converting it into eigenvector coordinates. I'm sure the author knows about the difference between the vector and a coordinate vector given a basis. I just don't like this wording. It's weird to say applying a function doesn't affect a change.

The thread also included a broader comment on the limits of intuition. @penguin_booze wrote:

Ah, "intuition": one man's cheese is another man's rotten milk.

That remark captures a recurring tension in discussions of mathematical primers: explanations that feel intuitive to one reader may leave another reader with unanswered questions.

Critiques and gaps readers raised

The main critiques concerned unexplained notation, imprecise coordinate-change language, and the absence of information theory as a foundational chapter. The notation and wording objections were specific and local. The information-theory critique was broader.

One commenter argued that a primer on generative AI foundations should put information theory first. @programjames wrote:

I got annoyed at yet another primer that does not put information theory as foundational, so I decided to write my own.

The same commenter then posted a long alternative outline covering losses, architecture, and learning theory. That outline is a commenter's own essay, not part of Chen's book. It is useful as evidence that some readers want a different foundational ordering, not as a summary of the book's contents.

Other readers asked for adjacent resources that the provided thread did not answer with specific titles. @amplifydev asked for books specifically on attention architecture. @nblgbg asked whether any deep learning books include problems to solve. No direct book recommendations for those two requests appear in the provided comments.

Related books recommended in the thread

Commenters recommended several adjacent books for readers who want a gentler introduction or a different mathematical path. These recommendations are not part of Chen's preprint, but they show how readers positioned the book against other learning resources.

@prideout recommended François Fleuret's The Little Book of Deep Learning, linking to fleuret.org/francois/lbdl.html.

@hasteg recommended Anil Ananthaswamy's Why Machines Learn: The Elegant Math Behind Modern AI for readers who want a gentler build-up of intuition. @hasteg wrote:

Another book which I would highly recommend for learning the math (or at least getting a very good idea of how it works) behind machine learning/transformers is "Why Machines Learn: The Elegant Math Behind Modern AI" by Anil Ananthaswamy. I read it this past year and I found it was extremely good at pacing and building up intuition to the "how and why" of most important math concepts and discoveries in ML that have lead to where we are today. For those who don't already have a fairly deep understanding of linear algebra, statistics, and multivariate calculus, this book will be a far more gentle introduction yet still has enough rigor for you to come away with something.

Another commenter described keeping a large personal collection of papers and books for retrieval. @minimaxa wrote:

Appreciated, I drop all of these into a RAG folder for when I need them later on in projects. 8Gb of whitepapers/books...

The thread also drifted into general book recommendations unrelated to generative AI, including Matt Dinniman's Dungeon Crawler Carl series. Those recommendations are not relevant to the technical content of Chen's preprint.

Access and citation

The book is available as arXiv:2605.29713v2, with a PDF, experimental HTML version, TeX source, and audio summary. The DOI is 10.48550/arXiv.2605.29713. The arXiv record lists the license as a nonexclusive distribution license.

For citation, the arXiv record gives the following core metadata:

Title: The Little Book of Generative AI Foundations: An Intuitive Mathematical Primer
Author: Tianhua Chen
arXiv identifier: arXiv:2605.29713
Current version: v2, revised 8 September 2026
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
Length: 195 pages

The book is a preprint, not a formally published textbook. Readers should treat the v2 PDF as the current reference and check the arXiv record for later revisions.