harleyszhang/llm_note

LLM notes, including model inference, transformer model structure, and llm framework code analysis notes.

What is this project?

llm_note is a personal knowledge base (in Chinese) about building and optimizing large language models (LLMs). It collects notes on:

Transformer model internals — how models like GPT, LLaMA, and ViT are structured and implemented.
LLM quantization — techniques to shrink model size (e.g., SmoothQuant, AWQ) with paper summaries and code walkthroughs.
Inference optimization — algorithm-level tricks (FlashAttention, online softmax, CUDA graphs) and parallelism strategies (tensor parallelism).
High-performance computing (HPC) — practical notes on writing GPU kernels with Triton and CUDA, plus curated external learning resources.
Framework analysis — overviews of production inference servers like vLLM, TGI, and LightLLM.

The project is organized as folders (1-transformer_model, 3-llm_infer, etc.) containing linked Markdown study notes. It also includes a Star History Chart showing community interest over time.

The paid course (the main product)

The README heavily promotes a 499 CNY course co-created with the author of "Self-made Deep Learning Inference Framework". The course teaches you to build your own lightweight LLM inference framework from scratch.

Claims from the README about that framework’s strengths:

Built with Triton + PyTorch — you write GPU kernels using PyTorch-like Triton syntax, avoiding complex CUDA code.
High-performance matrix multiplication kernels comparable to NVIDIA's cuBLAS library.
Efficient GPU memory management via PyTorch.
Supports FlashAttention V1/V2/V3, GQA (grouped-query attention), and PageAttention (vLLM's memory paging technique).
Includes fused operators (e.g., KV linear layer fusion).
Compatible with recent models: Qwen3, Qwen2.5, LLaMA3, LLaVA1.5.
Claims up to 4× speedup vs. HuggingFace Transformers on LLaMA3 1B/3B.

It also includes interview-focused materials — categorized LLM interview questions and 2025 high-performance-computing job interview questions.

What’s actually in the free GitHub repo?

Based solely on the README, the free repo consists of:

Study note links (Markdown files) covering the topics listed above.
Demo GIFs showing streaming generation from llama3.2-1.5B-Instruct, Qwen2.5-3B, and Llava1.5-7b-hf — presumably demonstrating the course framework's output.
A curated resources list for learning CUDA programming, Triton kernels, and LLM efficiency — including recommended tutorials and GitHub repos.
A promotional QR code for purchasing the course.

Important caveats

The free README contains table-of-contents-style links to notes, but the repository does not expose the framework source code itself — the actual framework appears to be gated behind the paid course.
Speedup claims (4×), performance parity with cuBLAS, and feature support (FlashAttention, PageAttention, etc.) are self-reported by the author in the README, not independently verified by this explainer.
The course price (499 CNY) and the claim that it's "very affordable" are promotional statements — treat them as marketing.

Bottom line

If you want free, well-organized notes on transformer internals, LLM inference optimization, GPU programming (Triton/CUDA), and serving frameworks like vLLM — this repo gives you a structured index of study material. If you want a hands-on inference framework project (for learning or interview prep), you'd need to buy the course — the repo itself leans toward documentation and promotion rather than open-source code.