wdndev/llm_interview_note

What is this?

llm_interview_note is a personal, open-source study guide and interview-prep collection focused on Large Language Models (LLMs) — the technology behind tools like ChatGPT. The author gathered concepts from online resources and wrote their own answers to common interview questions.

What's inside?

The repository is organized as a structured set of notes covering the full LLM lifecycle:

• LLM fundamentals — history, tokenization, word vectors, language models, deep learning basics
• Model architecture — Transformers, attention mechanisms, position encoding, BERT, and well-known models (LLaMA series, ChatGLM series)
• Training data — dataset formats and model parameters
• Distributed training — data parallelism, pipeline parallelism, tensor parallelism, DeepSpeed, Megatron, and memory issues
• Supervised fine-tuning — prompting, adapter tuning, LoRA, plus practical fine-tuning examples for LLaMA2 and ChatGLM3
• Inference — frameworks (vLLM, FasterTransformer, TensorRT-LLM), optimization techniques, and quantization
• Reinforcement learning — PPO, RLHF, DPO
• RAG & Agents — retrieval-augmented generation and agent technology
• Evaluation — model evaluation and LLM hallucination
• Applications — chain-of-thought prompting and LangChain
• Real interview questions — a dedicated section with actual interview questions

Each topic is a separate markdown file, and the whole thing can be read online as a website.

Who is it for?

People preparing for LLM-related job interviews or wanting a structured overview of LLM concepts. The author notes the answers are self-written and invites corrections.

Related hands-on projects

The README links to several companion repositories for practical learning:

• tiny-llm-zh — build a small Chinese LLM from scratch (pre-training, fine-tuning, RL)
• tiny-rag — build a simple RAG system with multi-recall and reranking
• tiny-mcp — implement MCP (Model Context Protocol) server/client using prompts and function calling
• llama3-from-scratch-zh — implement LLaMA3 from scratch, runnable on a laptop with 16GB RAM

There's also a broader AI engineer interview note covering deep learning, machine learning, and recommendation systems.

How to access it

• Online reading: LLMs Interview Note
• GitHub: the repository itself (star it if useful, per the author's request)
• The author also runs a WeChat public account for LLM content and interview experience (QR code in the README).

Key caveat

The content is self-written by the author, not an official or peer-reviewed source. The author explicitly asks readers to point out any inaccuracies.