jingyaogong/minimind

MiniMind: A Tiny LLM Trained From Scratch

MiniMind is a hands-on open-source project that shows you how to build a small but functional large language model (LLM) completely from zero. The headline claim is that you can train a ~64 million parameter model for about 3 RMB (less than $1) in compute cost and just 2 hours of training time.

Instead of just fine-tuning a huge existing model with LoRA, MiniMind provides the complete code pipeline to do everything yourself: collecting and cleaning data, training a tokenizer, pretraining the base model, supervised fine-tuning (SFT), and aligning it with modern reinforcement learning techniques like DPO, PPO, GRPO, and CISPO. It even includes experimental variants for Mixture-of-Experts (MoE), vision, and diffusion-based language models.

The project is designed for education and experimentation. It includes scripts for training, evaluation, and serving, and it provides an OpenAI-compatible API server so you can plug your trained model into standard chat UIs like FastGPT or Open-WebUI, or deploy it via Ollama or vLLM.