qibin0506/Cortex

What it solves

Cortex provides a complete, open-source pipeline for building a Large Language Model (LLM) from scratch, specifically designed to be accessible to individual developers by minimizing training costs and hardware requirements.

How it works

The project implements a full-lifecycle training flow divided into four sequential stages:

1. Pretraining: Learning basic knowledge with a short context window.
2. Midtraining: Adapting the model to longer text contexts.
3. Supervised Fine-Tuning (SFT): Giving the model conversational capabilities.
4. Preference Alignment: Using either Direct Preference Optimization (DPO) or Proximal Policy Optimization (PPO). The PPO stage utilizes an "LLM as Judge" approach, where an external LLM provides the reward signals for reinforcement learning.

Technically, it employs a lightweight Mixture-of-Experts (MoE) architecture with only 0.1B total parameters (and ~67M active parameters during inference) to ensure high throughput on low-power devices.

Who it’s for

Individual developers and researchers who want to practice the entire LLM creation process—from pretraining to RLHF—without needing massive industrial compute resources.

Highlights

• Full-Stack Open Source: Provides 100% of the training code for Pretrain, Midtrain, SFT, DPO, and PPO.
• Ultra-Lightweight MoE: A 0.1B parameter model designed for extreme efficiency on low-end hardware.
• LLM-as-Judge PPO: Integrates external LLMs to act as reward models for higher quality alignment training.
• Thinking Control: Supports /think and /no think tags to toggle the model's reasoning mode.
• Hardware Adaptability: Successfully tested and trained on domestic Chinese chips (MLU370).