vixhal-baraiya/microgpt-c

What it solves

It provides a minimal, dependency-free implementation of a GPT-style transformer model written in pure C. This allows for training and inference without the need for heavy machine learning frameworks or external libraries beyond the standard C library (libc).

How it works

The project implements a character-level transformer that includes a forward pass, backpropagation, the Adam optimizer, and sampling. It uses two distinct forward pass paths: one for training (gpt_forward) which stores activations for backpropagation, and a specialized, optimized path for single-token inference (gpt_forward_infer). To maximize performance, it leverages hardware-specific optimizations like NEON for ARM64 and AVX2 for x86-64.

Who it’s for

Developers and researchers who want to understand the inner workings of GPT models at the most atomic level or who need a lightweight, highly efficient C implementation for simple text generation tasks.

Highlights

• Dependency-free: Written in pure C using only libc.
• High performance: Optimized for ARM64 (NEON) and AVX2 (x86-64).
• Efficient: A small model (4192 parameters) that generalizes well on character-level tasks like name generation.
• Single-file implementation: The core logic is contained within one C file.