TrenTorch/TrenTorch

Tren⚡Torch – A From‑Scratch ML Framework for Learning the Internals

What it is – TrenTorch is an open‑source educational deep‑learning library built entirely on top of NumPy. It re‑implements the classic TinyTorch curriculum (Harvard CS249r) and extends it with many more modules, from basic tensors all the way to full‑blown transformer‑based LLMs, profiling, quantization and benchmarking. The codebase contains no external deep‑learning dependencies (no PyTorch, TensorFlow, JAX, etc.); every operation, autograd engine, optimizer and layer is hand‑written.

Why it exists – The authors want learners to own every piece of a modern ML stack. By building the framework themselves they can see exactly how loss.backward() works, how memory is allocated, and how production‑grade optimizations are applied. The project is positioned as a progressive curriculum: you start with tensors, then add convolutions, then attention, and finally LLM‑ops.

Core Features (as described in the README)

Area	What you get
Foundations (Modules 01‑08)	Tensor class, activation functions, linear layers, loss functions, dataloader, automatic differentiation, optimizers (SGD, Adam, AdamW, Lion, Muon), training loops.
Vision (Module 09)	`Conv2d`, `MaxPool2d`, complete CNN pipelines that can be run on standard image benchmarks.
Language (Modules 10‑13)	Tokenizer, embedding layers, multi‑head attention, full transformer blocks for GPT‑style language models.
Optimization (Modules 14‑20)	Profiling tools, quantization, pruning & distillation, hardware‑specific acceleration helpers, KV‑cache memoization, benchmarking suite, and a capstone project that ties everything together.
CLI (`tren`)	A command‑line interface that scaffolds module work‑flows, runs milestones, exports models, and launches Jupyter notebooks.
Historical Milestones	Scripts that let you recreate landmark achievements (Perceptron, XOR, back‑prop, CNNs, Transformers, MLPerf‑style benchmarks) using the framework you just built.

Quick‑Start Installation

# Clone and set up a virtual environment
git clone https://github.com/TrenTorch/TrenTorch.git
cd TrenTorch
python3 -m venv .venv
source .venv/bin/activate   # or .venv\Scripts\Activate.ps1 on Windows
pip install -r requirements.txt
pip install -e .            # install the package in editable mode
tren setup   # adds the `tren` command to your PATH
tren          # launches the CLI

The README notes that after tren setup you can run tren from any terminal without activating the venv again.

Intended Audience & Use Cases

Students & self‑learners who want a deep, hands‑on understanding of how modern deep‑learning frameworks are built.
Educators looking for a curriculum that progresses from low‑level tensor ops to production‑grade LLM pipelines.
Engineers interested in profiling, quantization and other performance tricks on a minimal codebase they can read entirely.
Hackers who want a lightweight, NumPy‑only stack for experiments where installing heavyweight libraries is undesirable.

Project Status

All 20 modules are implemented and have unit‑ and integration‑tests.
The CLI, documentation and milestone scripts are functional.
Ongoing work focuses on polishing docs, hardening edge cases, and performance tuning.
Future roadmap mentions community leaderboards, additional milestone exercises and more extensive benchmarking.

License & Community

Licensed under the MIT License (permissive, commercial‑friendly).
Contributions are welcomed; a Contributor Covenant governs conduct.
The Team Engineers section lists current maintainers and shows a live‑generated contributor badge.

Bottom Line

TrenTorch is a genuine, actively maintained open‑source project that provides a full‑stack, NumPy‑only deep‑learning framework for learning and experimentation. It is well‑documented, test‑covered, and organized as a step‑by‑step curriculum that culminates in building and optimizing modern models such as transformers and LLMs.