theaniketgiri/create-llm

What it solves

create-llm is a CLI tool designed to eliminate the boilerplate and complexity involved in training a custom Large Language Model (LLM) from scratch. It provides a standardized project structure and pre-configured templates, allowing users to move from a blank folder to a production-ready training pipeline in seconds.

How it works

The tool scaffolds a complete PyTorch-based training environment. Users can choose from four right-sized templates based on their hardware and data availability:

• NANO (~1M params): For learning and quick CPU tests.
• TINY (~6M params): For prototyping on CPU or basic GPUs.
• SMALL (~100M params): For production-grade domain-specific models.
• BASE (~1B params): For research-grade high-quality models.

Once scaffolded, the project includes scripts for tokenizer training (BPE, WordPiece, Unigram), data preprocessing, training with TensorBoard monitoring, evaluation, text generation, and deployment to Hugging Face or Replicate.

Who it’s for

It is aimed at developers and researchers who want to train their own language models without manually setting up the architecture, data pipelines, and training loops. It is particularly useful for those who want to start with small-scale experiments and scale up to larger models.

Highlights

• One-Command Setup: Quickly generates a full project structure using npx create-llm.
• Integrated Toolkit: Includes everything from tokenizer training to an interactive chat interface for testing the trained model.
• Smart Defaults: Automatically handles vocabulary size detection, sequence length mismatches, and provides overfitting warnings.
• Docker Support: Offers a full Docker and Docker Compose setup to run the entire pipeline without needing local Node.js or Python installations.
• Plugin System: Optional integrations for experiment tracking via WandB and model sharing via Hugging Face.