rodmarkun/SmolML

A fully functional and simple Machine Learning library made entirely from scratch with Python.

What it solves

SmolML is designed to demystify the internal workings of machine learning libraries. It removes the complexity and optimizations of production-grade tools like PyTorch or Scikit-Learn to provide a transparent, educational implementation of core ML concepts.

How it works

The library is built using pure Python and only standard library modules (collections, random, math), intentionally avoiding NumPy or C++ extensions. It implements the essential building blocks of ML from scratch, including:

  • Core Engine: A custom N-dimensional array implementation (MLArray) and an automatic differentiation engine (Value) for computing gradients.
  • Preprocessing: Tools for data scaling (Standard and MinMax scalers).
  • Neural Network Components: Implementations of activation functions (ReLU, Sigmoid, Softmax), weight initializers (Xavier, He), loss functions (MSE, Cross-Entropy), and optimizers (SGD, Adam, AdaGrad).
  • Model Architectures: Implementations of Linear and Polynomial regression, feed-forward neural networks, and tree-based models (Decision Trees, Random Forests), as well as K-Means clustering.

Who it’s for

It is intended for students and developers who want to learn how ML algorithms and data-handling tools are implemented in code, rather than just using them as black boxes.

Highlights

  • Zero External Dependencies: Built entirely with pure Python for maximum accessibility.
  • Custom Autograd: Includes a built-in engine for automatic differentiation.
  • Comprehensive Coverage: Spans from basic regression and clustering to complex neural networks and random forests.
  • Educational Focus: Prioritizes code readability and understanding over computational performance.

Related

  • Project
    amitshekhariitbhu/build-your-own-x-machine-learning

    A collection of pure‑Python, from‑scratch implementations of classic ML algorithms, deep‑learning models, and applied projects (recommendation systems, CV apps, NLP, etc.). Designed as a hands‑on learning resource for anyone who wants to understand the inner workings of machine‑learning techniques.

  • Project
    scikit-learn/scikit-learn

    A Python module for machine learning built on top of SciPy, providing a wide range of tools for implementing ML algorithms.

  • Project
    microsoft/FLAML

    A lightweight Python library for efficient automated machine learning (AutoML) and hyperparameter tuning to find high-quality models with minimal computational resources.

  • Project
    skrub-data/skrub

    A Python library that simplifies machine learning workflows by facilitating the use of dataframes as input for ML models.

  • Project
    clearml/clearml

    An open-source MLOps and LLMOps suite that provides experiment tracking, orchestration, and data management to streamline the deep learning development lifecycle.