csyhhu/Awesome-Deep-Neural-Network-Compression

What it solves

Training and running deep neural networks is expensive in memory, compute, and energy. This project is a curated collection that helps researchers and engineers find and compare ways to shrink neural networks without destroying their accuracy, covering techniques like quantization (using fewer bits per number), pruning (removing unnecessary connections), and distillation (training a smaller model to mimic a larger one).

How it works

The repository is organized as a set of Markdown files that group papers by topic (for example, Quantization, Pruning, Efficient Model Design, NAS, Adversarial Compression, NLP Compression, and Large Pretraining Models) and also by publication venue (conferences from 2018 to 2024). It includes surveys, links to free summary slide decks explaining key methods, and a list of external compression systems like DeepSpeed and ColossalAI. There is also a folder with the author's own re-implementations of state-of-the-art compression methods.

Who it's for

This is for machine learning researchers, graduate students, and practitioners who want a structured, up-to-date overview of neural network compression papers, tools, and code. It is especially useful for anyone deciding which compression technique to apply or looking for references for a project or paper.

Highlights

• Organized paper lists by topic (quantization, pruning, distillation, NAS, robustness, NLP, differentiable compression, large pretrained models)
• Papers also indexed by conference year (2018–2024) and dedicated survey lists
• Summary slide decks in PDF form for quantizaton and pruning theory
• Links to popular compression systems (DeepSpeed, ColossalAI, Distiller, PocketFlow)
• Author-provided re-implementations of recent compression methods in the Codes folder