ridgerchu/matmulfreellm

What it solves

It addresses the high computational cost of Matrix Multiplication (MatMul) in traditional language models. By eliminating MatMul operations, the architecture aims to be more efficient, particularly for deployment on neuromorphic hardware.

How it works

The project implements a language model architecture that replaces standard matrix multiplications with more efficient operations, utilizing ternary weights in several layers. It is designed to be compatible with the Hugging Face Transformers library, allowing it to be integrated into existing AI workflows using standard AutoModel and AutoModelForCausalLM APIs.

Who it’s for

Researchers and developers focusing on efficient AI architecture, neuromorphic computing, and those looking to reduce the compute overhead of large language models.

Highlights

• MatMul-Free Architecture: Completely removes the need for standard matrix multiplication operations.
• Hugging Face Integration: Fully compatible with the Transformers library for easy initialization and text generation.
• Scaling Efficiency: Demonstrates a steeper scaling projection compared to Transformer++, suggesting better performance gains as compute increases.
• Pre-trained Models: Provides a model zoo with sizes ranging from 370M to 2.7B parameters.