Emericen/tiny-qwen
A minimal PyTorch re-implementation of Qwen 3.8
What it solves
It provides a minimal, easy-to-read re-implementation of the Qwen model architecture (specifically Qwen 3.8) using PyTorch, serving as an alternative to the more complex Hugging Face codebase. It also includes tools for running these models with lower memory requirements via quantization.
How it works
The project implements the Qwen architecture in PyTorch and provides a Model and Processor class for loading and running inference. It supports loading models from Hugging Face repositories, local directories, or connecting to OpenAI-compatible API endpoints. To reduce memory usage, it includes built-in support for int4 quantization.
Who it’s for
Developers and researchers who want a clean, understandable implementation of Qwen models or those who want to run Qwen models with int4 quantization on their own hardware.
Highlights
- Minimal PyTorch re-implementation of Qwen 3.8.
- Built-in int4 quantization support to reduce model size.
- Includes a single-file agentic harness for quick deployment.
- Supports both local execution and remote OpenAI-compatible API endpoints.
- Supports multimodal inputs (text and images) as shown in the library usage example.
Related
- Dispatch
- Project
- Project
- Project
- Dispatch