MiaAI-Lab/Qwen3.8-27B-16gb-NVIDIA-GPUs-one-click-install

What it solves

This project provides a "one-click" installation and serving kit for the Qwen3.8-27B model on consumer NVIDIA GPUs with 12GB to 32GB of VRAM. It removes the complexity of manual environment setup, weight downloading, and quantization selection by automating the process based on the user's hardware.

How it works

The kit uses EXL3 quants (primarily from turboderp) to fit the large model into limited VRAM. A launcher script (called simplex) analyzes the user's GPU VRAM via nvidia-smi and automatically selects the best quantization level (bpw) and context size that will fit the card. It then creates a self-contained Python virtual environment, downloads the required weights, and serves the model via an OpenAI-compatible API endpoint. For the user interface, it integrates the DeepSeek Harness (dsh), a Node-based chat UI that automatically configures itself based on the loaded model's capabilities.

Who it’s for

Users with NVIDIA GPUs (Turing architecture or newer) who want to run Qwen3.8-27B locally on Windows or Linux without needing to manually manage CUDA toolkits, compilers, or complex configuration files.

Highlights

• Hardware-Aware Automation: Automatically picks the optimal model quant and context window based on detected VRAM (12GB, 16GB, 24GB, or 32GB+).
• Self-Contained Installation: Installs everything into its own folder (venv, models, logs) without requiring administrator rights or system-wide Python changes.
• **OpenAI-Compatible API: Serves a /v1 endpoint for integration with other tools.
• Resumable Downloads: Model weight downloads can be interrupted and resumed without loss of data.
• Integrated Chat UI: Deploys the DeepSeek Harness for a ready-to-use chat experience with automatic model configuration.
• Cross-Platform: Unified command-line interface (simplex) for both Windows and Linux.