MiaAI-Lab/Qwen3.8-27B-16gb-NVIDIA-GPUs-one-click-install

What it solves

This project provides a "one-click" installation and serving kit for the Qwen3.8-27B model, specifically optimized to run on consumer NVIDIA GPUs with 16GB to 32GB of VRAM. It removes the manual complexity of setting up Python environments, downloading specific quantized weights, and configuring VRAM budgets for different hardware profiles.

How it works

The kit uses EXL3 quants (primarily from turboderp) to fit the large model into limited VRAM. It includes a launcher (simplex) that automatically detects the user's GPU VRAM and recommends the best quantization level (bpw) and context size based on measured performance data. It sets up a local Python virtual environment, downloads the required weights, and serves the model via an OpenAI-compatible API endpoint. For the user interface, it integrates the DeepSeek Harness, a Node-based chat UI that automatically configures itself based on the model's loaded capabilities (such as vision support and context window).

Who it’s for

Users with NVIDIA GPUs (12GB VRAM or more, Turing architecture or newer) on Windows or Linux who want to run Qwen3.8-27B locally without manually managing dependencies, CUDA toolkits, or complex configuration files.

Highlights

• Automated Hardware Profiling: Automatically detects VRAM and suggests the optimal model size and context window.
• Cross-Platform: Unified behavior across Windows and Linux with dedicated scripts for installation and management.
• OpenAI-Compatible: Serves an API endpoint that can be used with other AI tools.
• Self-Contained: Installs all dependencies (Python venv, weights) within its own folder to avoid polluting system paths.
• Resumable Downloads: Model weight downloads can be interrupted and resumed without loss of data.
• Integrated UI: One-click deployment of a chat interface that dynamically adjusts to the model's loaded features.