Strix Halo Guide – Local LLM Setup for AMD Ryzen AI MAX+ 395

What the project is

Strix Halo Guide is a step‑by‑step, open‑source guide for installing and benchmarking large language models (LLMs) on a very specific class of AMD hardware – the Strix Halo laptops/workstations that ship with the Ryzen AI MAX+ 395 NPU and a Radeon 8060S GPU (gfx1151). The repository does not contain model weights or compiled binaries; it only provides:

• Ubuntu 24.04 installation instructions (BIOS tweaks, kernel parameters, driver setup).
• Scripts that configure the Ollama server and the llama.cpp Vulkan/RADV backend to use the integrated GPU/NPU.
• Re‑producible benchmark data (CSV files, raw logs, charts) for a range of models – from 8 B‑parameter MoE models up to a 284 B‑parameter GGUF model.
• A “quick‑start” 6‑step checklist for getting a working chat interface, plus detailed pages that explain which model/backend combo is best for a given use‑case.
• A transparent evidence chain: every headline claim links to the exact command line, hardware metadata, and raw log file that produced it.

In short, it is a practical, evidence‑driven handbook for anyone who owns a Strix Halo machine and wants to run LLMs locally, with a focus on reproducibility and community‑verified performance numbers.

---

Who it’s for

Audience	Why it helps
Owners of a Strix Halo / Ryzen AI MAX+ 395	Gives a copy‑and‑paste Ubuntu + Vulkan setup that actually works on the hardware’s integrated GPU/NPU.
Researchers or developers evaluating AMD’s local‑AI platform	Provides independent benchmark data (prompt‑throughput, generation‑throughput, power, memory usage) that can be compared with other hardware.
Buyers or reviewers	Summarises hardware‑fit, power/thermal expectations, and which models are feasible on 96 GB‑128 GB unified memory.
Contributors	Clear contribution guide, reproducibility checklist, and a list of open‑ended test items (e.g., ROCm/vLLM bugs).

---

Core features (as described in the README)

1. Full Ubuntu 24.04 + kernel configuration – BIOS UMA frame‑buffer, IOMMU, GRUB kernel parameters (amdgpu.gttsize, ttm.pages_limit, amdgpu.cwsr_enable).
2. GPU/NPU‑enabled Ollama – environment variables (OLLAMA_VULKAN=1, OLLAMA_IGPU_ENABLE=1, HIP_VISIBLE_DEVICES=-1) so the integrated Radeon 8060S is used for inference.
3. llama.cpp Vulkan/RADV backend – scripts to build and run llama.cpp with AMD’s RADV driver, giving direct‑GPU inference without a server.
4. Benchmark evidence – CSV files (headline_claims.csv, best_known_profiles.csv, etc.) and raw log directories that show exact token‑per‑second (t/s) numbers for each model/backend combo.
5. Model‑specific decision pages – e.g., QWEN38_STRIX_HALO.md explains why the measured 20.42 t/s generation speed for Qwen 3.8 27B differs from community‑reported numbers.
6. Cross‑OEM validation – results from 13 different systems and 10 community contributors are collected and kept separate from the author’s primary Beelink GTR9 Pro data.
7. Reproducibility workflow – setup.sh automates the installation, but the README stresses manual review, version pinning, and how to undo each change.
8. Transparency & independence – No affiliate links; all upstream contributions are listed (UPSTREAM_CONTRIBUTIONS.md).

---

How to get started (quick‑start 6 steps)

1. BIOS – Set UMA frame‑buffer to 512 MiB (or 2 GiB if that’s the minimum). Keep IOMMU enabled unless you deliberately want the desktop benchmark profile.
2. Install Ubuntu 24.04 LTS – X11 is only needed for a optional desktop UI; headless inference works fine.
3. Apply memory‑profile kernel args – amdgpu.gttsize=131072 ttm.pages_limit=31457280 amdgpu.cwsr_enable=0 (for 128 GB systems).
4. Install Mesa/RADV driver – via the kisak‑mesa PPA; optionally enable the tuned power profile for performance.
5. Install Ollama – set the Vulkan and iGPU env vars, then pull a model (e.g., ollama run qwen3.6:35b-a3b).
6. Run a test – the provided bench-ollama.sh script measures token‑throughput and compares it to the CSV evidence.

The repository also ships a setup.sh script that performs steps 3‑5 automatically; the README advises reviewing the script before execution.

Evidence & performance highlights (as of Oct 1 2026)

Model / backend	Token‑throughput (t/s)	What the number represents
Qwen 3.8 27B (Q4_K_M) – Ollama API with default MTP drafting	20.42 t/s generation (12.89 t/s without drafting)	Measured on Ollama 0.31.2; includes 50 k prompt tokens, image & tool use.
Qwen 3‑Coder 30B‑A3B (Q4_K_S) – direct llama.cpp Vulkan	100.99 t/s (tg128)	Fastest 30 B‑class direct route; a speed‑first quantisation.
DeepSeek V4 Flash 284B (UD‑IQ2_XXS) – direct GGUF	13.27 tg128	Proof‑of‑capacity; model loads as a 90.86 GB low‑bit artifact.
LFM2.5 8B‑A1B (Q4_K_M) – small‑MoE – llama.cpp	170 t/s generation‑only	Shows how small MoE models can exceed 100 t/s on this hardware.

---

Strengths & limitations (summary)

Strengths

• Very detailed, hardware‑specific instructions – rare for AMD integrated AI.
• Transparent benchmark pipeline; every headline claim is backed by raw logs.
• Community‑driven cross‑system data adds confidence beyond a single test box.
• Open‑source scripts (setup.sh, bench-ollama.sh) make reproduction straightforward.

Limitations

• Scope is limited to Strix Halo / Ryzen AI MAX+ 395 hardware; the guide will not work out‑of‑the‑box on other AMD laptops or on non‑Ubuntu OSes.
• Some scripts are marked “untested on hardware” for undo‑operations, so users must be comfortable with manual Linux configuration.
• Performance numbers are tied to specific Ollama/llama.cpp versions; newer releases need a fresh‑install qualification step.

---

Who maintains it

The maintainer has contributed to several upstream projects (e.g., llama.cpp PR #25643, AMD‑sponsored Lemonade server) and lists those contributions in UPSTREAM_CONTRIBUTIONS.md. The repository is MIT‑licensed and the author explicitly states no official AMD endorsement.

Bottom line

If you own a Strix Halo laptop or workstation and want to run LLMs locally, Strix Halo Guide gives you a reproducible, evidence‑backed path from a fresh Ubuntu install to a working Ollama or llama.cpp inference server, complete with benchmark data that lets you compare different models and back‑ends. It is a genuine, highly focused software‑engineering project in the AI‑local‑inference space.