AMD Ryzen AI Halo Review: 128 GB Unified Memory Enables 120 B‑Parameter Local AI

AMD Ryzen AI Halo Review: 128 GB Unified Memory Enables 120 B‑Parameter Local AI

TL;DR

The AMD Ryzen AI Halo’s 128 GB of unified LPDDR5X memory lets you run far larger local AI models—up to 120 B parameters—and multiple models simultaneously, making high‑quality generative AI feasible on a single desk‑top without the cost of cloud APIs.

What Makes the Ryzen AI Halo Different?

  • Unified Memory Architecture – CPU and GPU share a single 128 GB pool, allowing you to allocate most of it to the GPU for model storage. This eliminates the performance cliff that occurs when a discrete GPU runs out of VRAM and offloads to slower system RAM.
  • Chipset Specs – The system is powered by a Ryzen AI Max + 395 SoC with 16 CPU cores, a Radeon 8060S GPU (≈60 TFLOPs FP16), and an NPU delivering ~50 TOPS. Current LLM stacks rarely use the NPU, but future software may.
  • Scalable Memory Options – AMD announced a future Pro 495 variant with up to 192 GB of memory, expanding the ceiling for even larger workloads.

Out‑of‑the‑Box Experience

  • Two OS Flavors – A Windows edition and a Linux edition are available. The Linux version ships pre‑configured with ROCm, drivers, and a suite of AI tools, removing the need for manual driver installation.
  • AMD Ryzen AI Developer Center – Central UI to inspect system software versions, manage GPU memory allocation, and launch pre‑installed AI applications (e.g., Llama CPP, ComfyUI). Memory split defaults to 75 % GPU, adjustable at any time.
  • Remote Access – SSH support lets you treat the Halo as a headless AI server, ideal for background inference or training jobs.
  • Playbooks – Step‑by‑step guides for common tasks (LLM serving with LMStudio, image generation with ComfyUI, fine‑tuning with Unsloth, custom GPU kernel development). Playbooks are categorized for beginner, intermediate, and advanced users.

Large Language Model Performance

Model Type Tokens / s
Ornith 9B (dense) MTP drafter ~40
Qwen 3.6 35B MoE (3 B active) MoE + drafter >50
GPT‑OSS 120B (MoE) Mixture of Experts ~30
  • Observation: Even the 120 B parameter GPT‑OSS runs at a usable 30 t/s, suitable for chat or RAG applications. Smaller, faster models are preferable for agent workloads.
  • Memory Advantage: All these models load without hitting a VRAM ceiling, which was a limitation on the prior 32 GB Radeon AI Pro R9700 workstation.

Image & Video Generation with ComfyUI

  • Pre‑installed Models – Includes ImageZ and support for downloading additional models (e.g., Qwen image model, LTX video model).
  • Workflow – Using ComfyUI’s API, the author scripted a pipeline that:
    1. Generates varied prompts via an LLM.
    2. Renders dozens of kitten‑dancing images (≈19 s per image).
    3. Feeds the images into LTX to produce short video clips.
  • Cost Efficiency – Running overnight on the Halo costs only electricity, avoiding the $0.10 / s cloud API fees typical of proprietary services.

Running Local Agents (Hermes‑Agent)

  • Setup – Installed via LMStudio, using the Qwen 3.6 3.5B model with speculative decoding (MTP).
  • Capabilities – Handles function calling, skill execution, and can switch between multiple loaded models (e.g., Ornith 9B, Qwen 3.6) without reloading.
  • Use‑Case – A 24/7 local agent can process millions of tokens daily at zero token cost, limited only by power consumption.

Fine‑Tuning with Unsloth

  • Process – Playbook automates environment setup, dataset download, and LoRA training on a 4 B Gemma model.
  • Training Tricks – Gradient accumulation allows effective larger batch sizes on limited GPU memory (e.g., accumulate over 4 steps to simulate batch‑16).
  • Performance – Training completes in a few hours on the Halo; overnight jobs are stable, with fans ramping under load but no overheating observed.

Who Should Consider the Ryzen AI Halo?

  • Large‑Model Enthusiasts – Researchers or developers needing to run 40 B+ open‑weight models locally, especially when privacy or offline capability is required.
  • Multi‑Model Workloads – Scenarios that demand several LLMs or a mix of LLMs and diffusion models simultaneously.
  • Cost‑Sensitive Creators – Content producers who want to generate high‑volume images or videos without recurring cloud fees.
  • Budget‑Conscious Builders – The Halo is priced competitively relative to a discrete‑GPU workstation with comparable performance for large‑model inference.

Limitations & Future Outlook

  • NPU Utilization – Current software ecosystems rarely tap the 50 TOPS NPU; future AMD or third‑party frameworks may unlock it.
  • Memory Trade‑offs – Unified memory means you must balance CPU vs. GPU allocation; workloads that heavily tax system RAM may require careful tuning.
  • Scaling Beyond 128 GB – The upcoming Pro 495 variant promises up to 192 GB, further extending the model size ceiling.

Final Thoughts

The AMD Ryzen AI Halo demonstrates that unified‑memory architectures can democratize access to very large language and diffusion models. By eliminating VRAM bottlenecks and bundling a ready‑to‑use software stack, it enables developers to run 120 B‑parameter LLMs, multi‑model pipelines, and overnight generative projects entirely on‑premise, at a fraction of cloud costs.

Links

Sources