Google TPU 8tx and TPU 8i Max Release Notes

Overview of New TPU Systems

Google has expanded its eighth-generation Tensor Processing Unit (TPU) portfolio with two new specialized systems: the TPU 8tx and the TPU 8i Max. These additions bring the total number of eighth-generation TPU systems to four, joining the previously released TPU 8t and TPU 8i. The expansion aims to address specific requirements for the "agentic era," focusing on secure, confidential computing and the ability to handle massive context windows.

TPU 8tx for Confidential Inference

TPU 8tx is designed for regulated industries and government workloads that require hardware-level data security. It is built on the same silicon as the TPU 8t but integrates security directly into the hardware layer. By pairing with 5th Generation AMD EPYC CPU hosts, the system ensures that sensitive data and model weights remain isolated and protected even during active processing. The TPU 8tx also utilizes the Boardfly network to provide low-latency performance for multi-step autonomous agents, offering up to 2x lower latency compared to the TPU 8t.

TPU 8i Max for High-Density Reasoning

TPU 8i Max addresses the "memory wall" by combining two TPU 8i chips onto a single package. This configuration doubles the available compute, SRAM, and High Bandwidth Memory (HBM) bandwidth compared to the standard TPU 8i. The system is engineered for workloads involving massive context windows, such as agents analyzing entire software codebases or long-duration video. By fitting twice the silicon into the existing rack footprint, the TPU 8i Max increases compute density per square foot and per watt.

Software and Ecosystem Updates

Google is positioning these hardware releases alongside an updated, open-source software stack to improve accessibility for developers:

  • TorchTPU and Framework Support: Google has open-sourced TorchTPU to enable native PyTorch and Hugging Face model execution on TPUs, with backend support for inference engines like vLLM and SGLang.
  • TPU Agent Suite: A new preview suite of tools designed to automate onboarding, diagnose bottlenecks, and autotune custom kernels using natural language.
  • XLA Serving Engine: An open-source engine that allows models trained in Python to be served in a low-latency C++ runtime across TPUs, GPUs, and CPUs.
  • Agent Compute Engine: A software feature that suspends and resumes idle agent sessions to reduce operational costs during periods of inactivity.
  • TPU Sync: A new model-state layer that turns chip HBM, host DRAM, and data center networking into a unified memory hierarchy to reduce Time to First Token (TTFT) by over 90%.

Community Feedback and Industry Perspective

While Google emphasizes the accessibility of these new systems, community discourse has highlighted a desire for more granular technical data. Critics have noted the absence of specific performance metrics, such as watt-per-token or price-per-token figures, as well as direct comparisons to competitor hardware like NVIDIA GPUs.

"A product announcement page with 0 links to purchase or ways to purchase is useless." — HN User waterTanuki

"Where's the watt/token or $/token figures for each of these? And how does each compare to their NVDA price equivalent?" — HN User ur-whale

Additionally, developers have expressed frustration regarding the practical availability of these new TPU models for individual users, noting that current Google Colab offerings remain limited to older, smaller configurations.

Sources