OpenAI Jalapeño Inference Chip First Results

OpenAI has unveiled the first performance results for Jalapeño, its first custom inference chip. The hardware is designed to eliminate the traditional tradeoff between throughput and latency, enabling faster responses and higher power efficiency for large-scale AI models and interactive agents.

Performance Benchmarks and Efficiency

Jalapeño demonstrates industry-leading efficiency and speed across various model architectures, including GPT-OSS 120B, DeepSeek R1, and Kimi K2.5 1T. Using the InferenceX public benchmark from SemiAnalysis, OpenAI reports that Jalapeño consistently places on the Pareto frontier for performance per watt and latency.

Key performance gains include:

• Power Efficiency: Jalapeño delivers 1.5 to 1.9 times more AI work per watt at peak throughput compared to comparison systems.
• Latency Reduction: End-to-end latency is 1.7 to 3.6 times lower than existing systems.
• Interactive Workloads: For highly interactive use cases, the chip delivers 2.1 to 4.1 times higher performance.

Specific model results show that on Kimi K2.5 1T, the largest tested public model, Jalapeño achieved approximately 1.5 times higher peak performance per watt and 3.4 times lower end-to-end latency. The chip is rated at 700 watts, though measured sustained power remained at or below 550 watts during testing.

Full-Stack Architectural Design

Jalapeño is built as a full-stack solution where the chip, memory, networking, and software are co-designed to minimize data movement and communication delays. This approach addresses the specific bottlenecks of language model inference:

• Prefill Phase: Optimized for compute-intensive prompt processing.
• Decode Phase: Optimized for memory-bandwidth-constrained token generation.

By explicitly placing model state and the KV cache locally, the system reduces the time processing units spend idling while waiting for data. The integrated network allows the entire workload to remain within one connected system, creating a balanced accelerator capable of adapting to the changing requirements of agentic workloads.

AI-Driven Hardware Development

OpenAI utilized its own AI models to accelerate the design and programming of the Jalapeño chip, reducing the time from initial design to tapeout to nine months. AI was used to optimize arithmetic circuits and shorten verification loops.

Furthermore, the chip was designed to be a predictable programming target for AI. Using Codex with GPT-Astra, OpenAI engineers brought three open-weight models to high performance within two months. In specific tests on GPT-OSS attention and mixture-of-experts blocks, AI-generated implementations performed 1.5 to 1.8 times faster than those written by human experts.

Deployment Roadmap

OpenAI plans to begin deploying Jalapeño within its compute infrastructure by the end of the year. This chip represents the first generation of a multi-generational platform, with Gen 2 already in development and Gen 3 currently taking shape.

While deploying its own silicon, OpenAI stated it will continue to use accelerators from NVIDIA and other partners for both training and inference workloads to meet growing demand.