OpenAI's Jalapeño Chip Delivers Faster AI Inference Than Nvidia's Superchips

OpenAI has published benchmark results for its in-house Jalapeño chip showing it delivers faster AI inference and significantly better power efficiency than competing silicon, according to data released at the Hot Chips conference. The results represent a direct challenge to Nvidia's dominance in AI inference hardware and signal that the race to build efficient, purpose-built AI chips is accelerating beyond the GPU era.

Benchmark Performance and Design

Jalapeño, an Application-Specific Integrated Circuit (ASIC) developed by OpenAI in close collaboration with Broadcom, is designed specifically for AI inference workloads at scale. According to OpenAI hardware vice president Richard Ho, Jalapeño offers the "best of both worlds" with lower latency and higher throughput, as AI systems typically "have to make a trade-off between the two."

The benchmarks were tested on SemiAnalysis' InferenceX benchmark platform, which measures how well AI systems handle inference. The test compared Jalapeño's performance against the best results recorded at the time, which were with Nvidia's GB200 or GB300 superchips. The results demonstrate substantial efficiency gains: OpenAI says Jalapeño delivered 1.5 to 1.9 times more AI work per watt across GPT-OSS 120B, DeepSeek R1, and Kimi K2.5 1T than the comparison systems, while offering 1.7 to 3.6 times lower end-to-end latency across the three models.

That means the chip can provide users with "faster responses, more responsive agents, and more reliable access as the demand grows," according to Ho. The chip's architectural approach addresses specific bottlenecks in the inference process. OpenAI designed Jalapeño to minimize delays during the prefill and communication phases of processing, which often act as bottlenecks, with model state, including the KV cache used while generating a response, explicitly placed and kept local while the system activates the right combination of compute, memory, and networking for each inference phase.

Deployment Timeline and Strategic Vision

OpenAI estimates Jalapeño will deploy at the end of 2026 "in very small volumes," with more significant deployment coming in 2027. The company plans to deploy Jalapeño in "small volumes" by the end of this year, but will begin to "ramp the volume up" into 2027, Ho added, though the company doesn't say how many chips it plans to deploy next year. OpenAI plans to make Jalapeño a multigenerational platform, allowing AI products, models, chips, and memory all developed in concert.

Significantly, OpenAI's strategy does not envision replacing its entire chip portfolio. OpenAI doesn't expect to replace its entire chip lineup with Jalapeño, saying its overall compute strategy includes "very good partners," like Nvidia. OpenAI will continue developing the second and third generations of the new chip. This indicates that while custom silicon is becoming strategically important, the company sees continued value in partnerships with established hardware manufacturers for its broader infrastructure needs.

Why Inference Efficiency Matters

The Jalapeño chip story cuts to the heart of where AI infrastructure economics are heading. Training large models requires massive compute clusters deployed once, but inference—actually running models for users at scale—is where the ongoing electricity and hardware costs accumulate continuously. A chip purpose-built for fast, efficient inference addresses both performance and economics simultaneously.

The power consumption metric underlying these benchmarks carries material business significance. OpenAI says Jalapeño can perform more work while using less energy. This efficiency translates directly to operational cost savings when multiplied across hundreds of millions of daily inference requests. Every watt saved per inference query, compounded across millions of users generating billions of tokens daily, represents significant operational savings and reduced energy footprint. Lower per-token energy consumption also improves unit economics for inference-heavy businesses where response generation costs directly affect profitability.

This economic logic explains why custom silicon has become strategically important across the AI industry. Rather than relying on general-purpose GPUs optimized for broad workloads, companies developing their own inference chips can optimize specifically for their production workloads—the models they actually deploy, the token generation patterns they observe, and the latency requirements their users demand. The ability to reduce dependence on external silicon suppliers also addresses supply-chain fragility and cost exposure at the scale OpenAI operates. By designing its own inference chip in partnership with Broadcom, OpenAI reduces vulnerability to Nvidia's pricing power and supply constraints, giving it greater control over both hardware economics and deployment timing.

The Competitive Landscape and Independent Verification

The Verge noted that OpenAI's claims put Jalapeño ahead of rival offerings on response speed, though independent verification of the figures has not yet been published. This distinction matters: company-published benchmarks for proprietary chips are common, and third-party analysis frequently either confirms or narrows claimed advantages. Notably, that comparison is against an Nvidia Blackwell system — but by the time Jalapeño reaches full deployment, the competition may have advanced significantly.

The competitive dynamics here extend beyond raw performance figures. Jalapeño represents OpenAI's response to a structural challenge: as AI workloads mature and inference demand scales, companies that can optimize their hardware stack gain compounding cost advantages. Each generation of custom silicon can be designed with specific knowledge of production workloads and constraints. This full-stack integration—where chip design, model architecture, and deployment infrastructure are co-optimized—creates barriers to competition that pure hardware performance cannot capture.

Supply Chain and Manufacturing Context

The ability to design and manufacture competitive AI chips ultimately depends on access to the most advanced fabrication processes. ASML holds a monopoly on extreme ultraviolet lithography machines—the equipment without which no leading-edge chip, including those powering OpenAI's inference infrastructure, could be manufactured. This chokepoint position gives ASML, and by extension the geopolitics surrounding its export controls, an outsized influence on which companies and nations can access next-generation silicon.

Jalapeño's successful development signals that even companies without in-house foundry operations can access advanced manufacturing. OpenAI's partnership with Broadcom—which handles the actual chip fabrication through partner foundries like TSMC—demonstrates that the custom silicon strategy remains viable for companies with sufficient capital and technical depth, even in the face of ASML's structural role in the semiconductor supply chain.

Broader Industry Trajectory

The inference chip race also reflects longer structural shifts in how AI workloads differ from traditional compute. The GPU's evolution from graphics processing to AI acceleration explains why general-purpose graphics hardware became the default AI training platform—and why specialised inference chips are now challenging that default. GPUs were optimized for parallel floating-point computation across broad use cases; purpose-built inference chips can eliminate inefficiencies that matter specifically for language model inference: memory bandwidth constraints, communication bottlenecks between compute and storage, and latency-throughput trade-offs that affect end-user experience.

The timing of Jalapeño's announcement and deployment roadmap also suggests that OpenAI expects inference workloads to become an increasingly dominant share of its infrastructure costs. By committing significant engineering resources to a custom chip now, with deployment ramping in 2027, the company is positioning itself to manage cost growth as user scale increases. This mirrors decisions made by hyperscalers like Google, Meta, and Amazon, all of which have invested in custom silicon for inference workloads—a trend that underscores how fundamental this shift has become.