OpenAI Built Its Own Chip and It Just Beat Nvidia by 3.6x

Nvidia has dominated AI hardware so completely that the idea of a software company building a chip to compete with it seemed more like ambition than an actual plan. On August 25, OpenAI released the first benchmark data for Jalapeño, its custom-built inference chip. The tests were run against Nvidia's GB200 and GB300 systems — the same hardware most major AI companies currently pay to use, and the numbers came back in Jalapeño's favor across throughput, latency, and power efficiency.

What the Numbers Actually Say

OpenAI tested Jalapeño on InferenceX, a public inference benchmark from SemiAnalysis, across three separate models: GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T. Including DeepSeek and Kimi, both built entirely outside OpenAI, was a deliberate choice. It signals that Jalapeño is not simply tuned for OpenAI's own models, which would have been a far narrower and less convincing result. The standout number comes from the DeepSeek R1 670B test. Jalapeño completed end-to-end requests in 1.65 seconds. The Nvidia GB300-based system took 5.99 seconds. That is a 3.6x gap — not on a narrow synthetic test, but on the full cycle of receiving a real request and returning a completed response.

On Kimi K2.5 1T, the largest model in the test, Jalapeño posted a 3.4x latency lead and roughly 1.5x more throughput per watt at peak load. Against the GB200 on GPT-OSS 120B, latency came in 1.7x lower and peak throughput per kilowatt landed 1.9x higher. For highly interactive workloads — where a model is being called in rapid succession — the performance gap stretched to 2.1x to 4.1x across all three models.

One number that deserves a closer look: Jalapeño carries a 700-watt power rating. The GB200 runs at 1,200W. The GB300 at 1,400W. During the actual benchmark runs, Jalapeño's real sustained draw stayed at or below 550 watts — below its own rated ceiling — while competing against hardware running at nearly double the power.

Why Latency Is the Metric That Actually Changes Products

A 3.6x speed improvement on a single model call sounds impressive on paper. Inside an agentic workflow, it becomes something else entirely. When an AI agent runs a multi-step task — researching a topic, drafting an output, checking it, formatting the result — it calls the model repeatedly in sequence. Each step waits for the previous one to finish before it can move forward.

If one call takes 5.99 seconds instead of 1.65 seconds, that delay stacks across every single step. A ten-step workflow that takes around 15 seconds on Jalapeño could take well over a minute on the comparison system. That kind of gap is the difference between a product that feels responsive and one that feels broken. This is exactly why OpenAI highlighted interactive and agentic workloads specifically in the results. Speed at scale matters for infrastructure cost. Speed per request matters for the people actually using the product. Jalapeño appears built to chase both at the same time.

How the Chip Was Designed and What Helped Build It

OpenAI did not adapt a general-purpose architecture and tune it. Jalapeño was built from the ground up around one question: what does hardware look like when its primary job is serving language models?

LLM inference runs in two distinct phases with completely different bottlenecks — prefill, where the system processes the incoming prompt (compute-heavy), and decode, where it generates the response token by token (memory-bandwidth-heavy). Most hardware handles one of these phases well and compensates on the other. Jalapeño was designed to cut data movement between these phases, keeping the KV cache — the memory structure that tracks what the model has generated — placed locally so the decode stage does not stall waiting on transfers.

The development timeline is also worth paying attention to. OpenAI's own AI models were directly involved in the chip design process, which helped the team reach tapeout in just nine months. After production, Codex and GPT-Astra were brought in to get three open-weight models — none of which were in the original roadmap — running efficiently on Jalapeño within two months.

On specific GPT-OSS model blocks — attention layers and mixture-of-experts sections — kernels generated by AI outran what engineers had previously hand-coded by 1.5x to 1.8x. That figure covers selected components, not the full model. But it points to a clear direction: OpenAI is building infrastructure that its own AI can progressively optimize over time.

What This Actually Changes for Users and Developers

Jalapeño is not in production yet. OpenAI plans to begin deploying it within its own compute infrastructure before the end of 2026, so nothing changes today for API users or people using ChatGPT. But if these results hold at production scale, two things follow:

Inference costs have room to drop — serving more requests per watt means the cost per API call can fall without cutting quality. And response times on OpenAI products, especially in agentic use cases, could improve meaningfully once Jalapeño is running at full scale.

OpenAI confirmed it will keep purchasing and deploying Nvidia hardware and other third-party accelerators for both training and inference. Jalapeño is not a full replacement — it is a focused addition to the stack, handling inference where it can do so more efficiently. A second generation is already deep in development, and a third is in early design.