--- url: "https://xcademia.com/news/openai-s-jalape-o-chip-delivers-faster-more-efficient-ai-inference" title: "OpenAI’s Jalapeño Chip Delivers Faster, More Efficient AI Inference" description: "OpenAI says its Jalapeño chip delivers faster, more efficient AI inference across GPT-OSS, DeepSeek R1 and Kimi K2.5 workloads." publishedAt: "2026-08-26T11:42:35.598+00:00" updatedAt: "2026-08-26T12:12:46.152937+00:00" type: news category: "ai-ml" source_name: OpenAI source_url: "https://openai.com/index/jalapeno-first-results/" tags: - "#OpenAI" - "#Jalapeno" - "#AIChips" - "#AIInference" - "#ArtificialIntelligence" - "#AIInfrastructure" - "#CustomSilicon" - "#MachineLearning" --- # OpenAI’s Jalapeño Chip Delivers Faster, More Efficient AI Inference > OpenAI says its custom Jalapeño inference chip delivers higher throughput, lower latency and better performance per watt across GPT-OSS, DeepSeek R1 and Kimi K2.5 workloads. Source: **OpenAI** · 26 August 2026 ## OpenAI’s Jalapeño Shows What Custom AI Inference Hardware Can Deliver OpenAI has published the first performance results for **Jalapeño**, its first custom inference chip, showing a focus on three increasingly important areas of AI infrastructure: speed, power efficiency and the ability to serve growing workloads. According to OpenAI, Jalapeño can deliver more AI work per unit of power while also returning responses faster. The company says this combination is important because conventional infrastructure can involve trade-offs between throughput and latency. The results were measured using **InferenceX**, a public benchmark from SemiAnalysis, across three public models: **GPT-OSS 120B, DeepSeek R1 670B and Kimi K2.5 1T**. OpenAI reports that Jalapeño delivered between **1.5x and 1.9x more AI work per watt at peak throughput**, while end-to-end latency was between **1.7x and 3.6x lower** than the comparison systems. For highly interactive workloads, OpenAI reports **2.1x to 4.1x higher performance** across the tested models. These are OpenAI's reported benchmark results rather than independent verification. ## Why Inference Efficiency Matters AI inference is the stage where trained models actually process requests and generate responses. As AI assistants and agents perform more tasks, the infrastructure needs to handle both large volumes of requests and increasingly interactive workloads. OpenAI says it evaluated Jalapeño at matched user experiences, looking at how much useful AI work a system can complete per unit of power while still meeting latency requirements. This distinction matters for AI agents. An agent may need to perform several actions sequentially, meaning delays in individual steps can accumulate over the course of a task. Lower inference latency can therefore be particularly relevant for interactive systems. ![info-1](https://0a515t3ure77wbvx.public.blob.vercel-storage.com/articles/1787744469897-info1--43-.webp) ## Jalapeño’s Results Across Three Major Open Models OpenAI tested Jalapeño with three public model workloads to evaluate how its architecture performs across different inference requirements. The company says the chip remained on the Pareto frontier across the tested operating ranges for **GPT-OSS 120B, DeepSeek R1 670B and Kimi K2.5 1T**. For GPT-OSS 120B, OpenAI reports approximately: - **1.9x higher peak performance per watt** - **1.7x lower end-to-end latency** - **2.7x lower minimum time between tokens** - Higher throughput at the comparison system's previous time-between-token point For DeepSeek R1 670B, OpenAI reports approximately: - **1.7x higher peak performance per watt** - **3.6x lower end-to-end latency** - **4.1x lower minimum time between tokens** For Kimi K2.5 1T, the company reports approximately: - **1.5x higher peak performance per watt** - **3.4x lower end-to-end latency** - **3.8x lower minimum time between tokens** The comparison systems and test conditions differ by model. OpenAI says Jalapeño has a published package power rating of **700 watts**, while its measured sustained power remained at or below **550 watts** on the workloads tested. OpenAI says Kimi K2.5 1T, the largest public model included in its testing, delivered approximately 1.5x higher peak performance per watt and 3.4x lower end-to-end latency compared with the comparison system. ## A Different Approach to the Inference Bottleneck OpenAI's approach goes beyond designing an accelerator in isolation. The company says Jalapeño was designed alongside memory, networking, software and the broader rack-scale system around modern language-model workloads. One important consideration is that inference does not have a single consistent bottleneck. During **prefill**, the system processes the incoming prompt and is generally more compute-intensive. During **decode**, the model generates output token by token, making memory bandwidth more important. Communication between cores and chips can introduce another source of delay. OpenAI says Jalapeño was designed to reduce unnecessary data movement and communication delays by keeping model state, including the KV cache used during generation, local where possible. The architecture also integrates networking into the overall design so workloads can remain within one connected system. ![info-2](https://0a515t3ure77wbvx.public.blob.vercel-storage.com/articles/1787744487921-info2--44-.webp) ## Designing for Both Throughput and Latency A major point in OpenAI's announcement is that Jalapeño was designed to avoid optimising only one aspect of inference. A system can achieve high throughput by processing many requests efficiently, but highly interactive applications also need low response latency. OpenAI says Jalapeño was designed to address both requirements within one architecture. The company describes the resulting accelerator as balanced and fungible, allowing it to support changing model architectures while handling different combinations of prefill and decode workloads. This approach is particularly relevant to agentic AI, where workloads can shift dynamically depending on what an agent is doing. For enterprises, the broader implication is that AI infrastructure increasingly needs to be evaluated as a complete system rather than by accelerator specifications alone. ## AI Helped OpenAI Design Jalapeño OpenAI also says AI itself played a direct role in the development of Jalapeño. The company reports that AI-assisted development helped the team move from initial design to tapeout in **nine months**, including implementation exploration, design iteration, measurement and verification. AI was also used to optimise arithmetic circuits. The chip was designed to provide a predictable programming target. Engineers can work with local tensors, explicit communication and predictable synchronisation, while AI can help optimise how workloads are mapped, placed and scheduled across the system. OpenAI says this structure makes the parallel-programming problem more tractable for AI-assisted optimisation. The company also reports that its teams used **Codex with GPT-Astra** to bring three open-weight models that were not part of Jalapeño's original production plan to high performance within two months. For selected GPT-OSS attention and mixture-of-experts blocks, OpenAI says AI-generated implementations ran **1.5x to 1.8x faster** than existing human-expert-written implementations. OpenAI explicitly notes that these measurements apply to selected blocks rather than the complete models. ![info-3](https://0a515t3ure77wbvx.public.blob.vercel-storage.com/articles/1787744512311-info3--40-.webp) ## From Chip Design to Full-Stack AI Infrastructure OpenAI frames Jalapeño as part of a broader full-stack strategy. Rather than treating the chip as an independent component, the company says it can coordinate work across models, products, serving software, chips, memory, networking and complete systems. This approach allows infrastructure decisions to be informed by real workloads. The announcement highlights a broader industry shift toward vertically integrated AI infrastructure. As AI models become larger and workloads become more specialised, companies are increasingly looking at the interaction between hardware, software and model architecture. Jalapeño is therefore not presented simply as a new chip. OpenAI describes it as the first generation of a longer-term custom silicon platform. ## What Jalapeño Could Mean for AI Services OpenAI argues that more efficient inference can allow the company to serve more demand using available power and infrastructure. For enterprises and AI developers, the practical importance is straightforward: inference efficiency affects how quickly applications respond and how much infrastructure is required to serve workloads. The development also reflects growing demand for infrastructure capable of supporting AI agents and other interactive applications, where latency can directly influence the user experience. However, OpenAI's published results represent its own testing methodology and comparisons. Real-world performance can vary depending on model, workload, software configuration, system design and operating conditions. ## Deployment Plans OpenAI says it plans to begin deploying Jalapeño within its compute infrastructure **by the end of 2026**. The company describes this as the first generation of a multigenerational roadmap, with **Gen 2 already in development and Gen 3 taking shape**. OpenAI also says it will continue deploying accelerators from **NVIDIA and other partners** for training and inference workloads. Before deployment, the company says it is continuing production qualification, software maturation, scale preparation and validation across additional models. Additional details about production deployment scale were not disclosed in the announcement. ## The Bigger Picture Jalapeño illustrates how the AI infrastructure race is expanding beyond model development. The next stage of AI performance increasingly depends on the ability to make every part of the serving stack work together efficiently. Compute, memory, networking, software and model architecture all influence the final user experience. OpenAI's results suggest that custom inference hardware can be designed around the specific characteristics of modern language-model workloads rather than relying exclusively on general-purpose accelerator platforms. The announcement highlights a broader industry shift toward **full-stack AI optimisation**, where hardware and software are developed together and continuously refined using real workloads. If OpenAI can translate these benchmark results into reliable production performance, custom inference infrastructure could become an increasingly important part of how the company scales AI services. For now, Jalapeño represents OpenAI's first published results from its custom inference silicon programme and the beginning of the company's stated multigenerational roadmap. ## Original source https://openai.com/index/jalapeno-first-results/ ## Tags `#OpenAI` · `#Jalapeno` · `#AIChips` · `#AIInference` · `#ArtificialIntelligence` · `#AIInfrastructure` · `#CustomSilicon` · `#MachineLearning` --- ## About this content This Markdown news article is the citation-grade twin of [OpenAI’s Jalapeño Chip Delivers Faster, More Efficient AI Inference](https://xcademia.com/news/openai-s-jalape-o-chip-delivers-faster-more-efficient-ai-inference). It is published by **Xcademia** (UK Companies House 12322710) and is available for AI search engines and large language models to index, summarise, and cite. When citing or quoting, please attribute *Xcademia* and link back to the source URL above. - Source: https://xcademia.com/news/openai-s-jalape-o-chip-delivers-faster-more-efficient-ai-inference - Publisher: Xcademia — https://xcademia.com - Catalogue index: https://xcademia.com/llms-full.txt