a16z Podcast• Nov 28, 2025• 53:22Interview

How OpenAI Builds for 800 Million Weekly Users: Model Specialization and Fine-Tuning

From a16z Podcast

Sherman Wu•Lead, Engineering Team for OpenAI's Developer Platform

Executive Summary

OpenAI is pursuing a dual-pronged strategy, simultaneously scaling its first-party application, ChatGPT (with 800M weekly users), and its horizontal developer API platform.
The market is shifting away from a "one model to rule them all" paradigm towards a proliferation of specialized models.
OpenAI is enabling this through advanced features like reinforcement fine-tuning (RFT).
Reinforcement fine-tuning allows customers to achieve state-of-the-art performance on specific tasks using their proprietary data, with OpenAI offering discounted inference and free training in exchange for data sharing.
The concept of prompt engineering has evolved into "context engineering," focusing on providing models with the right tools, data, and structured logic to handle complex, real-world tasks, especially in enterprise settings.

12 quotes

Concerns Raised

Current models are not yet reliable enough for fully autonomous, unconstrained agentic behavior.
Inference at scale remains an extremely hard engineering and capital problem.

Opportunities Identified

Leveraging reinforcement fine-tuning to create state-of-the-art specialized models for customers.
Expanding on-premise deployments for government and high-security clients.
High developer retention on the API indicates a strong and sticky platform business.
Tapping into vast, unused enterprise data troves to create highly valuable AI applications.

Key Themes

Platform vs. Product Strategy

OpenAI strategically balances the development of its horizontal API platform for developers with its vertical, first-party ChatGPT application. This dual approach allows them to capture both the broad developer ecosystem and the mass consumer market, creating a powerful flywheel.

This strategy demonstrates a comprehensive approach to market capture, securing a foundational role in the AI ecosystem while also building a direct-to-consumer brand with massive reach.

The Rise of Specialized Models

The initial industry belief in a single, all-powerful general model has been replaced by the reality of a diverse ecosystem of specialized models. OpenAI is adapting by providing tools that allow developers and enterprises to fine-tune models for specific, high-value tasks.

This shift signifies a maturation of the AI market, creating opportunities for niche applications and vertically-focused AI companies to build state-of-the-art solutions on top of foundation models.

Data as a Strategic Asset and Incentive

OpenAI is creating a powerful data feedback loop with its reinforcement fine-tuning API. By offering significant cost incentives (discounted inference, free training) for customers who share their data, OpenAI can improve its core models while helping customers build superior, specialized ones.

This business model aligns incentives, turning customer data into a mutually beneficial asset that enhances both the customer's application performance and OpenAI's foundational model capabilities.

Evolution of Prompting to Context Engineering

Contrary to early predictions, prompt engineering has not become obsolete; it has evolved into a more sophisticated discipline called "context engineering." The focus is now less on simple instruction-following and more on providing models with the right tools, retrieved data (RAG), and structured logic to reason and act effectively.

This highlights that building effective AI applications requires more than just a powerful model; it demands sophisticated engineering of the model's operating environment and inputs to achieve reliable and valuable outcomes.

Enterprise AI and the Need for Control

For enterprise adoption, especially in regulated or procedural domains like customer support, AI agents require a high degree of reliability and control. OpenAI's approach, such as the deterministic, node-based Agent Builder, reflects the need to constrain model behavior with explicit code and logic rather than relying solely on unpredictable natural language.

This underscores the critical challenge of bridging the gap between the probabilistic nature of LLMs and the deterministic needs of business processes, a key hurdle for widespread enterprise AI integration.

Get started free

Topics

OpenAI API Strategy Developer Platform ChatGPT Large Language Models (LLMs)Model Specialization Fine-Tuning Reinforcement Learning (RL)Reinforcement Fine-Tuning (RFT)Prompt Engineering Context Engineering Retrieval-Augmented Generation (RAG)AI Agents Enterprise AI On-premise Deployment Government Contracts Unit Economics

Processed Apr 6, 2026 yt-dlp + mlx-whisper + Gemini