← All chaptersSecond Book (Draft)

Chapter 02 · 11 min read

Inference Economics

Why serving a model, not training it, is where the cost, the latency, and most AI engineering careers actually live.

CHAPTER 02/2,549 words/Drafting

The first book took the model as given and began after the weights froze. Chapter 1 broke that assumption on the training side: the model is increasingly something you build, not something you receive. This chapter breaks it on the other side. Once you have a model — trained, tuned, and ready — you still have to run it. Running it is where the money, the latency, and the hard constraints actually live.

That is the subject of inference economics. Training is a capital cost you pay once. Inference is a recurring cost you pay on every request, for as long as the product lives. It is also where the user waits, where the bill compounds, and where a good model can still fail because the system around it is slow or expensive. The center of the AI business is not the training run. It is the served token.

Alex Atallah built OpenRouter into a marketplace that routes traffic across dozens of model providers, so he watches the whole flow from the middle of it. He states the scale of it plainly: "Inference might be the largest market ever in software." (Alex Atallah, OpenRouter) The whole open-model ecosystem — the serving frameworks, the inference clouds, the on-device runtimes this chapter surveys — exists because serving is where recurring cost and competitive advantage live. If the model is the product, inference is the cost of goods sold.

This chapter runs in two movements. The first is serving performance: how you make a given model fast and cheap — the software layer of engines, compilers, quantization, and on-device runtimes that you control. The second is the compute substrate: the GPUs, networks, data centers, and economics underneath, the floor you cannot move, up to and including geopolitics. Movement A is the frontier you can push. Movement B is the ceiling that pushes back. Together they decide what a trained model can actually do once real users arrive. Chapter 1 fed this one — reinforcement learning produces models you then have to serve. Chapter 3 asks who can afford to build at the frontier. Chapter 4 meets the hardest latency budgets of all, in voice and media.

Movement A — Serving Performance

Speed and cost are a system property, not a model property

A model's latency and cost are not printed on the checkpoint. They are set by the system you run it on.

Kyle Kranen ran one of NVIDIA's largest inference deployments — tens of millions of dollars a quarter — and his central point is that the model alone tells you almost nothing about how it will perform in production. "A good model and a good system that takes into account the actual constraints for what you need from your deployment is actually key to the success of both your deployment and the application." (Kyle Kranen, NVIDIA) The same weights, served two ways, differ by an order of magnitude in throughput and price. The variable that moves is the deployment, not the checkpoint. Kranen frames it as an inference Pareto frontier — the curve trading latency against cost against quality — and the engineering work is knowing where on that curve your application needs to sit, then building the system that puts you there.

Mark Moyou, an NVIDIA architect who works this problem with enterprise clients, gives the same warning from the cost side: LLM inference is not a normal model deployment, and it is not trivial once you have to manage scale, performance, and cost at once (Mark Moyou, NVIDIA). The naive path — take the model, wrap it in a simple server, ship it — leaves most of the performance and most of the savings on the table. Dmytro Dzhulgakov of Fireworks puts a figure on the gap: making open models roughly ten times faster is a serving problem, solved with secure, latency-sensitive, low-cost serving rather than a new model (Dmytro Dzhulgakov, Fireworks). The lesson under all three is one : if you want a model to be fast and cheap, you optimize the system, not the model.

The serving stack is the lever

If the system sets the cost, the serving framework is the first thing you reach for.

A serving engine is the software that turns model weights into a running API endpoint — batching requests, managing the key-value cache, and squeezing the hardware. The open ecosystem has matured fast here. Charles Frye of Modal spent a year measuring how fast open inference engines actually are, and his framing captures the shift: open-weights models and open-source inference servers have made major strides, to the point where running a serious agent on open models is now a real option rather than a compromise (Charles Frye, Modal). SGLang, an open-source serving framework, aims to serve models like DeepSeek and Qwen at state-of-the-art speeds on launch day (SGLang, Baseten). TensorRT-LLM sits at the high-performance end — the fastest framework in Baseten's account, with a steep learning curve as the price of that speed (TensorRT-LLM, Baseten).

The point is not which framework wins. It is that the framework is now a design decision with a large payoff, and the payoff is open to anyone. Dzhulgakov's whole pitch at Fireworks is productionizing open models — taking a checkpoint anyone can download and serving it faster and cheaper than the naive path allows (Dmytro Dzhulgakov, Fireworks). A team that treats serving as an afterthought pays the order-of-magnitude penalty Kranen described. A team that treats it as a lever gets frontier-adjacent economics out of open weights.

Compilers and generated kernels are automating the expert layer

The deepest performance work used to require a scarce specialist. Increasingly, it is done by search and by models.

A kernel is the low-level code that runs one operation on the GPU. Hand-writing fast kernels is expert work, and experts are rare. Two shifts are removing that bottleneck. The first is search-based compilation: Joe Fioti's Luminal is a deep-learning compiler for CPUs, GPUs, and ASICs that discovers efficient kernels — flash attention among them — automatically, by search rather than by hand (Joe Fioti, Luminal). His theme is radical simplification through search: a far simpler library that gives up nothing on performance, because the compiler finds the fast path. The second shift is generation: Natalie Serrino's Gimlet Labs works on AI-generated kernels that can meaningfully speed up custom code with no human effort at all (Natalie Serrino, Gimlet Labs). The optimization that used to gate a deployment behind a specialist is becoming something a compiler or a model produces on demand.

Portability is the other half of this. Yusuf Olokoba of Muna frames the practical question as one of reach: "How do I use more models in more places without having to rebuild or extend?" (Yusuf Olokoba, Muna) Chris Lattner's MAX at Modular attacks the same seam — real applications span GPU compute, CPU preprocessing, and orchestration, often across different vendors, and the productivity win is a stack that spans them without a rewrite (Chris Lattner, Modular). The direction is consistent: the expert layer of inference optimization is being automated and abstracted, which widens the set of people who can deploy a model well.

Inference is moving onto the device

For a growing class of models, the cheapest and most private data center is the one already in your pocket.

The hardware crossed a threshold. Adrien Grondin runs Gemma 4 on an iPhone at 40 tokens per second through Apple's MLX framework — a capable open model, on a phone, fast enough to use (Adrien Grondin, Locally AI). Prince Canuma, a core MLX contributor, makes the economic case directly: "I want to convince you today to offload some of that subscription completely on device and then all you need to pay is your energy bill." (Prince Canuma, Neywa Labs) He backs it with working demos on consumer Macs — vision, speech, even a million-token context enabled by his own cache-compression work — and with a reason that is not about cost at all. His father went blind in rural Africa, where cloud subscriptions are unaffordable and connectivity is unreliable; for that user, on-device is not a preference but the only option. On-device inference changes who can reach AI, not just what it costs.

The pattern is not confined to Apple silicon. Mozilla's Llamafile, from Stephen Hood and Justine Tunney, makes open models run fast on ordinary consumer CPUs, with the explicit goal of democratizing access (Stephen Hood & Justine Tunney, Llamafile). Microsoft's Foundry Local, presented by Emma Ning, brings the same capability to cross-platform apps through the ONNX Runtime, and her first argument for it is the conference-Wi-Fi problem — cloud AI fails the moment the network does (Emma Ning, Microsoft). A per-month subscription becomes a one-time energy cost. A round trip to a data center becomes a local call. The move from demo to default is underway.

The tightest budget is real time

Some workloads have no slack. When a human is waiting to hear a reply, the latency budget is a few hundred milliseconds, and every part of the serving stack has to fit inside it.

Neil Dwyer of Gabber ran realtime text-to-speech in production, and reports the discipline it takes to serve voice at roughly a dollar an hour — open-source models, LoRAs, and careful load balancing, all bent toward latency (Neil Dwyer, Gabber). Voice is where serving performance stops being an optimization and becomes a hard gate: miss the budget and the product does not work at all. These realtime economics are the seam where this chapter meets Chapter 4, which takes up voice and media models on their own terms. Here the point is narrower and belongs to inference economics: the tighter the latency budget, the more the system — not the model — decides whether the product is possible.

Movement B — The Compute Substrate

Underneath the stack is a substrate that sets the ceiling

Every serving trick in Movement A runs on physical hardware, and that hardware sets a ceiling no software can lift.

For years most AI engineers built above an API boundary — the OpenAI call, the Anthropic call — and never touched the metal. Charles Frye argues that boundary is no longer enough: AI's extreme demand for mathematical throughput means engineers now need to understand the GPU underneath — its processors, its memory, its limits — the way every programmer once had to learn a little about hardware (Charles Frye, Modal). The substrate is not a footnote to the serving stack. It is the thing the serving stack is fighting against.

And the substrate is more than chips. Paul Gilbert of Arista Networks describes the full picture as infrastructure that must be built and wired together: "What we do is a plumbing… how you train models, what the infrastructure looks like, and how you do inferencing on the infrastructure." (Paul Gilbert, Arista Networks) Building an AI data center means power, cooling, networking, and the fabric that moves data between GPUs fast enough to keep them busy. That plumbing is now a first-order engineering discipline, because it sets the ceiling the software in Movement A is trying to reach.

The substrate is contested — in silicon and in geopolitics

Because the substrate sets the ceiling, whoever changes the substrate changes what is possible — which makes it a place of intense competition.

New silicon is one front. Sunny Madra of Groq frames his company's specialized inference chip against a historical marker: "Twenty-five years ago we crossed the 1 gigahertz speed barrier in microprocessors." (Sunny Madra, Groq) The analogy is deliberate. A step change in raw inference speed does not just make existing products faster; it unlocks applications that were impossible at the old speed, the way the gigahertz threshold once did. Purpose-built inference hardware is a bet that the GPU is not the last word on the substrate.

The other front is national. Dylan Patel of SemiAnalysis argues that the infrastructure behind AI — chips, data centers, power, supply chains — has become an arena for geopolitical competition, with China, the United States, and the Middle East each maneuvering, and players like Huawei closing the chip gap by engineering around export controls (Dylan Patel, SemiAnalysis). This is where inference economics stops being a purely technical discipline. The cost and availability of the substrate now depend on trade policy, fabrication capacity, and national strategy. The ceiling on what any serving stack can do is, in part, a political variable — a strange and important thing to have to say about the cost of running a model.

Trust is becoming a first-class inference requirement

For a large class of buyers, a model is unusable until they can run it on sensitive data without surrendering the data or the result. That is an inference problem, and it has become a product category.

Mike Bursell names the blocker directly: "AI is transforming everything. Healthcare, finance, automation, digital marketing. But one thing holds it back, and that's trust. How do you run models on sensitive data without handing it over?" (Mike Bursell) His answer is confidential computing — running inference inside a hardware-protected environment so the data stays private even from the operator of the machine. The demand is real, because the alternative — shipping regulated data to someone else's GPU — is a non-starter for the industries with the most to spend.

Sahil Yadav and Hariharan Ganesan of Telemetrak reach trust from the enterprise side, after a decade deploying AI in healthcare, industrial IoT, and telecom. Their recurring question is the one a CIO actually asks: can we trust the model's decisions when they drive mission-critical outcomes and the bottom line? (Yadav & Ganesan, Telemetrak) Between the two talks, trust splits into two requirements inference must now meet: confidentiality — the data is safe — and reliability — the decision can be relied upon. Neither is a property of the model in isolation. Both are properties of how and where it is served. Trust has moved from an afterthought to a design input, alongside speed and cost.

Inference is the discipline, not the afterthought

Put the two movements together and the chapter's is simple. A model's cost and speed profile is a design input as fundamental as its accuracy. You set that profile in the serving stack — the engines, compilers, quantization, and on-device runtimes of Movement A — and you set it against a substrate you do not fully control — the GPUs, networks, power, and politics of Movement B.

This is why inference, not training, is the discipline most AI engineers will spend their careers inside. Most engineers will never run a pre-training job. Nearly all of them will serve a model, and the moment they do, they inherit the whole problem: the Pareto frontier, the framework, the kernel, the device, the chip, the data center, the trust boundary. Atallah's line was not hyperbole. If inference is the largest market in software, it is because it is the part of AI that never stops charging — and the part where good engineering compounds on every request. The next chapter turns to who can afford to build at the frontier of that market at all.

Evidence — Source Anchors

Reinforcement learning is the post-training method that carries a model into production

  • reinforcement learning RL is not just any other algorithm for post-training, but is an an algorithm that at its core will bring models to production.
    #674 — Alessandro Cappelli, Adaptive MLconfidence: high
  • it's not about powering them up with the next biggest LLM in the market. All they need is simple data flywheels.
    #397 — Sylendran Arunagiri, NVIDIAconfidence: high

RL's real cost is operational, not conceptual

  • requires orchestrating not one, but four large language models at the same time.
    #674 — Alessandro Cappelli, Adaptive MLconfidence: high
  • you have to keep four copies of uh different models.
    #251 — Aakanksha Chowdhery, Reflection.aiconfidence: high
  • our rail optimized infin band cluster networking
    #475 — Crusoeconfidence: high
  • today we're going to talk about the deep dive into RL kernels, agents, and quantization.
    #230 — Daniel Hanconfidence: high

Inference, not training, may be the largest market in AI software

  • inference might be the largest market ever in software
    #335 — Alex Atallah, OpenRouterconfidence: high
  • possible with Open Weights models where they weren't before.
    #316 — Charles Frye, Modalconfidence: high

A model's speed and cost are a system property, not a model property

  • a good model and a good system that takes into account the actual constraints for what you need from your deployment is actually key to the success of both your deployment and the application
    #141 — Kyle Kranen, NVIDIAconfidence: high
  • controlling the cost of a deployment cuz that's often times the thing that's going to really prevent you from taking this taking this to any meaningful scale
    #514 — Mark Moyou, NVIDIAconfidence: high
  • quality while being you know up to 10 times faster
    #548 — Dmytro Dzhulgakov, Fireworksconfidence: high

On-device inference is crossing from demo to default

  • I want to convince you today to offload some of that subscription completely on device and then all you need to pay is your energy bill.
    #670 — Prince Canuma, Neywa Labsconfidence: high
  • it can run easily at 40 token 40 token per second.
    #007 — Adrien Grondin, Locally AIconfidence: high
  • LL file is an open source project from Mozilla that has the goal of democratizing access to AI
    #582 — Stephen Hood & Justine Tunney, Llamafileconfidence: high
  • how does cloud AI work in environments with low network bandwidth or even offline access?
    #307 — Emma Ning, Microsoftconfidence: high

Compilers and generated kernels are automating the expert layer of inference optimization

  • the fastest one in this case just happens to be flash attention.
    #388 — Joe Fioti, Luminalconfidence: high
  • this result achieved a 40% speed up over the baseline on the M4.
    #068 — Natalie Serrino, Gimlet Labsconfidence: high
  • How do I use more models in more places without having to rebuild or extend
    #097 — Yusuf Olokoba, Munaconfidence: high
  • max is two things one is an AI framework
    #578 — Chris Lattner, Modularconfidence: high

The compute substrate is a hard constraint — and now a geopolitical variable

  • they optimize for math bandwidth over memory bandwidth.
    #228 — Charles Frye, Modalconfidence: high
  • what we do is a plumbing uh so I'm not going to talk about agents but more kind of how you train uh models what the infrastructure looks like and how you do inferencing on on the infrastructure
    #402 — Paul Gilbert, Arista Networksconfidence: high
  • what's interesting about the geopolitics of this is that despite the fact that Huawei is a sanctioned entity, China is a sanctioned country,
    #345 — Dylan Patel, SemiAnalysisconfidence: high
  • 25 years ago we crossed the 1 gigahertz speed barrier uh in
    #547 — Sunny Madra, Groqconfidence: medium

Trust and confidentiality are becoming first-class inference requirements

  • AI is transforming everything. Healthcare, finance, automation, digital marketing. But one thing holds it back, and that's trust. How do you run models on sensitive data without handing it over?
    #361 — Mike Bursellconfidence: high
  • over the past 10 years we have deployed AI in various industries from from health um monitoring to industrial IoT to uh network automation in telecom networks
    #211 — Sahil Yadav & Hariharan Ganesan, Telemetrakconfidence: high