Module 16 of 16
Worked AI designs
Five complete AI system designs, each argued from its invariants and sized with arithmetic: chat at scale, retrieval over private documents, a coding agent platform, a batch inference pipeline, and an evaluation platform.
The earlier modules took one part of an AI system at a time. This module puts the parts together. Each lesson designs one system end to end and follows the same route: requirements, the invariants that must hold, a capacity estimate with the arithmetic shown, the architecture, how one request or job moves through it, how it fails, a security review, what it costs, and the decisions taken with the alternatives that were rejected.
The five systems cover the main shapes of AI workload: an interactive chat service, a retrieval assistant over permissioned documents, a long-running agent platform, an offline batch pipeline, and the evaluation platform that gates changes to all of them. The numbers are assumptions stated up front, and every program runs offline, so you can change the inputs and watch the conclusions move.
Lessons
- 16.aA chat assistant at scaleHow do you size and build a chat service so that every turn streams quickly, history stays private, and the GPUs are spent on new tokens rather than recomputed ones?24 min
- 16.bRetrieval over private documentsHow do you build an assistant over private documents that never answers from a document the user cannot open, and stays current as documents and permissions change?20 min
- 16.cA coding agent platformHow do you build a platform that runs thousands of long coding tasks at once, survives crashes without repeating side effects, and keeps a hijacked agent from reaching anything beyond one pull request?22 min
- 16.dA batch inference pipelineHow do you run a model over tens of millions of records on interruptible capacity, finish before the deadline, and publish exactly one result per record?22 min
- 16.eAn evaluation platformHow do you build an evaluation service that many teams trust to gate releases, at a cost they can afford, when the graders are themselves models?19 min