Module 6 of 16
Scale and performance
Why services get slow and how to make them fast again: queueing in pools and pipelines, caches, partitioning, load balancing against the tail, and profiling.
The capacity lesson in the first module estimated whether a design can carry its load. This module is about what happens once it does: where requests wait, which copies of the data absorb the reads, how the data and the traffic are spread over machines, and how to find the code that is actually slow. Each topic comes back to the same quantities, arrival rate, service time, and variability, because they decide latency long before the hardware runs out.
The five lessons cover queueing for pools and multi-stage services, caching and its failure modes, partitioning and rebalancing, load balancing and tail latency under fan-out, and a profiling workflow that starts from a symptom and ends with a measured fix. Every lesson has a short simulation whose output you can reproduce. By the end you should be able to say, for a slow service, which resource requests are queueing for, what would change the queue, and how to show the change worked.
Lessons
- 6.aQueueing and Little's lawWhere do requests wait inside a service, how many servers does a pool need, and why does variability matter as much as load?21 min
- 6.bCachingHow large a cache do you need, what does each point of hit ratio buy, and how do caches cause outages and stale data?21 min
- 6.cPartitioning and shardingHow do you split data across machines so that load stays even, queries stay local, and adding a node does not move everything?18 min
- 6.dLoad balancing and tail latencyHow should requests be spread over servers so that one slow server does not set everyone’s latency, and how do you cut the tail that remains?20 min
- 6.eProfiling a slow serviceWhen a service is slow, how do you find the code responsible and prove that a fix worked?16 min