Latency, throughput, and bottlenecks

Latency is time per operation, throughput is completed work per time, and a bottleneck is the constrained resource that currently limits the system. Improving one metric can worsen another.

At a glance

Question	Practical answer
When is it useful?	In a small real-world scenario, draw the parts involved, follow one request or decision from start to finish, and mark the evidence produced at each step.
What should you do?	Time 100 small operations at increasing concurrency, graph median and p95 latency plus throughput, and identify the first saturated resource.
How do you know it worked?	The result can be repeated from your notes, and each important claim is supported by an output, measurement, query result, or reviewable artifact.
Common failure	A common mistake is choosing a tool or pattern before stating the problem, constraints, and success measure.

flowchart LR
  A[Question] --> B[Latency, throughput, and bottlenecks]
  B --> C[Small example]
  C --> D[Evidence]

The important idea is not to stop at a definition: connect the concept to a small example and observable evidence.

Worked example

In a small real-world scenario, draw the parts involved, follow one request or decision from start to finish, and mark the evidence produced at each step.

Before acting, write the success signal. Change one condition at a time, observe the result, and record assumptions. For Latency, throughput, and bottlenecks, this separates what you know from what you are merely guessing.

Practice in 20–30 minutes

Goal: Time 100 small operations at increasing concurrency, graph median and p95 latency plus throughput, and identify the first saturated resource.

1. Record the starting state and your prediction.
2. Implement the smallest version without adding unnecessary tools.
3. Change exactly one input or constraint and repeat.
4. Save a command, screenshot, output, or checklist as evidence.

Expected result: The result can be repeated from your notes, and each important claim is supported by an output, measurement, query result, or reviewable artifact.

What can go wrong

A common mistake is choosing a tool or pattern before stating the problem, constraints, and success measure.

When the result differs from your prediction, do not change many things at once. Check inputs, versions, environment, permissions, and logs, then repeat from the smallest example.

Definition of done

• I can explain the concept in my own words.
• I completed the small example and kept evidence.
• I know one failure mode and how to check it.
• Someone else can repeat the work without guessing missing steps.

Go deeper

Use the linked resource or repository at the end of the page when you need a full implementation. Check current versions before applying commands to a real project.