# System Design Drill Day 1: Latency, Throughput and Availability

Your load balancer says both app servers are healthy. Users say the site is slow. A teammate says Redis "blipped" for a few seconds and half the fleet got pulled out of rotation. All three can be true at the same time. That's why this series starts with latency, throughput and availability: people quote these three numbers constantly and mix them up nearly as often. I'll apply them to something small that every Laravel API has, or should have: the health endpoint.

First the three numbers, then a health check for a Laravel app with MySQL and Redis behind it. Sketch yours before you look at mine.

## Core concepts

Latency is how long one request waits for its answer, measured in milliseconds. Don't put much faith in the average. Look at p99, the slow tail where your unluckiest users end up.

Throughput counts something else: how many requests the whole system finishes per second (RPS). The two don't move together. A server can be slow per request and still get through a lot of them, or fast per request and fall over when traffic doubles.

Availability is the share of time the system actually worked for users, as a percentage. "For users" is the important bit. A green dashboard means nothing if nobody can log in.

The three pull against each other, and retries show it best. Add more of them and one client sees a "more available" system, while everyone else gets extra load and higher latency.

> **Pause and think:** Say you bolt automatic retries onto every client call. Which of the three numbers starts looking better, and which ones quietly get worse?

**Answer:** From that one client's point of view, availability improves, since a call that failed the first time often goes through on the second try. The cost lands on everyone else. Each retry is extra work for servers that were already struggling, so useful throughput (goodput) drops and latency goes up for the rest of the traffic. The raw request count actually rises, because the retries are requests too.

## The design problem

Your Laravel API depends on MySQL and Redis, roughly what a typical online marketplace runs, and it needs a `GET /health` endpoint (or `/status`, if you prefer that name). Sketch it. Boxes and arrows on paper are enough.

The sketch should show the components (app, DB, Redis, and probably a load balancer in front) and how a successful check flows through them. It should also cover three bad days: the DB is down, Redis is down, or the sneaky one, where the app is up but slow. Then decide what you send back, both the HTTP code and the shape of the body, and how all of it behaves with 2 app instances behind the load balancer.

Keep it at sketch level. You're not building a monitoring product.

> **Try the problem before reading the solution.**

## Questions to think about

1.  p99 latency doubles overnight but throughput stays flat. What usually broke first: capacity, a dependency, or the network path? How would you prove it?
    
2.  Redis is down, but some read-only routes still work straight from the DB. Should `/health` fail? It depends on who reads the answer. A load balancer and a human want different things.
    
3.  "Retry on timeout" sounds harmless. How can it make one client see better availability while useful throughput drops for everybody else?
    

## Model solution

### Sketch

The setup in plain text, if you want to copy it into your notes:

```plaintext
Client → LB → App1 / App2
                ├─→ MySQL (SELECT 1 / ping)
                └─→ Redis (PING)
```

The same thing as a diagram:

![Architecture of the health check: client, load balancer, two Laravel app instances, MySQL and Redis](https://cdn.hashnode.com/uploads/covers/5fdde6221485f22e24b34a2c/7d315346-7c6c-4f4f-9703-f23ca0045055.png align="center")

*Figure 1: Two app instances behind the load balancer, each checking MySQL and Redis on its own.*

When everything is healthy, the check should be boring. The probe hits one instance. That instance runs something trivial against MySQL, pings Redis and returns `200` with a small JSON body. Keep the checks under roughly 50 to 100 ms in total, and don't run migrations or heavy queries in there. A health check that's slow or expensive is just one more thing that can break.

### Body shape (example)

A healthy instance might send back something like this. Keep it small; nothing reads a long body from a probe.

```json
{
  "status": "ok",
  "checks": { "app": "ok", "mysql": "ok", "redis": "ok" },
  "instance": "app-1"
}
```

### Three useful variants

A single `/health` can't satisfy everyone. The process supervisor wants to know whether to restart, the load balancer wants to know whether to send traffic, and the person on call wants to know what's wrong. So split it.

`/health/live` is the liveness check. An orchestrator or process supervisor calls it to decide whether to restart a dead process, so it should be as dumb as possible: if the PHP process is alive, it returns `200`. Nothing more. Keep the DB and Redis out of it completely, otherwise a dependency outage gets every healthy process restarted for no reason.

`/health/ready` is the one the load balancer routes on, and it's stricter. The DB has to answer. Redis has to answer too, but only if *every* request depends on it. When the check fails, return `503` so the load balancer drains that instance. If some routes can run without Redis, keep Redis out of this check, because failing an instance over a dependency it doesn't strictly need just throws away capacity.

`/status` is for humans and dashboards, so it can say more. It's fine for it to return `200` with `"status": "degraded"` when Redis is down and MySQL is up. A person needs that detail. A load balancer doesn't.

![Decision tree showing which health endpoint each caller uses: live for the process supervisor, ready for the load balancer, status for humans](https://cdn.hashnode.com/uploads/covers/5fdde6221485f22e24b34a2c/02d63d64-0779-4cd0-aef1-7510fdd35e07.png align="center")

*Figure 2: One endpoint per audience: liveness for restarts, readiness for the load balancer, and a human-readable status.*

### Failure paths

| Case | Live | Ready | Ops body |
| --- | --- | --- | --- |
| DB down | 200 | 503 | `status: down`, mysql fail |
| Redis down | 200 | 200 or 503 (policy) | `degraded` if the API can limp along without cache |
| App hung / too slow | probe times out, supervisor restarts it | probe times out, LB drains it | n/a |

> **Pause and think:** Redis is gone, MySQL is fine, and most routes get by without the cache. What do `/health/live`, `/health/ready` and `/status` each return?

**Answer:** `/health/live` returns 200. The process is up, and that's the only thing it checks, so nothing gets restarted. `/health/ready` also returns 200, as long as not every request needs Redis, so the load balancer keeps sending traffic. That part is a policy call: go to 503 only if Redis really is mandatory. `/status` returns 200 too, but with `"status": "degraded"`, so anyone looking can see right away that Redis is the problem.

With a second instance in the picture, keep in mind that the load balancer only knows what the readiness check of the instance it just probed told it. If app-1 goes bad, it gets pulled and app-2 keeps serving. That part works as intended.

The trap is a check tied to a dependency both instances share. Redis blips for a few seconds, both readiness checks fail at the same moment, and the load balancer pulls the whole fleet. A small cache hiccup has become a full outage. I'd only put Redis in readiness if literally every request needs it.

### Answers to the questions

1.  **Most of the time it's a dependency or a hot path, not raw capacity.** Flat throughput with a fat p99 tail usually means something is waiting: queues backing up, lock waits, a slow query, a cold cache. I'd check the dependency latency histograms and the app's queue depth first. CPU graphs alone rarely settle this one.
    
2.  **Split the endpoints, as above.** The load balancer should see readiness fail only when a dependency makes *this particular instance* useless. A human should see "degraded". If you fail readiness because Redis is down while reads still work, you're cutting capacity for nothing.
    
3.  **Retries multiply the work.** Say one client retries a timeout 3 times. One slow request has turned into 4. The extra load pushes more requests past the timeout, those get retried as well, and *useful* throughput (goodput) drops, even though the total request count goes up and that one client eventually gets its answer.
    

![Sequence diagram of one client retrying a timed-out request three times, turning one slow request into four queries](https://cdn.hashnode.com/uploads/covers/5fdde6221485f22e24b34a2c/856d7133-af0b-4507-a312-219bcaac5541.png align="center")

*Figure 3: How "retry on timeout" turns one slow request into four.*

## Interview one-liner

> "Latency, throughput and availability trade off against each other. I split health checks into a cheap liveness probe, a readiness check that only fails on dependencies that make this instance useless, and a detailed ops status that can report degraded."

## Conclusion

Latency, throughput and availability answer different questions, and improving one can quietly cost you another, with retries as the usual culprit. In a health check, the mistake I'd guard against is wiring a shared dependency like Redis into readiness when not every request needs it. That's how a few seconds of cache trouble takes the whole fleet down.

## Try it yourself

Take any Laravel project you have (a fresh `laravel new` is fine) and add two routes. `/health/live` just returns `200`. `/health/ready` runs `DB::select('select 1')` and `Redis::ping()` with short timeouts and returns `503` if the DB check fails. Neither call takes a timeout argument, so set the timeouts in `config/database.php`: the connection options for MySQL, and `timeout` and `read_timeout` for Redis.

Then stop Redis locally and hit both. Did you get what the table says? If not, which Redis policy did you end up with, and why?

* * *

This is part of my daily [System Design Drills](https://blog.anishansari.com.np/series/system-design-drills) series.