System Design Drill Day 5: Load Balancing and Horizontal Scaling
How to go from one Laravel server to three behind a load balancer: algorithms, health checks, stateless servers and safe deploys.
You add a second app server before a big sale. Users start getting logged out at random, product images show up for some people and not others, and the daily report email arrives twice. The code didn't change; there's simply more than one box now.
Day 4 took read pressure off the database, and the app tier needs the same treatment. This drill takes an online marketplace from one Laravel server to three behind a load balancer and goes through what breaks on the way, which is mostly state.
Core concepts
When one server gets slow, the first instinct is to buy a bigger one. That's vertical scaling, and it works until you hit the ceiling or the bill. The alternative is several identical boxes with a load balancer (LB) in front spreading requests between them, which is horizontal scaling.
An L4 balancer routes on IP and port. It's fast but can't see inside the request. An L7 balancer understands HTTP, so it can route by path or header, terminate TLS and add headers like X-Forwarded-For. Nginx and HAProxy can run at either layer, and together with AWS ALB they can all run at L7, which is what a Laravel app normally sits behind.
Round robin, the default, hands requests to servers in turn. Least-connections picks the least busy server, which works better when request times vary a lot. IP-hash or consistent hashing keeps a client on the same server.
The LB also needs to know which servers can take traffic. It probes each one on a readiness endpoint, /health/ready from Day 1, and stops routing to any that return 503. A separate liveness check, /health/live, only answers "is the process up?" and is what a process supervisor uses to restart a dead process. Keep the two apart, and keep shared optional dependencies out of readiness, or one Redis blip marks every server unready at once.
The rule underneath all of this is that any server must be able to handle any request. Whatever lives on a single box, such as sessions, uploaded files, an in-memory cache or a cron job, breaks as soon as a second box appears. Sticky sessions, which pin each user to one server, hide the problem for a while, but load becomes uneven and users get logged out when their server dies. In an interview, the expected answer is to make the servers stateless.
The LB itself can also be a single point of failure, so run a redundant pair or use a managed one.
Pause and think: You add a second server and users start getting logged out at random. Sessions are already in Redis. What did you most likely forget?
Answer:
APP_KEY. If it's different on each server, a cookie encrypted by one box can't be decrypted by the other, so the session looks invalid.
The design problem
The marketplace runs on one Laravel server and a big sale is coming. Add an LB and go to 3 app servers, using the primary plus replicas from Day 4 and Redis. Draw the request path and list every change.
Afterwards, a logged-in user must stay logged in whichever server answers, and product images uploaded in the admin must show up for everyone. php artisan schedule:run must not send three copies of the daily report email, deploys must not drop requests mid-checkout, and losing one server mid-sale should barely be noticeable.
Try the problem before reading the solution.
Questions to think about
- Why is stateless better than sticky sessions, and when is stickiness acceptable?
- What should the LB health check test, and what goes wrong if it checks a shared dependency like Redis?
- Round robin or least-connections for this marketplace, and why?
Model solution
The request path ends at 3 identical Laravel servers, all talking to the MySQL primary and replicas, Redis, and object storage.
Figure 1: Three stateless app servers. Everything that used to live on one box now lives in a shared store.
Set SESSION_DRIVER=redis (or database) so all servers share one session store. APP_KEY must be identical on all three, or cookies encrypted on one box won't decrypt on another. With that done, stickiness isn't needed.
Uploads go to S3 or an S3-compatible store (FILESYSTEM_DISK=s3) behind a CDN. User files should never be written to storage/app on a single box, where only a third of requests would find them.
For the scheduler, add ->onOneServer(), which takes a lock in Redis so only one box runs each task, or run the scheduler on a dedicated worker box. Use ->withoutOverlapping() for long tasks. Queue workers can run anywhere, because Redis hands each job to one worker at a time, with redelivery if that worker fails (at-least-once), and the Day 3 idempotency checks cover the rare repeat.
Deploys roll one server at a time: take it out of the LB, deploy, warm it up, put it back, move to the next. Connection draining lets in-flight checkouts finish. Migrations must be backward-compatible (add columns first, drop them later), since old and new code run side by side during the roll.
Figure 2: A rolling deploy, one server at a time, with draining so no checkout gets cut off.
If a server dies mid-sale, failing readiness probes take it out of rotation within seconds. Plan capacity for N+1, so two servers can carry the peak if one dies, and make the LB redundant. Retries on idempotent requests, together with the checkout idempotency keys from Day 3, cover requests cut off mid-flight.
A few smaller details still matter. Configure TrustProxies so Laravel sees the real client IP and the HTTPS scheme. Keep cache and rate-limiter state in Redis so limits are shared, and ship logs to one central place instead of leaving them on each server.
Answers to the questions
- Stateless servers can come and go freely: load stays even, and a dead server doesn't log anyone out. I'd only accept stickiness for things like WebSockets, or a legacy app you can't change yet.
- The LB should route on readiness, which checks the dependencies this instance needs, with timeouts. Liveness stays cheap and is for restarts, not routing. Never fail every server because of one shared dependency. When every target is unhealthy, many LBs "fail open" and send traffic everywhere anyway, so you gained nothing.
- Least-connections, because checkout and search requests take very different amounts of time here. Round robin is fine when requests are roughly uniform.
Interview one-liner
"Make the app tier stateless: sessions, cache and rate limits in Redis, files in object storage, scheduled tasks on one server, then put a redundant L7 load balancer with separate liveness and readiness checks in front and roll deploys one server at a time."
Conclusion
Horizontal scaling only works once every piece of per-box state has a shared home, so find the state before adding servers. Watch the readiness check in particular: if it fails on a dependency all servers share, the load balancer can empty the whole pool at once.
Try it yourself
Run two copies of a Laravel app locally on different ports, put a small Nginx upstream with both in front, and log in. Refresh a few times; with sessions on the file driver, the logouts appear right away.
Switch to SESSION_DRIVER=redis with the same APP_KEY on both and try again. Then schedule a command with and without ->onOneServer() and count how many times it runs.
This is part of my daily System Design Drills series.
Comments
Join the discussionOne gotcha I'd add from a rollout like this: when you sync APP_KEY across the new boxes, every existing encrypted cookie dies at the same moment. We watched a chunk of users get bounced mid session during the cutover. If you're on Laravel 11, APP_PREVIOUS_KEYS lets you rotate instead of flipping the switch, so old cookies keep decrypting until they expire naturally. Worth calling out before the big sale, not after.
Good catch Kamran, a hard APP_KEY swap logging everyone out in the middle of a sale is a painful way to learn that. Rotating with APP_PREVIOUS_KEYS so old cookies keep decrypting is the much safer move.
One crash fixture I would add: a worker sends the report email, then dies before recording completion. A scheduler lock stops concurrent starts, but a retry can still repeat that external side effect. Keep the same report/run ID on retry and test the provider's duplicate handling, where supported. Where it cannot deduplicate, record that remaining risk. Would you include that crash point in the Day 3/Day 5 exercise?
Yes Arnold, that crash point is worth adding, since the scheduler lock only stops two runs starting together and does nothing about a retry repeating the send. Reusing the same run ID and checking it before sending covers most of it, and where the provider can't dedupe you just note the leftover risk.
The readiness-vs-liveness split is the detail most first-time setups get wrong: folding Redis into readiness means one cache blip pulls every server out of rotation at once. The duplicate report email is the other classic: a scheduled job needs a single owner the moment more than one box exists.
Thanks Abhinav, keeping Redis out of readiness is exactly what stops one cache blip from emptying the whole pool. And the duplicate report email is the bug that only shows up the day you add a second server 馃檪
