# System Design Drill Day 17: File Uploads and Image Pipelines

A merchant on a slow mobile connection uploads a 4 MB product photo, and one PHP-FPM worker waits the whole time it trickles in. When enough merchants do that at once, the pool runs out of workers and every other request waits.

Workers aren't the only problem. A file may not be what it claims to be, and a CDN can keep serving an old photo long after the merchant replaced it. The pipeline here handles an online marketplace's merchant product photos and private KYC documents.

## Core concepts

### Keep the bytes off your app servers

A slow upload ties up a PHP-FPM worker for seconds, and saving to `storage/app` breaks as soon as there's more than one server behind the load balancer (Day 5). Files belong in object storage such as S3, Cloudflare R2, DigitalOcean Spaces or MinIO. MySQL keeps only a metadata row with the key, size, mime type, status and checksum.

### Presigned URLs

The server signs a short-lived URL that allows uploading to one exact key with a fixed content type, and the client uploads straight to the bucket with it. A presigned PUT, which is what Laravel's `temporaryUploadUrl()` gives you, can't enforce a size range. If you need the bucket itself to reject oversized files, use a presigned POST policy with a `content-length-range` condition; otherwise the HEAD size check after upload is what enforces the limit. Very large files use multipart upload, which is chunked and can resume after a dropped connection.

![Sequence diagram of a direct upload: the phone asks the Laravel API for an upload, gets a presigned URL, uploads straight to object storage, calls complete, and the server verifies the object and queues processing](https://cdn.hashnode.com/uploads/covers/5fdde6221485f22e24b34a2c/2930aa6b-1d3f-407e-a7f5-f84dfe1d29c1.png align="center")

*Figure 1: The bytes go phone to bucket. Laravel only signs, verifies and queues.*

### Treat an upload as a state machine

An upload moves through `pending`, `uploaded` and `processing`, and ends as `ready` or `failed`. The client calls a `/complete` endpoint, with a bucket event notification as a backup. The server confirms that the object exists with a HEAD request, checks its size, and reads the first bytes (the file's "magic number") to see what it really is. The Content-Type sent by the client is never trusted.

![State machine of an upload: pending, uploaded, processing, then ready or failed, with failed leading to a re-upload prompt](https://cdn.hashnode.com/uploads/covers/5fdde6221485f22e24b34a2c/c9028982-cef3-4662-aa65-3ae120f7af91.png align="center")

*Figure 2: Buyers only ever see images in the ready state.*

### Process in the background

A queued job produces the resized versions, for example 200, 600 and 1200 px wide, in WebP with a JPEG fallback. It also fixes rotation and strips EXIF metadata, which can include the phone's GPS location. Outputs go to fixed keys, so a retried job overwrites the same files. That makes the job idempotent: running it twice does no harm.

### Serving

A CDN sits in front of the bucket. When each key includes a hash of the content, the file behind a key never changes, so it can be cached for a year (`Cache-Control: max-age=31536000, immutable`). Changing a photo means writing a new key, and the CDN never needs a purge. Private files live in a private bucket without a CDN and are viewed through signed GET URLs that expire quickly.

> **Pause and think:** A merchant replaces a product photo. Why is "overwrite the same key" the wrong move?

**Answer:** The CDN (and browsers) have the old file cached under that key, possibly for a year. Buyers keep seeing the old photo until you purge it. Writing to a new content-hashed key means the new URL has never been cached, so it's live as soon as the product points to it.

## The design problem

Merchants upload product photos from their phones, about 20k uploads a day at around 4 MB each, and buyers load images about 30M times a day. Merchants also upload KYC documents that only admins may see.

Design the pipeline so that uploads never pass through PHP workers and buyers never see a broken or half-processed image. Replaced photos should go live quickly without CDN purges, KYC files must stay private, and abandoned uploads need cleaning up.

> **Try the problem before reading the solution.**

## Questions to think about

1. Why shouldn't a 4 MB photo pass through your Laravel app on its way to storage?
2. How do you make sure buyers never see a broken or half-processed image?
3. Where do KYC documents live, and how do admins view them?

## Model solution

### Upload, verify, process

The client asks the API for an upload. The server creates a `pending` row and returns a presigned URL for one exact key under a temporary prefix, `uploads/tmp/{uuid}`, and the client uploads directly to the bucket. Nothing under `uploads/tmp/` is ever served to buyers; processing writes the finished variants to their final keys.

Then the client calls `/complete`. The server HEADs the object, checks the size, reads the magic bytes and queues processing.

The processing job builds the resized WebP and JPEG variants, fixes rotation, strips EXIF, sets the row to `ready` and saves the variant URLs. The original moves to cold storage. Because the keys are fixed, a second run rewrites the same files. On a crash the job retries with backoff, and after the last retry it sets `failed`, which shows the merchant a re-upload prompt.

### Serving

The CDN sits in front of the public bucket, with files cached for a year as immutable. The product API returns the variant URLs, and `srcset` lets phones fetch the w600 WebP.

Replacing a photo creates a new row and a new key and clears the product's Redis cache, so the change is live within a minute without a CDN purge. The old files are deleted a few days later.

### Never broken

Buyer-facing queries include only images with status `ready`, and a product can't be published until it has at least one ready image. Meanwhile the merchant dashboard shows a local preview.

### KYC

KYC documents go in a separate private bucket with server-side encryption and no CDN. Admins view them through 5-minute `temporaryUrl` links, and every view is logged. In Laravel, `Storage::disk('kyc')->temporaryUrl()` generates those links.

### Cleanup

A lifecycle rule deletes `uploads/tmp/*` after one day, and a scheduled command marks `pending` rows older than 24 hours as abandoned.

### Numbers

Raw uploads come to 20k × 4 MB, about 80 GB a day, which is the traffic PHP workers should never carry. 30M image loads a day average about 350 per second, with peaks of perhaps 1.5k per second. At a CDN hit rate of 95% or more, the bucket sees fewer than about 75 requests per second.

> **Pause and think:** Why read the file's magic bytes when the client already sent a Content-Type header?

**Answer:** Because the client controls that header. Anyone can upload a script or an archive and label it `image/jpeg`. The first bytes of the file show what it actually is, and that's the only thing you should trust before processing or serving it.

### Common mistakes

- Storing files on local disk.
- Trusting the client's mime type.
- Overwriting the same key, which leaves stale copies in the CDN.
- Resizing inside the HTTP request.

### Answers to the questions

1. A slow phone upload holds a PHP-FPM worker for seconds, and 20k uploads a day at 4 MB is about 80 GB of traffic the workers would have to carry. Local disk also breaks once there's more than one server. With a presigned URL the phone uploads straight to the bucket, and Laravel only signs, verifies and queues.
2. Buyer-facing queries only return images whose status is `ready`, and a product can't be published until it has at least one ready image. Until then the merchant sees a local preview. If processing keeps failing, the row ends up `failed` and the merchant gets a re-upload prompt, so a half-made image is never shown.
3. In a separate private bucket with server-side encryption and no CDN. Admins open them through 5-minute signed links from `temporaryUrl`, and every view is logged.

## Interview one-liner

> "The client uploads straight to object storage with a presigned URL, a queue does the processing idempotently, and the CDN serves immutable, content-hashed keys. Private files go through a private bucket with short signed links."

## Conclusion

Two rules carry most of this design: file bytes never touch PHP workers, and buyers only ever see `ready` images. I'd also use content-hashed keys from the start, because a key that's cached for a year will keep showing the old photo if you overwrite it.

## Try it yourself

Use Laravel's `Storage::disk('s3')->temporaryUploadUrl()` (or MinIO locally) to give a test client a presigned upload URL, upload a file straight to the bucket with `curl` (the method returns both a `url` and a `headers` array, and the `curl` request must send those headers, or the signature won't match), then call a `/complete` route that checks the object's size and first bytes.

Try uploading a text file renamed to `.jpg` and confirm that the magic-byte check rejects it.

---

This is part of my daily [System Design Drills](https://blog.anishansari.com.np/series/system-design-drills) series.
