Handling Large Blobs

The moment your system handles user photos, video, documents or backups, a specific pattern applies. Large binary objects ("blobs") behave nothing like rows: they're big, they're read far more than written, and they shouldn't travel through your application servers at all. Getting this wrong — storing files in the database, or proxying every byte through your API — caps your scale early.

Rule one: blobs go in object storage, not the database

Store files in object storage (S3, R2, GCS), and store only metadata in your database — a row with the file's id, owner, size, type, and the storage key:

DB row:  { id, owner_id, filename, content_type, size, storage_key, created_at }
S3:      the actual bytes, at storage_key

Why not the database? Blobs bloat it, slow backups, blow out row sizes, and waste expensive database storage on data the DB can't meaningfully query. Object storage is cheap, effectively infinite, and built exactly for this.

Rule two: upload and download direct, with presigned URLs

Don't route the bytes through your servers — that makes your API the bottleneck and burns bandwidth. Instead, let the client talk to object storage directly, authorised by a short-lived presigned URL your server issues:

upload:   client → API "I want to upload" → API returns a presigned PUT URL
          client → PUT bytes directly to S3 → tells API "done"
download: client → API "give me file X" → API checks permission,
          returns a presigned GET URL → client reads directly from S3 (or CDN)

Your server only ever handles tiny JSON (the request and the URL); the gigabytes flow straight between the client and storage. The presigned URL carries time-limited, scoped permission, so the bucket itself stays private.

Rule three: serve downloads through a CDN

Reads of public or frequently-accessed files should come from a CDN edge, not from object storage on every request — faster for users and cheaper on egress. For private files, issue a fresh presigned URL (short expiry) each time the page renders; never store a signed URL, since it's a durable key to a private object.

Handle big uploads properly

• Multipart / chunked upload for large files, so a dropped connection doesn't restart from zero and you can upload parts in parallel.
• Validate after upload (type, size, a virus scan) in a background job — triggered by a storage event — rather than blocking the user.
• Process asynchronously: transcoding video, generating thumbnails, extracting text all happen in workers after the upload lands, not in the request.

Video is its own beast

Streaming video adds a layer: you don't send one giant file, you transcode it into multiple bitrates and split each into small segments, then let the player adaptively pick a quality based on bandwidth (HLS/DASH). The segments live in object storage and serve through a CDN. The database still only holds metadata and the manifest location.

What to say in an interview

Keep bytes out of your DB and off your servers:

"Photos go to object storage; the database only holds metadata and the storage key. Uploads use a presigned PUT so the client sends bytes straight to S3 — my API never touches them. Downloads go through a CDN for public images, or a short-lived presigned GET for private ones. Thumbnailing happens in a worker triggered by the upload event, not in the request."

Next up: Managing Long-Running Tasks — work that outlives a single request.