Video Streaming

A video streaming platform lets creators upload video and lets viewers watch it smoothly on any device and connection. It splits cleanly into two very different problems: a heavy, async processing pipeline that prepares uploaded video, and a massively scaled delivery path that streams it. The delivery side is almost entirely about CDNs and large blobs; the metadata side is an ordinary read-heavy service.

1. Requirements

Functional:

• Creators upload videos; the system processes them for streaming.
• Viewers stream video smoothly, adapting to their connection quality.
• Browse/search videos and see metadata (title, views, etc.).

Non-functional:

• Smooth playback — minimal buffering, fast start, adapts to bandwidth.
• Massive scale — millions of concurrent viewers; video is enormous (bandwidth and storage dominate).
• Global — viewers everywhere get low latency.
• Upload-to-watchable can take minutes (async); watching must be instant.

2. Upload and the processing pipeline

A raw upload is useless for streaming as-is. Uploaded bytes go straight to blob storage via presigned, resumable URLs (videos are huge — never through app servers). That triggers an async pipeline, a long-running task off a queue:

upload → blob store → enqueue "process video" →
  transcode into multiple resolutions/bitrates (1080p, 720p, 480p, ...)
  segment each into small chunks (a few seconds each)
  generate the manifest (HLS/DASH playlist)
  store segments + manifest in blob storage → mark video ready

Transcoding is CPU-heavy and parallelisable — fan it out across a worker fleet, often per-segment, so a long video processes quickly.

3. Adaptive bitrate: the key delivery idea

The reason video streams smoothly across connections is adaptive bitrate streaming (HLS or DASH). Each video is encoded at several qualities, and every quality is cut into small time-segments (e.g. 4-second .ts/.m4s chunks). A manifest lists the available qualities and their segment URLs.

The player drives it: it downloads segments one at a time and, based on measured bandwidth, picks the next segment's quality — stepping down to 480p when the network slows, back up to 1080p when it recovers. The server just serves static segment files; the intelligence lives in the client.

manifest ─► player picks quality per segment ─► fetch segment ─► play ─► repeat
          (bandwidth drops → request lower-quality next segment)

4. Delivery: the CDN does the heavy lifting

Serving video from your origin to millions is impossible and pointless. Push all segments to a CDN so they're cached at edge locations close to viewers (Handling Large Blobs). A viewer fetches the manifest, then pulls segments from the nearest edge. Because segments are immutable static files, they cache beautifully — the origin is barely touched. This is the single most important scaling decision on the watch path.

5. Metadata side

Separate from the bytes, there's an ordinary service for titles, descriptions, view counts, comments, and search — read-heavy, so cache it and index search in Elasticsearch. View counts are a high-write aggregation: increment asynchronously, not synchronously per play.

6. Deep dives

• Thumbnails / previews: generated in the same pipeline.
• Resumable uploads: chunked upload survives a dropped connection mid-upload (creators on flaky networks).
• DRM / access control: signed segment URLs or token-gated manifests for paid content.
• Live streaming: same ABR idea but the pipeline runs in near-real-time with a short segment buffer, trading latency for smoothness.
• Cost: storing every video at many bitrates is expensive — tier cold videos to cheaper storage, transcode popular formats eagerly and rare ones lazily.

What to emphasise

Separate processing from delivery, and make ABR + CDN the stars:

"I'd split this into an async processing pipeline and a CDN-based delivery path. Uploads go straight to blob storage via resumable presigned URLs, triggering a fan-out transcoding job that produces several bitrates, cuts each into short segments, and writes a HLS/DASH manifest. Playback is adaptive-bitrate: the player picks each segment's quality based on measured bandwidth, so it degrades gracefully instead of buffering. All segments are immutable static files served from a CDN edge near the viewer, so the origin barely gets touched — that's what makes millions of concurrent viewers feasible. Metadata and view counts are a separate read-heavy, async-aggregated service."

Next up: Top-K Trending — the most-frequent-items problem at scale.