Networking Essentials

Almost every system you design is a set of machines talking to each other over a network. You don't need the whole networking textbook to design well — you need a small, sharp set of ideas and the judgement to apply them. This lesson is that slice.

We'll follow a single request from a user's browser to your server and back, name the few layers that matter, memorise the latency numbers that shape every decision, and then line up the protocols you'll actually choose between.

The path of a request

When someone opens your app and it loads data, a lot happens before your code runs. Walking this path once makes the rest of system design click:

1. DNS turns api.yourapp.com into an IP address. It's cached aggressively, so this usually costs nothing — but a cold lookup is a real round trip.
2. The connection is established to that IP, typically through a load balancer that picks one of your many servers.
3. The request travels up to your application, which may talk to a cache, a database, and other services.
4. The response travels back down the same path.

Two takeaways shape everything later. First, every arrow in that diagram is a place to add a cache, a retry, or a failure. Second, the slowest arrow dominates — a 1ms server that waits 80ms on a database is an 80ms server.

The layers that matter

Networking is taught as seven layers. In interviews you reason about three:

Everything below the network layer (cables, Wi-Fi, Ethernet) is real, but you rarely design around it. Everything at and above the transport layer is where your choices live.

Numbers worth memorising

You can't reason about a design without a feel for how long things take. These are rounded, order-of-magnitude numbers — exactly the resolution you want for back-of-the-envelope work:

The one that changes designs most: a cross-region round trip is ~100,000× slower than a memory read. That gap is why we cache, why we co-locate data with compute, and why "just add another network call" is never free. If a request makes five sequential cross-service calls, you've spent five round trips before any real work happens.

A quick rule of thumb: anything under about 100 ms feels instant to a user; past ~1 s they notice; past a few seconds they leave. Design budgets backwards from those thresholds.

TCP vs UDP: reliable or fast

At the transport layer you pick between two defaults.

TCP guarantees that bytes arrive, in order, exactly once. It does this with a handshake to set up the connection, acknowledgements for every chunk, and retransmission of anything lost. That reliability costs round trips and buffering. It's the right default for almost everything: web pages, APIs, database connections, file transfers — anything where a missing byte corrupts the result.

UDP sends packets and forgets them. No handshake, no ordering, no retransmission. That makes it fast and cheap, at the cost of packets that can arrive late, out of order, or not at all. You reach for it when fresh beats complete: live video and voice, real-time game state, and metrics where one dropped sample doesn't matter. A video call would rather skip a stale frame than wait for it.

TCP  →  "deliver every byte, in order, no matter how long it takes"
UDP  →  "send it now; if it's lost, the next one is already more useful"

A useful interview instinct: default to TCP, and justify UDP only when you can tolerate loss and can't tolerate delay.

Choosing an application protocol

On top of TCP (mostly), you choose how client and server actually converse. This is the decision you'll make most often.

• HTTP / REST — request in, response out. Stateless, cacheable, universally understood. The default for client-to-server APIs. If you're unsure, this is the answer.
• gRPC — binary, schema-first (Protobuf), with streaming built in. Faster and stricter than JSON over HTTP. The default for service-to-service calls inside your own system where both ends are yours.
• Server-Sent Events (SSE) — a single long-lived HTTP connection the server pushes down. One-directional, simple, auto-reconnecting. Perfect for feeds, notifications, and token-by-token AI responses.
• WebSockets — a persistent, two-way channel. Use it when the client and server both need to send messages continuously: chat, multiplayer, collaborative editing, live cursors.
• Polling — the client just asks again on a timer. Crude, but with a sensible interval it's often enough, and it's the easiest thing to operate. Don't reach for WebSockets when polling every few seconds would do.

The trap is reaching for the fanciest option. Persistent connections (WebSockets, SSE) cost memory and complexity on the server — every open connection is state you have to hold, balance, and clean up. Only pay that cost when the interaction genuinely needs it.

What to say in an interview

When networking comes up, you rarely need depth — you need to show you know where the costs are. A strong, compact answer sounds like:

"Clients hit our API over HTTPS through a load balancer. Reads are served from cache to keep us well under our latency budget; a cache miss is one extra round trip to the database. For the live feed I'd use SSE since it's one-directional server push, and keep the rest as plain REST. Between our own services I'd use gRPC. The thing I'd watch is round trips — I'd avoid chaining synchronous cross-service calls on the hot path."

That's the whole game: name the path, respect the round trips, and pick the simplest protocol that meets the need.

Next up: API Design — turning these protocols into an interface that's actually pleasant to build against.