Coding Judge

A coding judge (think LeetCode / an online IDE's "Run" button) takes a user's source code, executes it against a set of test cases, and returns pass/fail with timing. The defining challenge is unique in this list: you are running untrusted, possibly malicious code on your own machines. Everything about the design is shaped by containing that risk while staying fast and scalable. It builds on Managing Long-Running Tasks.

1. Requirements

Functional:

• Submit code + language; run it against predefined test cases.
• Return a verdict: accepted / wrong answer / time-limit-exceeded / runtime error, with output and timing.

Non-functional:

• Safe — untrusted code must not touch the host, the network, other users' runs, or hog resources. This dominates the design.
• Scalable — handle bursts (a contest start is a thundering herd).
• Bounded — every run has strict CPU, memory, and wall-clock limits.
• Users tolerate a short wait; submissions are async, not request-response.

2. Shape: submit async, run on workers

Execution takes seconds and is bursty, so never run code inside the web request. Instead:

client → API: submit code → enqueue job, return submission_id (202 Accepted)
queue  → execution workers (sandboxed) → run against tests → write result
client ← poll / push ← result by submission_id

This is the classic async worker pool off a queue: the API is thin and fast, a queue absorbs bursts and smooths load, and a fleet of execution workers consumes jobs at whatever rate it can. The queue also gives you retries and backpressure for free.

3. The heart: sandboxing untrusted code

This is the part interviewers care about. Each run must be isolated:

• Containers / microVMs. Run each submission in its own locked-down container (or a lightweight microVM like Firecracker for stronger isolation). Fresh, ephemeral, destroyed after the run.
• No network. Disable networking inside the sandbox so code can't exfiltrate data or attack other hosts.
• Resource limits. Cap CPU, memory, processes, and file size (cgroups/ulimits). Enforce a hard wall-clock timeout and kill on breach → time-limit-exceeded.
• Minimal privileges / filesystem. Read-only base image, a small writable scratch dir, dropped capabilities, non-root user, restricted syscalls (seccomp).

The rule: assume the code is hostile, give it the absolute minimum, and throw the environment away afterward.

4. Running the tests

A worker: pulls the job, writes the code into the sandbox, compiles if needed, then runs each test case feeding input and capturing output. Compare output to expected; first failing case → wrong answer; exceeds time/memory → the matching verdict. Aggregate into a result and persist it keyed by submission_id.

Warm, pre-provisioned sandboxes (a pool kept ready) cut cold-start latency so the "Run" button feels quick.

5. Returning the result

The submission was async, so deliver the verdict by either:

• Polling: client polls GET /submissions/{id} until status is done — simplest.
• Push: a WebSocket/SSE pushes the verdict when ready — nicer UX.

6. Deep dives

• Contest bursts: a contest start floods submissions; the queue buffers them and workers autoscale. Fairness (don't let one user's 100 submissions starve others) via per-user concurrency limits.
• Caching compiles / warm pools: reuse language runtimes; keep sandboxes warm.
• Determinism: pin the runtime version and resources so timing verdicts are consistent across runs and machines.
• Abuse: cap submission rate per user (Rate Limiter); scan for fork bombs via the process limit.

What to emphasise

Lead with isolation, then the async worker architecture:

"The defining constraint is running untrusted code, so every submission executes in its own ephemeral, network-less sandbox — a container or microVM — with hard CPU, memory, process, and wall-clock limits, minimal privileges, and a read-only filesystem, destroyed after the run. Execution is async: the API enqueues a job and returns an id, a pool of sandboxed workers consumes the queue so contest bursts just buffer, and the verdict is delivered by polling or a push once the tests finish. Warm sandbox pools keep latency low."

Next up: Video Streaming — delivering huge media to millions.