Handling Large Files
Practice this topic in a realistic system design interview
A user uploads a 2 GB video file. After about 20 minutes, the connection drops at 80%. If the upload was handled as one long request, the user starts again from 0%.
The server can suffer too. A naive implementation may buffer too much data, tie up worker threads, hit load balancer timeouts, or proxy every byte through application servers before writing to storage.
Large-file handling is mostly about avoiding that all-or-nothing path. The interview patterns are straightforward: split files into parts, make retries safe, move bytes directly to object storage when possible, and keep metadata separate from raw file data.
Interview Answer Shape
In an interview, structure the answer around the data path:
- Clarify upload size, download traffic, file privacy, and whether uploads must resume after failure.
- Use resumable chunked or multipart uploads so retries resend only missing parts.
- Use pre-signed URLs so file bytes go directly to object storage while the application owns metadata and authorization.
- Store metadata in a database and file bytes in object storage; verify chunks and final checksums before marking the file complete. Finalize reliably using storage event notifications rather than depending only on a client call, and run heavy work like transcoding, scanning, and indexing asynchronously.
- For downloads, support range requests and CDN delivery, and gate private files behind short-lived signed URLs. Add deduplication or content-defined chunking only if storage efficiency or sync is part of the problem.
Where This Pattern Shows Up
Large-file handling shows up in systems that deal with media, documents, or data transfers:
| Problem | Why Large File Handling Matters |
|---|---|
| Design Google Drive/Dropbox | Users upload multi-GB files that need chunking, resume, and sync |
| Design YouTube | Video uploads can be hours long, requiring resumable uploads and transcoding |
| Design Slack/Teams | File sharing in chat requires efficient upload and CDN distribution |
| Design GitHub | Large repos with binary assets need efficient storage and cloning |
| Design Backup System | Terabytes of data require incremental uploads and deduplication |
| Design Netflix | Streaming large video files needs range requests and adaptive bitrate |