How to design a Dropbox-Style File Sync Service

The workload has two different costs. Small files are dominated by request, hashing and metadata overhead; large files are dominated by bytes on the wire and interruption risk. A design that treats every file as one HTTP body pays both costs badly.

Split file bytes into bounded blocks and identify each block by a cryptographic hash. A version manifest stores an ordered list of block IDs plus file metadata. Immutable blocks are simple to retry and cache; mutable meaning lives in the version manifest, not inside the block store.

For a large upload, the client first sends its proposed block list. The service returns which hashes are already durable; the client uploads only the missing blocks with bounded concurrency. After a disconnect, it can repeat that negotiation and resume. The server does not need to trust a remembered session to know which bytes exist: it checks the durable block index again.

Content hashes are identifiers and integrity checks, not authorization tokens. A block store still needs a policy for who may read a hash; otherwise a guessed or leaked identifier can expose data across users.

The safe sequence is: authenticate and validate the expected current revision; upload missing blocks and wait for durable acknowledgements; conditionally commit the new manifest and append its journal row in one metadata transaction; then notify online devices. If the process crashes before commit, the previous version is still visible and the new blocks are harmless orphans. If it crashes after commit, every referenced block was already acknowledged as durable. Notifications follow the commit and can be retried.

The commit should compare the current file revision with the revision the client edited. This is a compare-and-swap: only one writer based on revision 12 can publish revision 13. A second writer gets a conflict response and can preserve its content as a separate version instead of silently replacing the winner.

Give each namespace an ordered journal and each device a durable cursor. A client requests changes after its cursor in bounded pages, applies them, then saves the cursor. If it crashes while applying a page, it can safely repeat it using stable change IDs and idempotent local operations.

A cursor expires only if the server has compacted the journal past it. Make that response explicit: tell the client to rebuild from a snapshot at sequence S, then continue after S. Silently returning only recent changes would leave the device permanently inconsistent.

A last-write-wins update silently discards one valid edit. Compare the client's base revision to the current revision at commit. If they differ, preserve both versions or create an explicit conflict copy. A product-specific merge is appropriate only when the format has a safe, well-defined merge algorithm.

Check authorization at the metadata operation and at block retrieval. A signed URL is a bearer capability: anyone who has it can use it until it expires. Short lifetimes reduce revocation delay; proxying reads through an authorization service gives faster revocation but adds bandwidth, cost and a dependency on the hot path.

Make delete a metadata operation first: append a tombstone, remove the version from the active namespace, and preserve it for the product's retention window. A collector finds blocks unreachable from all retained manifests, but a stale mark is not permission to delete immediately. Recheck reachability and wait beyond the maximum metadata replication and in-flight upload delay.

At large scale, immutable blocks simplify reads and retries but move complexity into reclamation. Garbage collection identifies unreferenced content; compaction is the physical work that makes space reusable. Dropbox's public Magic Pocket posts describe this trade-off in its own storage system; the exercise uses it as an operational lesson, not as a claim that this design matches their implementation.