Designing Offline-First Photo Sync: Idempotency, Dedup, and Server-as-Truth

2026-08-14mobileoffline-firstsyncdynamodbsystem-design

The Problem

Offline-first backup has one hard requirement: no data loss, and when the network comes back, sync should resume exactly where it left off — not duplicate work, not silently drop a half-finished upload, and not require the app to guess whether something actually succeeded.

The interesting failure mode isn’t “network is down.” It’s ambiguity: the client starts a large upload, the network drops mid-transfer, and the client genuinely doesn’t know whether the server finished saving the file before the connection died. Guessing wrong in either direction is bad — retrying a successful upload wastes bandwidth and money (especially with Glacier-class storage), while assuming success on a failed upload loses data.

Scope for V1

Explicitly out of scope for the first version:

  • True byte-range multipart resume (S3 multipart doesn’t support “join back partial bytes” — resumption works at the granularity of whole parts via ListParts, not arbitrary offsets). V1 just retries the whole object.
  • Keeping a session “alive” for long-running uploads.
  • A full server-side orphaned-upload reconciliation subsystem.

These are legitimate simplifications — building partial-part resume and full orphan reconciliation is real complexity with low payoff for a V1.

Core Design

1. Client-generated idempotency key. The client generates an upload-id before starting an upload and sends it via an HTTP header. The server records it against a “live upload” record. This is the same pattern as Stripe’s idempotency keys — it lets the server distinguish “same attempt retried” from “a new upload.”

2. Server is the source of truth. On reconnect, the client doesn’t retry blindly — it asks the server: “here’s the upload-id I have, what’s its state?” The server answers with one of a small set of states rather than a vague yes/no:

PENDING → UPLOADING → COMPLETED / FAILED / EXPIRED

This closes a subtle race: the server may have actually finished saving the file, but the acknowledgment never reached the client because the network died right after the write succeeded. Without a server-truth check, the client would retry a completed upload. With it, the client just moves on to the next file.

3. Idempotency ≠ deduplication. These solve different problems and are easy to conflate:

  • The upload-id answers “was this specific attempt completed?”
  • A separate content identity key (e.g. photo asset local identifier + modification date, or a content hash for stronger guarantees) answers “have I already backed up this photo under a different attempt?”

Without the second key, retries and re-imports can silently create duplicate backups — and duplicate egress/storage cost on a Glacier-class backend is not free.

4. Persistence must survive app kill, not just network cut. The upload-id and the local file reference need to be written to disk before the first byte goes out, so a killed app or device restart doesn’t lose track of an in-flight upload. In-memory-only tracking doesn’t survive this.

5. Abandoned records need a TTL, not a hand-built cron. An “IN_PROGRESS” server record that nobody ever revisits is a silent leak. Rather than writing a cron job that scans for stale records (expensive, needs its own index), a TTL attribute on the record does this natively and for free on a store like DynamoDB — no reconciliation logic required, just automatic expiry.

One easy-to-miss trap here: if the storage layer also has its own cleanup (e.g. an S3 lifecycle rule that deletes orphaned upload prefixes after N days), that TTL and the metadata TTL need to agree. If the metadata record claims “maybe still resumable” for longer than the underlying storage actually keeps the partial data around, the client will confidently retry against data that’s already gone.

Why NoSQL (and Single-Table Design) Fits This Problem

Upload-tracking records have no need for joins, have a small number of fixed access patterns (fetch by user, look up by upload-id), and are extremely high-write, low-relationship data. That’s a textbook fit for a key-value/wide-column store over a relational database.

The concrete shape: a Partition Key (which “drawer” an item lives in) plus a Sort Key (ordering within that drawer) for the base table, and a Global Secondary Index — a second, independently-partitioned “cabinet” over the same data, keyed differently — for the alternate access pattern:

  • Base table: PK = USER#<uid>, SK = UPLOAD#<uploadId> → all of one user’s uploads, sorted by upload-id.
  • GSI: PK = uploadId alone → lets the server answer “what’s the state of upload-id X?” on reconnect without first knowing which user it belongs to.

This is the same overloaded-generic-key pattern (PK/SK/GSI1PK/GSI1SK) used across unrelated entity types in a single table — the trade DynamoDB makes is that every access pattern must be known upfront (no ad-hoc queries, no joins, no aggregations without extra work), in exchange for predictable, low-latency performance at any scale. For a narrow, well-defined problem like idempotent upload tracking, that trade is exactly right — it would be the wrong trade for something like an evolving analytics/reporting feature where query patterns aren’t known in advance.

Summary

The design that held up under review:

  1. Client-generated upload-id as an idempotency key, persisted to disk before the upload starts.
  2. Server as the single source of truth, with an explicit state machine (PENDING/UPLOADING/COMPLETED/FAILED/EXPIRED) instead of an ambiguous retry decision.
  3. A separate, cheaper content-identity key for true deduplication — not conflated with the idempotency key.
  4. TTL-based expiry for abandoned records, aligned with any underlying storage lifecycle rules.
  5. Deliberately deferred: true byte-range multipart resume and full orphan reconciliation — real complexity, low V1 payoff.