Skip to content
sysgeeks

Design a Video Processing Pipeline, stage 14 of 14: defend it

Defend the design

A staff engineer reviews your design and says:

"This is overbuilt. Use a managed transcoding service: hand it the object key, and it calls our webhook when it's done. Delete the workers, the leases, the reconciler, all of it."

They might be right. Respond as you would in the review.

System so far· 8 parts
123456789CLIENTInstructorbrowserSERVICEVideo APIDATABASEPostgresOBJECT STOREObject storageWORKERTranscodeworkersWORKERReconcilerEDGECDNCLIENTStudent player

Select a component to see what it is responsible for and which state it owns.

  1. 1Instructor browser → Video API: Create upload, report parts, poll status
  2. 2Instructor browser → Object storage: Upload parts via presigned URLs
  3. 3Video API → Object storage: Complete multipart upload, verify object
  4. 4Video API → Postgres: Video row and job row in one transaction
  5. 5Transcode workers → Postgres: Claim lease, heartbeat, fenced completion
  6. 6Transcode workers → Object storage: Read raw upload, write attempt output
  7. 7Reconciler → Postgres: Find abandoned uploads and orphaned output
  8. 8CDN → Object storage: Origin fetch on cache miss
  9. 9Student player → CDN: Manifest and segments
  • Request / response
  • Bulk data

What you need to know

0 of 2 checks done

  1. A managed service (here, a transcoding API) removes the parts it runs for you: workers, scaling, codecs. It doesn't remove guarantees that live at the boundary between it and your system.

    When evaluating "just use service X", list each guarantee your design provides and ask: does X provide it, or does it move to the code that talks to X?

  2. Check

    The managed service calls your webhook when a transcode finishes. Which guarantee is still yours?