Skip to content
sysgeeks

Examples/Developer tools

Hosted CI service (example)

A hosted CI service that runs a pipeline on every push. A webhook receiver records pipelines in Postgres, a scheduler expands them into job graphs, and ephemeral runner VMs lease jobs from a queue and stream logs to a live dashboard. Replace any part with how your own system works.

Scale: 8k pipelines/hour; 600 concurrent runners

1234567891011121314EXTERNALGitHubSERVICEWebhook receiverDATABASEPostgresSERVICESchedulerQUEUEJob queueWORKERRunner VMsLOG / STREAMLog streamOBJECT STOREArtifact storeSERVICEDashboard APICLIENTDashboard UIWORKERStatus reporter

Select a component to see what it is responsible for and which state it owns.

  1. 1GitHub → Webhook receiver: Push webhook
  2. 2Webhook receiver → Postgres: Create pipeline
  3. 3Webhook receiver → Scheduler: Pipeline created
  4. 4Scheduler → Job queue: Enqueue ready jobs
  5. 5Runner VMs → Job queue: Lease a job
  6. 6Runner VMs → Postgres: Heartbeat and step status
  7. 7Runner VMs → Log stream: Stream log lines
  8. 8Runner VMs → Artifact store: Upload logs and artifacts
  9. 9Runner VMs → Scheduler: Job finished
  10. 10Dashboard UI → Dashboard API: Open pipeline
  11. 11Dashboard API → Log stream: Tail logs
  12. 12Dashboard API → Dashboard UI: Live logs (SSE)
  13. 13Scheduler → Status reporter: Status changed
  14. 14Status reporter → GitHub: Set commit status
  • Asynchronous
  • Request / response
  • Bulk data
  • Server push

Select a part to trace its flows.

  • Asynchronous
  • Request / response
  • Bulk data
  • Server push

Parts 11

Flows 14

  1. 1GitHub → Webhook receiverPush webhookAsynchronous
  2. 2Webhook receiver → PostgresCreate pipelineRequest / response
  3. 3Webhook receiver → SchedulerPipeline createdAsynchronous
  4. 4Scheduler → Job queueEnqueue ready jobsAsynchronous
  5. 5Runner VMs → Job queueLease a jobRequest / response
  6. 6Runner VMs → PostgresHeartbeat and step statusRequest / response
  7. 7Runner VMs → Log streamStream log linesAsynchronous
  8. 8Runner VMs → Artifact storeUpload logs and artifactsBulk data
  9. 9Runner VMs → SchedulerJob finishedAsynchronous
  10. 10Dashboard UI → Dashboard APIOpen pipelineRequest / response
  11. 11Dashboard API → Log streamTail logsRequest / response
  12. 12Dashboard API → Dashboard UILive logs (SSE)Server push
  13. 13Scheduler → Status reporterStatus changedAsynchronous
  14. 14Status reporter → GitHubSet commit statusRequest / response

Invariants 3

  • A job attempt is run by one runner at a time

    Lease with a visibility timeout renewed by heartbeats; the attempt number is a fencing token, and status writes are conditional UPDATEs with WHERE attempt = $n.

    Kept by

  • A job starts only after all its upstream jobs succeed

    On job completion the scheduler takes a row lock on the pipeline (SELECT ... FOR UPDATE) and enqueues only jobs whose upstreams have all succeeded.

    Kept by

  • A redelivered webhook does not create a second pipeline

    Unique constraint on the GitHub delivery id; the insert for a redelivery conflicts and creates nothing.

    Kept by