How to design a Payment System

What a strong answer covers

• Most charges finish in seconds, so a short bounded wait gives most buyers an immediate answer.
• When the wait ends without a result the outcome is unknown, so the design needs an explicit processing state resolved later by a source of truth.
• The server learns outcomes from the provider, never from the client's claim.Supporting

Tradeoffs

What a strong answer covers

• Each attempt (e.g. a declined card, then a new one) needs its own record and its own idempotency key.
• The model includes an explicit state for outcomes that are not yet known.
• An append-only record of transitions and their causes makes the state auditable.Supporting

Tradeoffs

What the fix has to do

• The first request's outcome was unknown to the buyer; retrying without a stable idempotency key created a second charge.
• The status check races with concurrent requests; dedupe must be an atomic claim (unique key / insert-if-absent).
• The attempt must be durably recorded, with its key, before the provider is called.
• Follow-up effects need to be recorded in the same transaction as the status change (outbox) so a crash cannot drop them.Supporting

Reference implementation

app.post("/orders/:id/pay", async (req, res) => {
  const key = req.header("Idempotency-Key");
  if (!key) return res.status(400).json({ error: "Idempotency-Key required" });
  const order = await db.orders.find(req.params.id);

  // 1. Atomic claim: exactly one request creates the attempt for this key.
  const inserted = await db.oneOrNone(`
    INSERT INTO payment_attempts (order_id, idempotency_key, amount, currency, status)
    VALUES ($1, $2, $3, $4, 'processing')
    ON CONFLICT (idempotency_key) DO NOTHING
    RETURNING *`, [order.id, key, order.total, order.currency]);

  if (!inserted) {
    const existing = await db.one(
      `SELECT * FROM payment_attempts WHERE idempotency_key = $1`, [key]);
    if (existing.order_id !== order.id || existing.amount !== order.total) {
      return res.status(422).json({ error: "Key reused with different parameters" });
    }
    return res.status(existing.status === "processing" ? 202 : 200).json(view(existing));
  }

  // 2. Call the provider with the same key. Bounded wait.
  let result;
  try {
    result = await provider.payments.create(
      { amount: order.total, currency: order.currency,
        paymentToken: req.body.token, metadata: { attempt: inserted.id } },
      { idempotencyKey: inserted.idempotency_key, timeoutMs: 8000 });
  } catch (err) {
    if (isDefinitiveDecline(err)) {
      await applyTransition(inserted.id, "failed", { cause: "api", reason: err.code });
      return res.status(200).json({ status: "failed", reason: err.code });
    }
    // Timeout or network error: the outcome is unknown. Leave it processing.
    return res.status(202).json({ status: "processing" });
  }

  await applyTransition(inserted.id, result.status, { cause: "api", providerRef: result.id });
  return res.status(result.status === "processing" ? 202 : 200).json({ status: result.status });
});

// Shared by the API, the webhook receiver and the reconciler.
async function applyTransition(attemptId, to, cause) {
  const from = { succeeded: ["processing"], failed: ["processing"], refunded: ["succeeded"] }[to];
  if (!from) return false; // "processing" again: nothing to do
  return db.tx(async (tx) => {
    const moved = await tx.result(`
      UPDATE payment_attempts SET status = $2, provider_payment_id = COALESCE($3, provider_payment_id)
      WHERE id = $1 AND status = ANY($4)`, [attemptId, to, cause.providerRef, from]);
    if (moved.rowCount === 0) return false; // already moved by another path
    await tx.none(`INSERT INTO payment_events (attempt_id, kind, cause, provider_ref)
                   VALUES ($1, $2, $3, $4)`, [attemptId, to, cause.cause, cause.providerRef]);
    if (to === "succeeded") await enqueue(tx, ["grant-enrollment", "send-receipt"], attemptId);
    if (to === "refunded") await enqueue(tx, ["revoke-enrollment"], attemptId);
    return true;
  });
}

What a strong answer covers

• The attempt is inserted under a unique idempotency key in one atomic statement; a duplicate finds the existing attempt instead of creating one.
• The provider call carries the attempt's key, so the provider deduplicates retries too.
• A timeout or network error leaves the attempt in processing and returns 202, never failed.
• Results are applied with a conditional transition from processing, together with a ledger event and outbox rows in one transaction.
• Reusing a key with a different order or amount is rejected.Supporting
• A duplicate arriving while the first is still processing returns the current state rather than calling again.Supporting

What a strong answer covers

• The key identifies one payment attempt: retries of it share the key, and a genuinely new attempt gets a new one.
• Explains a concrete failure of a key that is too broad (stuck on a cached decline) or too narrow (new key on retry, so a double charge).
• The server's own uniqueness checks must not depend solely on the client behaving.Supporting
• Retries arriving after the provider's 24-hour retention would bypass its dedupe, so your own record must catch them.Supporting

What a strong answer covers

• Repeated events are deduplicated by event id, or the transition itself is idempotent.
• Out-of-order events cannot regress state, either through forward-only transitions or by fetching current state.
• Side effects are triggered by the state transition, not by receiving the event, and run outside the webhook request.
• Signatures are verified so forged events cannot mark payments succeeded.Supporting

Tradeoffs

What a strong answer covers

• The state change and the notification are writes to different systems; a crash between them loses one.
• Recording the obligation in the same transaction as the state change makes them atomic.
• Delivery from the outbox is at-least-once, so each effect must be idempotent.
• External effects like email cannot be rolled back, which is why they come after the commit, not inside it.Supporting

Reference answer

SELECT * FROM payment_attempts
WHERE status = 'processing' AND created_at < now() - interval '15 minutes';

What a strong answer covers

• Periodically finds attempts in processing older than a threshold and looks each up at the provider by its key or reference.
• Applies the provider's authoritative state through the same conditional transitions as the API and webhooks, so they cannot conflict.
• Treats 'provider has no record' carefully: waits until no in-flight request could still land before failing it, and escalates true anomalies to a human.
• Also compares the daily settlement report with the ledger in both directions (charged but not recorded, recorded but not charged).Supporting
• Alerts on the number and age of unresolved attempts, which would have caught the broken endpoint within hours.Supporting

What a strong answer covers

• A refund has the same uncertainty as a charge: it needs its own record, idempotency key and unknown state.
• State changes to refunded only on a confirmed provider outcome.
• History is appended (a refund event), never rewritten.Supporting

Reference answer

What a strong answer covers

• Every charge is made under an idempotency key belonging to an attempt that is durably recorded before the call.
• Duplicate requests, retries and events converge through unique constraints and conditional transitions.
• Unknown outcomes are resolved from the source of truth (webhooks, lookup, reconciliation), never guessed.
• Names residual risks honestly, e.g. retries after the provider's key retention, manual dashboard actions, failover to another provider, or a client generating new keys on retry.
• Distinguishes the guarantee (no double charge) from what is not guaranteed (instant resolution, emails exactly once).Supporting