Skip to content
sysgeeks

Design a Distributed Job Queue, stage 9 of 9: defend it

Defend keeping Redis

Your interviewer: "You now run Kafka and Redis and two new services. Kafka alone could be the queue, or you could have used SQS. Why this?"

System so far· 7 parts
123456SERVICEWeb serversSERVICEEnqueue gatewayLOG / STREAMKafkaWORKERRelayQUEUERedis queuesWORKERWorkersDATABASEDatabasesand services

Select a component to see what it is responsible for and which state it owns.

  1. 1Web servers → Enqueue gateway: Enqueue job
  2. 2Enqueue gateway → Kafka: Append to topic
  3. 3Relay → Kafka: Read topics
  4. 4Relay → Redis queues: Push at a controlled rate
  5. 5Workers → Redis queues: Lease jobs
  6. 6Workers → Databases and services: Do the work

What you need to know

0 of 1 checks done

  1. "Why not use X instead?" is a standard interview follow-up. A strong answer has three parts:

    1. The constraint that decided it, which is often not technical elegance but risk, time or what already exists.
    2. What each part of your design is for, so nothing looks accidental.
    3. What the alternative does better, and when you would choose it.

    Conceding real costs reads as judgement, not weakness.

  2. Check

    Which reason best explains putting Kafka in front of Redis instead of replacing Redis?