Kafka and Message Queue Interview Questions

Model answer

At-most-once: each message is delivered once or not at all. The consumer acknowledges before processing, so a crash loses the message. Fine for metrics you can afford to drop.

At-least-once: the consumer acknowledges after processing, so a crash before the ack means the message is delivered again. Nothing is lost, but duplicates happen. The usual default.

Exactly-once: each message has its effect exactly once. Across real networks it is achieved as at-least-once delivery plus idempotent or transactional processing, not as a delivery promise.

Model answer

On restart the consumer resumes from the last committed offset, so it processes the message again. With at-least-once delivery this will happen, so the consumer must be idempotent.

• Record each processed message id in a table with a unique constraint, in the same database transaction as the result. A duplicate fails the insert and is skipped.
• Or make the write naturally idempotent: an upsert keyed by the event id, or "set status to shipped" rather than "add one".
• Or store the consumed offset in the same transaction as the result, and seek to it on startup.

Model answer

This is the dual write problem. Database commit then publish fails: the order exists but nobody downstream hears about it (no email, no stock reserved). Publish then commit fails: downstream acts on an order that does not exist. No ordering of the two calls is safe.

Use the transactional outbox: in the same transaction as the order, insert the event into an outbox table. A separate relay reads the outbox and publishes to Kafka, marking rows sent. Either both the order and its event are saved or neither is.

The relay can poll the table, or tail the database log with CDC (Debezium). The relay may publish an event twice after a crash, so consumers stay idempotent.

Model answer

Kafka keeps order within a partition only. Publish with the order id as the message key: every event for an order hashes to the same partition, and each partition is read by one consumer in the group at a time. Different orders still run in parallel across partitions.

Watch the things that break it:

• Processing messages from one partition concurrently inside the consumer (a thread pool per message) reorders them; parallelise by key, not by message.
• Sending a failed message to a retry topic lets later events for the same order overtake it. Park the whole key until the failed one succeeds, or make handlers tolerate out-of-order events with a version check.
• Adding partitions changes which partition a key maps to for new messages.

Model answer

A topic is a named stream of events, such as orders. It is split into partitions: ordered, append-only logs spread across brokers. Each message in a partition has an increasing offset.

A consumer group is a set of consumers sharing the work: each partition is assigned to exactly one consumer in the group, and the group tracks its committed offset per partition. A second group (say, analytics) reads the same topic independently from its own offsets.

Messages are not deleted when read; they stay for the topic's retention period, so a consumer can replay from an older offset.

Model answer

Six consumers each get one partition and four sit idle: a partition is never split between consumers in a group. The partition count is the ceiling on parallelism.

Adding consumers also triggers a rebalance, during which consumption pauses briefly (less so with cooperative rebalancing).

To use more consumers you need more partitions, so choose a partition count with growth in mind. Adding partitions later is possible but changes the key-to-partition mapping for new messages, which matters when ordering by key matters.

Model answer

Lag means producers write faster than consumers commit. Find where the time goes:

• Slow processing: each message waits on a database or API call. Batch the writes, process in parallel by key, or speed up the dependency.
• Frequent rebalances: consumers take longer than max.poll.interval.ms between polls, get kicked out, and the group keeps reshuffling. Process smaller batches or raise the limit.
• A hot partition: one key gets most of the traffic, so one consumer is behind while others are idle. Check lag per partition, not just the total.
• Too few consumers or partitions: scale up, within the partition count.

For a one-off backlog, temporarily add consumers and skip non-essential work; if old events are worthless (stale prices), reset offsets past them deliberately.

Model answer

Never retry forever in place. Retry a few times for transient errors, then move the message aside:

• Publish it to a dead letter topic with the error and original offset, commit, and carry on.
• Use retry topics with delays for errors that might clear (a dependency was down), and the dead letter topic for ones that will not (bad data).
• Alert on anything reaching the dead letter topic, and have a way to replay it once fixed.

If messages for one key must stay in order, skipping one breaks that order for its key, so record the key as blocked and park its later messages too.

Model answer

When the user does not need the result to continue, and doing it in the request would make them wait or make the request fragile: sending emails, generating PDFs or reports, resizing uploads, calling slow third-party APIs, recomputing search indexes.

The request records the job on a queue (BullMQ on Redis, SQS, RabbitMQ, or a database table) and returns at once with 202 Accepted. Workers process jobs separately, can retry failures, and can be scaled on their own.

Keep in the request anything the response depends on: you cannot tell the user "payment done" from a job that has not run yet.

Model answer

Queues are at-least-once. A worker takes a job, which becomes invisible for a visibility timeout; if the worker crashes or simply runs longer than the timeout, the job reappears and another worker runs it too.

Make the job idempotent: before sending, insert a row such as (user_id, 'welcome') with a unique constraint, or set welcome_sent_at with a conditional update (WHERE welcome_sent_at IS NULL), and send only if your update won. Pass an idempotency key to the email provider if it supports one.

Also set the visibility timeout comfortably above the job's normal run time, and extend it (heartbeat) for long jobs.

Model answer

• Run scheduling in one place: a separate scheduler (Kubernetes CronJob, EventBridge Scheduler) that enqueues one job, which any worker then takes.
• Or let every server try but only one win: take a database lock first (pg_try_advisory_lock, or a row in a job_runs table keyed by job name and date with a unique constraint).
• Use a job library with built-in scheduling that already does this (BullMQ repeatable jobs, Quartz in clustered mode).

Key the run by date as well, so a retry or a slow server the next minute still finds "today's report already sent" and stops.

Model answer

• Rate-limit the workers, not just the queue: a shared token bucket (in Redis) at, say, 90 requests a second across all workers, so adding workers cannot exceed the limit.
• Respect 429 and Retry-After from the API, with backoff.
• Prioritise: put new user-facing jobs (password resets) ahead of the backlog with a separate high-priority queue.
• Drop what is stale: jobs carry an expiry; an OTP from two hours ago is useless and should be discarded, not sent.

At 100 a second, 2 million jobs take about 5.5 hours. Say that number out loud to the product team; it may change what gets dropped.