How do you build an idempotent message consumer?

With at-least-once delivery, a consumer can see the same message twice: the broker redelivers after a timeout, a rebalance moves a partition, or an operator replays a batch. Idempotency is not optional; it is part of the consumer contract.

BEGIN;

INSERT INTO processed_messages (message_id)
VALUES ('evt_01J8Z3')
ON CONFLICT (message_id) DO NOTHING;

UPDATE orders
SET status = 'paid'
WHERE id = 1042 AND status = 'pending';

COMMIT;

Patterns that work

• Use a producer-generated message id, not a random value created on each attempt
• Store processed ids in the same database and transaction as the side effect
• Make external calls idempotent with an idempotency key
• Use conditional updates or unique constraints instead of blind increments
• Retain dedupe records long enough to cover the broker retention and replay window
• Handle ordering explicitly; a later event must not overwrite newer state

A separate dedupe cache is weaker than a transactional inbox because the cache write and the database commit can diverge. If you must use one, accept a small duplicate window and design the side effect to tolerate it. For payments and inventory, prefer the transactional approach.