Idempotency
Prevent duplicate handler execution when Kafka redelivers messages using Redis-backed idempotency tracking.
Idempotency
In an at-least-once delivery system, the same message can be delivered more than once — after a worker restart, a rebalance, or a failed offset commit. Idempotency ensures your handler only executes once per unique message, even if the message arrives multiple times.
How It Works
Every message published by EventBus carries a unique event_id header (a UuidV7 UUID). When ConsumerRunner processes a message, it checks a Redis cache before invoking each handler. Whether the check skips the handler depends on the handler's idempotent flag:
Message arrives with event_id: "019d3806-3b1c-7c2b-..."
│
▼
Build cache key: "vortos_idempotency_{handlerId}_{eventId}"
│
├── handler has idempotent: false (default)?
│ │
│ ▼
│ cache.has(key)?
│ ├── YES → SKIP handler (already processed, protect from duplicate side-effects)
│ └── NO → run handler
│ │
│ SUCCESS → cache.set(key, true, 86400s)
│ FAILURE → do NOT cache → message will retry
│
└── handler has idempotent: true?
│
▼
Always run handler — cache is never checked
SUCCESS → cache.set(key, true, 86400s) ← written but never read
FAILURE → do NOT cache → message will retryThe cache key includes both the handlerId and event_id — so the same event can be handled by multiple handlers, each with independent idempotency tracking.
Cache TTL
Idempotency keys expire after 24 hours (86400 seconds) by default. After expiry, the handler will execute again if the same message arrives. This is intentional — keeping keys forever would cause unbounded cache growth.
Cache Backend
Vortos uses Redis as the idempotency cache via PSR-16 (Psr\SimpleCache\CacheInterface). Redis is the correct choice for production — it persists across worker restarts and is shared across multiple worker processes.
The binding is registered automatically in MessagingExtension using Symfony Cache's RedisAdapter wrapped in Psr16Cache:
Redis (redis:6379)
└── RedisAdapter (namespace: vortos_messaging)
└── Psr16Cache
└── CacheInterface binding
└── ConsumerRunner injects itConfiguration
By default the framework connects to redis://redis:6379. To use a different Redis instance, bind CacheInterface yourself in your container config before MessagingExtension runs — the extension checks hasAlias(CacheInterface::class) and skips registration if already bound:
// in your services.php or a container extension
$container->setAlias(CacheInterface::class, 'your.custom.cache.service');Marking a Handler as Idempotent
By default (idempotent: false), all handlers participate in deduplication — if the cache key for an (eventId, handlerId) pair already exists, the handler is skipped and the message is treated as already processed. This protects handlers with side effects (sending emails, charging cards, creating records) from running twice when Kafka redelivers a message.
If a handler is truly idempotent by design — it always produces the same result no matter how many times it runs — mark it with idempotent: true. The cache check is bypassed and the handler runs on every delivery, including duplicates:
#[AsEventHandler(
handlerId: 'user.registered.sync-read-model',
consumer: 'user.events',
idempotent: true, // upsert — running twice produces identical state, so skip dedup
)]
final class SyncUserReadModelHandler
{
public function __invoke(UserRegisteredEvent $event): void
{
// INSERT INTO read_model ... ON CONFLICT DO UPDATE SET ...
// Idempotent: running this twice leaves the database in the same state
}
}Default is Safe
The default (idempotent: false) is the safer choice. Non-idempotent handlers are protected from duplicate execution automatically. Only opt out with idempotent: true when you are certain running the handler twice produces no additional side effects — same database state, no duplicate emails, no duplicate charges.
Testing Idempotency
To verify idempotency works, publish two messages with the same event_id. The cleanest way is to set a fixed event_id header when producing a test message directly to Kafka, or to produce the same message twice using the InMemory driver in tests:
// In a test using the InMemory driver, publish the same event twice
// with the same event_id header to simulate Kafka redelivery
$headers = ['event_id' => '019d3806-3b1c-7c2b-ac39-1be1b02ac8e1', 'event_class' => UserCreatedEvent::class];
$this->producer->produce('user.events', new UserCreatedEvent('user-1', 'test@example.com'), $headers);
$this->producer->produce('user.events', new UserCreatedEvent('user-1', 'test@example.com'), $headers);Check logs after processing both:
First message:
app.INFO: UserCreatedHandler executed {...}
app.DEBUG: Event dispatched {...}
Second message (same event_id, handler has idempotent: false):
app.DEBUG: Skipping duplicate handler execution
app.DEBUG: Event dispatched {...}The handler log line does not appear the second time — the handler was skipped by the idempotency cache.
Idempotency TTL
The default TTL is 86400 seconds (24 hours). There are two ways to change it.
Global default — set in your bootstrap via VortosMessagingConfig. Applies to all consumers that do not override it:
use Vortos\Messaging\DependencyInjection\VortosMessagingConfig;
$config = new VortosMessagingConfig();
$config->consumerDefaults()
->idempotencyTtl(43200); // 12 hours for all consumersPer-consumer override — set on the consumer definition. Takes precedence over the global default for that consumer only:
#[RegisterConsumer]
public function paymentConsumer(): KafkaConsumerDefinition
{
return KafkaConsumerDefinition::create('payment.events')
->groupId('payment-service')
->retry(RetryPolicy::exponential(attempts: 3, initialDelayMs: 500))
->idempotencyTtl(604800); // 7-day dedup window for payment events
}Use a longer TTL for critical consumers where duplicate execution has high cost (payments, inventory). Use a shorter TTL for high-volume consumers where Redis memory is a concern.