How to implement eventual delivery between ERP and online store using queues and idempotency (no duplicate orders)

By Kevin Marcondes · Published on Sep 29, 2026 · Updated on Oct 10, 2026 · 7 min read

Need to implement eventual delivery between your ERP and online store using queues and idempotency to prevent duplicate orders? The solution combines messaging, a consistent event design, and application-level deduplication. Below you’ll find the root causes of duplicates, a recommended architecture, and a practical step-by-step plan to enable this safely.

The real problem: duplicate orders and incorrect stock

Duplicate orders happen when integrations retry without proper control. Timeouts, instability, and concurrency lead to reprocessing. Without idempotency, the same message can create multiple orders, deplete stock, and disrupt finance. Below are common symptoms and their typical causes.

Core concepts: eventual delivery, queues and idempotency

Eventual delivery accepts small delays in exchange for resilience. A message queue decouples systems and enables retries. Most queues operate on at-least-once semantics: the same message can be delivered more than once. That’s why the application must be idempotent—processing N times should produce the same final result.

FIFO queues with deduplication help reduce redeliveries but don’t remove the need for idempotency at the application edge. Idempotency belongs close to the database and business rules.

Recommended architecture for ERP ↔ e-commerce

A robust topology combines event-driven integration patterns with consumer-side deduplication.

Practical components:

- Outbox pattern in the ERP or store: the system writes the event and business state in the same DB transaction. An async publisher reads the outbox and pushes to the queue.

- Queue (FIFO or standard) with DLQ: retries with backoff and isolation for problematic messages.

- Idempotent consumer: before creating/updating an order, check a unique key and record processing.

If your ERP doesn’t expose changes easily, consider CDC (Change Data Capture) with an outbox. See Strategy for zero-downtime database migration using CDC.

Step-by-step implementation

An incremental approach reduces risk and provides visibility.

• Map critical flows: order creation, stock decrement, invoicing and cancellations.
• Define the event contract: immutable identifiers, event version and origin.
• Implement outbox or CDC to publish events reliably.
• Provision the queue and DLQ; tune retry and visibility policies.
• In the consumer, create an idempotency repository (table) with a unique index.
• Use transactional upsert to create/update orders idempotently.
• Implement exponential backoff and idempotency with TTL where appropriate.
• Test under failures: simulate timeouts and network drops. Performance tips in How to reduce REST API latency: cache and SQL optimization.

Idempotency in practice: keys, upsert and deduplication

The idempotency key should uniquely identify the business intent. Examples: the store’s orderId, the payment provider’s paymentIntentId, or a stable hash of order fields.

Best practices for the consumer:

• Persist the idempotency key before performing the action: INSERT with a unique constraint; if it already exists, treat it as a retry.
• Use upsert (INSERT ... ON CONFLICT / ON DUPLICATE KEY) for idempotent changes.
• Record processing status (in progress, completed, failed) and timestamps.
• Avoid side effects outside the transaction (emails, webhooks) without their own idempotency checks.
• Useful docs: FIFO queues and deduplication in AWS SQS (aws.amazon.com/sqs/), idempotency keys in APIs such as Stripe (stripe.com/docs/idempotency).

Retries, DLQ and at-least-once guarantees

Failures will happen. Retries with exponential backoff reduce retry storms. Use a DLQ to isolate consistently failing messages and avoid blocking the flow.

Do not confuse idempotency with rate limiting or authentication—they complement each other. For strengthening the API edge, see Guide to secure APIs with OAuth2 and rate limiting.

• Define retry limits by error type (transient vs permanent).
• Apply exponential backoff with jitter to avoid thundering herd.
• Monitor DLQ rate, end-to-end latency and queue backlog.
• Implement assisted DLQ reprocessing with rich observability.

Testing and observability before flipping the switch

Without telemetry, idempotency becomes a black box. Instrument counters for prevented duplicates, latency per step and backlog size.

Load tests should include duplicate messages and out-of-order delivery. Prove that the same event processed N times does not change the final state.

Log correlations: traceId, idempotencyKey and orderId. Dashboards and alerts reduce MTTR during incidents.

Signals it’s time to change and next steps

Some signs clearly indicate the need for eventual delivery with idempotency. If several apply, prioritize implementation.

To plan the integration protocol, choose REST or GraphQL depending on use case. See Comparison: GraphQL vs REST for integration between e-commerce and ERP (catalog and stock sync).

• Duplicate orders, invoices or webhooks during traffic peaks.
• Frequent timeouts and retries in ERP or store.
• Stock divergence after cancellations or refunds.
• Growing integration backlog and manual correction windows.

Frequently asked questions

What is the best idempotency key for e-commerce orders?

Choose an immutable identifier that represents the business intent, for example the store’s orderId, a payment provider’s paymentIntentId, or a stable hash of the order payload. Persist the key with a unique constraint and record processing status (in-progress, completed, failed) so retries are detected and treated as idempotent operations without creating duplicate orders.

Do FIFO queues eliminate message duplication?

FIFO queues reduce out-of-order delivery and can provide deduplication within a configured time window, but they do not guarantee zero duplicates in all failure scenarios. Because most queues operate at-least-once, the consumer must still implement idempotent processing and deduplication logic to ensure repeated deliveries do not create duplicate orders or side effects.

How to avoid duplicate emails or webhooks when reprocessing?

Treat side effects as first-class idempotent actions: store a send-key for each outgoing event and check it before sending, or require an idempotency-key from the client. Avoid performing external side effects inside the main transaction unless those effects have their own deduplication. Use TTL on stored keys when appropriate and ensure the consumer checks the repository before resending.

What belongs in the DLQ and how to safely reprocess?

Place messages in the DLQ that exceed retry limits or fail due to persistent data errors that require human or corrective action. Reprocess only after fixing the root cause, with full traceability (traceId, idempotencyKey, orderId). Monitor DLQ rates, inspect failure reasons, and provide assisted reprocessing tools that avoid duplicating already-processed work.

Do I need strong consistency for stock?

It depends on business requirements: many high-volume systems tolerate short delays and use eventual delivery with reservations or periodic reconciliation. If immediate consistency is critical for some flows, add synchronous checks or temporary reservations for those operations while keeping the bulk of integration asynchronous and idempotent to preserve resilience and throughput.

See also

• How to recover corrupted statistics and indexes in PostgreSQL that caused slow queries
• REST API latency spikes from N+1 queries and PHP GC — how to diagnose and fix
• Postgres 'Too many connections' at billing peak — practical diagnosis and remediation guide
• Reduce costs migrating Magento to OpenMage with a rollback plan and acceptance tests