Skip to main content
Guides

Handling Webhook Failures

Understand how XRNotify retries failed deliveries, how to recover from extended outages with the Replay API, and how to build an idempotent handler that safely processes retried events.

Retry policy overview

XRNotify automatically retries a delivery whenever your endpoint returns a non-2xx HTTP status or does not respond within the 10-second timeout. Retries follow an exponential backoff schedule with ±10% jitter:

AttemptDelayTotal elapsed
1 (initial)Immediate0s
21 minute~1m
35 minutes~6m
430 minutes~36m
52 hours~2.6h
66 hours~8.6h

After 6 attempts the delivery status becomes failed. No further automatic retries occur, but you can trigger a manual retry at any time. See the full schedule in the Retry Policy reference.

Checking failed deliveries

Query the deliveries API with a status filter to see everything that failed for a given webhook:

curl "https://api.xrnotify.io/v1/deliveries?status=failed&webhook_id=wh_abc" \
  -H "X-XRNotify-Key: xrn_live_xxx"

Each delivery object in the response includes last_error, attempts, and next_retry_at so you can understand exactly what went wrong. You can also view this on the Deliveries page in the dashboard.

Manual retry (single delivery)

If a delivery has exhausted its automatic retries, or you want to force an immediate re-attempt before the next scheduled retry, call the retry endpoint:

curl -X POST https://api.xrnotify.io/v1/deliveries/dlv_abc123/retry \
  -H "X-XRNotify-Key: xrn_live_xxx"

The delivery will be dispatched immediately. The response includes the new status and response_code once the attempt completes.

Bulk recovery with the Replay API

If your endpoint was down for several hours and many deliveries failed, retrying them individually is impractical. The Replay API re-delivers all events matching a time window and optional event type filter in a single call:

curl -X POST https://api.xrnotify.io/v1/replay \
  -H "X-XRNotify-Key: xrn_live_xxx" \
  -H "Content-Type: application/json" \
  -d '{
    "webhook_id": "wh_abc123",
    "filters": {
      "since": "2024-01-15T00:00:00Z",
      "until": "2024-01-15T06:00:00Z",
      "event_types": ["payment.xrp"]
    }
  }'
Because the Replay API re-delivers events, your handler will receive X-XRNotify-Delivery-Id values that differ from the original delivery IDs. Ensure your idempotency key is derived from the event_id field in the body (which is deterministic) rather than the delivery ID header.

Idempotency

Because XRNotify retries deliveries, your endpoint may receive the same event more than once. Use the X-XRNotify-Delivery-Id header to record processed deliveries and skip duplicates:

app.post('/webhooks', express.raw({ type: 'application/json' }), async (req, res) => {
  // 1. Verify signature first
  if (!verifySignature(req.body, req.headers['x-xrnotify-signature'], process.env.WEBHOOK_SECRET)) {
    return res.status(401).end();
  }

  const deliveryId = req.headers['x-xrnotify-delivery-id'];

  // 2. Check if already processed
  const processed = await cache.get(`delivery:${deliveryId}`);
  if (processed) return res.sendStatus(200); // Already done — safe to ack

  // 3. Process event
  const event = JSON.parse(req.body);
  await processEvent(event);

  // 4. Mark as processed with a 24-hour TTL
  await cache.set(`delivery:${deliveryId}`, 'done', 86400);

  res.sendStatus(200);
});

Any in-memory store, Redis, or database table works for the idempotency cache. The 24-hour TTL is sufficient because XRNotify's retry window is under 9 hours.

Making your endpoint resilient

A well-designed webhook endpoint minimises the chance of generating failures in the first place:

  • Respond within 10 seconds. XRNotify's delivery timeout is 10 seconds. If your processing takes longer, push the event to an internal queue (SQS, BullMQ, etc.) immediately and acknowledge with 200 OK. Process asynchronously.
  • Always return 200 on receipt. Acknowledge the delivery as soon as you've safely queued the event — even if downstream processing fails. Track your own processing failures in a dead-letter queue separate from XRNotify's retry mechanism.
  • Monitor your own queue. XRNotify's retry mechanism ensures delivery to your endpoint, but it cannot guarantee that your internal processing succeeds. Set up alerts on your dead-letter queue depth.
  • Use a valid TLS certificate. Expired or self-signed certificates cause immediate failures. Use a certificate from a public CA and set up automatic renewal (e.g., Let's Encrypt with Certbot).
Warning: If your endpoint returns 5xx errors consistently, XRNotify may automatically disable the webhook after 100 consecutive failures. You will receive an email notification when this happens. Re-enable the webhook from the dashboard or via PATCH /v1/webhooks/{id} once your endpoint is healthy.

Delivery health dashboard

The Stats endpoint returns aggregate delivery metrics for a webhook over the last 24 hours:

curl "https://api.xrnotify.io/v1/webhooks/wh_abc123/stats" \
  -H "X-XRNotify-Key: xrn_live_xxx"

You can also view a real-time success rate chart and per-delivery logs on the Deliveries page in the dashboard — no API call needed.

Next steps