How I Handle Webhook Concurrency in NestJS (Idempotency First, Panic Never)

If webhooks had a personality, they’d be the type to say:

“Hey, this is important.”

…and then send the same message five times just to be safe.

That’s not a bug. That’s how webhooks work.

So instead of asking “Why is this duplicated?”, I design my system around one rule:

Every webhook must be idempotent.

Everything else exists to support that rule.

First Principle: Webhooks Are Not Trustworthy

Webhook providers will:

• Retry on timeout
• Retry on network failure
• Retry even when you already processed it
• Send duplicates with the same event ID

So concurrency handling without idempotency is just coping.

Let’s do it properly.

1. Idempotency Keys: The Main Character

Every webhook payload usually has:

• event_id
• id
• request_id
• or some unique reference

That becomes my idempotency key.

idempotency_key = webhook:{provider}:{eventId}

If I’ve already processed this key, I do nothing.

No re-processing. No side effects. No drama.

Idempotency Storage Strategy

I store processed keys in Redis via @nestjs-redis/lock:

• Fast
• TTL-based
• Shared across instances

TTL example:

• 24 hours
• Or based on the provider retry window

NestJS Idempotency Service

import { Injectable } from "@nestjs/common";
import { RedisService } from "@liaoliaots/nestjs-redis";
import Redis from "ioredis";

@Injectable()
export class WebhookIdempotencyService {
  private readonly redis: Redis | null;

  constructor(private readonly redisService: RedisService) {
    this.redis = this.redisService.getOrThrow();
  }

  async isProcessed(key: string): Promise<boolean> {
    return (await this.redis.exists(key)) === 1;
  }

  async markProcessed(key: string) {
    await this.redis.set(key, "1", "EX", 60 * 60 * 24);
  }
}

2. Controller: Accept Fast, Judge Later

Webhook controllers should:

• Validate
• Enqueue using BullMQ
• Respond 200 OK

Speed matters. Providers hate slow endpoints.

import { Controller, Post, Body } from "@nestjs/common";
import { WebhookQueueService } from "./webhook-queue.service";
import { WebhookPayload } from "./types";

@Controller("webhooks")
export class WebhookController {
  constructor(private readonly queue: WebhookQueueService) {}

  @Post()
  async handle(@Body() payload: WebhookPayload) {
    await this.queue.enqueue(payload);
    return { ok: true };
  }
}

3. Queue + EventEmitter: Decouple Processing

Use @nestjs/bullmq for persistence and retries. Use @nestjs/event-emitter for domain events.

import { Injectable } from "@nestjs/common";
import { Queue } from "bullmq";
import { InjectQueue } from "@nestjs/bullmq";
import { EventEmitter2 } from "@nestjs/event-emitter";
import { WebhookPayload } from "./types";

@Injectable()
export class WebhookQueueService {
  constructor(
    @InjectQueue("webhooks") private queue: Queue,
    private readonly emitter: EventEmitter2
  ) {}

  async enqueue(payload: WebhookPayload) {
    await this.queue.add("process", payload);
  }

  async processJob(payload: WebhookPayload) {
    this.emitter.emit("webhook.received", payload);
  }
}

4. RxJS: Debounce the Noise, Not the Truth

Idempotency stops duplicates. RxJS handles event storms.

I use RxJS to:

• Group events by resource
• Process only the latest state
• Avoid useless repeated work

import { Injectable } from "@nestjs/common";
import { Subject } from "rxjs";
import { groupBy, mergeMap, debounceTime, takeLast } from "rxjs/operators";
import { WebhookPayload } from "./types";
import { WebhookIdempotencyService } from "./webhook-idempotency.service";
import { RedlockService } from "@nestjs-redis/lock";

@Injectable()
export class WebhookService {
  private webhook$ = new Subject<WebhookPayload>();

  constructor(
    private readonly idempotency: WebhookIdempotencyService,
    private readonly redlock: RedlockService
  ) {
    this.webhook$
      .pipe(
        groupBy((e) => e.resourceId),
        mergeMap((group) => group.pipe(debounceTime(300), takeLast(1)))
      )
      .subscribe((payload) => this.process(payload));
  }

  async enqueue(payload: WebhookPayload) {
    this.webhook$.next(payload);
  }

  async process(payload: WebhookPayload) {
    const key = `webhook:${payload.provider}:${payload.eventId}`;

    if (await this.idempotency.isProcessed(key)) return;

    try {
      const lock = await this.redlock.lock(
        [`lock:${payload.resourceId}`],
        3000
      );

      try {
        if (await this.idempotency.isProcessed(key)) return;

        await this.handleBusinessLogic(payload);
        await this.idempotency.markProcessed(key);
      } finally {
        await lock.unlock();
      }
    } catch (error) {
      // Lock not acquired, skip
    }
  }

  private async handleBusinessLogic(payload: WebhookPayload) {
    // Your domain logic here
  }
}

5. Database Versioning: The Final Judge

Idempotency prevents duplicates. Locks prevent concurrency. Database versioning prevents logic mistakes.

version INT NOT NULL

Safe update example (TypeORM):

const result = await this.orderRepository.update(
  { id: orderId, version: currentVersion },
  { status: newStatus, version: currentVersion + 1 }
);

If nothing updates:

• Data already changed
• Webhook is outdated
• Move on

How Everything Fits Together

Or in one sentence:

Idempotency first, concurrency second, database last.

Final Thoughts

Most webhook bugs come from one assumption:

“This request will only arrive once.”

It won’t.

Designing around idempotency keys, Redis locks, RxJS, BullMQ, and database versioning turns webhooks from unpredictable chaos into boring infrastructure — and boring is good.

Now when a provider retries the same webhook five times:

• My system shrugs
• Does nothing
• Goes back to minding its business