Skip to main content

Cloudsoft Zone Build, deploy and scale with practical guides to PHP, Node.js, React, DevOps and AI automation. Explore the guides

Cloudsoft Zone
SaaS and CloudDevOpsJavaScriptNode.jsTroubleshooting guide

BullMQ jobs stuck or stalled? How to find and fix the cause

Diagnose BullMQ jobs stuck in waiting or active, stalled twice-run jobs and vanishing keys: event-loop blocking, shutdown, Redis eviction and retry fixes.

Cloudsoft Zone

Updated 8 min read

Black-and-white photo of an empty airport baggage carousel under a sign for customs and lost and found
Photograph: djedj / CC0, via Wikimedia Commons
In this article

When BullMQ jobs sit in "waiting", hang in "active", run twice or fail with "job stalled more than allowable limit", the cause is almost always one of five things: no worker is consuming the queue, the worker blocks the Node.js event loop, workers are killed mid-job during deploys, Redis is evicting BullMQ's keys, or retries are configured in a way that hides the real error. This guide shows how to tell which one you have and how to fix it.

Everything below applies to BullMQ 6 (6.3.12 is the latest release on npm at the time of writing, October 2026) and most of it to BullMQ 5 as well. The code samples were type-checked against BullMQ 6.3.

Long line of students queuing along a brick building under green trees, waiting for a university library to open
Jobs in waiting are like a queue at a locked door: nothing moves until a worker opens it.

First, look at the queue

Before changing settings, get facts. This short script prints whether the queue is paused, how many workers are connected, the count of jobs in each state and the reasons for recent failures:

TypeScript
import { Queue } from 'bullmq';

const queue = new Queue('email', { connection: { host: '127.0.0.1', port: 6379 } });

console.log('paused:', await queue.isPaused());
console.log('workers:', await queue.getWorkersCount());
console.log(await queue.getJobCounts('waiting', 'active', 'delayed', 'failed', 'completed'));

for (const job of await queue.getFailed(0, 9)) {
  console.log(job.id, job.attemptsMade, job.failedReason);
}
await queue.close();

Then match what you see to the symptom table.

SymptomMost likely causeJump to
Jobs pile up in waiting, workers: 0No worker running, or it listens to a different queue name, prefix or RedisCause 1
Jobs pile up in waiting, paused: trueSomeone paused the queueCause 1
stalled events, jobs processed twice, "job stalled more than allowable limit"Event loop blocked longer than the lock, or workers killed mid-jobCauses 2 and 3
Jobs stuck in active after a crashNo running worker to detect the stall and move them backCause 3
"Missing key for job" or "Missing lock for job" errors, jobs vanishingRedis evicting keys under memory pressureCause 4
Failures retried instantly, or never retriedNo backoff, attempts left at 1, or errors that should not be retriedCause 5

Cause 1: Nothing is consuming the queue

If getWorkersCount() returns 0, the producer and the worker are not talking to the same queue. Check, in this order:

  • The worker process is running and did not crash on start-up. Look at its logs, not the web server's.
  • Queue names match exactly, including case. new Queue('emails') and new Worker('email', ...) are two different queues.
  • The key prefix matches. BullMQ prefixes Redis keys with bull by default; if one side sets a custom prefix, both must.
  • Both sides use the same Redis host, port and database number. A worker reading database 0 never sees jobs added to database 1.
  • The queue is not paused. If isPaused() is true, call queue.resume() and find out who paused it.

Cause 2: The worker blocks the event loop

This is the classic stalled-job cause. BullMQ gives each active job a lock in Redis and the worker keeps renewing it in the background. The documentation explains that an active job must continuously tell the queue the worker is still alive; if it does not, the job is treated as stalled, moved back to waiting (or to failed), and the queue emits a stalled event.

The defaults in BullMQ 6 are a lockDuration of 30 seconds, a stalled check every 30 seconds (stalledInterval) and maxStalledCount of 1. After more stalls than maxStalledCount allows, the job fails permanently with "job stalled more than allowable limit".

Lock renewal is a timer on the same event loop as your job code. If a job spends 40 seconds in synchronous work (resizing images, parsing a huge CSV, generating PDFs, a tight loop), the renewal never runs, the lock expires and another worker picks up the same job while the first copy is still running. That is why stalls often show up as duplicate emails or double-charged webhooks.

Fixes

  • Move CPU-heavy work into a sandboxed processor. BullMQ can run the processor in a separate process or worker thread, so the main worker keeps renewing locks. Its documentation says sandboxed processors "will not result in stalled jobs as easily as standard workers".
  • Break large jobs into smaller ones, such as one job per 1,000 rows instead of one job per file.
  • Raise lockDuration only as a stopgap for jobs that legitimately block for a while. It also delays recovery from real crashes.
  • Keep concurrency realistic. Fifty concurrent jobs on one Node.js process share one event loop.
TypeScript
// worker.ts: the heavy work runs in a separate process
import { Worker } from 'bullmq';
import { fileURLToPath } from 'node:url';

const processor = fileURLToPath(new URL('./render-pdf.processor.js', import.meta.url));
new Worker('pdf', processor, { connection, concurrency: 2 });

The processor file default-exports an async function that receives the job, exactly like an inline processor.

Travellers collecting suitcases from a baggage belt in the arrivals hall at Zurich Airport
A healthy queue keeps moving: workers pick up jobs, finish them and release their locks.

Cause 3: Workers are killed mid-job

Deploys, autoscaling and out-of-memory kills all stop workers. If the process exits without releasing its jobs, those jobs stay "active" until another worker's stalled checker notices the expired lock. If no worker is running at all, they stay active indefinitely; they recover only when a worker starts again.

BullMQ's production guide asks you to close workers on both SIGINT and SIGTERM with await worker.close(), which lets the current jobs finish before exit. Give the process enough time: Docker sends SIGKILL 10 seconds after SIGTERM by default, so set a longer stop grace period for workers whose jobs run longer than that, or keep jobs short.

Here is a complete worker with sensible defaults, retries and graceful shutdown:

TypeScript
import { Queue, Worker, QueueEvents, UnrecoverableError, type Job } from 'bullmq';

const connection = {
  host: process.env.REDIS_HOST ?? '127.0.0.1',
  port: Number(process.env.REDIS_PORT ?? 6379),
};

type EmailJob = { to?: string; template: string };

export const emailQueue = new Queue<EmailJob>('email', {
  connection,
  defaultJobOptions: {
    attempts: 5,
    backoff: { type: 'exponential', delay: 2_000 },
    removeOnComplete: { count: 1_000 },
    removeOnFail: { age: 7 * 24 * 3600 },
  },
});

async function sendEmail(data: EmailJob): Promise<void> {
  // call your email provider here
}

const worker = new Worker<EmailJob>(
  'email',
  async (job: Job<EmailJob>) => {
    if (!job.data.to) throw new UnrecoverableError('missing recipient');
    await sendEmail(job.data);
  },
  { connection, concurrency: 10, lockDuration: 60_000 },
);

worker.on('failed', (job, err) => {
  console.warn(`job ${job?.id} failed after ${job?.attemptsMade} attempts: ${err.message}`);
});
worker.on('error', (err) => console.error('worker error', err));

const events = new QueueEvents('email', { connection });
events.on('stalled', ({ jobId }) => console.warn(`job ${jobId} stalled`));

const shutdown = async () => {
  await worker.close();
  await events.close();
  await emailQueue.close();
  process.exit(0);
};
process.on('SIGTERM', shutdown);
process.on('SIGINT', shutdown);

Two lines in it prevent common incidents. The error listener stops Redis connection errors from surfacing as unhandled exceptions, and removeOnComplete and removeOnFail keep finished jobs from accumulating in Redis forever.

Cause 4: Redis is evicting BullMQ's keys

BullMQ stores every job, lock and list as Redis keys. If Redis reaches its memory limit with an eviction policy such as allkeys-lru, it deletes keys to make room, and BullMQ loses locks and job data. The symptoms are confusing: jobs disappear, "Missing key for job" or "Missing lock for job" errors appear, and stalls happen even though workers are healthy.

BullMQ's production guide states that maxmemory-policy must be noeviction, the only setting that guarantees correct behaviour. It also recommends enabling AOF persistence so jobs survive a Redis restart. Check your instance:

Shell
redis-cli CONFIG GET maxmemory-policy
redis-cli INFO memory | grep -E 'used_memory_human|maxmemory_human|maxmemory_policy'
redis-cli INFO stats | grep evicted_keys

# Fix on a self-managed Redis (also set it in redis.conf so it survives restarts)
redis-cli CONFIG SET maxmemory-policy noeviction

A non-zero evicted_keys count on a queue Redis is a red flag. On managed Redis services, the policy is usually a parameter in the provider's console rather than something you can change with CONFIG SET. Avoid sharing one Redis instance between a cache (which wants eviction) and a queue (which must never evict); run two instances or two clusters instead.

Cause 5: Retries that hide the real problem

BullMQ only retries a job when attempts is greater than 1. Without a backoff, retries happen immediately, which hammers a struggling third-party API and burns all attempts within a second. With { type: 'exponential', delay: 2000 }, BullMQ waits 2 ^ (attempts - 1) * delay milliseconds: roughly 2, 4, 8 and 16 seconds for the retries in the example above.

Some errors should never be retried: invalid input, a deleted user, a permanently rejected card. Throw UnrecoverableError for those so the job fails straight away instead of retrying four more times. Always throw real Error objects; BullMQ's documentation notes that it relies on them to work correctly.

Make jobs safe to run twice

Even with every fix above, a crash at the wrong moment can run a job twice. Design processors to be idempotent:

  • Use a stable jobId when adding jobs that must happen once per business event (for example invoice-123-receipt); BullMQ ignores a second job with the same ID while the first is still in the queue. Once removeOnComplete or removeOnFail deletes it, the ID can be used again, so keep your own completion record too.
  • Record completion in your database inside the same transaction as the side effect where you can, and check it at the start of the job.
  • Pass idempotency keys to payment and email APIs that support them.

What to do next

  1. Run the inspection script against production and save the output as a baseline.
  2. Confirm maxmemory-policy is noeviction and AOF is enabled on the queue Redis.
  3. Add the stalled and failed listeners to your logging and alert when stalls exceed zero for more than a few minutes.
  4. Find the slowest synchronous job and move it into a sandboxed processor or split it.
  5. Check your worker's stop grace period against your longest job.

Frequently asked questions

  • What does “job stalled more than allowable limit” mean in BullMQ?

    The job's lock expired more times than maxStalledCount allows (1 by default), so BullMQ failed it permanently. Usually the worker blocked the event loop for longer than lockDuration (30 seconds by default) or was killed while processing.

  • Should I just increase lockDuration?

    Only as a stopgap. A longer lock hides event-loop blocking but also delays recovery when a worker really crashes. The lasting fix is to move CPU-heavy work into a sandboxed processor or split it into smaller jobs.

  • Why are my BullMQ jobs processed twice?

    A job whose lock expires is moved back to waiting and picked up again, even if the first worker is still running it. Fix the stall, and make processors idempotent with stable job IDs and completion checks so a second run does no harm.

  • What Redis eviction policy does BullMQ need?

    noeviction. BullMQ's production guide says it is the only setting that guarantees correct behaviour. With any eviction policy, Redis can delete locks and job data under memory pressure.

  • How do I stop BullMQ retrying an error that will never succeed?

    Throw UnrecoverableError from the processor. The job moves to failed immediately instead of using its remaining attempts.

  • Do I need a QueueScheduler in BullMQ?

    No. The QueueScheduler class is not needed since BullMQ 2.0; workers run the stalled-job checks themselves. That is also why stalled active jobs only recover when at least one worker is running.

Advertise on CloudSoftZone

Your next customer could start right here.

Introduce your tool, product or business to readers exploring technology.

Advertise with us

Explore placements and request a private quote

Sources

  1. Stalled Jobs — BullMQ
  2. Going to production — BullMQ
  3. Sandboxed processors — BullMQ
  4. Retrying failing jobs — BullMQ
  5. Changelog — BullMQ
  6. Job Ids — BullMQ
  7. docker container stop — Docker

Written with substantial AI assistance. Facts checked against cited sources. Editorial policy.

Comments

No comments yet. Be the first to join the conversation.

Leave a comment

Every comment is read by a moderator before it appears. Your email address is never published; it is used only to contact you about this comment.

Never published. Used only to contact you about this comment.

Loading the security check…

Comments are moderated before publication.