Utilizing BullMQ in Combination with NestJS for Queue Name Management

Question

Utilizing BullMQ in Combination with NestJS for Queue Name Management

Currently, I am in the process of developing an application with BullMQ and NestJS. Everything seems to be working smoothly, but there is a particular issue that is bothering me.

Whenever I register a new queue within my application, I typically follow this approach:


    @Module({
       imports: [             
             BullModule.registerQueueAsync({
              name: 'email-queue',
           }),
      ],
      controllers: [EmailController],
      providers: [ ],
    })

Subsequently, when utilizing it within a NestJS service, my code would look something like this:


@Injectable()
export class WhatsappQueueService {
  constructor(
    @InjectQueue('email-queue') private queue: Queue,
  ) {}

  async addMessage(dtoSendMessage: any) {
    const included = await this.queue.add(
      'send-message',
      dtoSendMessage,
    );
    return included;
  }

}

The dilemma arises when I attempt to create a constant with the value 'email-queue', export it from the service, and incorporate it into the Module definition. This way, manual management of queue names can be avoided.

However, upon using the constant, an error is triggered indicating that NestJS is unable to locate the Queue. I suspect this issue is related to the @InjectQueue() decorator.

Is there a means by which constants can be utilized for naming queues?

javascript typescript nestjs bull bullmq

Answer 1

Answer №1

Absolutely, constants can be utilized

src/queue/constants/queue.constants.ts

export const QUEUE_NAME = 'add-email-vectors-queue';
export const QUEUE_PROCESS = 'save';

export const EMAIL_CACHE_QUEUE_PROCESS = 'email-cache';
export const EMAIL_CACHE_QUEUE_NAME = 'email-cache-queue';

 BullModule.registerQueue({
      name: QUEUE_NAME,
      defaultJobOptions: {
        delay: 1000,
        removeOnComplete: true,
        removeOnFail: true,
        attempts: 3,
      },
    }),

import {
  EMAIL_CACHE_QUEUE_NAME,
  EMAIL_CACHE_QUEUE_PROCESS,
} from './constants/queue.constants';

@Processor(EMAIL_CACHE_QUEUE_NAME)
export class EmailCacheQueueProcessService {
  private readonly logger = new Logger(EmailCacheQueueProcessService.name);

  constructor(
    private readonly outlookService: OutlookService,
    private readonly cacheStoreService: CacheStoreService,
  ) {}

  @Process(EMAIL_CACHE_QUEUE_PROCESS)
  async saveEmailsCacheProcess(job: Job) {
    const emails = await this.outlookService.getEmailsForCaching(
      job.data.token,
      1,
      100,
    );

   
    this.logger.log(`Job completed for ${emails.length} emails in ${EMAIL_CACHE_QUEUE_NAME}`);
  }
}

import { InjectQueue } from '@nestjs/bull';
import { Injectable } from '@nestjs/common';
import { Queue } from 'bull';
import { QUEUE_NAME, QUEUE_PROCESS } from './constants/queue.constants';

@Injectable()
export class QueueService {
  constructor(
    @InjectQueue(QUEUE_NAME)
    private queue: Queue,
  ) {}

  /**
   * @param text email body and the id of the email
   * @param userId user id of the database
   * @description Save emails to vector database from queue
  */
  async saveVectorsFromQueue(
    text: { text: string; id: string }[],
    userId: string,
  ) {
    await this.queue.add(QUEUE_PROCESS, {
      text,
      userId,
    });

    console.log('Job added to queue');
  }
}

Utilizing constants is recommended for naming queues.

Answer 2

Absolutely, constants can be utilized

src/queue/constants/queue.constants.ts

export const QUEUE_NAME = 'add-email-vectors-queue';
export const QUEUE_PROCESS = 'save';

export const EMAIL_CACHE_QUEUE_PROCESS = 'email-cache';
export const EMAIL_CACHE_QUEUE_NAME = 'email-cache-queue';

 BullModule.registerQueue({
      name: QUEUE_NAME,
      defaultJobOptions: {
        delay: 1000,
        removeOnComplete: true,
        removeOnFail: true,
        attempts: 3,
      },
    }),

import {
  EMAIL_CACHE_QUEUE_NAME,
  EMAIL_CACHE_QUEUE_PROCESS,
} from './constants/queue.constants';

@Processor(EMAIL_CACHE_QUEUE_NAME)
export class EmailCacheQueueProcessService {
  private readonly logger = new Logger(EmailCacheQueueProcessService.name);

  constructor(
    private readonly outlookService: OutlookService,
    private readonly cacheStoreService: CacheStoreService,
  ) {}

  @Process(EMAIL_CACHE_QUEUE_PROCESS)
  async saveEmailsCacheProcess(job: Job) {
    const emails = await this.outlookService.getEmailsForCaching(
      job.data.token,
      1,
      100,
    );

   
    this.logger.log(`Job completed for ${emails.length} emails in ${EMAIL_CACHE_QUEUE_NAME}`);
  }
}

import { InjectQueue } from '@nestjs/bull';
import { Injectable } from '@nestjs/common';
import { Queue } from 'bull';
import { QUEUE_NAME, QUEUE_PROCESS } from './constants/queue.constants';

@Injectable()
export class QueueService {
  constructor(
    @InjectQueue(QUEUE_NAME)
    private queue: Queue,
  ) {}

  /**
   * @param text email body and the id of the email
   * @param userId user id of the database
   * @description Save emails to vector database from queue
  */
  async saveVectorsFromQueue(
    text: { text: string; id: string }[],
    userId: string,
  ) {
    await this.queue.add(QUEUE_PROCESS, {
      text,
      userId,
    });

    console.log('Job added to queue');
  }
}

Utilizing constants is recommended for naming queues.

Utilizing BullMQ in Combination with NestJS for Queue Name Management

Answer №1

Similar questions

Define the data type for the toObject function's return value

HackerRank Challenge: Strategies for Efficiently Solving Minimum Swaps 2

Expecting commitment on the horizon with the utilization of async/await in a TypeScript

Revamping the website to become a Progressive Web App

What is the best way to bring a nested object to the top level without deleting the original top level

Can someone please explain the result of console.log(error) and how can I convert it into a string?

What could be the reason for my function not being executed in this particular scenario with my calculator HTML code?

My form does not receive the Bootstrap classes when using the jQuery script

reveal concealed section and seamlessly glide to specified location inside the secret compartment

Submitting data twice through AJAX POST requests

Adding fields from one collection to another collection in MongoDB based on specific conditions for a considerable amount of data

Components in Angular modules that are loaded lazily with identical names

Even though I have the image button's ID, I am unable to successfully click on it

What is the best way to configure the default entry point for a package.json file in a React

Navigate to a new page on button click using Row with Tanstack / React-Table and Typescript (2339)

Trouble setting custom attribute tags in Ionic 4

Is there a way to navigate to the home page using a PNG icon without refreshing the page when clicked on, while also utilizing NavLink with a route of '/'?

Incorporating URL addresses and pagination features using React.Js and Material-UI components

Error: Attempting to use Moment with Bootstrap Date Time Picker results in TypeError, as Object(...) is not recognized as

Tips for navigating through elements generated with ng-repeat as I scroll