Message Queues

RabbitMQ Queues & n8n Integration

Overview

This page details the implementation and registry of RabbitMQ message queues within our system. Our RabbitMQ infrastructure is primarily utilized by the n8n automation system to decouple processes, manage event-driven workloads, and prevent rate-limiting during large data operations.

Naming Convention Strategy

To ensure consistency and make debugging easier, all n8n-related queues follow a strict naming pattern:

dpe.internal.trigger.[workflowname]

Here is the breakdown of this naming taxonomy:

• dpe: Stands for Data Processing Unit. This namespaces the queue to data-specific operations.
• internal: Indicates that this queue is exclusively used internally by the DPE and data tasks. It is fully isolated and not shared with the main backend application.
• trigger: Denotes the operational purpose of the queue—messages pushed here are specifically meant to trigger an n8n workflow.
• [workflowname]: Directly maps the queue to the downstream workflow it activates. This 1:1 mapping makes it incredibly easy for developers to identify and trace connected workflows.

Queue Registry

The table below tracks our active RabbitMQ queues, their intended purpose, and the specific n8n workflows they trigger.

Note: When adding new queues to the system, please ensure this table is updated.