Understanding RabbitMQ Classic Mirrored Queues and Quorum Queues

💚💚💚Understanding RabbitMQ Classic Mirrored Queues and Quorum Queues💚💚💚

 I understood RabbitMQ only after I understood why Classic Mirrored Queues were replaced.

When I first started learning RabbitMQ, I kept hearing:

  • Classic Queues

  • Classic Mirrored Queues

  • Quorum Queues

  • Raft Consensus

Everyone said: "Use Quorum Queues."

But I couldn't understand why.

To understand Quorum Queues, I first had to understand how Classic Mirrored Queues worked.

💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚  

RabbitMQ Queue Types

Historically, RabbitMQ supported:

  1. Classic Queue

  2. Classic Mirrored Queue

  3. Quorum Queue

  4. Stream Queue

Today, Classic Mirrored Queues are being phased out and RabbitMQ recommends Quorum Queues for replicated workloads.


Why RabbitMQ Exists

Imagine a developer opens a GitHub Pull Request.

Without RabbitMQ:


Problems:

  • What if the worker is unavailable?

  • What if all workers are busy?

  • What if hundreds of PRs are opened at once?

  • What if a worker crashes while processing a build?

Build requests could be delayed or lost.


RabbitMQ Solution

Instead of sending requests directly to workers:






RabbitMQ acts as a message broker.

Its job is to:

  • Receive build requests

  • Store them safely

  • Deliver them to available workers

  • Ensure requests are not lost

 RabbitMQ Core Flow



Using  CI/CD example:

  • Producer = GitHub Webhook

  • Exchange = RabbitMQ Router

  • Queue = Build Queue

  • Consumer = Build Worker


1. Classic Queue

A Classic Queue is the simplest queue type.

Messages are stored in one queue.

If the node hosting the queue fails, the queue may become unavailable.

Why Classic Mirrored Queues Were Introduced

What happens if the RabbitMQ node crashes?

Build requests may be lost.

To solve this problem, RabbitMQ introduced Classic Mirrored Queues.


2. Classic Mirrored Queue

Suppose we have five RabbitMQ nodes.

Node1
Node2
Node3
Node4
Node5

Policy: ha-mode = all

The queue is replicated across all nodes.

example: Classic Mirrored Queue

Every queue has:

  • One Master

  • Multiple Mirrors

Every queue has its own master.

Example:

builds       -> Master Node3
deployments  -> Master Node1
emails       -> Master Node5
notifications -> Master Node2

One node can be master for many queues.

Node3 : builds, deployments, notifications

This can create hotspots.


Operational Challenges

Imagine Node3 is master for many busy queues.

Node3 handles:

  • Writes

  • Consumer delivery

  • ACK tracking

  • Replication

  • Mirror synchronization

Memory and CPU usage can become much higher on Node3 than other nodes.

During maintenance you need to know:

Which queues are mastered here?
Are mirrors synchronized?
Can I safely restart this node?
 

Mirror Synchronization Problem

Suppose Node5 is down for several hours. Meanwhile millions of build jobs are processed.

When Node5 returns:

Node5: it must synchronize again. This synchronization can be expensive and time 
consuming.


💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚 Enter Quorum Queues

Quorum Queues solve many of these problems using the Raft Consensus Algorithm.

Instead of: Master , Mirrors

we now have: Leader , Followers

 

Quorum Queue Architecture

Quorum Queue 


Each queue has:

  • One Leader

  • Multiple Followers

Every Queue Still Has Its Own Leader

Quorum Queues do NOT have one leader for the entire cluster. Each queue has its own leader.

Example:

builds        -> Leader Node3
deployments   -> Leader Node5
emails        -> Leader Node1
notifications -> Leader Node2

This is very similar to Classic Mirrored Queues.


What Is Different?

The biggest difference is not replication. Both systems replicate data. The biggest difference is consensus.

Raft ensures:

  • One leader at a time

  • Safe failover

  • Majority agreement

  • Faster recovery

What Is a Quorum?

A quorum means a majority of replicas.

Example:

3 replicas:

Node1
Node2
Node3

Majority: 2

Example:

5 replicas:

Node1
Node2
Node3
Node4
Node5

Majority: 3

A message is considered safe only after a majority acknowledges it.


Why Odd Numbers?

RabbitMQ commonly recommends: 3 replicas or 5 replicas....instead of: 4 replicas

With five replicas you can lose two nodes while still maintaining a majority.


Leader Election

Suppose:

Leader:
Node3

Followers:

Node5
Node7

Node3 crashes.

Followers automatically hold an election.

When Node3 returns: 

Node3 becomes Follower and synchronizes with the new leader. 

This process is automatic.

 💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚 

Why RabbitMQ and Amazon MQ Recommend Quorum Queues

Quorum Queues provide:

  • Better resiliency

  • Faster failure detection

  • Predictable failover

  • Safer data replication

  • Easier operations

  • Better handling of node failures

  • Automatic leader election

For this reason RabbitMQ and Amazon MQ recommend Quorum Queues as the preferred replicated queue type.

https://aws.amazon.com/blogs/compute/introducing-quorum-queues-on-amazon-mq-for-rabbitmq/

💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚 

→💚💚

Classic Mirrored Queue:

Master
   |
Mirrors

Focus: Replication

→ 💚💚

Quorum Queue:

Leader
   |
Followers

Focus: Replication + Consensus + Majority Agreement

The architecture looks similar, but the way failures, recovery, synchronization, and leadership are handled is fundamentally different.

 

💚💚💚The biggest lesson is:💚💚💚

Classic Mirrored Queues replicate data.

Quorum Queues replicate data and ensure all replicas agree on that data using the Raft Consensus Algorithm.

💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚💚 

Popular posts from this blog

☁️ AWS Global Accelerator (GA) + Route 53

🟧☁️ AWS + 🌍 Global Accelerator (GA) + 🌐 Route 53 what they are and how they work together (especially for ALB/EKS). ☁️ AWS = Cloud platform 🌍 GA = Fast global routing + static IPs 🌐 Route 53 = DNS ☁️ What is AWS? AWS (Amazon Web Services) is Amazon’s cloud platform where you run servers, containers, databases, networking, DNS, monitoring, and more. EC2 (servers), EKS (Kubernetes), RDS (databases) ALB/NLB (load balancers), S3 (storage), Route 53 (DNS) What is AWS Global Accelerator? Global Accelerator (GA) improves availability and performance by routing users through AWS’s private global network to the nearest healthy endpoint . Key idea: Users hit GA (static Anycast IPs). GA picks the best Region/endpoint automatically. Static Anycast IPs (2 fixed IP addresse...
Read more

🐳 Docker Filesystem Internals (AdvancEd)

Docker Filesystem Internals (Advanced) This post is for readers who already understand Docker basics and want to learn how Docker manages files internally. If you are new to Docker, read the beginner guide first. Containers Do NOT Have Their Own Disk A common misunderstanding is that Docker containers have their own disk. In reality: Containers do not have a separate physical filesystem. Docker creates a filesystem view using layers stored on the host machine. This layered filesystem is implemented using a Union Filesystem , most commonly OverlayFS on Linux. Docker Images Are Read-Only Layers A Docker image is made up of multiple read-only layers. Each instruction in a Dockerfile creates a new layer. FROM python:3.11 RUN pip install flask COPY app.py . This results in layers lik...
Read more

🐳 Docker Tutorial for Beginners: Step-by-Step with a Simple Example

Docker Explained Using a Very Simple App (Beginner Friendly) Goal: Build a tiny Python app, run it normally, then run the same app using Docker, and understand why Docker helps. Step 1: Create a Very Small Python App Create a file named app.py : print("Hello I am running inside Docker!") Step 2: Run the App WITHOUT Docker Run this on your machine: python app.py Common real-life problems: Python not installed, wrong Python version, missing dependencies, different OS behavior this is where the “works on my machine” issue starts. Step 3: Introduce Docker Instead of relying on your system, Docker lets you package the runtime (Python) together with your app. We’ll do that using a Dockerfile . Step 4: Create a Dockerfile Create a file named Dockerfile (no extension): FROM python:3.11 WORKDIR /app COPY app.py . CMD ["py...
Read more

☸️What’s Inside EKS? A Beginner’s Guide to Its Core Components

Image
 EKS (Elastic Kubernetes Service) EKS is a managed service by AWS that runs Kubernetes for you. It takes care of setting up and managing the Kubernetes control plane so you can focus on running your applications. Kubernetes  Kubernetes is an open-source system for automating the deployment, scaling, and management of containerized applications. It helps run apps reliably across a cluster of machines, automatically handling scheduling, scaling, updates, and recovery. Pod A Pod is the smallest unit in Kubernetes. It can hold one or more containers that share the same network and storage. All containers in a pod are scheduled and managed together. Container A Container is a lightweight, standalone package that includes everything an app needs to run — code, runtime, libraries, and dependencies. It ensures the app runs the same no matter where it’s deployed. 👀1 pod can have multiple containers but 1 container cannot belongs to multiple pods.
Read more

☸️ Kubernetes Taints and Tolerations(with Node Affinity)

Kubernetes Taints and Tolerations(with Node Affinity + EKS Upgrade Context) Kubernetes scheduling is the process of deciding which node runs a given Pod . This page explains how Kubernetes keeps Pods away from “inappropriate” nodes using taints (node-side restrictions) and tolerations (pod-side permissions), and how that relates to node affinity (pod-side attraction). 1) Core idea: attraction vs repulsion Node affinity (attraction) Node affinity is a property of Pods that attracts them to a set of nodes. You can express it as: Hard requirement (must match), or Preference (nice to have). Taints (repulsion) Taints are the opposite of node affinity. They allow a node to repel a set of Pods. Tolerations (permission) Tolerations are applied to Pods. A toleration allows the scheduler to schedule Pods onto nodes with matching taints. Important...
Read more

AWS Load Balancer Controller Upgrade Guide: v2.x to v3.3

Image
AWS Load Balancer Controller Upgrade Guide: v2.x to v3.3 What is AWS Load Balancer Controller? AWS Load Balancer Controller is a Kubernetes controller that integrates Amazon EKS with AWS Elastic Load Balancing services. It watches Kubernetes resources and automatically creates or updates AWS load balancing resources based on the desired state defined inside the cluster. Instead of manually creating load balancers, listeners, target groups, and security group rules in AWS, teams can define Kubernetes resources such as Ingresses and Services. The controller then reconciles those resources with AWS. What Does It Manage? AWS Load Balancer Controller commonly manages: Application Load Balancers (ALB) Network Load Balancers (NLB) Listeners and listener rules Target groups Target registration and deregistration Security group rules Ingress resources TargetGroupBinding resources Beca...
Read more

🐳 Build a Tiny Flask Web App in Docker (with Ports)

docker run -p 8080:5000 flask-hello Then open in your browser: http://localhost:8080 Step 6: Stop the container Press Ctrl + C in the terminal. Quick Troubleshooting Browser says “This site can’t be reached”: Make sure the container is running and you used the -p port mapping. Flask runs but not accessible: Confirm Flask is running with host="0.0.0.0" inside Docker (not 127.0.0.1 ). Port already in use: Try using a different host port, for example: docker run -p 9090:5000 flask-hello Next Post Ideas Docker image layers and caching (explained simply) Docker volumes : keeping data after a container stops Running the same container in Kubernetes (Pod basics)
Read more

☁️ Amazon S3 Explained: More Than Just Object Storage

Amazon S3 Explained: More Than Just Object Storage When people hear Amazon S3 , they usually think it is only used to store files like images, logs, backups, or documents. That is true, but S3 has grown into much more than simple object storage. Today, S3 can be used for object storage, file-like access, immutable backups, lifecycle management, static websites, event-driven workflows, analytics, and data lake storage. In this post, let us understand what S3 is and how it is used in real-world AWS environments. 1. What is Amazon S3? Amazon S3 stands for Simple Storage Service . It is mainly an object storage service . In S3, data is stored as objects inside buckets. Bucket → Object Each object usually contains: The actual data Metadata A unique object key Example: Bucket name: my-app-logs Object key: logs/2026/app.log Even though this looks like a folder path, S3 does not work like a traditional filesystem internally. The folder-like str...
Read more

☸️ CoreDNS and AWS VPC CNI in EKS

CoreDNS vs AWS VPC CNI in EKS: What Each Component Actually Does When working with Amazon EKS , two important components help your Kubernetes cluster communicate properly: CoreDNS and AWS VPC CNI . Both are part of the networking story, but they do very different jobs. CoreDNS helps pods find services by name, while AWS VPC CNI gives pods network connectivity using IP addresses from your AWS VPC. In simple words: CoreDNS answers: “What IP address belongs to this service name?” AWS VPC CNI answers: “How does traffic reach that pod over the network?” 1. What is CoreDNS? CoreDNS is the DNS server used inside Kubernetes. Its main job is to resolve Kubernetes service names into IP addresses. For example, if one pod wants to call another service using a name like: user-service.default.svc.cluster.local CoreDNS resolves that service name into the correct Kubernetes service IP. What CoreDNS Does Resolves Kubernetes service names Supports pod-to...
Read more