Rabbitmq Fast Revision
RABBITMQ FAST REVISION
GOAL:
Understand RabbitMQ internals, not memorize commands.
---
1. CORE ARCHITECTURE
---
Producer
↓
Exchange
↓ routing key + binding
Queue
↓
Consumer
Producer normally knows:
- Exchange
- Routing key
Producer does NOT need to know:
- Queue
- Consumer
- Binding
RabbitMQ admin defines queues + bindings.
Exchange = routing decision
Queue = stores messages
Consumer = processes messages
---
2. EXCHANGES
---
DIRECT:
Exact routing-key match.
topic.sms → queue A
topic.email → queue B
IMPORTANT:
Direct exchange does NOT support * or # wildcards.
"sms.*" is literal.
TOPIC:
Pattern matching.
* = exactly ONE word
= ZERO or MORE words
sms.* → sms.email
sms.* → sms.email.audit ❌
sms.# → sms
sms.# → sms.email
sms.# → sms.email.audit
FANOUT:
Ignores routing key.
Sends to every bound queue.
HEADERS:
Routes using message headers.
x-match = all / any
DEFAULT EXCHANGE:
Routing key is queue name.
Producer can directly publish to a queue.
---
3. QUEUE TYPES
---
CLASSIC:
Simple/lightweight queue.
QUORUM:
Replicated HA queue.
Uses Raft.
Example 3-member quorum:
Leader + Follower + Follower
3 members → tolerate 1 failure
5 members → tolerate 2 failures
IMPORTANT:
RabbitMQ Cluster ≠ automatic queue replication.
Cluster shares topology/metadata.
Queue replication depends on queue type/configuration.
STREAM:
Designed for high-throughput stream-like workloads.
---
4. CLUSTER
---
Cluster = multiple RabbitMQ nodes acting as one logical cluster.
Example:
rabbit1
rabbit2
rabbit3
Cluster shares metadata such as:
- exchanges
- bindings
- users
- vhosts
- permissions
NOT every queue/message is automatically replicated.
Quorum queue provides queue-level replication.
Node failure:
Quorum leader fails
→ remaining members elect new leader
→ clients connected to failed node lose connection
→ application must reconnect/recover
Cluster failover ≠ client connection failover.
---
5. RABBITMQ PORTS
---
5672 = AMQP
5671 = AMQP TLS
15672 = Management HTTP/API
15671 = Management HTTPS
4369 = EPMD
25672 = Erlang distribution (commonly)
Distribution port range can be configured, e.g.:
35672-35682
Always verify actual configuration before opening firewall ports.
Useful:
ss -lntp
rabbitmq-diagnostics listeners
epmd -names
---
6. CONNECTION vs CHANNEL
---
Connection = TCP connection.
Channel = lightweight logical session inside connection.
One connection:
├── Channel 1
├── Channel 2
└── Channel 3
Avoid creating a new TCP connection for every message/request.
Connection failure:
→ all channels on it are affected.
Channel failure:
→ other channels can continue.
---
7. PRODUCER CONFIRM
---
Publisher Confirm:
RabbitMQ confirms broker accepted the publish.
It does NOT mean:
"Consumer successfully processed message."
Publisher Confirm ≠ Consumer ACK
Flow:
Producer
↓ publish
RabbitMQ
↓ confirm
Producer
Consumer:
RabbitMQ
↓ message
Consumer
↓ ACK
RabbitMQ
---
8. ACK / NACK / REJECT
---
ACK:
Processing successful.
NACK/reject:
Processing failed.
requeue=true:
Message can be delivered again.
requeue=false:
No same-queue requeue.
Can go to DLX if configured.
BAD:
NACK + requeue=true forever
→ redelivery loop.
---
9. PREFETCH + CONSUMERS
---
Consumer count = number of workers.
Prefetch = outstanding message window per consumer.
Example:
5 consumers
prefetch = 10
≈ 5 × 10 = 50 outstanding messages
Higher prefetch:
+ throughput
- less fair distribution
- more outstanding work
prefetch=1:
Usually more balanced/fair.
---
10. COMPETING CONSUMERS
---
Same queue + multiple consumers:
Queue
/ | \
C1 C2 C3
Messages are distributed among consumers.
Multiple queues with matching bindings:
Exchange
├── Queue A → Consumer
└── Queue B → Consumer
Message can reach BOTH queues.
That is different from competing consumers.
---
11. MESSAGE ORDER
---
Single consumer:
Usually queue order.
Multiple consumers:
No global processing-order guarantee.
Requeue/failure/prefetch can affect application-observed order.
---
12. MESSAGE PROPERTIES
---
Common properties:
delivery_mode
content_type
message_id
correlation_id
reply_to
expiration
headers
Routing key is NOT a message property.
It is used by the exchange for routing.
---
13. DLX / DLQ
---
DLX = Dead Letter EXCHANGE
DLQ = Dead Letter QUEUE
DLX is NOT itself a queue.
Example:
Main Queue
↓ failure
DLX
↓
DLQ
Typical reasons:
- reject/nack with requeue=false
- TTL expiry
- max length/other dead-letter conditions
---
14. RETRY WITH TTL + DLX
---
Main Exchange
↓
Main Queue
↓ failure
Retry Exchange
↓
Retry Queue
TTL = 30 sec
↓ expiry
Main Exchange
↓
Main Queue
IMPORTANT:
DLX ≠ retry automatically.
Need explicit retry design.
Avoid infinite retry.
Use maximum attempts/business logic.
---
15. DURABILITY
---
Durable queue:
Queue survives broker restart.
Persistent message:
Message is stored persistently according to RabbitMQ's persistence mechanisms.
For durable messaging:
Use durable queue + persistent messages appropriately.
Quorum queues additionally provide replication.
Persistence ≠ replication.
---
16. TRANSACTIONS
---
AMQP transactions:
tx.select
publish
tx.commit
or
tx.rollback
Publisher confirms are generally preferred for performance-sensitive publishing.
---
17. RPC
---
RabbitMQ RPC commonly uses:
reply_to
correlation_id
Request:
Producer → request queue
Response:
Consumer → reply queue
correlation_id identifies which response belongs to which request.
---
18. AUTHENTICATION vs AUTHORIZATION
---
Authentication:
WHO are you?
Authorization:
WHAT are you allowed to do?
Local user:
rabbitmqctl add_user appuser password
Vhost:
rabbitmqctl add_vhost /production
Permissions:
rabbitmqctl set_permissions -p /production appuser "." "." ".*"
Permission order:
configure
write
read
User tags mainly affect management/UI capabilities.
---
19. LDAP / AD
---
LDAP can provide authentication.
RabbitMQ still needs authorization rules/permissions.
Can configure:
internal
LDAP
LDAP:
389
LDAPS:
636
Keep a local break-glass admin when appropriate.
---
20. TLS
---
AMQP TLS:
5671
Important files:
CA certificate
Server certificate
Private key
Typical:
ssl_options.cacertfile
ssl_options.certfile
ssl_options.keyfile
ssl_options.verify
mTLS:
verify_peer
fail_if_no_peer_cert = true
"unable to get local issuer certificate"
→ client cannot build a trusted certificate chain.
curl -k:
Disables certificate verification.
Useful only for testing, NOT a security fix.
Hostname/SAN must also match.
---
21. FEDERATION
---
Federation connects INDEPENDENT RabbitMQ brokers/clusters.
Cluster:
One logical RabbitMQ cluster.
Federation:
Independent brokers connected together.
Federation mental model:
Upstream
↓
Federation
↓
Downstream
Federated queue:
Upstream Queue
↓
Federation
↓
Downstream Queue
↓
Application Consumer
Federated exchange:
Upstream Exchange
↓
Federation
↓
Downstream Exchange
↓
Queue
↓
Consumer
Downstream federated queue behaves somewhat like a consumer of upstream.
SHOVEL:
Source Queue
↓
Shovel
↓
Destination Exchange/Queue
Easy memory:
FEDERATION = topology/following relationship
SHOVEL = explicit message movement
Upstream definition:
WHERE/HOW to connect.
Federation policy:
WHAT object to federate.
Policy is configured on DOWNSTREAM.
---
22. FEDERATION FAILURE
---
If upstream is unavailable:
Federation link breaks
↓
Reconnect attempts
↓
Already transferred downstream messages remain downstream
↓
Messages existing only upstream remain upstream
Downstream can continue processing messages it already has.
---
23. MONITORING — BASIC HEALTH
---
rabbitmq-diagnostics ping
→ Is RabbitMQ responding?
rabbitmq-diagnostics status
→ What is happening on this node?
rabbitmqctl cluster_status
→ What does the cluster look like?
rabbitmq-diagnostics listeners
→ Which ports is RabbitMQ listening on?
---
24. QUEUE MONITORING
---
rabbitmqctl list_queues name messages messages_ready messages_unacknowledged
READY:
Waiting for delivery.
UNACKED:
Delivered to consumer but ACK not received.
MOST IMPORTANT:
READY ↑
→ consumer capacity/availability problem likely.
UNACKED ↑
→ consumers received messages but aren't ACKing fast enough.
---
25. MESSAGE RATES
---
Publish rate:
Producer → RabbitMQ
Delivery rate:
RabbitMQ → Consumer
ACK rate:
Consumer → RabbitMQ
Redelivery rate:
Messages being delivered again.
Example:
Publish = 1000/s
Consumer/ACK = 400/s
→ backlog grows.
Example:
Delivery = 500/s
ACK = 100/s
→ UNACKED grows.
Always correlate:
READY + UNACKED + rates.
---
26. CONNECTION MONITORING
---
rabbitmqctl list_connections
Connection = TCP.
Too many connections may indicate:
- connection leak
- bad application design
- reconnect storm
- excessive application instances
Do not create a new RabbitMQ TCP connection for every message.
---
27. CHANNEL MONITORING
---
rabbitmqctl list_channels
Channel = lightweight logical session inside a connection.
Useful when investigating:
- excessive channels
- consumer channels
- unacked messages
- channel failures
---
28. CONSUMER MONITORING
---
rabbitmqctl list_consumers
Check:
- Is consumer connected?
- Which queue?
- Consumer count
- ACK behavior
- Prefetch
- Channel
Scenario:
READY = 50,000
UNACKED = 0
Consumers = 0
→ Consumer unavailable/configuration/connectivity problem likely.
Scenario:
READY = increasing
Consumers exist
→ Consumer capacity may be lower than producer rate.
Scenario:
UNACKED = increasing
→ Consumer processing/ACK problem likely.
---
29. MEMORY / DISK ALARMS
---
RabbitMQ protects itself from resource exhaustion.
Memory alarm:
Memory usage exceeds configured threshold.
Disk alarm:
Available disk space becomes too low.
Check:
rabbitmq-diagnostics alarms
Important:
RabbitMQ process UP
≠
RabbitMQ accepting publishes normally
A resource alarm can cause publishers to be blocked.
---
30. UNACKNOWLEDGED TROUBLESHOOTING
---
Message flow:
Queue
↓
Consumer receives
↓
UNACKED
↓
Application processes
↓
ACK
↓
RabbitMQ removes message
If consumer is slow:
UNACKED ↑
If DB/API is slow:
UNACKED ↑
If consumer hangs:
UNACKED ↑
If consumer crashes before ACK:
Message can be redelivered.
High prefetch can increase outstanding messages.
---
31. PRODUCTION TROUBLESHOOTING FLOW
---
Problem:
"RabbitMQ messages are stuck."
STEP 1:
Is RabbitMQ alive?
rabbitmq-diagnostics ping
STEP 2:
Node healthy?
rabbitmq-diagnostics status
STEP 3:
Cluster healthy?
rabbitmqctl cluster_status
STEP 4:
Check listeners/network.
rabbitmq-diagnostics listeners
ss -lntp
STEP 5:
Check queue.
messages_ready
messages_unacknowledged
STEP 6:
Check rates.
publish
delivery
ACK
redelivery
STEP 7:
Check consumers.
rabbitmqctl list_consumers
STEP 8:
Check connections/channels.
rabbitmqctl list_connections
rabbitmqctl list_channels
STEP 9:
Check resource alarms.
rabbitmq-diagnostics alarms
STEP 10:
Check consumer application logs,
DB/API dependencies and processing time.
---
32. FAST PRODUCTION PATTERNS
---
READY ↑ + UNACKED low
→ Consumers unavailable/slow capacity.
READY ↑ + UNACKED ↑
→ Incoming rate > processing capacity.
READY low + UNACKED ↑
→ Consumer processing/ACK problem.
Consumers = 0
→ Consumer/application/config/network problem.
RabbitMQ UP + publishing blocked
→ Check resource alarms.
Consumer crashes before ACK
→ Unacked messages can be redelivered.
Redelivery ↑↑
→ Check failures/requeue loops.
---
33. GOLDEN MENTAL MODEL
---
When troubleshooting RabbitMQ:
PRODUCER
↓
PUBLISH RATE
↓
EXCHANGE
↓
ROUTING/BINDING
↓
QUEUE
↓
READY
↓
DELIVERY
↓
CONSUMER
↓
PROCESSING
↓
ACK
↓
MESSAGE COMPLETE
At every stage ask:
"Where is the message currently stuck?"
---
34. MOST IMPORTANT COMMANDS
---
rabbitmq-diagnostics ping
rabbitmq-diagnostics status
rabbitmq-diagnostics listeners
rabbitmq-diagnostics alarms
rabbitmqctl cluster_status
rabbitmqctl list_queues
rabbitmqctl list_consumers
rabbitmqctl list_connections
rabbitmqctl list_channels
ss -lntp
epmd -names
journalctl -u rabbitmq-server
---
35. INTERVIEW GOLD
---
RabbitMQ Cluster:
Multiple nodes forming one logical cluster.
Quorum Queue:
Replicated queue using Raft.
Exchange:
Routes messages to queues.
Binding:
Defines exchange → queue routing relationship.
Routing key:
Input used by exchange for routing.
READY:
Waiting for consumer delivery.
UNACKED:
Delivered but not ACKed.
Prefetch:
Limits outstanding messages per consumer.
Publisher Confirm:
Broker accepted publish.
Consumer ACK:
Consumer successfully processed/accepted responsibility.
DLX:
Exchange receiving dead-lettered messages.
DLQ:
Queue storing dead-lettered messages.
Federation:
Connects independent RabbitMQ brokers/clusters.
Shovel:
Moves messages explicitly from source to destination.
Authentication:
Who are you?
Authorization:
What can you do?
TLS:
Protects communication.
---
ONE-LINE TROUBLESHOOTING RULE
DON'T RESTART FIRST.
CHECK:
HEALTH
↓
CLUSTER
↓
QUEUE
↓
READY / UNACKED
↓
RATES
↓
CONSUMERS
↓
CONNECTIONS
↓
ALARMS
↓
APPLICATION / DB / NETWORK