Message Broker Questions/Concerns

Since the last post, I have narrowed my queue search down to two brokers; Qpid and RabbitMQ.  The following is a list of questions that focus on the management aspect of each broker.  What I have found is that management doesn’t seem to be a major focus of either broker, but both do have some support built in.  RabbitMQ suggests users develop plugins to meet their specific needs, but since the plugins need to be written in Erlang, it would require learning a new language (which I’m all for, it will just take a little time).  The management console that I refer to for Qpid is the Red Hat Enterprise MRG Management Console. Ideally I would have liked to be able to control everything from a single GUI, but that doesn’t seem to be the case with either broker.

1. How are the brokers managed?

·        [Qpid] Web UI and command line

·        [RabbitMQ] Web UI, command line, and REST API

2. Does the broker support replaying messages in queue ‘A’ through queue ‘B’ ?

·        [QPID] Messages can be moved through the web management console (although I received errors when I tried to do it).

·        [RabbitMQ] No

3. Does the broker support purging of queues?

·        [Qpid] Messages can be purged through the web management console (although I received errors when I tried to do it).

·        [RabbitMQ] Messages can be purged through the web management UI

4. What type of error monitoring does the broker support? Email notifications? System alerts?

·        [Qpid] Possibly

·        [RabbitMQ] Possibly

·        If yes, how are they setup?

·        [Qpid] From the command line it is possible to use qpid-printevents. This could be used within a script and send an email if a specific problem arises, such as the server goes down. Alternatively, we could create a script/application that monitors specific queues and sends email notifications if a specific queue reasons a predefined threshold.

·        [RabbitMQ] There are a few third party plugins that could be used to monitor the system. Alternatively, we could do something similar to the Qpid approach.

5. What happens if the queues exceed a certain threshold? Exceeds given memory?

·        [Qpid] Application connection is closed by broker.

·        [RabbitMQ] The message broker stops accepting messages.

6. How to start/stop/restart brokers?

·        [Qpid] server command line

·        [RabbitMQ] server command line

7. Where are the logs located?

·        [Qpid] /var/log/messages/

·        [RabbitMQ] /var/log/rabbitmq/

8. Where are configuration files located?

·        [Qpid] /etc/qpid/qpidc.conf

·        [RabbitMQ] /etc/rabbitmq/rabbitmq.config NOTE: the file is not created by default, but can be created manually if needed.

9. How do we add additional queues? Delete queues?

·        [Qpid] Can be done through web management UI

·        [RabbitMQ] Can be done through web management UI

10. Are the queued messages visible?

·        [Qpid] No

·        [RabbitMQ] The first X number of messages are visible from the web UI.

11. Does the broker provide a way to view broker/system statistics?

·        [Qpid] Provides a plug-in that allows users to view the server statistics from the web UI. The web UI also displays some statistics about messages queued/dequeued, number of messages, size of messages, and the number of consumers.

·        [RabbitMQ] The web UI provides visibility to the same information Qpid provides plus users can view IP address of the open producer/consumer connections.

12. What kind of clients (i.e. languages/runtimes) are supported/available?

·        [Qpid] Java, C++, .NET, Python, Ruby

·        [RabbitMQ] Java, C/C++, web, .NET, Python, Ruby, and many others. Complete list can be found here.

13. What language is the broker written in?

·        [Qpid] Two implementations, Java and C++. The RHN version is C++

·        [RabbitMQ] Erlang

14. How do you add additional users?

·        [Qpid] server command line

·        [RabbitMQ] web UI

15. Can queues be paused?

·        [Qpid] No

·        [RabbitMQ] No

16. Can test messages be added from the management console?

·        [Qpid] No

·        [RabbitMQ] Yes, through the web UI

17. How are the queue maximums set?

·        [Qpid] there are four fields that need to be set: file-count(nbr files in queue's persistence journal), file-size (file size in pages), max-queue-size, max-queue-count. The broker appears to go by the smallest value set. ex: if the max-queue-count is set to 200k messages, but the file-count is set to its default (8) and that size can't hold 200k messages, the limit is what the file-count can hold (10k messages).

·        [RabbitMQ] Maximums cannot be set. RabbitMQ states that it shouldn't be needed because if the queue depth starts to grow exceptionally large, it writes the messages to disk so they will not be lost.

18. Can users view the maximum number of messages for a queue?

·        [Qpid] not from web UI. The queue file sizes can be viewed from the command line

·        [RabbitMQ] N/A

19. Can messages be routed between life-cycles?

·        [Qpid] Supported by using qpid-route

·        [RabbitMQ] Supported by using the Shovel plugin.

20. Can queues be setup to route failed messages to an invalid (dead-letter) queue?

·        [Qpid] Yes, but only from the command line

·        [RabbitMQ] Yes, from the web UI and from the command line