Showing posts with label JMS. Show all posts
Showing posts with label JMS. Show all posts

Monday, August 27, 2012

Dynamic queue creation with WMQ


Today's task is dynamically creating persistant queues in WMQ(IBM's Websphere MQ) from within Java code. Out of all of the message brokers that I have researched, WMQ definetely was the hardest to figure out how to create queues from code.  Although, on the positive side, at least it is possible.  

In order to invoke the broker from Java, you must use WMQ's API which they call PCF (Programming Command Format).  PCF allows you to do a wide range of admin tasks such as viewing current queues and creating new queues. In order to perform a specific action, you must create a PCFMessage and add all of the needed parameters to it (You also need to make sure that you add them in the correct order!?). A list of all of the required and optional parameters can be found here. After setting all of the parameters, all you need to do is create a PCFMessageAgent and send the parameters. Done. It seems easy now, but most of my time was spent fishing through IBM documentation looking for the correct syntax and parameter names.

If you are interested in other functionality provided by PCF, there are samples included with the installation of the WMQ client found here.  After installation, by default the samples are location at: C:\Program Files (x86)\IBM\WebSphere MQ\tools\pcf\samples
Source:
   public void createQueue(String queueName) throws Exception {
      PCFMessageAgent agent = null;

      // Create the PCF message type for the create queue.
      // NOTE: The parameters must be added in a specific order or an exception (3015) will be thrown.
      // The required order is Name, type, then all optional parameters
      PCFMessage pcfCmd = new PCFMessage(MQConstants.MQCMD_CREATE_Q);

      // Queue name - Mandatory.
      pcfCmd.addParameter(MQConstants.MQCA_Q_NAME, queueName);

      // Queue Type - Optional.
      pcfCmd.addParameter(MQConstants.MQIA_Q_TYPE, MQConstants.MQQT_LOCAL);

      // Add description.
      pcfCmd.addParameter(MQConstants.MQCA_Q_DESC, "Queue created by PCF test");
     
      //set the max queue depth
      pcfCmd.addParameter(MQConstants.MQIA_MAX_Q_DEPTH, 999999);
     
      //Make queue persistant
      pcfCmd.addParameter(MQConstants.MQIA_DEF_PERSISTENCE, MQConstants.MQPER_PERSISTENT);

      //backout queue
      pcfCmd.addParameter(MQConstants.MQCA_BACKOUT_REQ_Q_NAME, "INVALID." + queueName);
     
      //harden backout count-determines if backout count is saved after restart
      pcfCmd.addParameter(MQConstants.MQIA_HARDEN_GET_BACKOUT,MQConstants.MQQA_BACKOUT_NOT_HARDENED);
     
      try {
          agent = new PCFMessageAgent(BROKER_HOST, BROKER_PORT, DEFAULT_CHANNEL_NAME);
          //An exception will be thrown if the queue already exists
          agent.send(pcfCmd);
          System.out.println("Successfull created queue: " + queueName);
      }
      catch (PCFException pcfe) {
       System.out.println("Failed to create queue");
       if (pcfe.reasonCode == MQConstants.MQRCCF_OBJECT_ALREADY_EXISTS) {
        System.out.println("The queue " + queueName + " already exists on the queue manager.");
       }
      }
      finally{
       // Disconnect the agent.
          agent.disconnect();
      }
      return;
   }

Monday, July 16, 2012

Dynamic messaging with Qpid and Spring Integration

One of the key functions of a message broker that I would like to be able to do is to dynamically send messages to queues at runtime. This means that I don’t want to have to create queues ahead of time within the broker and then alter my application’s configuration to connect to the new queue.  What I want to do is have the application determine when it needs to create a new queue (either by customer interaction, database values, or some of arbitrary logic), and then have new messages sent to it.

With RabbitMQ, if you send a message to a queue that doesn’t exist, it creates the queue for you and then adds the message to it. Qpid, on the other hand, doesn’t work like this.  If you try to send a message to a non-existent queue, you’ll get an error.

While fishing through Qpid documentation I read that if a consumer is created for a queue that doesn’t exist, then the broker will create the queue.  This is better than nothing, but still not really what I want.

Then while testing a few different settings, I realized that from within Java, you can specify arguments (similar to the arguments you would use if you were to create a queue from the command line) when you specify the message destination.  This changes everything.  Now I can create durable queues on the fly, which is just what I wanted.

Qpid isn’t directly supported by Spring Integration in the same way that RabbitMQ is, but since it can be accessed using JMS, you can use the existing JMS classes within SI do to send and receive messages.This can be done by creating a service activator that has a DynamicJmsTemplate which will send the messages.  




The JmsTemplate will need a connection factory.  The Qpid examples use a JNDI connection factory, but I would rather have my connection defined in my config and not have to worry about JNDI.  For this, I used the AMQConnectionFactory provided in the qpid client library.  I’m not sure if this is the best class to use, but for now it will work.


 



 
  
   
  
 

At this point all you will need to do is to implement a sendMessage method within the handler class and use the JmsTemplate to send the message.  But wait, how do we guarantee that the queue will be created if it doesn’t exist when we send the message?  The trick here is specify the create arguments with the destination.

String destinationName = queueName + "; {create: always, node:{type:queue,durable:True}}";
this.jmsTemplate.convertAndSend(destinationName, objectToSend);


And there you have it. You can now create queues on the fly without having to worry about errors.  Just make sure you create a consumer for the queue you just created, otherwise your queues will fill up pretty quick.

Tuesday, July 3, 2012

Message Broker Comparison

Recently I have been given the task to find a new message broker. I was given a large list of vague requirements such as it needs to be fast, persistant, durable, etc. Below is a list that I have put together that describes a few brokers that I have researched. As I started to research more and more, I found that these are just a few of the more popular message brokers out of a very large and growing list. I you have experiance with any of these queues, or any other queues that are not mentioned here, feel free to share your thoughts.
  1. RabbitMQ (AMQP)
    • Pros:
      1. Can setup retry logic using spring’s retry template
      2. Supports dynamic queue creation.
      3. JSON API to retrieve list and manage queues
      4. Web UI for queue management
      5. Very fast (10x ActiveMQ)
      6. Spring supported through Spring-AMQP package
      7. Large community support
      8. Can configure broker with exchanges/queues to manage performance and route messages based on content
      9. supports STOMP protocol
    • Cons:
      1. Broker is implemented using Erlang and not Java (impacts setup, config)
      2. AMQP is still new
  2. ActiveMQ (JMS)
    • Pros:
      1. Can configure connection factory to handle retry logic (delays/redelivery)
      2. Supports dynamic queue creation.
      3. Web UI for queue management
      4. Can retrieve a list of queues through JMX
      5. Spring supported with Spring-JMS
      6. Large community support
      7. Broker is configurable through XML
    • Cons:
      1. Not as fast as RabbitMQ
      2. Messages must be sent to either queues or topics
      3. Doesn’t appear to be completely stable.
  3. Qpid (AMQP)
    • Pros:
      1. Java implementation
      2. Durable queues, persistent messages using Derby DB
      3. Windows .NET SDK
    • Cons:
      1. No queue management UI, must use command line, or JMX
      2. Not completely supported by Spring. Can use Spring-AMQP, but not completely supported. NOTE: could use supported spring rabbit libraries, but would need to be using the same version of AMQP as RabbitMQ
      3. Little community support
      4. AMQP is still new
  4. HornetQ (JMS)
    • Pros:
      1. Can be installed as a standalone server or with JBoss
      2. REST api to manage queues
      3. XML configuration
    • Cons:
      1. Recommended to use Java 6, although there are special client jars for Java5 (ran into issues using Java5 library)
  5. ZeroMQ (AMQP)
    • Pros:
      1. Very fast
    • Cons:
      1. Queues are not durable
      2. Messages are not persistent
      3. Not really a message broker
  6. WMQ (JMS)
    • Pros:
      1. Spring support
      2. Can dynamically create listeners
      3. Desktop UI management
      4. Can view messages in UI
    • Cons:
      1. Does not support dynamic queue creation (out of the box).
      2. Does not support consumer backoff (retry delay)
      3. Unless using version 7 or newer, wmq libraries does not support a way to get a current list of queues. We would need to keep track of the queues ourselves.