Messaging and Madness: Sending Messages with AMQP and Amazon MQ

Table Of Contents

Introduction

AMQP

AMQP and Amazon MQ

Serialization

Conclusion

Introduction

How do software systems talk to each other? Back-end systems can scale into giant melted together Cronenberg monsters, often making up different tools and languages. So, communicating between these services can become an untenable challenge without some shared vocabulary. We can communicate in many ways, but today I wanted to talk about asynchronous messaging protocols and figure out how AWS can help.

AMQP

AMQP stands for Advanced Message Queuing Protocol. I’ve been working to implement it for some back-end software suites I’m building out to enable them to talk to each other. AMQP utilizes these things called brokers to publish messages on, then on the other end, a receiving service subscribed to the same “channel” that we posted to can pick up that message.

Let’s dive a little further down; the publisher service publishes a message to an exchange on a broker. This exchange has routes that lead to queues, or “channels,” where the payload is published. We make sure to include the sending information with our message to be routed to the correct queue. The broker cannot see the message, although it might look into any metadata attached to the message from the publisher. This workflow asynchronously sends messages. Imagine a server version of a mail sorting machine shooting letters into the correct mail slot based on the address.

When referring to a publisher, I mean some code that we utilize to connect and send a message. AMQP is programmable, so I can shape it to fit most situations. In this case, we need to send messages to our different software suites to trigger actions to happen. Learning this took some time, but it’s been simple to implement.

There are different types of exchanges that we can use to make these services fit our needs. I’m going to explain what we use briefly.

We use a direct exchange utilizing routing keys to bind queues to exchanges. Our code can use direct exchanges to distribute tasks to many different endpoints, but we used these direct exchanges to make direct routes between our services. Other types of exchanges can be used to broadcast messages. More information can be found here. For now, we’re going to focus on direct exchanges.

AMQP and Amazon MQ

We touched on all that because I wanted to talk about Amazon MQ. Amazon MQ is a fully managed platform for setting up message brokers. Amazon MQ utilizes both RabbitMQ and Apache Active MQ for creating brokers. We’re sticking with Rabbit MQ for the time being.

Here above, you can see you can easily set up a broker in just a few clicks. I left most of the settings on default, except for choosing “RabbitMQ” for our broker engine and setting some security up for accessing our management console.

Once we get that, we have access to the RabbitMQ dashboard Amazon MQ created and is managing. Now that we have a broker set up, we can play with some code.

Above I use the library Kombu to create some connections and send some stuff. I started by setting up our environment variables. Then created exchange and queue objects. Finally, I made our connection object and the producer object, and then we sent a simple “Hello” message.

Serialization

Serialization is another blog post, but I chose to use JSON to serialize the payload. In the production software, I use a combination of JSON and Pickle to serialize things like image data.

Now we can see our message published on the queue I declared in our publisher service. An identical receiving service would be set up on the other side to read out messages sent to that queue.

Conclusion

In conclusion, using Amazon MQ allows us to set up managed brokers for us to send messages. With AMQP as the broker engine, we have a lightweight message-sending workflow. Thanks for reading.

-George