learn rabbitmq with python in 90mins

Larry cai <larry.caiyu@gmail.com>

Agenda Introduction Exercise 1: Hello World Exercise 2: Work queues - Direct Exercise 3: Publish/Subscribe - Fanout Exercise 4: Routing - Direct Exercise 5: Topics Reference

2 04/15/23

Mostly are based https://www.rabbitmq.com/getstarted.html

https://www.rabbitmq.com/

Environment Preparation (docker) Boot2docker Installer (127M)

Contains latest docker already, fast Container persistence via disk automount on /var/lib/docker $ docker -v User/Passwd: docker/tcuser (192.168.59.103)

Download python/rabbitmq:2 docker images $ docker pull python:2 $ docker pull rabbitmq:3-management

(Windows/Mac) Clone the code from github to your user directory (~/git) https://github.com/larrycai/codingwithme-rabbitmq

Notepad++ & MobaXterm are recommended

3 04/15/23

http://boot2docker.io/

Introduction RabbitMQ is a lightweight, reliable, scalable

and portable message broker. (Apache ActiveMQ/Apollo, ZeroMQ)

It supports AMQP (Advanced Message Queuing Protocol)

Written in Erlang/OTP Messaging: Pattern (asynchronous)

Producer: create message Consumer: Handle a message Queue: Stack the message

4 04/15/23

Exercise 1: Hello World Run first app inside docker environment

$ cd /c/Users/<id>/git/codingwithme-rabbitmq

$ docker run -d -e RABBITMQ_NODENAME=rabbit --name rabbit -p 15672:15672 rabbitmq:3-management

$ docker run -v $PWD:/code -w /code --name worker --link=rabbit:rabbit -it python:2 bash

# pip install pika

# cd exer1 && ./send.py

Browse: http://192.168.59.103:15672 See overview and Check Exchange/QueueGet message

Run in another shell inside docker$ docker exec –it worker bash

# cd exer1 && ./receive.py

Check queue again in GUI Update the “Hello world” to “Hello Larry”

5 04/15/23

Docker HostDocker Host

RabbitMQ Server

(rabbit)

15672

Python:2(worker)Python:2(worker)

RabbitMQ - Concept Publisher/Consumer is app (running in python

container in exercise 1) Publish & consume the messages

In RabbitMQ, An exchange receives messages from producers and pushes them to queues.

6 04/15/23

Rabbit Libraries - Python Pika module Pika is RabbitMQ server python module Procedure for Publish

1. Establish connection

2. Define Queue (“Hello”)3. Publish (routing_key is

queue name) Consume

1,2 same for publish 3: Call back

7 04/15/23

Queue in depth Round-robin dispatching

Scale to distribute to different consumers Message acknowledgment

An ack(nowledgement) is sent back from the consumer to tell RabbitMQ that a particular message had been received

Message durability & Fair dispatch (self-learning)

8 04/15/23

Exercise 2: Work queues How to scale the twitter message image

processing twitter.py – publish twitter message image.py – process the image if twitter has Env# cd ../exer2#./twitter.py “I see the nice picture 1.png”#./image.py

Did we receive the message in queue ? Why ? Run more image.py to see how it is distributed

(another shell, docker exec -it .. ) Kill some image.py

9 04/15/23

Exchange types Exchange type is the rule on how

to deliver message to the queues: direct, fanout, topic and headers

Direct: if the routing key matches, then the message is delivered to the corresponding queue.

Fanout: multicast the received message to the bound queues. Publish/Subscribe Patten Broadcast

10 04/15/23

Exercise 3: Fanout - Publish/Subscribe How to broadcast the message for different

consumers twitter.py – publish twitter message image.py – manipulate the image if twitter has

Env# cd ../exer3#./twitter.py “I see the nice picture 1.png”#./image.py

Did we receive first twitter & Check in the browser for the queue

Implement extra tag.py check whether twitter has tag Define queue `tag` and bind to `twitter` exchange Search for “#xxx#” keywords and print it out ./twitter.py “#codingwithme# is good, send picture 1.png”

11 04/15/23

Binding A binding is a relationship between an

exchange and a queue.  One to one mapping

Multiple mapping with Routing_key

12 04/15/23

Exercise 4: Direct - Routing How to handle different log system Env# cd ../exer4# ./emit_log_direct.py # ./receive_logs_direct.py

Exercise: Check GUI for queue name and bindings More receiver

# ./receive_logs_direct.py warning error > logs_from_rabbit.log# ./receive_logs_direct.py info warning error

Define fixed queue name

13 04/15/23

Topics User pattern as routing key (“a.b.c.d”) Queues can use wildcard characters in binding

* matches a single word, # zero or more words. “rabbit.*” matches for rabbit.info/rabbit.error

Topic exchange is powerful and can behave like other exchanges. When a queue is bound with "#" (hash) binding key - it

will receive all the messages, regardless of the routing key - like in fanout exchange.

When special characters "*" (star) and "#" (hash) aren't used in bindings, the topic exchange will behave just like a direct one.

14 04/15/23

Exercise 5: Topic handle different log system more Env# cd ../exer5# ./emit_log_direct.py # ./receive_logs_direct.py “#”

Exercise Check the queue name in GUI Create for rabbit.*/apache.*/*.info Publish message for rabbit.info/error & apache.info/error

15 04/15/23

Summary RabbitMQ is the message broker support

AMQP Handle async messages

AMQP Model Different Exchange types Messages Queues Bindings Rule for Binding ..

16 04/15/23

Reference http://www.rabbitmq.com/ Exercises are based on

http://www.rabbitmq.com/getstarted.html Book:

RabbitMQ in Action (2012):http://www.manning.com/videla/

Code: https://github.com/larrycai/codingwithme-rabbitmq

17 04/15/23

ChangeLog 2015/05/11: first release

18 04/15/23