how ibm mq is evolving

How IBM MQ is EvolvingOctober 2021

David WareIBM MQ Chief Architect

© 2021 IBM Corporation

Messaging is essential for building fully connected, efficient and scalable solutions. More now than ever before

Messaging is essential for building fully connected, efficient and scalable solutions. More now than ever before

Critical exchange of information from one system to another

Messages must get through, no questions asked. The system must be secure and distributed

Real-time event notification for microservice scalability

Must be frictionless, scalable and lightweightSimple to exploit, invisible to the application

IBM MQ, supporting the most demanding messaging and event driven applications


github.com/cncf/landscape#trail-map

Focus: IBM MQ is the cloud native choice for enterprise messaging

How can IBM MQ be cloud native? What is cloud native?






q Container native

q Loosely coupled architecture

q Horizontally scalable

q Replicated data

q Declarative deployments

q Open APIs and protocols

3 mins 46 seconds

3 days

0 seconds

What success looks like

IBM MQ deployment lead time

IBM MQ version and maintenance adoption

IBM MQ down time


On-premise, software and the MQ Appliance, exactly as you need it

Run MQ yourself in public or private clouds, virtual machines or containers

Let IBM host MQ for you with its managed SaaS MQ service in public clouds,IBM Cloud and AWS

A focus on where you need MQ today and tomorrow


Kubernetes

AWS

IBM Z

WindowsAIX

IBMi

Linux

Appliance

zLinuxHPE NonStop

Azure

AWS

MQ in containers

Highly available

Deploy highly available, replicated

queue managers with ease

Active/active, horizontally scaled

deployment patterns for always on

systems

Enterprise

Production ready certified container image with Cloud

Pak for Integration and OpenShift

Operator managed deployments for

native Kubernetes experience

Loosely coupled

Building large networks of application

centric messaging servers is fundamental

to MQ

MQ Clustering enables building of dynamic

networks with automatic routing from

source to target

Light footprint

MQ is designed to be lightweight and to scale to run at any

sizeIBM itself maintains

½ core queue managers in production





A focus on where you need MQ today and tomorrow


Kubernetes

AWS

IBM Z

WindowsAIX

IBMi

Linux

Appliance

zLinuxHPE NonStop

Azure

AWS




IBM MQ, a true hybrid cloud solution


Kubernetes

AWS

IBM Z

WindowsAIX

IBMi

Linux

Appliance

zLinuxHPE NonStop

Azure

AWS


Original MQ systemsTrust in the system


Queue Manager

Original MQ systemsTrust in the system


Queue Manager

Traditional high availabilityRely on an external system


Native high availabilityBuilt in data replication and quorum consensus


Active/active availabilityAutomatically partition and traffic balancing


Availability and ConsistencyConsistent message availability and system availability

consistency

partitiontoleranceavailability

CPCA

AP

CAP theory


consistency

partitiontoleranceavailability

CPCA

AP

CAP theory

Availability and ConsistencyConsistent message availability and system availability


Message availabilityProtecting your critical data


System managed HA

Multi-instance queue

managers

MQ Appliance

Dependencies

Replicated data queue manager

z/OS Queue Sharing Groups

availability

Constantly evolving to meet your availability needs


System managed HA


managers

MQ Appliance

Dependencies



Constantly evolving to meet your availability needsLinux

availability

Replicated Data Queue Managers


RHEL x86, MQ Advanced HA solution with no need for a shared file system or HA cluster

All components are provided and supported through MQ Advanced

Synchronous replication and fast quorum based take over for HA scenarios, recovery in seconds.

Asynchronous replication between quorum groups to support long distance DR deployments

All nodes support concurrently running multiple different active queue managers, with bidirectional asynchronous replication, supporting active/active HA and DR topologies

IBM MQ 9.2 AdvancedRHEL x86

Node 2 Node 3Node 1

sync syncactive

synchronous replication

Node 5 Node 6Node 4

async asyncasync

asynchronous replication

sync activesync

async asyncasync

synchronous replication

asynchronous replication


System managed HA


managers

MQ Appliance

Dependencies



Constantly evolving to meet your availability needsCloud

availability


System managed HA


managers

MQ Appliance

Dependencies



Constantly evolving to meet your availability needsContainers

availability


System managed HA


managers

MQ Appliance

Dependencies



Constantly evolving to meet your availability needsCloud native containers

Native HA

MQ 9.2.3 CD inCP4I

availability


Queue manageractive

Queue managerreplica

Instance 1storage

Instance 2storage

Instance 3storage


MQ Native HA

Messages persisted in three locationsExact replicas, maintaining configuration, message order, transactional stateQuorum ensures consistency and rapid failure detection and recovery

App

Exact message replication

Automatic availability

New in MQ 9.2.3Cloud Pak for Integration in OpenShift

Node NodeNode

Pod 0 Pod 1

OpenShift service

Pod 2

StatefulSet, Replicas=3MQ Operator


Queue manageractive


Instance 1storage

Instance 2storage

Instance 3storage


MQ Native HAOpenShift native

Availability: cross AZ RPO=0, RTO “a few seconds”Compatibility: simple RWO block storage requirementCost: Included in CP4I license (MQ Advanced ratio)Complexity: No external services to managePerformance: Network + block storage

App

New in MQ 9.2.3Cloud Pak for Integration in OpenShift

Always-onBuilding scalable, active-active, solutions


Active/active message distribution

IBM MQ Uniform Cluster enables applications to be workload balanced acrossloosely coupled queue managersUniform Cluster detects application imbalance and automatically moves connectionsEnsures constant availability of the system and seamless scaling out

Automatic rebalancing

Constant system availability

New in MQ 9.2 LTSDistributed

App App App App App App

Uniform Cluster

Application awareness


To provide an active/active, solution you need to consider multiple active queue managers acting as a single messaging service

Applications should treat the queue managers as interchangeable and want to connect to the group in the most efficient and available distribution

With IBM MQ 9.2 LTS, queue managers can form a uniform cluster, each queue manager provides the same messaging capabilities

Application language and environment support has been growing ever since MQ first delivered uniform clusters.

IBM MQ 9.2.3 Resource Adapter adds JEE Message Driven Bean support to automaticallybalance your clustered MDB applications.

Always-on MQ

New in MQ 9.2.3Resource Adapter


Active/active message distribution


Combine the twoUniform Cluster with replicated HA

ApplicationsMaking it easy to benefit from MQ in your applications


Application autoscaling in Kubernetes

IBM MQ supports any number of application instances, with no need to pre-configure the system. This allows application instances to grow and shrink as needed before needing to scale the messaging layer

KEDA enables Kubernetes-based event-drivenautoscaling, automatically increasing and decreasing the number of instances of an application based on the number of messages that need to be processed

App AppApp AppApp App

keda.sh/docs/2.0/scalers/ibm-mq/


Developer enablement & optimizationThe way in which developers utilize messaging resources can impact the flow of data between applications.

Our learn-mq resources get developers up and running quickly

Open-source dev-patterns repo provides developers with sample code for common MEPs

Provide developers with a greater insight into solution complexity

Core MQ objects and how they interact

Optimizing code: Persistence, Expiry, QoS, Flow Control, Error Handling, Acks & Responses

developer.ibm.com/components/ibm-mq/


Expanding application choice

MQ supports many protocols and APIs. MQ has been expanding these to meet new requirements and environments

REST Messaging Provides a very simple way to get messages in and out of your MQ system(IBM MQ 9.1)

Support for AMQP 1.0 clients to connect and interoperate with any other MQ application.Messaging behaviour follows Apache Qpid JMS, widening the choice of open source clients even further (IBM MQ 9.2.1)

Define your MQ messaging endpoints for applications with AsyncAPI

REST{ }

MQI *

* IBM MQ Advanced

Insight to your dataStream MQ data to new applications


MQ Streaming QueuesTap into the value of existing data flowing over MQ by making message data available to Kafka, AI, and analytics applications with zero impact to the existing applications or their messages, and without a need for re-architecting your message flows.

1. Streaming Processing to accelerate time to insight from existing data.

2. Real world data to accurately simulate production workloads to test the impact of architectural changes on applications.

3. Auditing and Replay of data in the event of disasters. Auditing and replay use cases require exact duplicates of message content as well as message attributes including Message IDs, Correlation IDs etc.

Application Application

Streaming Queue

New in MQ 9.2.3Distributed

Expand your networkBuilding hybrid cloud MQ systems

fasp

Gate

way

Data goes through the

gateway

fasp

Gate

wayTCP/IP TCP/IP

Advanced Connectivitywith IBM Aspera fasp.io Gateway

UDP

Accelerate the speed of data transfer across long distances and/or poor networks with the IBM Aspera fasp.io Gateway:• The Fast and Secure Protocol (FASP) is a network

optimized protocol at the heart of IBM Aspera.

• MQ Advanced brings you the benefits of this when communicating between distant queue managers

• Dramatically increase capacity without requiring any network changes - go from millions to billions of messages per day!

• For more information about performance, see: ibm-messaging.github.io/mqperf/MQ914_fasp_gw.pdf

Bypass: 0ms network latency (no packet loss)N1: 25ms network latency (no packet loss)

N2: 40ms network latency (0.1% packet loss)N3: 50ms network latency (0.5% packet loss)© 2021 IBM Corporation

ibm-messaging.github.io/mqperf/MQ914_fasp_gw.pdf

Managing MQAdministering, automating and securing your MQ system



Deployment pipelinesDevOps and CI/CD

Automate: the customization, build and deployment of IBM MQ using industry standard tooling.

Opinionated: prescriptive best practice and experience built into every layer

Empower: individual development teams to own their configuration, allowing then to react to change.

Evergreen: assure new MQ releases can be deployed across the enterprise with minimal impact on operations and development teams

GitHub

Queue ManagerCustom Resource Definition

MQSCConfig Map

Automate deployment of standardized,but application centric, MQ systems

Applicationpipeline

Base IBM MQ Image

Red Hat UniversalBase Image

IBM MQ

IBM MQcontainer

Benefit from IBM’s certifiedOpenShift MQ container

Enterprise IBM MQ Image

Base IBM MQ Image

Red Hat UniversalBase Image

IBM MQ

IBM MQcontainer

Enterpriseconfiguration

Enterprisepipeline

GitHub

Enterprisestandard

configuration

Build your own enterprisestandards into your MQ images

QMgr config

MQSC



Automation with MQScripting is key to automation. MQ has supported scripting through MQSC scripts.

MQ has been evolving to make this even easier• Remote runmqsc enables scripts to be

deployed from a system remote to the queue manager

• MQSC commands are now more idempotent

• Queue managers can now automatically pull in updated MQSC scripts and ini file settings at start time (MQ 9.1.4)

• New REST API support opens upadministration over HTTP using JSON(MQ 9.1.3)

AutoCluster:Type=UniformClusterName=UNIDEMO

DEFINE CHANNEL(…..DEFINE QLOCAL(….….

deploy

IBM MQ 9.2 LTS


REST administrationJSON format, MQSC style, REST commands

Send request body in HTTP POST to admin/action/qmgr/{qmgrName}/mqscresource

New command type of “runCommandJson”

Existing command type of “runCommand” can still be used to run a plain text MQSC command {

"type": "runCommandJSON","command": "define","qualifier": "qlocal","name": ”Q1","parameters": {"descr": "My queue"

}}

DEFINE QLOCAL(Q1) DESCR(‘My queue’)

OptionaladditionalparametersPrimary

argument(often anobject name)

Primarycommandkeyword Secondary

commandkeyword

JSON equivalent

IBM MQ 9.2 LTSAll Platforms

MQ 9.1.5 CD carried these APIs over into a new V2 of the REST API. Earlier APIs for per-object manipulation have been stabilised at V1www.ibm.com/support/knowledgecenter/en/SSFKSJ_9.1.0/com.ibm.mq.pro.doc/q133690_.htm#q133690___restapiv2

New Web Console

MQ 9.2 replaces the existing web console with a new web console across all platforms

Focus is on user experience and consistency across IBM products


IBM MQ 9.2 LTS

https://community.ibm.com/community/user/imwuc/blogs/callum-jackson1/2020/04/09/enhanced-web-console-in-ibm-mq-915


Central Web Console

Originally, the web server component of MQ that underpins the web console was collocated with the queue managers. A simple way to point at each MQ installation and see the queue managers there.

With IBM MQ 9.2.3 CD you can point a browser at a single system, one that just hosts the MQ web server, and now manage multiple queue managers across multiple systems, of any type.

HTTP

New in MQ 9.2.3All installable platforms





q Container native

q Loosely coupled architecture

q Horizontally scalable

q Replicated data

q Declarative deployments

q Open APIs and protocols

IBM MQ, continually evolving

Thank you

© Copyright IBM Corporation 2021. All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. Any statement of direction represents IBM’s current intent, is subject to change or withdrawal, and represent only goals and objectives. IBM, the IBM logo, and ibm.com are trademarks of IBM Corp., registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available at Copyright and trademark information.


David WareIBM MQ Chief Architect

[email protected]

https://www.ibm.com/legal/copytrade

how ibm mq is evolving

Documents