mq deployments on cloud pak for integration
TRANSCRIPT
MQ deployments on Cloud Pak for Integration
Ashlin Joseph and Justin Deane
IBM Lab Service
IBM Cloud © 2020 IBM Corporation
Why IBM Cloud Integration Expert Labs?
Expedite the successful deployment of IBM’s Cloud Integration Solutions
Through our deep technical expertise, methodology, repeatable patterns, learning services and mentoring
Faster time to value from your IBM Cloud Integration Solutions
Our purpose How wedrive success
How you benefit
IBM Cloud © 2020 IBM Corporation
Contact us through email: [email protected]
Why IBM Cloud Integration Education?Invest in the skills of your workforce with 3 options:
Cost effective, digital-based technical training
Understanding skill gaps and the paths to close those gaps
Develop professional acumen
Individual and Enterprise Digital Learning Subscriptions
Skill Assessments & Augmentation with skill roadmap recommendations
Validate your skills with IBM Cloud Certification and Digital Credentials
Increased employee productivity
Motivatedemployees
Trained employees
To learn more, contact us: [email protected]
Faster ROI
Topics we will cover for MQ in CP4I
• Availability
• Resilience
• Dynamic scaling
• Scaling by numbers
• Deployment patterns on CP4I (or other)
• Demonstrations and walkthroughs
• Questions
IBM Cloud © 2020 IBM Corporation
MQ availability in CP4i
‘Traditional’ and ‘Modern’ architectures
• Traditional options? (MQ running directly in an operating system.)
• Multiple queue managers, Multi-instance, RDQM (on Red Hat), and OS clustered (e.g. HACMP), MQ Appliance, z/OS (shared queues)
• Modern options? (MQ running in a container environment.)
• Replicated queue managers, scaled as needed by k8s
• Single resilient queue manager, which offers a simple approach for HA in the cloud, using networked storage.
• Multi-instance queue manager, which is an active-standby pair, using a shared, networked filesystem.
IBM Cloud © 2020 IBM Corporation
Two basic classes of available configurations
Active/Passive
• An active/passive configuration is a basic HA configuration in which one node performs work while the other node acts only as standby.
Active/Active
• An active/active configuration is one where a number of nodes participate in highly available (movable) work. This type of HA configuration is also sometimes referred to as "mutual takeover”.
IBM Cloud © 2020 IBM Corporation
Resilience vs availability
MQ resilience refers to automated methods that ensure the availability of the service.
• (active/passive) Restarts in the case of a failure
• (active/active) Available by virtue of numbers
IBM Cloud © 2020 IBM Corporation
Resilient and scalable configurations
Dynamic scalability in Kubernetes promises to deliver both…• Resilience: Restart of a queue manager in the event of a failure
• Dynamic scalability: Replica sets can be scaled up/down, right?
However ….?
MQ is stateful!
IBM Cloud © 2020 IBM Corporation
Dynamic scaling: MQ clusters
• Deleting a queue manager in an MQ cluster • Normally requires performing steps to remove it
• Else QM information held in cluster up to 90 days
• MQ objects such as cluster queues• Can cause problems if they are not removed carefully
• E.g. messages on cluster transmission queues destined for deleted queue managers
IBM Cloud © 2020 IBM Corporation
Scale and state in MQ containers
Tension between scalability (dynamic) and statefulness.
• To retain state in MQ we require independent disk storage to be associated with the queue manager (Persistent volume - PV).
• To dynamically scale a container we really need to stick with local storage.
IBM Cloud © 2020 IBM Corporation
Multiple queue managers: Message ordering
• Horizontal scaling - messages processed concurrently
• Messages might be received out of sequence
IBM Cloud © 2020 IBM Corporation
Multiple queue managers
• Each queue manager is independent of the other so and loss of one queue manager will not allow the other to take over its in-flight transactions.• Similar use case to non-persistent messaging
• Message may arrive out of sequence• Applications responsible for sequencing
• Applications connecting to the queue managers now need to be able to recover from loss of connection and reconnect to one of the alternative queue managers.• CCDTs and Uniform clusters
IBM Cloud © 2020 IBM Corporation
MQ in a container
A “single resilient queue manager”
• Single instance of a queue manager in a container
A “multi-instance queue manager” in a container
• Active and standby pods
Both require: Stateful Set with PVs (backed by network storage)
IBM Cloud © 2020 IBM Corporation
Single resilient queue manager
Single resilient queue manager, requires storage classes containing PVs. Ideally, these should shared file systems, to allow the pod to start on a different node.
IBM Cloud © 2020 IBM Corporation
Single resilient queue manager: recovery times
Restart is handled Kubernetes
• Liveness probe time to detect, Scheduler time to move to new node
• Should pre-pull images on nodes (use and IfNotPresent pull policy)
• New instance start up time
• Readiness probe (for service availability)
• Time taken for a regular queue manager to start up
• Client reconnection time
IBM Cloud © 2020 IBM Corporation
Multi-instance queue manager
• Failover support provided by MQ
• Faster failover than single resilient queue manager
• Provides transactional integrity
• active and the standby Pods to have concurrent access to the persistent volume
• Persistent Volumes with access modeset to ReadWriteMany
• MQ relies on the automatic release of file locks to instigate a queue manager failover
• List of standard requirements for file system
IBM Cloud © 2020 IBM Corporation
Multi-instance queue manager: recovery times
Multi-instance standby process for a queue manager is already running
• Time taken similar to regular queue manager start-up.
• Client reconnection time
IBM Cloud © 2020 IBM Corporation
Demonstrations and walkthroughs
IBM Cloud © 2020 IBM Corporation
Within the Cloud Pak for Integration Platform Navigator click on Runtimes, and select Create Instance:
IBM Cloud © 2020 IBM Corporation
Select the MQ Queue Manager tile:
IBM Cloud © 2020 IBM Corporation
Select the Quick start tile and click Next
IBM Cloud © 2020 IBM Corporation
Accept the license and and click Create to deploy a new Queue Manager:
IBM Cloud © 2020 IBM Corporation
Create a config map for MQSC:
Create a config map for MQSC:
Create a config map for MQSC:
You will be returned to the Runtime and Instances list, MQ will take a few seconds to deploy and the status will change to Ready. You may need to refresh the status by using the table refresh button:
IBM Cloud © 2020 IBM Corporation
Select the newly created entry and you will be redirected to the MQ Console:
IBM Cloud © 2020 IBM Corporation
Click on the manage button from the left menu to view the details of the Queue Manager:
IBM Cloud © 2020 IBM Corporation
We will test MQ by sending a message to a queue, for this a new queue will be created, click on Create +:
IBM Cloud © 2020 IBM Corporation
Select the Local queue type:
IBM Cloud © 2020 IBM Corporation
Fill in app1 as the Queue Name and click Create:
IBM Cloud © 2020 IBM Corporation
A MQ channel needs to be defined for communication into MQ, select ‘Manage’:
IBM Cloud © 2020 IBM Corporation
and then the Communication tab, App channels section and click Create +:
IBM Cloud © 2020 IBM Corporation
Click Next
IBM Cloud © 2020 IBM Corporation
Enter QUICKSTART as the channel name (as this will match the Queue Manager), and click Create:
IBM Cloud © 2020 IBM Corporation
By default MQ is secure and will block all communication without explicit configuration. We will allow all communication for the newly created channel. Click on Configuration in the top right:
IBM Cloud © 2020 IBM Corporation
Select the Security tab, Channel authentication section and click Create +:
IBM Cloud © 2020 IBM Corporation
We will create a channel auth record that blocks nobody and allows everyone. Select Block from the pull down, and the Final assigned user ID tile:
IBM Cloud © 2020 IBM Corporation
•Enter the following value:Channel name: ’QUICKSTARTAJ’, User list: ‘nobody’Click on the plus (+) sign next to the User list text field and then click on Create +:
IBM Cloud © 2020 IBM Corporation
IBM Cloud © 2020 IBM Corporation
Testing
IBM Cloud © 2020 IBM Corporation
Return to the OpenShift Console, navigate to Workload --> Pods and type quicks in the search bar:
IBM Cloud © 2020 IBM Corporation
Change to the terminal tab which will automatically log you into the Queue Manager container:
IBM Cloud © 2020 IBM Corporation
Run the following commands to send a message to app1queue:export MQSERVER='QUICKSTART/TCP/quickstart-cp4i-ibm-mq(1414)' /opt/mqm/samp/bin/amqsputc app1 QUICKSTART
IBM Cloud © 2020 IBM Corporation
Return to the MQ Console and navigate back to the queue manager view by clicking on Back to Queue Manager:
IBM Cloud © 2020 IBM Corporation
Select the app1 queue:
IBM Cloud © 2020 IBM Corporation
You will then see the message sent:
IBM Cloud © 2020 IBM Corporation
Connecting MQ Explorer to a deployed Queue Manager in OpenShiftConsists of two parts
• Finding the information within the OpenShift console
• Configuring the MQ Explorer
IBM Cloud © 2020 IBM Corporation
Finding the information within the OpenShift console• Navigate to the OpenShift Console
by selecting the menu at the top left of the screen and selecting OpenShift Console:
IBM Cloud © 2020 IBM Corporation
Navigate to Networking --> Routes
IBM Cloud © 2020 IBM Corporation
Configuring the MQ Explorer
IBM Cloud © 2020 IBM Corporation