configuring a tibco enterprise message service environment · 2020. 10. 19. · tibco software inc....

TIBCO Software Inc. Global Headquarters

3307 Hillview Avenue

Palo Alto, CA 94304

Tel: +1 650-846-1000

Toll Free: 1 800-420-8450

Fax: +1 650-846-1005

www.tibco.com

TIBCO fuels digital business by enabling better decisions and faster, smarter actions through the TIBCO Connected Intelligence Cloud. From APIs and systems to devices and people, we interconnect everything, capture data in real time wherever it is, and augment the intelligence of your business through analytical insights. Thousands of customers around the globe rely on us to build compelling experiences, energize operations, and propel innovation. Learn how TIBCO makes digital smarter at www.tibco.com.

Configuring a TIBCO Enterprise Message ServiceTM in a Azure Kubernetes Service (AKS) Environment This document provides the steps for configuring TIBCO Enterprise Message Service in a Azure Kubernetes Service (AKS) environment without requiring a shared storage device.

Version 1.0 October 2020 Initial Document

©2020 TIBCO Software Inc. All Rights Reserved. 2

Copyright Notice COPYRIGHT© 2020 TIBCO Software Inc. All rights reserved.

Trademarks TIBCO, the TIBCO logo, TIBCO Enterprise Message Service, and TIBCO FTL are either registered trademarks or trademarks of TIBCO Software Inc. in the United States and/or other countries. All other product and company names and marks mentioned in this document are the property of their respective owners and are mentioned for identification purposes only.

Content Warranty The information in this document is subject to change without notice. THIS DOCUMENT IS PROVIDED "AS IS" AND TIBCO MAKES NO WARRANTY, EXPRESS, IMPLIED, OR STATUTORY, INCLUDING BUT NOT LIMITED TO ALL WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. TIBCO Software Inc. shall not be liable for errors contained herein or for incidental or consequential damages in connection with the furnishing, performance or use of this material.

For more information, please contact:

TIBCO Software Inc. 3303 Hillview Avenue Palo Alto, CA 94304 USA


Table of Contents

Configuring a TIBCO Enterprise Message ServiceTM in a Azure Kubernetes Service (AKS) Environment .......................................................................................................................................... 1

1 Overview ........................................................................................................................................ 5 1.1 EMS Architecture ................................................................................................................................ 5 1.2 Supported Versions ............................................................................................................................. 5 1.3 Prerequisites ....................................................................................................................................... 6 1.4 Prepare Local Environment ................................................................................................................. 6 1.5 Prepare Preliminary Azure Account and Kubernetes Configuration ................................................... 6

2 Azure AKS Setup ............................................................................................................................. 7 2.1 Create a New Azure Kubernetes Service (AKS) ................................................................................... 7 2.2 Configuring Kubectl to connect to Azure Kubernetes Service ............................................................ 8

2.2.1 Configure Kubectl to connect to AKS ............................................................................................... 8

3 Building the EMS Docker image .................................................................................................... 9 3.1 Creating the Base Docker Image ......................................................................................................... 9 3.2 Extending the Base Docker Image ..................................................................................................... 10

3.2.1 Provisioning FTL Client Libraries to Use the Corresponding Transports ........................................ 10 3.2.2 Provisioning Custom JAAS Authentication or JACI authorization Modules ................................... 10

3.3 Create and Configure the Azure Container Registry ......................................................................... 11 3.3.1 Configure the Azure Container Registry .......................................................................................... 11 3.3.2 Push the EMS Docker Image to ACR ................................................................................................ 11

3.4 Update the AKS Cluster to access the ACR ........................................................................................ 12

4 Configuring EMS in AKS ............................................................................................................... 13 4.1 Configuring EMS for Kubernetes ....................................................................................................... 13

4.1.1 Main Configuration File ................................................................................................................ 13 4.1.2 Apply the configurations in Kubernetes ........................................................................................ 14

4.2 Stopping or Deleting the EMS Server process ................................................................................... 14 4.3 Connecting to the EMS Server Pod ................................................................................................... 15

5 Accessing and Testing EMS on AKS ............................................................................................. 16 5.1 Internal Access to the EMS Server .................................................................................................... 16 5.2 External Access to the EMS Server .................................................................................................... 16

5.2.1 Access the EMS Server .................................................................................................................. 16 5.2.2 Connection Factory Update .......................................................................................................... 17


Table of Figures FIGURE 1 - CREATE AKS KUBERNETES CLUSTER ............................................................................................................................ 8 FIGURE 2 - CONFIGURE KUBECTL ............................................................................................................................................... 8 FIGURE 3 - VERIFY CONNECTING TO THE KUBERNETES CLUSTER ........................................................................................................ 8 FIGURE 4 - RUN TIBEMSCREATEIMAGE ...................................................................................................................................... 10 FIGURE 5 - CREATE ACR REPOSITORY ....................................................................................................................................... 11 FIGURE 6 - LOGIN INTO THE ACR ............................................................................................................................................. 11 FIGURE 7 - TAG AND PUSH EMS DOCKER IMAGE ........................................................................................................................ 11 FIGURE 8 - UPDATE THE AKS FOR THE ACRS ............................................................................................................................. 12 FIGURE 9 - SUCCESSFUL STARTUP OF EMS ON AKS .................................................................................................................... 14 FIGURE 10 - TO STOP AND START THE EMS STATEFULSET ............................................................................................................ 14 FIGURE 11 - POD ACCESS EXAMPLE ......................................................................................................................................... 15 FIGURE 12 - INTERNAL CONNECTION TO EMS ............................................................................................................................ 16 FIGURE 13 - EXTERNAL ACCESS TO THE EMS SERVER .................................................................................................................. 17 FIGURE 14 - CREATE AN EXTERNAL CONNECTION FACTORY ........................................................................................................... 17


1 Overview

The purpose of this document is to provide a guide to install, configure, and run TIBCO Enterprise Message ServiceTM (EMS) in a Azure Kubernetes Service (AKS) environment without requiring shared storage. Instead, a persisted SSD volume will be used. EMS High Availability is still provided via the Kubernetes Cluster.

Running TIBCO Enterprise Message Service (EMS) on Azure AKS involves:

• Configuring the Azure Kubernetes Service (AKS) for TIBCO Enterprise Message Service (EMS).

• Configuring an Azure Container Registry (ACR) for the Docker® image registry • Creating a Docker® image embedding EMS and hosting it on ACR • Configuring and creating EMS Kubernetes containers based on the EMS Docker image

1.1 EMS Architecture

Using this document, the following architecture can be created: • AKS cluster • ACR Registry for the EMS container • Load Balancer (Kubernetes) for external access to EMS • Node Port (Kubernetes) for internal access to EMS • One (1) TIBCO EMS server instance

1.2 Supported Versions

The steps described in this document are supported for the following versions of the products and components involved:

• TIBCO EMS 8.5.1 or later • TIBCO FTL 6.x or later • Docker Community/Enterprise Edition should be most recent version. • Kubernetes 1.17 or newer • CentOS 7.5 or newer running as part of the Docker Container


1.3 Prerequisites

The reader of this document must be familiar with:

• Docker concepts • Azure console and the Azure CLI (az) • Kubernetes installation and administration • Kubernetes CLI, kubectl • TIBCO EMS configuration • All necessary downloads discussed in the next section • The appropriate TIBCO licenses for EMS and FTL (if used)

1.4 Prepare Local Environment

The following infrastructure should already be in place:

• A Linux or macOS machine equipped for building Docker images • The following software must already be downloaded to the Linux or macOS machine

equipped for building Docker images. Note: All software must be for Linux!

• TIBCO EMS v8.5.1 installation package. The Enterprise Edition must be used to access the necessary script required to build the EMS Docker image. Download the EMS installation package from edelivery.tibco.com .

• The ems_aks_files_8.5.zip. The zip file contains the necessary Kubernetes build files. Download from https://community.tibco.com/wiki/tibcor-messaging-article-links-quick-access

• Create a directory, such as ems_aks_files_8.5 • Unzip ems_aks_files_8.5.zip to ems_aks_files_8.5.

1.5 Prepare Preliminary Azure Account and Kubernetes Configuration

Use the following to prepare the preliminary environment to install EMS on AKS.

• An Azure account is required. If necessary, create an account at http://portal.azure.com and follow the on-screen instructions.

• Install the Azure CLI on the workstation used.

• Install Docker on the workstation to build the TIBCO EMS images.

• Install the kubectl command-line tool do manage and deploy applications to Kubernetes in AZURE from a workstation.


2 Azure AKS Setup

2.1 Create a New Azure Kubernetes Service (AKS)

A new Kubernetes cluster must be created in AKS. Use the following to build a new Kubernetes Service in Azure. This can be created via the Azure Portal of the Azure CLI. This document will outline building the cluster via the Azure portal.

• Sign into the Azure portal at https://portal.azure.com/ • In the top left-hand corner of the Azure portal, select Create a resource > Kubernetes

Service. • Select a Subscription and Resource group. These should be the same subscription and

Resource group used for the Storage Account created previously. • Provide a new Kubernetes Cluster Name, Region, and Kubernetes version (must be at least

1.17.11). • For Scale, select the node size. Recommend a DS3_v2 (4 vCPUs / 14 Gb RAM). Can be

larger, if desired. • Select a node count of 2. • Set virtual nodes and VM scale sets to disabled, if desired, since autoscaling is not required

with EMS. • Click on Next: Authentication • Select to create a new service principal • Click on Yes to Enable RBAC • AKS-managed Azure Active Directory is not required • Click on Next: Networking • Choose either Yes or No for application routing • Choose either Basic or Advanced for Network configuration. Recommend using Basic. • Use the defaults for monitoring • Wait for the Running the Validation to complete, with validation passed. Fix any issues

before continuing! • Click on Create. It will take several minutes to complete.


Figure 1 - Create AKS Kubernetes Cluster

2.2 Configuring Kubectl to connect to Azure Kubernetes Service

With AKS, the Kubernetes command line tool, kubectl, is used to configure the Kubernetes cluster for EMS on AKS.

2.2.1 Configure Kubectl to connect to AKS After the Kubernetes cluster has been built, kubectl must be configured to connect to the cluster on AKS. Use the following example to set the credentials for kubectl. > az aks get-credentials --resource-group --name

Figure 2 - Configure Kubectl

Use kubectl get nodes as shown in the following example to verify connecting to the cluster. NAME STATUS ROLES AGE VERSION aks-agentpool-40625860-vmss000000 Ready agent 116s v1.17.11 aks-agentpool-40625860-vmss000001 Ready agent 2m40s v1.17.11

Figure 3 - Verify connecting to the Kubernetes Cluster


3 Building the EMS Docker image

3.1 Creating the Base Docker Image

The content of the container that will run on Kubernetes derives from a Docker image that first needs to be created and then hosted in a Docker registry. To create an EMS Docker image, use the tibemscreateimage script on a machine equipped for building Docker images. Note: CentOS 7.5 is used for the base OS. This can be changed, but other modifications (not documented) may be required. Use the following steps to prepare the environment:

• In a separate directory, unzip and if necessary, install EMS from the EMS installation package (TIB_ems_8.5.1_linux_x86_64.zip). This is necessary to access the tibemscreateimage build script. Note: Only Linux EMS installation packages contain this script. See the TIBCO EMS installation guide for details.

• Copy /opt/tibco/ems/8.5/samples/docker/tibemscreateimage to ems_aks_files_8.5, or where ever the ems-aks yaml files are located.

• Copy to ems_aks_files_8.5: o Optional EMS hotfixes o Optional Java package

• The tibemscreateimage script should be modified for running in AKS. The changes are minor, but should be completed for EMS to work properly in AKS.

o In the cat > ${DOCKER_BUILD_DIR}/tmp/tibemsd-configbase.json


-j .tar.gz \ -u 1000 \ -g 1000

Figure 4 - Run tibemscreateimage

This example creates a Docker image based on the EMS 8.5.1 Linux installation package, adding a JVM, the 1000 uid and the 1000 gid. If you are curious to run this image stand-alone: > docker run -p 7222:7222 -v `pwd`:/shared ems:8.5.1 tibemsd

This creates a sample EMS server folder hierarchy and configuration in the current directory and starts the corresponding server. You can override the creation and use of the sample configuration with your own setup:

> docker run -p 7222:7222 -v :/shared \ ems:8.5.1 tibemsd -config /shared/

This starts an EMS server using the / configuration. The tibemscreateimage script can be modified to meet your specific needs.

3.2 Extending the Base Docker Image

The base Docker image can be extended to include FTL client libraries and custom JAAS authentication and JACI authorization modules.

3.2.1 Provisioning FTL Client Libraries to Use the Corresponding Transports

1. Copy the FTL client library files to a temporary folder. 2. If customizing your EMS configuration, make sure to include

/opt/tibco/ems/docker/ftl in the Module Path property. Note: tibemscreateimage must be modified, and ran again to ensure /opt/tibco/ems/docker/ftl is in the Module Path.

3. From the temporary folder, use a Dockerfile based on the example given below to copy these files into the base Docker image: FROM ems:8.5.1 COPY --chown=tibuser:tibgroup . /opt/tibco/ems/docker/ftl

> docker build -t ems:8.5.1_ftl .

3.2.2 Provisioning Custom JAAS Authentication or JACI authorization Modules

1. Copy your custom JAAS or JACI plugin files, including the static configuration files they may rely on, to a temporary folder.


2. From the temporary folder, use a Dockerfile based on the example given below to copy these files into the base Docker image: FROM ems:8.5.1 COPY --chown=tibuser:tibgroup . /opt/tibco/ems/docker/security

> docker build -t ems:8.5.1_security .

3. Upon customizing your EMS configuration, make sure to include the relevant paths to those files in the Security Classpath property. Note: The other required files are in their usual location: /opt/tibco/ems//bin and /opt/tibco/ems//lib For example: /opt/tibco/ems/docker/security/user_jaas_plugin.jar:/opt/tibco/ems/8.5/bin/tibemsd_jaas.jar:/opt/tibco/ems/8.5/lib/tibjmsadmin.jar, etc.

3.3 Create and Configure the Azure Container Registry

3.3.1 Configure the Azure Container Registry A new ACR repository must be created to host the EMS Docker image.

• Create a new ACR repository, such as tibems. The repository can be created via the Azure CLI or via the console. Please note the loginServer of your ACR repository. > az acr create --resource-group --name tibems --sku Basic

Figure 5 - Create ACR Repository

• Login into the newly created Azure ACR from the Azure CLI. > az acr login --name

Figure 6 - Login into the ACR

3.3.2 Push the EMS Docker Image to ACR • Tag the image and push the Docker image to the ACR repository using the loginServer

name noted above. Note: Name of Docker image may differ depending on setup.

> docker tag ems:latest /ems:latest Figure 7 - Tag and Push EMS Docker image

• Push the EMS Docker image to ACR. Replace the name of the loginServer > docker push /ems:latest


3.4 Update the AKS Cluster to access the ACR

In Azure, the Azure Kubernetes Service (AKS) must be updated to access the ACR containing the EMS Docker container. Note: a newer version of the Azure-cli must be installed. Version 2.7.0 or higher is required. Use az –version to determine which version if the azure-cli is installed. Use the following to update your AKS. Note: this is the ACR name, and not the LoginServer name. In the examples above, this would be tibems. az aks update -n -g --attach-acr

Figure 8 - Update the AKS for the ACRs


4 Configuring EMS in AKS

After the EMS Docker image is pushed to ACR, Kubernetes can be configured to run the EMS container.

4.1 Configuring EMS for Kubernetes

There are two templates used for the AKS configuration. The ems-aks.yaml template is the main EMS K8 template, and ems-aks-storage.yaml is used to create the required persisted storage. The ems-aks-storage.yaml file will create the persisted storage with using Azure’s premium managed disk, which will be locally redundant. There are other options available on Azure. See the Azure documentation for details. No changes are required to the file.

4.1.1 Main Configuration File The ems_aks_files_8.5/ems-aks.yaml is used to configure the K8 emserver statefulset/pod, and the services for access. There are some necessary changes required to this file. This section will outline these modifications.

• The ACR image for each container. The statefulset defined in ems-aks.yaml requires the image be update to reference the ACR created in section 3.3. The name and location of the Azure Container Registry (ACR) where the EMS Docker container is located will need to be updated. Ensure the proper permissions are set. The image maybe something different than latest, depending on how it was tagged in Docker.

image: /ems:latest

• The storage size requests for the EMS persisted storage. The default storage value for the

nodes is set to 25 Gi. While this should be sufficient for most environments, it may be too large/small for all environments. Under the emsserver container, modify the storage resource request to a smaller/larger value if required.

storageClassName: ems-ssd

resources: requests: storage: 25Gi

• In the service section, under the LoadBalancer, a change is required for the trusted IP range. The trusted IP range will determine what IP addresses can connect to the load balancer, and it is recommended not to be configured to 0.0.0.0./0. The example below shows the change required to ems-aks.yaml.

sessionAffinity: None

externalTrafficPolicy: Cluster loadBalancerSourceRanges:


-

4.1.2 Apply the configurations in Kubernetes Once the ems-aks yaml file has been updated, this file, along with the storage yaml file can be applied using kubectl to AKS. Use kubectl apply –f ems-aks-storage.yaml,ems-aks.yaml to apply both files. Use kubectl get storageclass,svc,pods to verify the storage classes, the services, and the emsserver pod are available. Do not continue until the load balancer has been assigned an External IP address as shown in the following example. NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE storageclass.storage.k8s.io/azurefile kubernetes.io/azure-file Delete Immediate true 23h storageclass.storage.k8s.io/azurefile-premium kubernetes.io/azure-file Delete Immediate true 23h storageclass.storage.k8s.io/default (default) kubernetes.io/azure-disk Delete Immediate true 23h storageclass.storage.k8s.io/ems-ssd kubernetes.io/azure-disk Retain Immediate false 83m storageclass.storage.k8s.io/managed-premium kubernetes.io/azure-disk Delete Immediate true 23h NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/emsserver NodePort 10.0.71.83 30722:30722/TCP 83m service/emsserverlb LoadBalancer 10.0.81.73 40.76.162.125 30724:30724/TCP 83m service/kubernetes ClusterIP 10.0.0.1 443/TCP 23h NAME READY STATUS RESTARTS AGE pod/emsserver-0 1/1 Running 0 15m

Figure 9 - Successful Startup of EMS on AKS

4.2 Stopping or Deleting the EMS Server process

To stop the EMS Server process without deleting it, use the kubectl scale operation to set its number of replicas to 0. For example: > kubectl scale --replicas=0 statefulset emsserver

To start the process again, set its number of replicas back to one (1). Note: Do not set the replicas higher than one (1)! > kubectl scale --replicas=1 statefulset emsserver

Figure 10 - To Stop and Start the EMS Statefulset

To delete the statefulset, storage, and services entirely, use the kubectl delete operation: > kubectl delete –f ems-aks.yaml,ems-aks-storage.yaml


The corresponding pod, statefulset, storage class, services will be deleted. The PVC and PV will not be deleted, nor will the corresponding data. To delete the data, PV, and PVC, use the following:

> kubectl delete pvc,pv –all

4.3 Connecting to the EMS Server Pod

The EMS server logs and configuration can be accessed directly through the kubectl exec command using the name of the EMS Server pod name. The default name is emsserver-0. This can be useful for viewing the logs, modifying the configuration file, etc.

> kubectl exec -it emsserver-0 -- /bin/bash

Figure 11 - Pod Access Example


5 Accessing and Testing EMS on AKS

When ems-aks.yaml was applied to the K8 cluster in AKS, two K8 Services were created, as shown below: emsserver NodePort 10.4.12.85 30722:30722/TCP 4d18h emsserverlb LoadBalancer 10.4.8.147 34.67.32.216 30724:30724/TCP 4d17h The emsserver service is a NodePort service allowing access internally, while the emsserverlb service will provide a LoadBlancer which can provide external access to the EMS Server.

5.1 Internal Access to the EMS Server

The EMS Server running in AKS can be accessed via the NodePort Kubernetes service and port. The default is tcp://emsserver:30722. Any K8 process running in the same cluster and name space can access the EMS Server using this URL. The following example shows an EMS client running in AKS connecting to the EMS Server running in AKS via the NodePort.

Figure 12 - Internal Connection to EMS

5.2 External Access to the EMS Server

5.2.1 Access the EMS Server The LoadBalancer K8 service, emsserverlb, will provide external access to the EMS server via the LoadBalancer Kubernetes IP address and port. The default port is 30724. Access will also be based on which trusted IP range defined in section 4.1. The following example shows access the EMS Server from an external source.


Figure 13 - External Access to the EMS Server

5.2.2 Connection Factory Update When the EMS Server is created in the Kubernetes cluster, the default connection factories are all created using emsserver as the hostname. While this works fine for internal EMS connections, the connection factories will fail with external connections. A new EMS connection factory should be created to allow for connection factor/JNDI support from external sources. Using the connection string used in the above example, an new connect factory can be created.

Figure 14 - Create an External Connection Factory

configuring a tibco enterprise message service environment · 2020. 10. 19. · tibco software inc....

Documents