bco 90 architecture scalability

8/17/2019 BCO 90 Architecture Scalability

1/53

© Copyright 10/31/2012 BMC Software, Inc 1BMC Proprietary & Confidential

Int'l Toll: 913-312-0828

US/CAN Toll free: 866-816-1982

Argentina: 0800 444 6440

Australia: 1 800 612 415

Austria: 0800 295 780

Bahamas: 1 800 389 0491

Belgium: 0 800 75 636

Brazil: 0800 891 0266

Bulgaria: 00 800 115 1141Chile: 123 0020 6707

China, Northern Region: 10 800 714 1509

China, Southern Region: 10 800 140 1376

Colombia: 01 800 518 1171

Czech Republic: 800 700 715Denmark: 80 883 277

Dominican Republic: 1 888 752 0002

France: 0 800 914 176

Germany: 0 800 183 0299

Greece: 00 800 161 2205 6440Hong Kong: 800 968 066

Hungary: 06 800 112 82

India: 000 800 1007 613

Indonesia: 001 803 017 6440

Ireland: 1 800 947 415

Israel: 1 80 925 6440

Italy: 800 789 377

Japan: 00348 0040 1009

Latvia: 8000 3523Lithuania: 8 800 3 09 64

Luxembourg: 800 2 3214

Malaysia: 1 800 814 723

Mexico: 001 800 514 6440

Monaco: 800 39 593Netherlands: 0 800 022 1465

New Zealand: 0 800 451 520

Norway: 800 138 41

Panama: 00 800 226 6440

Peru: 0800 54 129

Philippines: 1 800 111 010 55

Poland: 00 800 112 41 42

Portugal: 800 827 538

Russia: 810 800 2915 1012

Singapore: 800 101 2320

Slovenia: 0 800 80439

South Africa: 0 800 982 304

South Korea: 003 0813 2344Spain: 900 937 665

Sweden: 02 079 3266

Switzerland: 0 800 894 821

Taiwan: 00 801 127 186

Thailand: 001 800 156 205 2068Trinidad and Tobago: 1 800 205 6440

United Kingdom: 0 808 101 7156

Uruguay: 0004 019 0348

Venezuela: 0 800 100 8540

INTERNATIONAL TOLL FREE

BMC Capacity Optimization Best Practice


2/53

BMC Proprietary & Confidential

BMC Capacity Optimization Best PracticesBCO Architecture and Scalability

With 9.0 updates


3/53


Legal Notice

The information contained in this presentation is the confidential information of BMC,

Inc. and is being provided to you with the express understanding that without the priorwritten consent of BMC, customers and partners may not discuss or otherwise disclosethis information to any third party or otherwise make use of this information for anypurpose other than for which BMC intended.

All of the recommendations and information described herein are at the sole discretionof BMC and are subject to change and/or cancellation, and in no way should this contentbe viewed as guarantees or warrantees on BMC’s part.


4/53


Overview

Best Practice vs. How To

Does not address every scenario

Prior knowledge of BCO components and terms

Updates with 9.0 SP1


5/53


BCO 9.0 best practice update

Topic Session Speaker

BCO Architecture and Scalability This session Sudheer

Architecture and Scalability ReviewScalability for ETL engines – local and remote

HA deployments of BCO

LDAP deployments and upgrades

BCO Data Integration Dec 2012 Giuseppe

ManagingBPA to BCO data integration Jan 2013 Mike

Deploying and configuring BCO-CLM integration Feb2013 Sudheer

StorageCapacity Management Mar 2013 Sudheer


6/53



Architecture and Scalability Review

Scalability for ETL engines – local and remote




7/53


BCO main components

BCO Application Server

• providescommunicationbus forall other components

• all analytical, modeling and reporting tasks

• supports BCO presentation layers

BCO ETL Engine Server

BCO Database Server

• supportsall data integration tasks

• providessecondaryscheduling services

• Data management and warehousing

• Data export for third-party reporting tools

supported on:

• any OS supported by Oracle

supported on:

• RedHat Enterprise Linux

• SuSELinux Enterprise

• CentOS

• Solaris (x86 architectures)

supported on:

• Same as Application Server


8/53


… and their major sizing drivers


9/53


Component responsibilities

ETL Engine AS ‐ Data hub Database Server AS‐ web

Extract /

parse / poll

Transform

Load

Transfer and load

for remote EE

Calculate

custom

stats

Summarize

Run analyses

and models

Run reports

Materialize

data marts

Run

background

tasks

Run services

Build and navigate

user context

Serve data

If you separate these two, be sure to share the repository between them!


10/53


AS installation more flexible OOTB


11/53


Throughput Load = samples per day

Entities- Monitored devices or servers that performance metrics are collected from- Examples

Server OSDatabase ApplicationNetwork Device

- Typically 90% of the data collected

Business KPIs- Non technical performance metrics- Examples

Financial DataBusiness Transaction VolumesFacility Data

- Typically 10% of the data collected


12/53


Throughput load examples

Example Data Processing Volumes

Samples/day Number of entitiesNumber of metrics

(average by entity)Time

granularity

5 million

1250 40 15 minutes

2500 40 30 minutes

5000 40 1 hour

10 million

2500 40 15 minutes

5000 40 30 minutes

10000 40 1 hour

20 million

5000 40 15 minutes

10000 40 30 minutes

20000 40 1 hour


13/53


Minimum Hardware

Role CPU RAM DiskDatabase Server 2 cores @2GHz up to 5Mln rows/day;

1 additional core @2GHz everyadditional 5Mln rows/day

2GB (SCSI or external storagerecommended)

ApplicationServer 2 cores @2GHz 8GB 8GB free+ 20GB for the repository (SCSIrecommended)

ETL Engine 2 cores@2GHz 2GB 8GB free


14/53


Database Server

Major Considerations- Horizontal and Vertical scalability- Number of entities

- Number of metrics- Sample rate- Retention periods

Minimum Hardware- CPU –2 cores at 2 GHz,- Memory –2 GB RAM- SCSI Disk minimum- At least two sets of disks

- Disk Space –50 GB for every 1 million samples per daybased on BCO standard summarization policies


15/53


Database Server

TablespaceSizing- Should be increased for large environments- Default Sizing

BCO -- 150 MB BCO_IDX -- 150 MB BCO_DATA -- 8 GB BCO_DATA_IDX -- 8 GB BCO_DASH -- 100 MB

- Typical Increased Sizing for large environments BCO -- 512 MB (with 128 KB extents) BCO_IDX -- 512 MB (with 128 KB extents) BCO_DASH -- 384 MB (with 128 KB extents) BCO_DATA -- 60% of remaining space (with 8 MB or 16 MB extents)

BCO_DATA_IDX -- 40% of remaining space (with 8 MB or 16 MB extents)

- Temporary Tablespaceshould be greater than 3GB

- UNDO Tablespaceshould be greater than 3GB


16/53


Database Server

Oracle Express Edition (XE)- Database size is limited by the Oracle licensing- Different Oracle license agreements have different quotas

- Use only for very small (or pilot) installations

Additional Recommendations- Use a dedicated instance

- Physical server improves I/O compared to virtual server - Use Oracle Partitioning Option (Enterprise Edition)- Use Oracle Automatic Storage Management (ASM)

- Use database JVM option installed and enabled.Must be enabled on the database instance.


17/53


Database Server

Minimum Hardware Scalability- Up to 5 million data samples a day- Default retention policies

Scaling- Increase hardware resources- For each additional 5 million samples per day

1 CPU cores at 2 GHz1 Gig RAM

- Disk50 GB of storage is required for every 1 million of samples a dayGrows linearly

Default summarization policies


18/53


Database Server

IOPS- 100 IOPS are required for every 1 million of samples a day- Grows linearly

- Default retention policies

Out-sync samples- Add additional processing & Storage- Manage as corrective factors for the estimates- Minimum estimate 0.1% of overall samples for entity metrics- 1.0% of overall samples for business metrics


19/53


Database Server

Example- Conditions

5000 systems

100 metrics per systemHourly sample rateDetailed aging policy 95 days

- Analysis

12 million daily samples (initial estimate)600 GB of storage (initial estimate) Assume 1.0% out-of synch (pessimistic)Corrected samples = 12 * 1.01 = 12.12Corrected storage = 12.12 * 50 GB = 606 GB 610 GB Account for aging policy 670 GB


20/53


Application Server

Major Considerations- Horizontal and Vertical scalability- Number of users

- Number reports- Required data volume processing throughput

Minimum Hardware- CPU –2 cores at 2GHz,- Memory 8GB RAM- SCSI Disk- Disk Space - 8GB- Repository Disk Space - 20GB


21/53


Application Server

Minimum Hardware Scalability- 100 users- 100 reports

- up to 10 million data samples a day

Vertical Scaling- Increase hardware resources- 20 million samples per day- 4 CPU cores at 2 GHz- 16 Gig RAM- A single application server should be limited to 20 million samples per day


22/53


Application Server

Horizontal Scaling- Add additional Application Servers- Requires load balancing on the front-end

See product documentation for configuration

Additional Considerations- One primary scheduler for the environment- One Data Hub for the environment- Remote Engine ETL Servers

Requires additional resources for the Data HubSee more on this later.


23/53


Application Server Deployment

Supported Operating Systems – newer OS support added, also dropped EL4- SUSE 11 added, etc.

See next slide

Additional Software- JRE –automatically installed- Apache web server –automatically installed

- Oracle Client 11.2.x recommended, both for 32 bit and 64 bit.Seeproduct documentation for requiredoptions and libraries

OS Account Requirements- The system user configured to run BCO and ETL components must have rights tocrontab

- Cpit is the default OS ID who owns the BCO files, etc.- The install is run as non-privileged account- If you have a cron.denypolicy, make sure that the cpit user is not included- If you have a cron.allowpolicy, include cpit in it


24/53


Operating system versions

Operating

system Version Architecturex86

Architecturex86_64

Red Hat Enterprise Linux

5. x Yes Yes

6. x Yes

Oracle Linux 6. x

Yes

Linux CentOS 5. x Yes Yes

SUSE Linux Enterprise

10. x Yes

11. x Yes

Oracle Solaris 10. x

Yes

11. x yes


25/53


Application Server Deployment

Recommendations- BCO base installation directory

$CPITBASE

suggested value /opt/bmc/BCO A frontend web server is configured OOTBUses Apache Allows cachingSupports compression

- Multiple Application ServersRepository directory needs to be shared

–NFS share (i.e. by configuring on all servers a mount point that maps to external storage) –SAN LUN.


26/53


ETL Engine Server

Major Considerations- Horizontal and Vertical scalability- Number of connectors

- Number entities- Number of metrics- Sample rates- Required data volume processing throughput

More on this in the next topic


27/53


General Installation Notes

BCO Installation Versions- Please make sure that you downloaded the correct installation package for your(supported) platform and architecture.

-

Install only on supported platforms- Installation may successfully complete on a non-supported platform, but BCO servicesmay not start and the product may not work as expected.

Installation Order- Database- Application Server - ETL Servers- Remote ETL Servers

- Perform Initial Configuration- Install and configure additional solution components

CLM integration, Chargeback, Consolidation, Virtual Farm


28/53








29/53


ETL engine sizing guidelines

ETL Engine configuration Disk space No. of connectors Samples per day

2 CPU cores @ 2GHz, 4 GB RAM 8 GB free 100 instances 10 million

4 CPU cores@ 2 GHz, 8 GB RAM 16 GB free 100 instances 20 million

Limit each connector task scheduled to 2 million samples.

Avoid scheduling more than 1 connector task per CPU at the same time. Allow twice as much memory for Java ETLs as for perl ETLs.

Vertical scaling:

Horizontal scaling:

The limit is the size of the BCO database.Special considerations for remote ETL engines (see next slide).


30/53


Local versus remote ETL Engine

Local ETL Engine

Remote ETL Engine Data hub

Oracle DB server extractload

Local ETL Engine

Remote ETL Engine

Oracle DB server

Remote ETL Engine increases CPU, memory, disk I/O on Data hub machine

JMS serviceextract transfer save

Data hub

JMS queue

load


31/53


Why use a remote ETL engine?

Only when:- A limited bandwidth or unstable connection is present between data

source and the BCO database. – Remote ETL Engine separates extraction phase from load phase. – Data extraction occurs reliably close to the datasource, and the load phase

occurs at the Data hub. – Store-and-forward messaging infrastructure makes transfer reliable.

- It is not possible to expose the needed TCP ports on the BCOdatabase. – Remote ETL Engine communicates with the BCO Data hub via HTTP or

HTTPS ports.

Don’t use a remote ETL Engine just because the data source is remote. – The name "local" in a local ETL Engine does not imply that it must be on thesame LAN as the database. As long as the ETL Engine can reach thedatabase, a local ETL Engine is to be preferred.


32/53


Resource impact of remote EE on Data hub

Remote ETL engines use a two-step process: – A store-and-forward messaging infrastructure transfers data to the Data hub

machine (usually the same as the Application Server machine).

– Data is loaded into the data warehouse by connecting to the Oracle database.

This two-step process involves use of the following resources: – Reads and writes to the disk on the Data hub machine. – Disk space on the EE machine to accumulate pre-transfer data for many days.

» (Note: today there is no check!) Accumulate for ever! – Enough disk space on the Data hub machine to accumulate transferred data

for 3 days.» Allows for space in case of down time for a weekend.

– CPU and memory on the Data hub machine to parse the transferred data, do

lookups, and format the samples for loading into the database. – Database connection pools for the Data hub component.

Even if there are sufficient resources for all of the above, the overallprocess also takes longer to complete than for local ETL Engines.


33/53


Setting the Data hub JVM heap size

WarningBe careful when modifying the heap size of the JVM! Wrong settings maycause unpredictable and hard to diagnose failures.

Edit the filecustomenvpre.sh.

Uncomment the lines:- #DATAHUB_HEAP_SIZE="1024m"- #export DATAHUB_HEAP_SIZE

and replace the value with "8192m" before restarting the Data hubservice.


34/53


Increasing the database connection pool size

WarningWhen modifying the database connection pool size, it is important toverify that the Oracle database also has corresponding sizes for the

number of concurrent sessions.

Edit the filecaplan-ds.xml.

Look for the element

- CaplanDHDSReplace the value for the element- 40.

WarningMake sure to edit the setting for theCaplanDHDSdatasourceonly!


35/53








36/53


Component single points of failure

Component Single

point

of

failure? Approach

for

high

availability

Data warehouse Yes Use multiple machines and Oracle RAC option

Web application Yes, if only one instance Use commercial cluster manager for A/P

Install on multiple machines, and use load balancer for A/A

Data

hub

and

Primary

scheduler Yes Use commercial cluster manager for A/P

ETL Engine and scheduler Yes, for its portion of data

connectors Use commercial cluster manager for A/P

BCO installation directory Yes Use either storage replication or host‐based

mirroring

BCO Repository Yes Use either storage replication or host‐based mirroring


37/53


BCO machine configurations

Configuration BCO

components

installed

AS‐ALL Web application, Data hub, Primary scheduler

AS‐WEB Web application

AS‐HUB Data hub, Primary scheduler

EE ETL Engine, scheduler


38/53


HA for AS-ALL configuration (1)

Configure two machines M1 and M2 in A/P failover using OS clusteringsoftware.

Configure the following resources to be shared and enabled among M1and M2: – A floating IP address associated with the BMC Capacity Optimization URL via

DNS – Front-end web server (Apache), when configured

– BMC Capacity Optimization Console JSP engine (Apache Tomcat) – Primary scheduler – Data hub (JBoss)


39/53


HA for AS-ALL configuration (2)

You need: – Primary machine M1, Secondary machine M2 – Third-party clustering software

– Shared or synchronized file system between M1 and M2 for installation files – Shared or synchronized directory for BCO repository

Steps: – Configure clustering software (typically scripts) for failover and giveback.

– Temporarily change the hostname of M1 to the cluster nameusinghostnamecommand as root. – Install BCO on M1. Wait for the post-installation to complete. – Change the hostname of M1 back to the individual machine name. – Shut down BCO services.

– Move installation directory into shared or synchronized mount point, and leavea symbolic link in the original installation directory.

– Create a similar symbolic link on machine M2. – Restart BCO services on M1 and start using BCO. – Now M1 will be used as primary, and M2 as secondary.


40/53


HA for AS-WEB configuration

Option 1: Configure two machines M1 and M2 in A/P failover:- Configure the following resources to be shared and enabled among

M1 and M2: – A floating IP address associated with the BMC Capacity Optimization URL via

DNS – Front-end web server (Apache), when configured – BMC Capacity Optimization Console JSP engine (Apache Tomcat)

-

Then, proceed exactly as in AS-ALL configuration.Option 2: Configure as many machines as needed in A/A using aload balancer.


41/53


HA for AS-HUB configuration


Configure the following resources to be shared and enabled among M1and M2:- A floating IP address- Primary scheduler

- Data hub (JBoss)Then, proceed exactly as in AS-ALL configuration above.


42/53


HA for EE configuration


Configure the following resources to be shared and enabled between M1and M2:- Scheduler - Data accumulator

Then, proceed exactly as in AS-ALL configuration above.


43/53







LDAP native mode


44/53


LDAP native modeUse LDAP only for authentication

BCO admin creates user and authorizes user

1

LDAP

server

User logs in with name and password

3

LDAP bind, search

sequence4

BCO authorizes user

5

BCO records user and authorization

2

User authentication as part ofbind in sequence

4a

Authentication

server

LDAP managed mode


45/53


LDAP managed modeCreate user account automatically

LDAP

server

User logs in with name and password

1

LDAP bind, searchsequence

2

4

BCO creates user and authorization on firstlogin

User authentication as part ofbind in sequence

2a

Authentication

server

Return user entry3

BCO LDAP sequence


46/53


BCO LDAP sequencebind directly with BCO user account

LDAP

server

Name and password

LDAP search using user bind context (search for groups)

Return user entry information

LDAP bind using username and password

Return success for bind

Site-specificauthentication

Use user entry info forauthorization

BCOuser

BCO LDAP sequence (in BCO 4 0-9 0)


47/53


BCO LDAP sequence (in BCO 4.0 9.0)bind using separate account

LDAP

server

Name and password

LDAP bind using admin account

LDAP search using user bind context


Second bind using DN and password


Site-specific

authentication


BCOuser

LDAP search using admin context (retrieve user DN from attributes)

Return user DN for next bind

BCO LDAP sequence 9.0 SP1


48/53


BCO LDAP sequence 9.0 SP1search LDAP through separate account

LDAP

server

Name and passwordLDAP bind using admin account


Site-specificauthentication


BCOuser

LDAP bind using DN and password

LDAP search using admin context (retrieve user DN from attributes)

Return user DN for bind

LDAP search using admin bind context (search for groups)


LDAP gro p mapping


49/53


LDAP group mapping

BCO version LDAP group mapped to BCO

role

LDAP group mapped to BCO access

group

4.5 “External name” on role (not available)

9.0 SP1 “External name” on role “External name” on access group

LDAP integration in 9 0 SP1


50/53


LDAP integration in 9.0 SP1

4.5- LDAP groups can be mapped as “external names” for a role.

– Semicolon-separated list of names specified in configuration screen.

- When logged-in BCO user is found in LDAP, and the user is part ofany of these groups, then the user is automatically assigned that role.

9.0- Roles AND access groups can be assigned “external names”.- When the user is part of any of these groups, then the user is

automatically added to the ac group.

For migration from 4.5 to 9.0

- You want to set up LDAP once. – Sign up once for the right groups in LDAP. – When users log into BCO, automatically get their access rights.

Migrating from 4 5


51/53


Migrating from 4.5

Today:- Identify which acgroups a user should be member of.- Assign explicitly the user to the acgroups in BCO.

For preparation for 9.0:- Create LDAP groups corresponding to the acgroups- Add these users to the appropriate LDAP groups.

After migrating to 9.0:- Specify external names for the acgroups.- These users will automatically be added to the acgroups on login.

– If a user was not added in LDAP, then he will be removed from the acgroup.

– An acgroupcan either be manually populated with users, or automaticallyusing LDAP external names. Not both.

Workaround if LDAP modifications are delayed:- Create a separate acgroupwith mnaual membership.

Additional Resources & Information


52/53


Additional Resources & Information

Product Documentation- https://docs.bmc.com/docs/display/public/bcmco90/Home

BMC Communities (public forum)-

https://communities.bmc.com/communities/community/bmcdn/service_assuranceBMC Support Knowledge Articles

- https://communities.bmc.com/communities/docs/DOC-18600

Chatter BCO Group –All about BCO- https://na13.salesforce.com/_ui/core/chatter/groups/GroupProfilePage?g=0F9300000004FIN


53/53


bco 90 architecture scalability

Documents