oow2011 rittman architecture

7/27/2019 Oow2011 Rittman Architecture

1/47

T : +44 (0) 8446 697 995 or(888) 631 1410 (USA) E : [email protected] W: www.rittmanmead.com

Mark Rittman, Rittman Mead BI Forum 2011ODTUG KScope11, Long Beach, California

OBIEE11g Architecture & Internals

Saturday, 1 October 11
http://www.rittmanmead.com/http://www.rittmanmead.com/mailto:[email protected]:[email protected]


2/47


Agenda

Overview of OBIEE 11gR1 (and whats new in 11.1.1.5)

High-Level Product Architecture, and Role within Oracle Fusion Middleware 11g

The Oracle BI Domain, WebLogic Server, and the Oracle BI Instance

Use of Oracle Fusion Middleware and Oracle Database

Filesystem and Environment Changes What do WebLogic and Enterprise Manager Do?

How do Diagnostics and Systems Metrics work with 11g?

How does the Systems API work?

Clustering & Multi-Tenancy



3/47


Oracle Business Intelligence 11gR1

The latest release of Oracles business intelligence platform

Derived from Siebel Analytics; second major Oracle release

Now at version 11.1.1.3 (hopefully 11.1.1.5 by the time of the BI Forum)

11gR1 new features focused on manageability and middleware integration



4/47


Key New Features introduced with OBIEE 11gR1

New Fusion Applications / ADF look and feel for front-end components

WebLogic Server and Oracle Enterprise Manager

Some enhancements to the Oracle BI Repository

Action Framework

KPIs and Scorecards



5/47


OBIEE 11g Basic Architecture

Overall system called anOracle BI Domain

Made up of Java and non-Javacomponents

Java components hosted inWebLogic Server 11g

Managed using OEM andWebLogic Admin Server

Supporting database schemafor repositories, metadata



6/47


OBIEE 11g Simple Install Architecture (Developer Sandbox)

OBIEE 11.1.1.5 now gives the optionto perform a Simple Install

Places all Java components in theone, administration, server

No install of the Node Manager

Reduces memory requirement byaround 1GB

Useful for laptops, smaller servers

Drawback is no ability to performhorizontal scale-outs in future



7/47


OBIEE 11g System Components, Java Components and OPMN

OBIEE components divided into System and Java components

System components are still C/C++ executables, are controlled by OPMN,and are managed by Fusion Middleware Control

Java Components are JEE applications, are usually installed in the

managed server, and are controlledby Fusion Middleware Control

mailto:[email protected]://www.rittmanmead.com/http://www.rittmanmead.com/mailto:[email protected]:[email protected]


8/47


Oracle Process Manager and Notification Server

Used for distributed processstart/stop/restart

Process recovery (optional),process ping

Managed via EM, or fromcommand line

Can also be used forperformance collection (DMS)

Required on everyOBIEE server

Standard FMW processcontrol tool

C:\Middleware\Oracle_BI1\opmn\bin>opmnctl status

Processes in Instance: instance1---------------------------------+--------------------+---------+---------ias-component | process-type | pid | status---------------------------------+--------------------+---------+- --------coreapplication_obiccs1 | OracleBIClusterCo~ | 3364 |Alivecoreapplication_obisch1 | OracleBIScheduler~ | 1012 | Alivecoreapplication_obijh1 | OracleBIJavaHostC~ | 4400 | Alivecoreapplication_obips1 | OracleBIPresentat~ | 2092 | Alivecoreapplication_obis1 | OracleBIServerCom~ | 5040 | Alive

mailto:[email protected]:[email protected]://www.rittmanmead.com/http://www.rittmanmead.com/mailto:[email protected]:[email protected]


9/47


Oracle Enterprise Manager Fusion Middleware Control

Manage System Components (BI Server, BI Presentation Server etc)

Start, Stop and Restart all System Components and Managed Servers

Configure Preferences and Defaults

Scale out System Components

Performance Monitoringand Diagnostics



10/47


The WebLogic Domain

Each OBIEE 11g Oracle BI Domain contains a single WebLogic Domain

Contains one Admin Server, and at the start,one Managed Server

New in 11.1.1.5 - ManagedServer is optional

Can add managed serversthrough Horizontal Scaleout

WebLogic Domain administeredthrough Admin Console,and WLST (Weblogic scripting)

Managed servers can be

started manually, or throughNode Manager



11/47


Oracle Instances

A collection of System Components, on a single host

Managed together using OPMN (instances are OPMN organisational units)

Starts with one of each systemcomponent, on each host

Can be added to by useofVertical Clustering

An Oracle BI Domain can containmultiple instances, one per host



12/47


What is a Farm?

All instances across the Oracle BI Domain

Each Enterprise Manager Fusion Middleware Controlinstallation manages a single farm, made up of

One or more instances

A single WebLogic domain



13/47


Based on Fusion Middleware 11g Platform

Fully-integrated with Oracle Fusion Middleware 11g

Common security (Oracle Identity Management, Oracle Access Manager)

Built on WebLogic Server

Managed through Fusion Middleware

Control (Enterprise Manager) Built using multiple technologies

including JDeveloper (ADF)



14/47


11g Architecture Compared to 10g Architecture

Main difference is wrapping components in WLS platform and EM management

Individual servers, components are still the same (BI Server, Presentation Server,BI Publisher J2EE application etc)

Some reworking has taken place in 11gR1 (unified logging, management of

config files using EM etc) Basic concepts, plus clustering (OBIEE), scheduling etc are the same as in 10g

Web Server

(IIS, Tomcat, Websphere, iPlanet)

Oracle BI ServerOracle BI Server

Intelligent Request GenerationIntelligent Request Generation

Data Source AdaptersData Source Adapters

NavigatorNavigator

Logical Request GenerationLogical Request Generation

AggregateNavigator

FragmentationOptimization

Multi-Pass /Sub-Request Logic

Optimized Query RewritesOptimized Query Rewrites

Session ManagementSession Management

MetadataInterchange

MetadataInterchange

Logical SQL ODBC/JDBC (Logical Business Model)Logical SQL ODBC/JDBC (Logical Business Model)

CacheCache

ServicesServices

Oracle BI AdministrationOracle BI Administration

Analytical and Operational Data Sources

Metadata Documentation ServicesMetadata Documentation Services

Metadata Management ServicesMetadata Management Services

Multi-User Development ServicesMulti-User Development Services

Server Management ServicesServer Management Services

ODBC over TCP/IP (SSL)

System/

System/

Perf

PerfMonitoring

Monitoring

ODBC, CLI, OCI, XML, MDX

Oracle BI Presentation ServicesOracle BI Presentation Services

SOAP Web Services, XML and URL InterfaceSOAP Web Services, XML and URL Interface

Cache Services (Web) & ConnectionCache Services (Web) & ConnectionMngmtMngmt

XMLXMLFrameworkFramework

WebWebCatalogCatalogServiceService

User Profiling, Security and SessionUser Profiling, Security and Session MngmtMngmt

Oracle InteractiveOracle InteractiveDashboardsDashboards

Oracle AnswersOracle Answers

Load BalancerLoad Balancer

SecurityServices

SecurityServices

QueryGovern.

QueryGovern.

TCP/IP

(SSL)

Javascript

for Usability & Interactivity

Web Browser

XML,HTML,

XLS,PDF,TXT

overHTTP/HTTPS

External Applications and Portals

HTML,SOAP

overHTTP/HTTPS

Execution EngineExecution Engine

SAWBridge (J2EE/ISAPI)SAWBridge (J2EE/ISAPI)

TCP/IP (SSL)Oracle BI PublisherOracle BI Publisher

Delivery ServerDelivery Server

Layout InterfacesLayout Interfaces

Data LogicData LogicXSL

Externalized

Authentication

LDAP

DB Authentication

Custom Authenticator

Oracle BIOracle BI

Cluster ControllerCluster Controller

Oracle Delivers ServerOracle Delivers Server

Scheduling/Event ServicesScheduling/Event Services

Agent Execution LogicAgent Execution Logic

Device Adaptive ContentDevice Adaptive Content

vs.



15/47


Core OBIEE Components Still Play Same Role

Once you strip away all of the middlewarecomponents, OBIEE 11g still works infundamentally the same way as OBIEE 10g

Users queries via the Presentation Server

The Oracle BI Server converts thesequeries to physical SQL/MDX, via theOracle BI Repository

Queries are passed to the underlyingphysical databases and OLAP cubes

Data returned to users in the form of

dashboards and reports



16/47


Oracle BI Server Still Works as the Analytic Middleware

Parses incoming logical SQL queries

Uses the navigator to generate optimizedphysical queries

Manages the query cache

Handles row and subject-area security

Provides load-balancing informationto the Cluster Controller

But now delegates authorization/authentication to Fusion Middleware



17/47


OBIEE 11g Security and Oracle Fusion Middleware 11g

OBIEE 11g delegates security to Oracle Fusion Middleware 11g

Leverages Oracle Platform Security Services

Users and Groups in RPD now moved toembedded WLS LDAP Server

RPD and Webcat groups replaced byFMW11g Application Roles

Comprehensive SSL and Credentials Management

Encrypted RPD, plus optional report encryptionand watermarking

Flexible authorization model through

WLS and OPSS Still backwards compatible with

LDAP model in OBIEE 10g

Infrastructure &Management

Database

Middleware

Applications



18/47


WLS Embedded LDAP Server

By default, OBIEE 11g users and groups are now held in the WLS LDAP Server

More robust directory for storing user details

Recommended for >1000 users

WLS Admin Server Console now used for

creating and maintaining users BI Server outsources all authentication,

authorization to FMW11g

BI Administration tool now used forsubject area and row-level security,connection pool passwords only

WLS LDAP Server can be swapped outfor alternative directories (MS AD etc)



19/47


Application Roles and Application Policies

Application roles introduce an indirection between LDAP groups and BI groups

LDAP server provides users, groups

FMW11g provides application roles

Application roles are granted to

LDAP users, groups Permissions are assigned to

application roles

Breaks direct link between groupsand roles

Application roles can be exported

between FMW11g environmentsAll RPD, webcat permissions secured

against application roles

Oracle

Business

Intelligence

Roles Groups

ApplicationRoles

/Policies

UsersUsers



20/47


OBIEE Security Providers

OBIEE 11g (through FMW11g) uses three Security Providers

Authentication Provider (defaults to WLS LDAP Server)

Policy Store Provider (defaults to WLS)

Credential Store Provider (defaults to WLS)

Flexible security framework that allows easy linking to external securityAll enabled through Oracle Platform Security Services



21/47


So Whats New in OBIEE 11.1.1.5?

Simple install option installs Java components in WLS administration server,doing away with node manager and managed server

64-bit Linux install lays down the 64-bit JVM

64-bit Windows install lays down a working 64-bit BI Administration tool (sort of)

Option to install BI Composer, a simplified analysis generation tool

Support for Oracle OLAP andSAP/BW as data sources

In-place and out-of-placeupgrades from 11.1.1.3



22/47


Upgrading from 11.1.1.3 to 11.1.1.5

To options for upgrading; out-of-place and in-place upgrade

Out of Place Upgrade involves installing 11.1.1.5 in a new MW Home

Manually migrate RPD, catalog and other BI objects

Manually migrate security, roles, users to new environment

Advantage : conceptually simple, but resource-intensive In-Place Upgrade involves following the FMW11g patching process

Detailed at http://www.rittmanmead.com/2011/05/in-place-upgrading-of-obiee-11-1-1-3-to-11-1-1-5/

Upgrade WLS from 10.3.3 to 10.3.5 via metalink-only Upgrade Installer

Do software-only install of OBIEE 11.1.1.5 into existing MW Home

Perform pre- and post-patching steps

Advantage : upgrades in-place, but a non-trivial exercise

http://www.rittmanmead.com/2011/05/in-place-upgrading-of-obiee-11-1-1-3-to-11-1-1-5/http://www.rittmanmead.com/2011/05/in-place-upgrading-of-obiee-11-1-1-3-to-11-1-1-5/http://www.rittmanmead.com/2011/05/in-place-upgrading-of-obiee-11-1-1-3-to-11-1-1-5/http://www.rittmanmead.com/2011/05/in-place-upgrading-of-obiee-11-1-1-3-to-11-1-1-5/http://www.rittmanmead.com/2011/05/in-place-upgrading-of-obiee-11-1-1-3-to-11-1-1-5/http://www.rittmanmead.com/2011/05/in-place-upgrading-of-obiee-11-1-1-3-to-11-1-1-5/http://www.rittmanmead.com/http://www.rittmanmead.com/mailto:[email protected]:[email protected]


23/47


Common OBIEE 11g Architecture & Internals Questions

What is the new directory structure for an OBIEE installation (and why are thereso many duplicated directories?)

What does Oracle WebLogic Server actually do?

What does Oracle Enterprise Manager Do?

And What Happens When I Press the Activate Button? How Does Logging Now Work in OBIEE 11g? What is ODL and DMS?

Where Does EM and WLS Keep Its Settings? What Secret Files Are There?

So How Does the Systems Management API Work?

Whats the Difference Between WebLogic and OBIEE Clustering?

Can You Run Multiple RPDs and Webcats on the Same OBIEE11g Box?

How Has Change Management Changed?



24/47


OBIEE 11g Directory Structure

Binaries in the BI Oracle Home

Configuration files (RPD,config files, diagnostics,WebCat) in the BI Instance Home

WLS binaries in theWL Home

WLS domain filesin Domain Home

Separation of binariesfrom configuration/instance

files



25/47


Why So Many Duplicated Directories?

Throughout the /instances/instancensub-directory structure, there are lots ofduplicated directories for BI Server, PS etc

These are for the System Components

One per system component

Divides config files, repositories etcinto subfolders, one for each component

Found under the following /instancenfolders

/bifoundation : for RPD, webcat etc

/config : for NQSConfig.INI etc

/diagnostics : for logs etc

/tmp : for logs etc



26/47


What Does Oracle WebLogic Server Do?

Oracle WebLogic Server, along with Oracle Fusion Middleware 11g, provides thefollowing key areas of functionality

Hosts the Oracle BI Analytics plugin (as per OC4J and IIS in OBIEE 10g)

Hosts the other java components

Provides the host platform forOracle Fusion Middleware 11g

Provides the LDAP server, and othersecurity stores for the OPSS securityproviders

Hosts the WLS Admin Console and

EM Fusion Middleware Control

mailto:[email protected]://www.rittmanmead.com/http://www.rittmanmead.com/http://www.rittmanmead.com/mailto:[email protected]:[email protected]


27/47


What Does Oracle Enterprise Manager Do?

Enterprise Manager uses the OBIEE 11g Systems Management API to performadministration tasks

You can also use this API through WLST scripting, or through Java code

Uses a set of JMX MBeans (Java Management Extensions) to perform tasks

MBeans are Managed Beans, Java utilitiesthat have properties and methods

Every EM task has a correspondingMBean method

Some MBeans have yet to beexposed through EM



28/47


The MBean Browser

The list of OBI 11g administration MBeans can be viewed in the MBean Browser

Using EM, right-click on the AdminServer, then selectfarm_bifoundation_domain > Weblogic Domain> bifoundation_domain > MBean Browser

OBI11g entries under several headings: oracle.biee.admin

oracle.biee.local

oracle.as.management.mbeans.opmn



29/47


The MBean Browser

The list of OBI 11g administration MBeans can be viewed in the MBean Browser

Using EM, right-click on the AdminServer, then selectfarm_bifoundation_domain > Weblogic Domain> bifoundation_domain > MBean Browser

OBI11g entries under several headings: oracle.biee.admin

oracle.biee.local

oracle.as.management.mbeans.opmn



30/47


The Oracle BI Systems Management API

A collection of MBeans used for providing systems management functions forOBIEE

A subset of the wider range of MBeans in Fusion Middleware 11g

Key Systems Management API MBeans include

ServerConfigurationMBean : upload and register new repositories

BIDomainMBean : lock, commit and rollback changes to OBIEE config

BIInstanceMBean : Start, stop and restart components

BILogConfigurationMBean, EmailConfigurationMBean etc



31/47


What Happens When You Press Activate?

When you press Apply, values are copied to these MBeans attributes

When you press Activate, the MBeans methods are called

In the background, changes are then made by the MBeans, on EMs behalf



32/47


So Can This All Be Scripted?

All calls to MBeans can be scripted through WLST

Command-line tool using Jython as the scripting language

connect("weblogic","welcome1","localhost:7001")domainCustom()

cd ('oracle.biee.admin')cd ('oracle.biee.admin:type=BIDomain,group=Service')biinstances = get('BIInstances')biinstance = biinstances[0]cd ('..')cd ('oracle.biee.admin:type=BIDomain,group=Service')objs = jarray.array([],java.lang.Object)strs = jarray.array([],java.lang.String)invoke('lock',objs,strs)cd('..')



33/47


So Can This All Be Scripted?

All calls to MBeans can be scripted through WLST

Command-line tool using Jython as the scripting language

connect("weblogic","welcome1","localhost:7001")domainCustom()

cd ('oracle.biee.admin')cd ('oracle.biee.admin:type=BIDomain,group=Service')biinstances = get('BIInstances')biinstance = biinstances[0]cd ('..')cd ('oracle.biee.admin:type=BIDomain,group=Service')objs = jarray.array([],java.lang.Object)strs = jarray.array([],java.lang.String)invoke('lock',objs,strs)cd('..')

Lock method

Create lock on Oracle BIDomain, prior to configurationchange



34/47


WLST Script Continued

cd (biinstance.toString())biserver = get('ServerConfiguration')cd('..')cd(biserver.toString())ls()

argtypes =jarray.array(['java.lang.String','java.lang.String'],java.lang.String)argvalues = jarray.array(['C:/SampleAppLite.rpd','Admin123'],java.lang.Object)invoke('uploadRepository',argvalues,argtypes)

cd('..')cd('oracle.biee.admin:type=BIDomain,group=Service')objs = jarray.array([],java.lang.Object)

strs = jarray.array([],java.lang.String)invoke('commit',objs,strs)



35/47







uploadRepository Method

Obtain next RPD sequence no.

Copy the RPD name andpassword into the MBeanproperties



36/47









37/47







commit Method

Activate configuration changes

Upload the RPD Update config files Store password in Credstore



38/47


Other Useful Uses of WLST Scripting

Stopping, starting and restarting the BI Instance

Stopping, starting and restarting individual components

Changing the web catalog path

Creating roles and mapping them to LDAP groups

Enabling and disabling caching

Setting other performance parameters

Vertical and horizontal scaleout



39/47


How Have Diagnostics and Logging Changed?

OBI 11g uses DMS (Dynamic Monitoring Services) to record key system metrics

Metrics and counters accessed via MBeans

OBI 11g logging is still carried out to nqserver.log, nqquery.log

Logs are now aggregated and parsed through ODL (Oracle Diagnostic Logging0

Logs can now be viewed in aggregate using the Log Viewer Enterprise Manager> Capacity Management > Diagnostics



40/47


Clustering in OBIEE 11g

OBIEE 11g Domains can be scaled-out (horizonal clustering)and/or scaled-up (vertical clustering)

Adds resilience, reduces single points of failure

Allows OBIEE systems to take advantage of more/larger hardware

Installer sets up single-node OBIEE 11g Domain; cluster can then be extendedout and upwards using EM and Universal Installer



41/47


Vertical Scaleout of System Components

Spare capacity on an individual host can be used by adding additionalsystem components

Can add additional BI Servers, Presentation Servers and Java Hosts

Useful for resilience, and usage of spare capacity

Doesnt protect against the whole server failing though



42/47


Horizontal Scaleout of Java Components

Used for adding additional managed servers (Java components) andsystem components to an existing cluster

Used for scalability and failover

Set up via the Universal Installer> Scale Out BI System option



43/47


WebLogic Clustering vs. OBIEE Clustering

Vertical clustering uses just OBIEE clustering features (Cluster Controller)

Cluster controller routes queries to clustered BI Server

Presentation Server, Cluster Controller, Java Host and Scheduler can also beclustered (active/active and active/passive)

Horizontal clustering uses WebLogic clustering Uses MBean method to pack/unpack domain and clone to separate server

Does not itself use/require OBIEE clustering

Does require WebLogic Server EE license



44/47


So ... Can We Run Multiple RPDs and Webcats on a Single Box?

No (or at least, weve not got it working)

Internal restrictions within OBIEE 11.1.1.3 that stops complete isolation of RPD /webcat pairs

Possibly coming (Multi-Tenancy) in OBIEE 11.1.1.5/6?

Otherwise, need to multiple Middleware Home installations



45/47


Summary

OBIEE11g has had a major architectural overhaul, adopting FMW11g technologies

Core servers stay the same, but new middleware wrapper

Uses WebLogic for J2EE application hosting, security, integration

Major security enhancements and changes

Enterprise Manager used for systems managemnet

WLST scripting can be used for process automation

EM and WLST both leverage MBeans for modularized administration

Clustering now extends to WebLogic clustering for horizontal scale-out

Diagnostics leverage MBeans and ODL for cross-node diagnostics and metrics



46/47


More Information

Thank you for attending this presentation

More information can be found at http://www.rittmanmead.com

Contact us at [email protected] [email protected]

Look out for our book, Oracle Business Intelligence Developers Guide due Q1 2012

Follow-us on Twitter (@rittmanmead) or Facebook (facebook.com/rittmanmead)

mailto:[email protected]://www.rittmanmead.com/mailto:[email protected]:[email protected]:[email protected]:[email protected]://www.rittmanmead.com/http://www.rittmanmead.com/http://www.rittmanmead.com/http://www.rittmanmead.com/mailto:[email protected]:[email protected]


47/47


Mark Rittman, Rittman Mead BI Forum 2011ODTUG KScope11, Long Beach, California

OBIEE11g Architecture & Internals

oow2011 rittman architecture

Documents