1 copyright © 2012, oracle and/or its affiliates. all ... · title: powerpoint presentation...


E-Business Suite Release 12.1 Upgrade Best Practices - Technical Insight

Isam Alyousfi Senior Director, Applications Performance

Udayan Parvate Director, EBS Release Engineering

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 3 3 Copyright © 2012, Oracle and/or its affiliates. All rights

reserved.

The following is intended to outline our general

product direction. It is intended for information

purposes only, and may not be incorporated into any

contract. It is not a commitment to deliver any

material, code, or functionality, and should not be

relied upon in making purchasing decisions. The

development, release, and timing of any features or

functionality described for Oracle's products remains

at the sole discretion of Oracle.


And so these men, disputed loud and long,

Each in his own opinion, exceeding stiff and strong,

Though each was partly in the right and all were in the wrong

Six blind men and the (Upgrade) Elephant..


Program Agenda

R12.1 Upgrade Overview

R12.1 Supported Upgrade Paths

R12.1 Upgrade Resources

R12.1 Upgrade Best Practices to

Minimize Downtime

Reference (Customer Upgrade

Snapshots and more)

Q&A


R12.1 Upgrade Overview


R12.1 Rapid Install (RI) R12.1 Maintenance Pack (MP)

R12.1 was generally available (GA) in May 2009

– Via Oracle Software Delivery Cloud (formerly known as Electronic

Product Delivery (EPD)) and Oracle Stores

Can be used by new and upgrading customers (11.5.9

and above) to go directly to R12.1

– If you are on R11i, use R12.1 RI from the software delivery cloud.

Follow instructions from the “Upgrade Guide: 11i to 12.1” and “12.1.1

Release notes”

– If you are on R12.0.X, use R12.1 MP (7303030) from My Oracle

Support (MOS). Follow instructions from “R12.1 Maintenance Pack Install

Instructions (752619.1)”

http://download.oracle.com/docs/cd/B53825_04/current/acrobat/121upgrade.pdf


EBS R12.1 Release Update Pack (RUP) 3 (12.1.3) Release

EBS 12.1.3 was released in Jul 2010 and delivers

bugfixes and targeted functionality enhancements

Available from My Oracle Support (MOS) as a patch

Can ONLY be applied after upgrade to R12.1

Currently, EBS 12.1.3 is the latest suite wide RUP

available for R12.1

EBS 12.1.3 Installation instructions : 1080973.1


R12.1 Technology Stack

For details “Oracle E-Business Suite Release 12.1 Maintenance Pack Installation Instructions” (752619.1)

TECHNOLOGY

COMPONENT

VERSION

INCLUDED

11i10CU2

VERSION

INCLUDED

12.0.4 RI

VERSION

INCLUDED

12.1 RI

VERSION CERTIFIED

WITH

MINIMUM REQUIRED

VERSION

• Apps Mid tier-

Forms/Reports 6.0.8.25 10.1.2.2 10.1.2.3 - 10.1.2.3

• Apps Mid tier-

Java Oracle

Home/

• Apps Mid tier-

JDK

1.0.2.2/1.4.2 10.1.3.0/1.5 10.1.3.4/1.6.0 10.1.3.5 10.1.3.4/1.6

• Database 9.2.0.6 10.2.0.3 11.1.0.7

10gr2: 10.2.0.5

11gr2: 11.2.0.1, 11.2.0.2,

11.2.0.3

10.2.0.4


11i10CU2 R12.0.4 RI R12.1 RI 12.1.3 RUP

#of Product

Schemas 209 195

(25 removed,11 added)

201

(25 removed,17 added)

No changes

#file calls in DB

portion of the U

driver

104242 144940 156622 23408

PROD db size

File system size

31 GB

26 GB

45 GB

28 GB

50 GB

28 GB

NA

#files shipped in

RI / RUP 268359 357778 385755 52103

#of Jobs (Changed

+ New) in DB

portion of the U

driver

NA ~95488 ( Vs 11.5.10.2 ) ~104057 ( Vs 11.5.10.2 )

~31843 ( Vs 12.0.4 )

~23474 ( Vs 12.1.1)

R12.1 Key Facts


R12.1 Supported Upgrade Paths


R12.1 Upgrade Paths Minimum EBS suite level for direct upgrade to R12.1

– 11i9, 11i9CU1, 11i9CU2 or above

– 11i10, 11i10CU1, 11i10CU2 or above

– R12.0 and above

Minimum EBS suite level required for database versions

– 10.2.0.4 requires 11i9CU2

– 10.2.0.5/11.1.0.7/11.2.0.1/11.2.0.2/11.2.0.3 require 11i10CU2/R12.0.4

Certified upgrade path options (761570.1) can be categorized into

A.Upgrade database and EBS level in a single downtime

B.Upgrade database and EBS level in separate downtimes

C.Apply upgrade interoperability DB patches and then upgrade EBS


R12.1 Upgrade Paths Continued.. 11.0/11i.X => 12.1

< = 11.5.8

11.5.9/cu1

11.5.10/cu1

12.1

10.2.0.4

12.1

10.2.0.5

12.1 11.2.0.1/11.2.0.2

11.2.0.3

12.1

11.1.0.7

11.0

11i10cu2

A

B ,C B ,C

B ,C

B ,C

B ,C

A. Upgrade database and EBS level in a single downtime

B. Upgrade database and EBS level in separate downtimes

C. Apply upgrade interoperability DB patches and then upgrade EBS

SOURCE

TARGET

11i9cu2


R12.1 Upgrade Paths Continued.. 12.0.X => 12.1

12.1

10.2.0.4

> = 12.0.4

12.1

10.2.0.5

12.1 11.2.0.1/11.2.0.2

11.2.0.3

12.1

11.1.0.7

< = 12.0.3 A

B ,C

B ,C

B ,C

B ,C

B ,C

A

A. Upgrade database and EBS level in a single downtime

B. Upgrade database and EBS level in separate downtimes

C. Apply upgrade interoperability DB patches and then upgrade EBS

SOURCE

TARGET


R12.1 Upgrade Resources


R12.1.3 EBS Data Model Comparison Report

Per product database object comparison between two releases for

the following object types (1290886.1)

–Regular tables, Partitioned tables, Index organized tables, Global

temporary tables, Queued tables

–Views, Materialized views, Materialized view logs

–Indexes, Sequences, Advanced queues, Packages, Triggers

Available for R12.1.3 Vs (11i9,11i10cu2,12.0.4,12.0.6,12.1.1,12.1.2)

Benefits

–Customers can focus on what has changed

–Easier to analyze impact on customizations, planned test coverage

–Differences viewable for all products in the same report via simple UI


R12.1.3 EBS ATG Seed Data Comparison Report

Per product EBS ATG Seed data type comparison between two

releases (1327399.1)

– Supports most of the EBS delivered seed data types. e.g. Menus,

Functions, Profiles and many more

Available for R12.1.3 Vs (11i10cu2,12.0.4,12.0.6, 2.1.1)

Benefits

– Meant for Advanced user with prior knowledge about EBS Seed data

delivery

– Easier for developers/consultants/testing team to analyze impact on

customizations, planned/desired test coverage

– Post Go-live, to answer end-user questions


EBS File Comparison Report (1446430.1)

EBS file system comparison per product for R12.1.3 Vs 11i10CU2

–Supports most of the file types found in EBS file system

–For ASCII files, provides new, removed and changed files compared to

prior release

–For binary files, provides new, removed files compared to prior release

Benefits

–In upgrade planning phase, customers can get information about new,

removed and changed files

–Assistance in analyzing impact on customizations, personalization and

planning functional flow tests

–Differences viewable for all products in the same report via a simple UI


EBS “pre-install” Patches Report (1448102.1)

What is a pre-install patch ?

– Any patch that fixes upgrade itself (critical upgrade failures, upgrade

performance issues)

– Must be applied to the R12.1 RI file system using “adpatch preinstall=y”

option

Report (revised ~monthly) logistics and usage

– EBS wide Report (grouped by product family)

– Patches categorized ( failure, performance, data fix, upgrade integrity )

–Merge and apply R12.1 CUP1 with listed patches. Then apply merged

upgrade driver

– Helps avoid time/effort chasing individual SRs for known upgrade fixes


R12.1 Upgrade Best Practices


Reduce amount of “work” for production upgrade

Use proven best practices for the commissioned “work”

Optimize upgrade for your hardware

Tune individual long runners and use special techniques

U

P

T

I

M

E

How to minimize upgrade downtime ?


Upgrade Best Practices

Identification of required patches

– Prepare a complete list of pre and post patches and recommended

code levels

Keep the system current on AD/ATG/OAM code e.g. latest AD/ATG

RUPs on 11i/R12.0 and once on R12.1

High priority patches from MOS.

Consolidated Upgrade Patches (CUP)

– EBS R12.1 CUP1 (7303029:12.1.0)

– Latest pre-install patches (1448102.1)

Review “Known-issues” sections from key “living” documents such

as Release notes, MP Install Instructions


Upgrade Best Practices Continued..

Patch merging, sequencing and adpatch options

– Use non-interactive patching

– Merge patches (228779.1). Merge NLS patches per language.

– Perform uptime maintenance when possible

hot patching of iHelp, NLS patches, upload patch history

Apply HRGLOBAL as a hot patch (1330470.1)

– Use adpatch options such as nomaintainmrc, phtofile, nolink,

nogenform, nogenrep, nocompile jsp, noautoconfig, novalidate

(1078973.1)



Downtime reduction procedures

– Run autoconfig in parallel on a multi-node system (387859.1)




– Use “Shared APPL_TOP” (384248.1) with “Distributed AD” for (236469.1) upgrades and regular maintenance for multi-node instances

No need to apply the same patch on multiple tiers

Distributed AD adds to the degree of parallelism by distributing AD workers across application tier nodes and improves timing for D/G portion of the patch driver.


adpatch workers 1-10




Admin/CM

Server

Web

Server

Web

Server

Forms

Server

Database

Server





Upgrade Best Practices Continued.. Shared APPL_TOP and Distributed AD

Shared

APPL_TOP


Upgrade Best Practices Continued.. Staged APPL_TOP

Production

Stage

1. Clone 1. Clone 4. Sync Patch History

2. Upgrade to R12

2. Upgrade DB

3. Sync APPL_TOP

3. Update DB




– Use “Staged APPL_TOP” (734025.1) for regular maintenance and upgrade

Saves time to patch the file system (C/G portion) by using a patched up copy of production instance file system

Use in 11i => R12.1 upgrade to avoid applying NLS C/G portion

Can use for R12.0.X => R12.1 upgrade and once on R12.1



Downtime reduction procedures continued..

– Use TUMS (“The Upgrade Manual Script”)

To avoid running tasks not relevant to your system

– Import statistics

Export statistics gathered during test runs, and import during final run

– Automate as much as possible

– Time every activity and deal with bottlenecks



Downtime reduction procedures continued..

– For the duration of the upgrade, consider…

Disabling custom triggers and business events

Disabling auditing if enabled

If possible run in noarchivelog mode

Disabling flashback DB

Removing TDE from volume tables


Pre-upgrade activities

– Identify and execute tasks that could be completed in a separate downtime period, prior to the production upgrade

Use applicable steps mentioned in the "Downtime reduction" and “Upgrade By Request” appendices E and G of the R12.1 upgrade guide

Upgrade RDBMS version to latest certified for the current APPS level ( 11.2.0.2 / 11.1.0.7 / 10.2.0.5 )

Convert to Oracle Applications Tablespace Model (OATM)

– Compacts data, optimizes storage settings and I/O

Other planned HW or OS upgrades



Pre-upgrade activities continued...

– Convert to Multiple Organization architecture (Document 210193.1)

– Drop MRC schema ( 11.5.10 and above )

– Assign post upgrade jobs to specialized CM queue (by

request_type, see Document 399362.1)




– Flush all the interfaces, such as Autoinvoice, Journal entry

import, order import etc..

– Review and disable all debug, logging at site, responsibility, user

level

– Gather stats with GATHER_AUTO option for all schemas close

to the start of downtime




– Pre-create new large indexes created by upgrade

constrained to indexes on existing columns

– Minimize historical data to be upgraded as per business requirements –

“Upgrade By Request”

Post-upgrade “hot-patch” of additional historical data outlined in

604893.1

– Purge old and/or transient data before upgrading

Over 260 standard purge programs in R12

Use new “Purge Portal” in OAM to administrate



Purging

– Use OAM to configure, initiate and monitor purge programs

Set the execution frequency and view program history

Programs tagged with the “Purge” program type

System Administrator >Oracle Applications Manager >Purging/Critical Activities



Key recommended fixes

– Affecting Upgrade Performance

AD-Parallel improved scalability: 8917381 (Part of 9179588 AD

CUP)

Forms/reports generation regression with 11g: 8557019

Optimizer:Dynamic Sampling on indexes 12942119 (11.2.0.2)

– General

Review 244040.1 for latest EBS recommended performance fixes



Database tier configuration

– Maximize SGA and PGA sizing

An upgrade only involves 10's of concurrent sessions;

starting rules of thumb ...

– log buffer = 30 to 100 Mb

– shared pool = 1 to 4 Gb

– pga target = 3 to 20 Gb

– buffer cache = multi Gb, be generous without causing

excessive paging or swapping

Adjust with help from AWR pool advisories



Database tier configuration

– Other upgrade specific init.ora changes

If specified, remove db_file_multiblock_read_count

– Maximize multiblock I/O sizes

Set job_queue_processes = # of CPUS

– adobjcmp.sql (Phases : plb+90 and last+63)

Set parallel_max_servers = 2 X CPUs

– Helps with large index creation, stats gathering and some large upg+

phase jobs

– Need to test with production-like DB server and I/O subsystem

– Shutdown other RAC instances



Batch size and #workers

– Batch size

10K is suitable for most installs, you can test other values from 1K up to

100K if time allows

– # Workers

Starting rule-of-thumb is between 1 and 1.5 x #COREs

– It is critical to do multiple rounds of testing, adjusting above settings to

maximize server utilization, but constrained by factors such as

Memory utilization (no swapping/ excessive paging)

CPU utilization (scale down if at 100%)

I/O response times (scale down if averages > 20 ms)



Performance testing, monitoring and additional optimizations...

– Analyze long runners via timing report, ad_task_timing analysis

– Mine ad_task_timing to identify low worker utilization due to phasing

waits and review responsible culprits

– Review targeted AWR Instance and SQL reports

awrrpt.sql and awrsqrpt.sql

– Use My Oracle Support to check for known issues and workarounds

for the longest running jobs



Performance testing, monitoring and additional optimizations

continued...

– Outside-the-box optimizations – Requires Support Approval*

Long running index creation and Statistics gathering

– As these tasks run with little or no concurrent tasks, consider using alter system commands to increase parallel_max_servers and pga_aggregate_target

(Need to test so as to not exhaust machine resources)

– Large indexes: Pre-create, but do not alter column definitions

– For stats (adsstats.sql ; phase: last+63) : Consider stats import or on RAC using all nodes when gathering stat



Performance testing, monitoring and additional optimizations

continued...

– Outside-the-box optimizations – Requires Support Approval*

Long running mv related xdf’s (“en” phases)

– Cleanup/truncate large MV logs (requires MV complete refresh)

Review long running jobs that you think are NA for your system

– Thousands of jobs and customer product/data/setup configurations

– Some heavy lifting work may not be required for a specific site

– Check with Oracle Support to validate

– A fix maybe available or possible to optimize for your case



Customer Upgrade Snapshots


• Toyota Motor Europe – Release: 11.5.9+ on IBM AIX to R12.1.3 on Oracle Linux 5 Appl tier

– DB size: 800 GB

– #Workers: 32

– #CPUs on DB server: 8 cores

– Downtime reduction measures

• Online NLS patch application

– #hrs for the 12.1.1 D driver: 21 hrs

– #hrs for the 12.1.3 US upgrade: 4 hrs

– #hrs for the 4 languages NLS patching 11 hrs

Customer Upgrade Snapshots Continued...

11i to 12.1.3


Customer Upgrade Snapshots 11i to 12.1

CPS (Chicago Public Schools)

– Release: 11.5.10.2 to 12.1

– DB size: 900GB

– #Workers and batch size: 32, 10000

– #CPUs on DB server: 2 node RAC, 8 CPUs per node


Distributed AD

Upgrade RDBMS to 10.2.0.4 in a separate downtime

# hrs for the D driver: 22 hrs

– Customer snapshot http://www.oracle.com/customers/snapshots/chicago-public-schools-ebs-snapshot.pdf


Customer Upgrade Snapshots Continued... 11i to 12.0.6

Cisco

– Release: 11i to R12.0.6

– DB size: 600GB


– #CPUs on DB server: 16


Distributed AD

– #hrs for the D driver: 5.5 hrs


Customer Upgrade Snapshots Continued... 11i to 12.1.3

Dell

– Release: 11i10 to R12.1.3

– DB size: 15TB , 16 node RAC Cluster




Distributed AD

Pre-create large indexes

– #hrs for the D driver: ~30 hrs


Customer Upgrade Snapshots Continued... 11i to 12.1

GE

– Large EBS HRMS Implementation (~300K employees,10 Lang)

– Release:11.5.10.2 to R12.1.3, DB size: 838 G

– Hardware: App Tier- 2 SUN T5240’s(64x64),

DB Tier - SUN M8000 (12 Dual Cores)

– #Workers and batch size per App Server: 48, 10000


Distributed AD, Staged APPL_TOP

– #hrs for D driver:~10 hrs US, ~13 hrs NLS (11.5.10.2 -> R12.1.1)

– #hrs for DB Portion:~2.5 hrs US, ~1 hr NLS (R12.1.1 -> R12.1.3)


Customer Upgrade Snapshots Continued... 12.0 to 12.1

Zebra Technologies Corporation

– Release: 12.0.6 to 12.1

– DB Size: 106GB




Staged APPL_TOP

– #hrs for the U driver: 12 hrs

– Customer snapshot http://www.oracle.com/customers/snapshots/zebra-technologies-corporation-ebs-snapshot.pdf


Oracle GSI

– Release: 12.0.3+ to R12.1

– DB size: 17TB


– #CPUs on DB server: 88 processors


Staged APPL_TOP for US and ten languages

Ran data fixes for problems found in test upgrades prior to production upgrade to minimize stoppages

Distributed AD (4 servers,15 workers each)

Pre-create large indexes (6 hrs savings)

– #hrs for the D driver: 14 hrs


12.0 to 12.1


Oracle GSI

– Release: 12.1+ to R12.1.3

– DB size: 17TB


– #CPUs on DB server: 150 processors



Ran data fixes for problems found in test upgrades prior to production upgrade to minimize stoppages

Distributed AD (4 servers, 50 workers each)

Pre-create large indexes



12.1 to 12.1.3


AT&T

– Release: 12.0+ to R12.1.2

– DB size: 10 TB


– #CPUs on DB server: 32 Processors



Distributed AD



12.0 to 12.1.2


References


References R12.1 documentation roadmap (790942.1)

Oracle E-Business Suite Release 12.1 Info center (806593.1)

Database preparation guidelines for R12.1 upgrade (761570.1)

Patching FAQs (459156.1, 225165.1)

Staged or shared APPL_TOP and distributed AD (734025.1, 384248.1, 236469.1)

OAM “Patch Wizard” overview and FAQ (976188.1, 976688.1)

AD Command Line Options for Release R12 (1078973.1)

Recommended Performance Fixes (244040.1)

R12 Upgrade Sizing & Best Practices (399362.1)


Additional Resources


Additional Resources EBS R12.1 Upgrade related Reports

– EBS 12.1.3 Data Model Comparison Report (1290886.1)

– EBS ATG Seed Data Comparison Report (1327399.1)

– EBS File Comparison Report (1446430.1)

– EBS pre-install patches Report (1448102.1)

White paper

– Planning Your Oracle E-Business Suite Upgrade from Release 11i to

Release 12.1 (987516.1)

– R12 Upgrade considerations by product: Financials (889733.1)

– Oracle E-Business Suite Upgrades and Platform Migration

(1377213.1)

Have Upgrade questions ? Please post on OTN R12 upgrade forum

http://forums.oracle.com/forums/forum.jspa?forumID=395&start=0

http://forums.oracle.com/forums/forum.jspa?forumID=395&start=0


Oracle Demos

Moscone West

End-to-End Management of Oracle

E-Business Suite W-063

SOA-Based Integration for Oracle


New User Productivity Capabilities in Oracle


Oracle E-Business Suite 12.1 Technical

Upgrade Best Practices W-066

Advanced Architecture and Technology Stack

for Oracle E-Business Suite W-067


Related Sessions

Wednesday

10:15 am

CON8809 - Oracle E-Business Suite 12.1 Upgrade Best

Practices: Technical Insight

Isam Alyousfi, Udayan Parvate

Moscone West

3011

10:15 am

CON9032 - Upgrading Your Customizations of Oracle

E-Business Suite 12.1

Sara Woodhull

Moscone West

2016

10:15 am

CON9259 - Oracle E-Business Suite Internationalization and

Multilingual Features

Maher Al-Nubani

Moscone West

2018

11:45 am

CON9029 - Oracle E-Business Suite Directions:

Slashing Downtimes with Online Patching

Kevin Hudson

Moscone West

2016


Related Sessions

Wednesday

1:15 pm

CON9027 - Personalize and Extend Oracle E-Business Suite

Applications with Rich Mashups

Gustavo Jimenez, Padmaprabodh Ambale

Moscone West

2016

3:30 pm

CON9036 - Advanced Oracle E-Business Suite Architectures:

Maximum Availability, Security, and More

Elke Phelps

Moscone West

2016

5:00 pm

CON9047 - Efficiently Scaling Oracle E-Business Suite on

Oracle Exadata and Oracle Exalogic

Isam Alyousfi, Nishit Rao

Moscone West

2016


Related Sessions

Thursday

11:15 am

CON8716 - Web Services and SOA Integration Options for

Oracle E-Business Suite

Rekha Ayothi, Veshaal Singh

Moscone West

2016

11:15 am

CON9030 - Recommendations for Oracle E-Business Suite

Performance Tuning

Isam Alyousfi, Samer Barakat

Moscone West

2018

11:15 am

CON3429 - Using Oracle ADF with Oracle E-Business Suite:

The Full Integration View

Siva Puthurkattil, Juan Camilo Ruiz, Sara Woodhull

Moscone West

3003

12:45 pm

CON9020 - Integrating Oracle E-Business Suite with Oracle

Identity Management Solutions

Sunil Ghosh, Elke Phelps

Moscone West

2016


Related Sessions

Thursday

2:15 pm

CON9019 - Troubleshooting, Diagnosing, and Optimizing

Oracle E-Business Suite Technology

Gustavo Jimenez

Moscone West

2016

2:15 pm

CON9031 - The Top 10 Things You Can Do to Secure Your

Oracle E-Business Suite Instance

Eric Bing, Erik Graversen

Moscone West

2018


Q&A

1 copyright © 2012, oracle and/or its affiliates. all ... · title: powerpoint presentation...

Documents