performance testing and obiee
DESCRIPTION
Performance Testing is a crucial part of any system development process, but one which is not widely documented for OBIEE. The multi-tier architecture and web client end-user interface also makes it more complex to correctly build and analyse a performance test. This technical presentation discusses approaches to performance testing OBIEE including scripting with OBIEE utilities, and Load Runner. It suggests a methodology to follow, and describes pitfalls to avoid. More information on my blog at rnm1978.wordpress.comTRANSCRIPT
![Page 1: Performance Testing and OBIEE](https://reader033.vdocuments.mx/reader033/viewer/2022061223/54c332e84a79597d2f8b45d5/html5/thumbnails/1.jpg)
Performance Testing and OBIEE
Robin Moffatt, WM Morrisons plc http://rnm1978.wordpress.com
![Page 2: Performance Testing and OBIEE](https://reader033.vdocuments.mx/reader033/viewer/2022061223/54c332e84a79597d2f8b45d5/html5/thumbnails/2.jpg)
Introduction
Oracle BI specialist at Morrisons plc
Big IT development programme at its early stages implementing OBIEE, OBIA, ORDM, all on Oracle 11g & HP-UX
![Page 3: Performance Testing and OBIEE](https://reader033.vdocuments.mx/reader033/viewer/2022061223/54c332e84a79597d2f8b45d5/html5/thumbnails/3.jpg)
The aim of this presentation
A Performance Tuning Methodology
OBIEE techie stuff
Learn from my mistakes!
![Page 4: Performance Testing and OBIEE](https://reader033.vdocuments.mx/reader033/viewer/2022061223/54c332e84a79597d2f8b45d5/html5/thumbnails/4.jpg)
What is performance testing all about?
Response times Report ETL batch OLTP transaction
System impact Resource usage Scalability
![Page 5: Performance Testing and OBIEE](https://reader033.vdocuments.mx/reader033/viewer/2022061223/54c332e84a79597d2f8b45d5/html5/thumbnails/5.jpg)
Why performance test?(Isn’t testing just for wimps?)
Check that your system performs Are the users going to be happy?
Baseline How fast is fast? ▪ How slow is slow?
Validate system design Do it right, first time
Capacity planning
![Page 6: Performance Testing and OBIEE](https://reader033.vdocuments.mx/reader033/viewer/2022061223/54c332e84a79597d2f8b45d5/html5/thumbnails/6.jpg)
Why performance test?
It’s never too late “You’ll never catch all your
problems in pre-production testing. That’s why you need a reliable and efficient method for solving the problems that leak through your pre-production testing processes.”
— Cary Millsap - Thinking Clearly About Performance
![Page 7: Performance Testing and OBIEE](https://reader033.vdocuments.mx/reader033/viewer/2022061223/54c332e84a79597d2f8b45d5/html5/thumbnails/7.jpg)
Why performance test?
Because it makes you better at your job
“At the very least, your performance test plan will make you a more competent diagnostician (and clearer thinker) when it comes time to fix the performance problems that will inevitably occur during production operation.”
— Cary Millsap - Thinking Clearly About Performance
![Page 8: Performance Testing and OBIEE](https://reader033.vdocuments.mx/reader033/viewer/2022061223/54c332e84a79597d2f8b45d5/html5/thumbnails/8.jpg)
Performance Testing – What & Why
Quantifying response times System impact
User expectations Problem diagnosis Design validation
![Page 9: Performance Testing and OBIEE](https://reader033.vdocuments.mx/reader033/viewer/2022061223/54c332e84a79597d2f8b45d5/html5/thumbnails/9.jpg)
Define
Measure
Analyse
Review
Implement
Timebox!
Performance Testing - How?
Evaluate design / config options
Do it rightDon’t “fudge it”
Do more testing
Iterative
approach
Be Methodical
Redefine test Do more
testing
![Page 10: Performance Testing and OBIEE](https://reader033.vdocuments.mx/reader033/viewer/2022061223/54c332e84a79597d2f8b45d5/html5/thumbnails/10.jpg)
Define & build your test
Define – what are you going to test• Aim of the test• Scope• Assumptions• Specifics• Data, environment, etc
Build – how are you going to test it OBIEE specific
Define
Measure
Analyse
Review
Implement
•E.g. :• Check that the system
performs• Baseline performance• Prove system capacity• Validate system design
![Page 11: Performance Testing and OBIEE](https://reader033.vdocuments.mx/reader033/viewer/2022061223/54c332e84a79597d2f8b45d5/html5/thumbnails/11.jpg)
Consider your test scopeMore components = more complex = more variables =
larger margin of errorFewer components = easier to manage = more precise =
more efficient
Define
Measure
Analyse
Review
Implement
![Page 12: Performance Testing and OBIEE](https://reader033.vdocuments.mx/reader033/viewer/2022061223/54c332e84a79597d2f8b45d5/html5/thumbnails/12.jpg)
OBIEE stack
Database
Presentation Services
BI Server
Report / Dashboard
Logical SQL
Physical SQL
statement(s)
Data set(s)
Data set
Rendered report
Excludes App/Web server & presentation services plug-in
Define
Measure
Analyse
Review
Implement
![Page 13: Performance Testing and OBIEE](https://reader033.vdocuments.mx/reader033/viewer/2022061223/54c332e84a79597d2f8b45d5/html5/thumbnails/13.jpg)
OBIEE testing options
Database
Presentation Services
BI Server
nqcmd
SQL Client
LSQL
Physical
SQL
Data set(s)
Data set
Rendered
report
LSQL
PhysicalSQL
User & Stopwatc
h
Load Testing tool(eg.
LoadRunner, OATS)
Define
Measure
Analyse
Review
Implement
Report /
Dashboard
![Page 14: Performance Testing and OBIEE](https://reader033.vdocuments.mx/reader033/viewer/2022061223/54c332e84a79597d2f8b45d5/html5/thumbnails/14.jpg)
OBIEE testing options
Database
BI Server
nqcmd
Physical
SQL
Data set(s)
LSQL
Define
Measure
Analyse
Review
Implement
![Page 15: Performance Testing and OBIEE](https://reader033.vdocuments.mx/reader033/viewer/2022061223/54c332e84a79597d2f8b45d5/html5/thumbnails/15.jpg)
nqcmd
Define
Measure
Analyse
Review
Implement
Usage Tracking or
NQQuery.log
Test
sc
ript
BI Serve
r
Data
nqcmd
Logical SQL
Logical SQL Logical
SQL
![Page 16: Performance Testing and OBIEE](https://reader033.vdocuments.mx/reader033/viewer/2022061223/54c332e84a79597d2f8b45d5/html5/thumbnails/16.jpg)
Mast
er
test
scr
ipt
nqcmd
Define
Measure
Analyse
Review
Implement
Test
sc
ript
BI Serve
r
Data
nqcmd
Logical SQL
Test
sc
ript nqcmd
Test
sc
ript nqcmd
Test
sc
ript nqcmd
![Page 17: Performance Testing and OBIEE](https://reader033.vdocuments.mx/reader033/viewer/2022061223/54c332e84a79597d2f8b45d5/html5/thumbnails/17.jpg)
LoadRunnera.k.a. HP Performance Centre
Simulates user interaction – HTTP traffic
Powerful, but can be difficult to set up Ajax complicates things
Do you really need to use it?
Tools Fiddler2 FireBug
Reference: My Oracle Support – Doc ID 496417.1 http://rnm1978.wordpress.com/category/loadrunner
Define
Measure
Analyse
Review
Implement
![Page 18: Performance Testing and OBIEE](https://reader033.vdocuments.mx/reader033/viewer/2022061223/54c332e84a79597d2f8b45d5/html5/thumbnails/18.jpg)
Defining your test - summary
Be very clear what the aim of your test is
You probably need to define multiple tests
Different points on the OBIEE stack to interface Pick the most appropriate one
Write everything down!
Define
Measure
Analyse
Review
Implement
![Page 19: Performance Testing and OBIEE](https://reader033.vdocuments.mx/reader033/viewer/2022061223/54c332e84a79597d2f8b45d5/html5/thumbnails/19.jpg)
Define
Measure
Analyse
Review
Implement
![Page 20: Performance Testing and OBIEE](https://reader033.vdocuments.mx/reader033/viewer/2022061223/54c332e84a79597d2f8b45d5/html5/thumbnails/20.jpg)
OBIEE measuring & monitoring
Database
Presentation Services
BI Server
Presentation Services plug-
in
App Server
Web Server
Apache log
OAS log
Analytics log
sawserver.log
NQServer.log
NQQuery.log
systems management
Enterprise Manager
BI Management Pack
Usage Trackin
g
PerfMon(windows
only)
jConsole etc
Presentation
services
Enterprise Manager
ASH, AWR, SQL
Monitor
Define
Measure
Analyse
Review
Implement
Server metricse.g. : IO, CPU,
Memory
PerfMon (Windows)
Oracle OS Watcher (unix)
Enterprise Manager (Oracle)
![Page 21: Performance Testing and OBIEE](https://reader033.vdocuments.mx/reader033/viewer/2022061223/54c332e84a79597d2f8b45d5/html5/thumbnails/21.jpg)
Oracle SQL Monitor
Define
Measure
Analyse
Review
Implement
![Page 22: Performance Testing and OBIEE](https://reader033.vdocuments.mx/reader033/viewer/2022061223/54c332e84a79597d2f8b45d5/html5/thumbnails/22.jpg)
Measure - summary
Lots of different ways to measure
Build measurement into your test plan Automate where possible▪ Easier▪ Less error
Define
Measure
Analyse
Review
Implement
![Page 23: Performance Testing and OBIEE](https://reader033.vdocuments.mx/reader033/viewer/2022061223/54c332e84a79597d2f8b45d5/html5/thumbnails/23.jpg)
Define
Measure
Analyse
Review
Implement
![Page 24: Performance Testing and OBIEE](https://reader033.vdocuments.mx/reader033/viewer/2022061223/54c332e84a79597d2f8b45d5/html5/thumbnails/24.jpg)
Analysing the data
Define
Measure
Analyse
Review
Implement
![Page 25: Performance Testing and OBIEE](https://reader033.vdocuments.mx/reader033/viewer/2022061223/54c332e84a79597d2f8b45d5/html5/thumbnails/25.jpg)
Analysing the data
Define
Measure
Analyse
Review
Implement
![Page 26: Performance Testing and OBIEE](https://reader033.vdocuments.mx/reader033/viewer/2022061223/54c332e84a79597d2f8b45d5/html5/thumbnails/26.jpg)
Analysing the data
Response time
1
1
9
3
2
10
2
1
2
3
Response time
1
1
1
2
2
2
3
3
9
10
Average (mean)
3.4
50th percentile (Median)
290th percentile
9.1
Define
Measure
Analyse
Review
Implement
![Page 27: Performance Testing and OBIEE](https://reader033.vdocuments.mx/reader033/viewer/2022061223/54c332e84a79597d2f8b45d5/html5/thumbnails/27.jpg)
Recording data about the test
Define
Measure
Analyse
Review
Implement
Dashboard
Requests
•ORA_HASH(QUERY_TEXT)
Logical SQL
•SQL IDs
Physical SQL
•Execution plan hash id
Execution plan
![Page 28: Performance Testing and OBIEE](https://reader033.vdocuments.mx/reader033/viewer/2022061223/54c332e84a79597d2f8b45d5/html5/thumbnails/28.jpg)
Extending Usage Tracking
S_NQ_ACCTSTART_TS ROW_COUNT TOTAL_TIME_SEC NUM_DB_QUERY QUERY_TEXT QUERY_SRC_CD SAW_SRC_PATH SAW_DASHBOARD
OBIEE_REPLAY_STATEMENTSqt_ora_hashquery_textsaw_pathdashboard
OBIEE_REPLAY_STATStestidtestenvqt_ora_hashstart_ts response_time row_count db_query_cnt
Define
Measure
Analyse
Review
Implement
![Page 29: Performance Testing and OBIEE](https://reader033.vdocuments.mx/reader033/viewer/2022061223/54c332e84a79597d2f8b45d5/html5/thumbnails/29.jpg)
Define
Measure
Analyse
Review
Implement
Analyse
Evaluate design / config options
Do it rightDon’t “fudge it”
Iterative
approach
Timebox!
![Page 30: Performance Testing and OBIEE](https://reader033.vdocuments.mx/reader033/viewer/2022061223/54c332e84a79597d2f8b45d5/html5/thumbnails/30.jpg)
Define
Measure
Analyse
Review
Implement
Review
Iterative
approach
Redefine test
Continue testing
Implement
![Page 31: Performance Testing and OBIEE](https://reader033.vdocuments.mx/reader033/viewer/2022061223/54c332e84a79597d2f8b45d5/html5/thumbnails/31.jpg)
Review
Define
Measure
Analyse
Review
Implement
![Page 32: Performance Testing and OBIEE](https://reader033.vdocuments.mx/reader033/viewer/2022061223/54c332e84a79597d2f8b45d5/html5/thumbnails/32.jpg)
Define
Measure
Analyse
Review
Implement
Implement
Iterative
approach
![Page 33: Performance Testing and OBIEE](https://reader033.vdocuments.mx/reader033/viewer/2022061223/54c332e84a79597d2f8b45d5/html5/thumbnails/33.jpg)
Lessons Learnt
You won’t get your testing right first time There’s no shame in that Don’t cook the books ▪ Better to redefine your test than invalidate its results
Stick to the methodology Don’t move the goalposts Very tempting to pick off the “low-hanging fruit” ▪ If you do, make sure you don’t get indigestion…
Timebox
Test your implementation!
![Page 34: Performance Testing and OBIEE](https://reader033.vdocuments.mx/reader033/viewer/2022061223/54c332e84a79597d2f8b45d5/html5/thumbnails/34.jpg)
Define
Measure
Analyse
Review
Implement
Performance Testing OBIEE
Evaluate design / config options
Do it rightDon’t “fudge it”
Do more testing
Iterative
approach
Be Methodical
Redefine test Do more
testing
[email protected] · http://rnm1978.wordpress.com · http://twitter.com/rnm1978
![Page 35: Performance Testing and OBIEE](https://reader033.vdocuments.mx/reader033/viewer/2022061223/54c332e84a79597d2f8b45d5/html5/thumbnails/35.jpg)
#EOF