redgatewebinar - where did my cpu go?
DESCRIPTION
TRANSCRIPT
![Page 1: RedGateWebinar - Where did my CPU go?](https://reader034.vdocuments.mx/reader034/viewer/2022042515/54820c4bb47959f10c8b46e9/html5/thumbnails/1.jpg)
“Where did my CPU go?”“Where did my CPU go?”
Presented by:
Karl Arao1
![Page 2: RedGateWebinar - Where did my CPU go?](https://reader034.vdocuments.mx/reader034/viewer/2022042515/54820c4bb47959f10c8b46e9/html5/thumbnails/2.jpg)
www.enkitec.com 2
![Page 3: RedGateWebinar - Where did my CPU go?](https://reader034.vdocuments.mx/reader034/viewer/2022042515/54820c4bb47959f10c8b46e9/html5/thumbnails/3.jpg)
www.enkitec.com 3
![Page 4: RedGateWebinar - Where did my CPU go?](https://reader034.vdocuments.mx/reader034/viewer/2022042515/54820c4bb47959f10c8b46e9/html5/thumbnails/4.jpg)
whoami
Karl Arao
• Senior Technical Consultant @ Enkitec
• Performance and Capacity Planning Enthusiast
7+ years DBA experience
Oracle ACE, OCP-DBA, RHCE, OakTable
Blog: karlarao.wordpress.com
Wiki: karlarao.tiddlyspot.com
Twitter: @karlarao
www.enkitec.com 4
![Page 5: RedGateWebinar - Where did my CPU go?](https://reader034.vdocuments.mx/reader034/viewer/2022042515/54820c4bb47959f10c8b46e9/html5/thumbnails/5.jpg)
www.enkitec.com 5
200+
3
![Page 6: RedGateWebinar - Where did my CPU go?](https://reader034.vdocuments.mx/reader034/viewer/2022042515/54820c4bb47959f10c8b46e9/html5/thumbnails/6.jpg)
Agenda
• HOWTO compare CPU speeds
• Cores vs Threads
• The different CPU events
• CPU Monitoring/Capacity Planning on consolidated environments
www.enkitec.com 6
![Page 7: RedGateWebinar - Where did my CPU go?](https://reader034.vdocuments.mx/reader034/viewer/2022042515/54820c4bb47959f10c8b46e9/html5/thumbnails/7.jpg)
12:27:15 SYS@DEMO1> show parameter cpu_count
NAME TYPE VALUE------------------------------------ ----------- --------
www.enkitec.com 7
------------------------------------ ----------- --------cpu_count integer 16
![Page 8: RedGateWebinar - Where did my CPU go?](https://reader034.vdocuments.mx/reader034/viewer/2022042515/54820c4bb47959f10c8b46e9/html5/thumbnails/8.jpg)
Socket0
Core0
CPU0 CPU8
Core1
CPU1 CPU9
Core2
CPU2 CPU10
Core3
CPU3 CPU11
CPU4 CPU12 CPU5 CPU13 CPU6 CPU14 CPU7 CPU15
Exadata V2 => 2s8c16t
www.enkitec.com 8
Socket1
Core0 Core1 Core2 Core3
![Page 9: RedGateWebinar - Where did my CPU go?](https://reader034.vdocuments.mx/reader034/viewer/2022042515/54820c4bb47959f10c8b46e9/html5/thumbnails/9.jpg)
PART1: compare CPU speeds
www.enkitec.com 9
![Page 10: RedGateWebinar - Where did my CPU go?](https://reader034.vdocuments.mx/reader034/viewer/2022042515/54820c4bb47959f10c8b46e9/html5/thumbnails/10.jpg)
Different methods:
• Published benchmarks
– TPC-C
– SPECint_rate2006
– SPECpower– SPECpower
• Actual Benchmarking
– cputoolkit
– SLOB (lio test)
www.enkitec.com 10
![Page 11: RedGateWebinar - Where did my CPU go?](https://reader034.vdocuments.mx/reader034/viewer/2022042515/54820c4bb47959f10c8b46e9/html5/thumbnails/11.jpg)
TPC-C• Transaction Processing Performance Council (TPC)
• Throughput => transactions per minute (tpmC)
• Price/Performance => USD / tpmC
www.enkitec.com 11
• CPU performance => tpmC / core
• 1609186.39 / 16 = 100574
![Page 12: RedGateWebinar - Where did my CPU go?](https://reader034.vdocuments.mx/reader034/viewer/2022042515/54820c4bb47959f10c8b46e9/html5/thumbnails/12.jpg)
SPECint_rate2006• Standard Performance Evaluation Corporation (SPEC)
• SPECint_rate2006
• Integer performance
• All CPUs are used
• Used by OEM12c Consolidation Planner (SYSMAN.EMCT_SPEC_RATE_LIB)
• CPU performance => SPECint_rate2006/core
• 702/16 = 43.875
www.enkitec.com 12
$ cat spec.txt | grep -i sun | grep -i x3-2 | sort -rnk144.0625, 16, 2, 8, 2, 632, 705, Oracle Corporation, Sun Blade X3-2B (Intel Xeon E5-2690 2.9GHz)44.0625, 16, 2, 8, 2, 630, 705, Oracle Corporation, Sun Server X3-2L (Intel Xeon E5-2690 2.9GHz)43.875, 16, 2, 8, 2, 628, 702, Oracle Corporation, Sun Server X3-2 (Intel Xeon E5-2690 2.9GHz)
![Page 13: RedGateWebinar - Where did my CPU go?](https://reader034.vdocuments.mx/reader034/viewer/2022042515/54820c4bb47959f10c8b46e9/html5/thumbnails/13.jpg)
2007 vs 2012tpmC/core, System, tpmC, Price/Perf, Total System Cost, Currency, Database Software, Server CPU Type, Total Server Cores, Cluster, Date Submitted
Result/# Cores, # Cores, # Chips, # Cores Per Chip, # Threads Per Core, Baseline, Result, Hardware Vendor, System
www.enkitec.com 13
![Page 14: RedGateWebinar - Where did my CPU go?](https://reader034.vdocuments.mx/reader034/viewer/2022042515/54820c4bb47959f10c8b46e9/html5/thumbnails/14.jpg)
Actual Benchmarking• cputoolkit and SLOB (lio test)
• LIOs/sec
CPU1
CPU2
CPU1
CPU2
cputoolkit./runcputoolkit-auto <start CPU> <end CPU> <db name>
./runcputoolkit-auto 1 2 dw
SLOB./runit.sh <writers> <readers>
while :; do ./runit.sh 0 2; done
www.enkitec.com 14
CPU2
CPU3
CPU4
CPU5
CPU6
CPU7
CPU8
CPU2
CPU3
CPU4
CPU5
CPU6
CPU7
CPU8
Both at 25%
CPU utilization
![Page 15: RedGateWebinar - Where did my CPU go?](https://reader034.vdocuments.mx/reader034/viewer/2022042515/54820c4bb47959f10c8b46e9/html5/thumbnails/15.jpg)
V2 and X2 CPU perf comparison
3.6M LIOs/sec
2.1M LIOs/sec
www.enkitec.com 15
![Page 16: RedGateWebinar - Where did my CPU go?](https://reader034.vdocuments.mx/reader034/viewer/2022042515/54820c4bb47959f10c8b46e9/html5/thumbnails/16.jpg)
V2 -> X2 migration
V2 X2
www.enkitec.com 16
chip efficiency factor = (source LIOs/sec) / (destination LIOs/sec)
= 2.1M / 3.6M
= .5833
X2 CPU requirement = source host CPUs * utilization * chip efficiency factor
= 16 * .46
= 7.36 * .5833
= 4.29 CPUs
X2 CPU Utilization = CPU requirement / CPU capacity
= 4.29 / 24
= 17.8 %
![Page 17: RedGateWebinar - Where did my CPU go?](https://reader034.vdocuments.mx/reader034/viewer/2022042515/54820c4bb47959f10c8b46e9/html5/thumbnails/17.jpg)
v2, x3, x3
www.enkitec.com 17
![Page 18: RedGateWebinar - Where did my CPU go?](https://reader034.vdocuments.mx/reader034/viewer/2022042515/54820c4bb47959f10c8b46e9/html5/thumbnails/18.jpg)
PART2: Cores vs Threads
Socket0
Core0 Core1 Core2 Core3
PART2: Cores vs Threads
www.enkitec.com 18
Socket0
Core0
CPU1 CPU5
Core1
CPU2 CPU6
Core2
CPU3 CPU7
Core3
CPU4 CPU8
![Page 19: RedGateWebinar - Where did my CPU go?](https://reader034.vdocuments.mx/reader034/viewer/2022042515/54820c4bb47959f10c8b46e9/html5/thumbnails/19.jpg)
www.enkitec.com 19
![Page 20: RedGateWebinar - Where did my CPU go?](https://reader034.vdocuments.mx/reader034/viewer/2022042515/54820c4bb47959f10c8b46e9/html5/thumbnails/20.jpg)
~30% depends on the workload
cputoolkit SLOB
www.enkitec.com 20
17% 21%
Intel HT Technology Technical User's Guide http://goo.gl/3Ec5Z
![Page 21: RedGateWebinar - Where did my CPU go?](https://reader034.vdocuments.mx/reader034/viewer/2022042515/54820c4bb47959f10c8b46e9/html5/thumbnails/21.jpg)
PART3: Different CPU events
CPU
CPU WaitCPU Wait
CPU Scheduler
www.enkitec.com 21
![Page 22: RedGateWebinar - Where did my CPU go?](https://reader034.vdocuments.mx/reader034/viewer/2022042515/54820c4bb47959f10c8b46e9/html5/thumbnails/22.jpg)
AAS CPU
www.enkitec.com 22
![Page 23: RedGateWebinar - Where did my CPU go?](https://reader034.vdocuments.mx/reader034/viewer/2022042515/54820c4bb47959f10c8b46e9/html5/thumbnails/23.jpg)
www.enkitec.com 23
![Page 24: RedGateWebinar - Where did my CPU go?](https://reader034.vdocuments.mx/reader034/viewer/2022042515/54820c4bb47959f10c8b46e9/html5/thumbnails/24.jpg)
CPU Wait
www.enkitec.com 24
![Page 25: RedGateWebinar - Where did my CPU go?](https://reader034.vdocuments.mx/reader034/viewer/2022042515/54820c4bb47959f10c8b46e9/html5/thumbnails/25.jpg)
www.enkitec.com 25
![Page 26: RedGateWebinar - Where did my CPU go?](https://reader034.vdocuments.mx/reader034/viewer/2022042515/54820c4bb47959f10c8b46e9/html5/thumbnails/26.jpg)
CPU Scheduler
www.enkitec.com 26
![Page 27: RedGateWebinar - Where did my CPU go?](https://reader034.vdocuments.mx/reader034/viewer/2022042515/54820c4bb47959f10c8b46e9/html5/thumbnails/27.jpg)
www.enkitec.com 27
![Page 28: RedGateWebinar - Where did my CPU go?](https://reader034.vdocuments.mx/reader034/viewer/2022042515/54820c4bb47959f10c8b46e9/html5/thumbnails/28.jpg)
Putting it all together
www.enkitec.com 28
Instances Caged
at 12 CPUs
SQL Applied to lock
in good plan.
Problem: A single SQL Stmt. overwhelming
CPU resources.
![Page 29: RedGateWebinar - Where did my CPU go?](https://reader034.vdocuments.mx/reader034/viewer/2022042515/54820c4bb47959f10c8b46e9/html5/thumbnails/29.jpg)
PART4: CPU monitoring and
Capacity PlanningCapacity Planning
www.enkitec.com 29
![Page 30: RedGateWebinar - Where did my CPU go?](https://reader034.vdocuments.mx/reader034/viewer/2022042515/54820c4bb47959f10c8b46e9/html5/thumbnails/30.jpg)
OS Tools• The usual Operating System commands
– vmstat
– top
– mpstat –P ALL 1 5
• Cool tools
– collectl –sC (http://collectl.sourceforge.net)– collectl –sC (http://collectl.sourceforge.net)
– turbostat.c
– dcli (Exadata)
• dcli -l oracle -g /home/oracle/dbs_group --vmstat 2
• dcli -l oracle -g /home/oracle/dbs_group uptime
www.enkitec.com 30
![Page 31: RedGateWebinar - Where did my CPU go?](https://reader034.vdocuments.mx/reader034/viewer/2022042515/54820c4bb47959f10c8b46e9/html5/thumbnails/31.jpg)
Load Map
www.enkitec.com 31
![Page 32: RedGateWebinar - Where did my CPU go?](https://reader034.vdocuments.mx/reader034/viewer/2022042515/54820c4bb47959f10c8b46e9/html5/thumbnails/32.jpg)
Performance Page – Historical View
www.enkitec.com 32
![Page 33: RedGateWebinar - Where did my CPU go?](https://reader034.vdocuments.mx/reader034/viewer/2022042515/54820c4bb47959f10c8b46e9/html5/thumbnails/33.jpg)
AWR Toolkit
• DIY performance data warehouse
run_awr
run_extract
Extract AWR data points as csv files
Package all the csv filesCustomer site
FRESH_LOAD
CHECK_LOAD
DELTA_LOAD
Create new client ““““dimension”””” tables
Check new data points
Load new data points
DIY DW server
1
2
www.enkitec.com 33
DELTA_LOAD Load new data points
awr_topevents_(ClientNameX)
awr_cpuwl_(ClientNameX)
awr_iowl_(ClientNameX)
3 Tableau Analytics
awr_topevents_(ClientNameY)
awr_cpuwl_(ClientNameY)
awr_iowl_(ClientNameY)
awr_topevents_(ClientNameZ)
awr_cpuwl_(ClientNameZ)
awr_iowl_(ClientNameZ)
![Page 34: RedGateWebinar - Where did my CPU go?](https://reader034.vdocuments.mx/reader034/viewer/2022042515/54820c4bb47959f10c8b46e9/html5/thumbnails/34.jpg)
AWR data• Top Events
– AAS CPU, latency, wait class
• SYSSTAT
– PGA, SGA, physical memory, Executes/sec
• IO
– IOPS breakdown, MB/s
• CPU
– Load Average, NUM_CPUs, – Load Average, NUM_CPUs,
• Storage
– total storage size, per tablespace size
• Services
– distribution of workload/modules
• Top SQL
– PIOs, LIOs, modules, SQL type, SQL_ID, PX
Correlate across months of workload data! http://goo.gl/7uCk7w
![Page 35: RedGateWebinar - Where did my CPU go?](https://reader034.vdocuments.mx/reader034/viewer/2022042515/54820c4bb47959f10c8b46e9/html5/thumbnails/35.jpg)
www.enkitec.com 35
![Page 36: RedGateWebinar - Where did my CPU go?](https://reader034.vdocuments.mx/reader034/viewer/2022042515/54820c4bb47959f10c8b46e9/html5/thumbnails/36.jpg)
• Tableau auto creates a time dimension for the time
column “MM/DD/YY HH24:MI:SS” of AWR csv output
www.enkitec.com 36
![Page 37: RedGateWebinar - Where did my CPU go?](https://reader034.vdocuments.mx/reader034/viewer/2022042515/54820c4bb47959f10c8b46e9/html5/thumbnails/37.jpg)
• Summary and Underlying data
www.enkitec.com 37
1-2AM
2-3AM
![Page 38: RedGateWebinar - Where did my CPU go?](https://reader034.vdocuments.mx/reader034/viewer/2022042515/54820c4bb47959f10c8b46e9/html5/thumbnails/38.jpg)
CPU usage across half rack Exadata
www.enkitec.com 38
![Page 39: RedGateWebinar - Where did my CPU go?](https://reader034.vdocuments.mx/reader034/viewer/2022042515/54820c4bb47959f10c8b46e9/html5/thumbnails/39.jpg)
CPU usage per host
www.enkitec.com 39
![Page 40: RedGateWebinar - Where did my CPU go?](https://reader034.vdocuments.mx/reader034/viewer/2022042515/54820c4bb47959f10c8b46e9/html5/thumbnails/40.jpg)
CPU redistribution across nodes
www.enkitec.com 40
![Page 41: RedGateWebinar - Where did my CPU go?](https://reader034.vdocuments.mx/reader034/viewer/2022042515/54820c4bb47959f10c8b46e9/html5/thumbnails/41.jpg)
Wrap up!• HOWTO compare CPU speeds
o SPECint_rate2006, TPC-C, Actual benchmarking
• Cores vs Threads
o Always have HT on
o ~30% performance benefit after core count
• The different CPU events
o 1 AAS CPU = 1 CPU thread
o Oracle CPU may not correlate with Host CPU if you have a lot ofCPU activity outside of the database
• CPU Monitoring/Capacity Planning on consolidated environments
o AWR analytics
www.enkitec.com 41
![Page 42: RedGateWebinar - Where did my CPU go?](https://reader034.vdocuments.mx/reader034/viewer/2022042515/54820c4bb47959f10c8b46e9/html5/thumbnails/42.jpg)
Resources• White paper: http://goo.gl/eq9Sn
• cputoolkit - http://karlarao.wordpress.com/scripts-resources/
• AWR Tableau and R toolkit Visualization Examples - http://goo.gl/xZHHY
• AAS investigation - http://goo.gl/5WaAg
• Cores vs Threads - http://goo.gl/1MLFf
• Turbostat.c - http://goo.gl/jDUKg
• cpu_topology - http://goo.gl/EUDG7
• CPU centric benchmark comparisons - http://goo.gl/nR9Yy
• SLOB - http://goo.gl/yKa45
• Kyle Hailey - http://dboptimizer.com/2011/07/21/oracle-cpu-time/
• Book: Computer Architecture: A Quantitative Approach 5th Ed - Chapter1 Section1.10 Putting it all together Perf, Price, Power http://goo.gl/MXigAQ
• Book: The Art of Scalability - Ch11 “Headroom” http://theartofscalability.com
• The mindmap of this presentation - http://goo.gl/XeY0e
www.enkitec.com 42
![Page 43: RedGateWebinar - Where did my CPU go?](https://reader034.vdocuments.mx/reader034/viewer/2022042515/54820c4bb47959f10c8b46e9/html5/thumbnails/43.jpg)
43
Fastest Growing Companies
in Dallas