frits hoogland - oracle usergroup norway 2013 exadata … · exadata and oltp frits hoogland -...
TRANSCRIPT
EXADATA AND OLTPFrits Hoogland - Oracle Usergroup Norway 2013
Thursday, April 18, 13
Who am I?• Frits Hoogland
– Working with Oracle products since 1996
• Blog: http://fritshoogland.wordpress.com• Twitter: @fritshoogland• Email: [email protected]• Oracle ACE Director• OakTable Member
2
Thursday, April 18, 13
Note!!
• This presentation has been made benchmarking an older version of the Oracle appliances.
• The versions are explicitly stated.
• The message of the presentation doesn’t change!
3
Thursday, April 18, 13
What is Exadata?• Engineered system specifically for oracle database.• Ability to reach high number of read IOPS and huge
bandwidth.• Has it’s own patchbundles.• Validated versions and patch versions across database,
clusterware, o/s and storage, firmware.• Dedicated, private storage for databases.• ASM.• Recent hardware & recent CPU.• No virtualisation.
4
Thursday, April 18, 13
Exadata versions• Oracle database 64 bit version >= 11 (.2)
• ASM 64 bit version >= 11 (.2)– Exadata communication is using in skgxp code
• Linux OL5 x64– No UEK kernel used (except X2-8)‣Update! Exadata 11.2.3.2.0 uses
› 2.6.32-400.1.1.el5uek
5
Thursday, April 18, 13
Exadata hardware• Intel Xeon server hardware
• Infiniband 40Gb/s
• Oracle cell (storage) server– Flash to mimic SAN cache– High performance disks or high capacity disks‣ 600GB 15k RPM / ~ 5ms latency‣ 2/3TB 7.2k RPM / ~ 8ms latency
6
Thursday, April 18, 13
Flash• Flashcards are in every storage server
– Total of 384GB per storage server*
• Do not confuse exadata STORAGE server flashcache with oracle database flashcache!
7
Thursday, April 18, 13
Flash• Flash can be configured either as cache or as
diskgroup or both.– cache: flashlog, flashcache*
• When flash is used as diskgroup latency is ~ 1 ms– Faster than disk– My guess was < 400µs‣ 1µs infiniband‣ 200µs flash IO time‣ some time for storage server
8
Thursday, April 18, 13
Flash• Flash is restricted* to 4x96GB = 384GB per storage server.
– Totals: – Q:1152GB, H:2688GB, F:5376GB
• Net (ASM Normal redundancy): – Q: 576GB, H:1344GB, F:2688GB
• That is a very limited amount of storage.
9
Thursday, April 18, 13
Exadata specific features• The secret sauce of exadata: the storage server
– smartscan
– storage indexes
– EHCC *
– IO Resource manager
10
Thursday, April 18, 13
Exadata specific features• The secret sauce of exadata: the storage server
– smartscan
– storage indexes
– EHCC *
– IO Resource manager
10
Thursday, April 18, 13
Exadata specific features• The secret sauce of exadata: the storage server
– smartscan
– storage indexes
– EHCC *
– IO Resource manager
10
Thursday, April 18, 13
Exadata specific features• The secret sauce of exadata: the storage server
– smartscan
– storage indexes
– EHCC *
– IO Resource manager
10
Thursday, April 18, 13
OLTP• How does OLTP look like (in general | simplistic)
– Fetch small amount of data‣ Invoice numbers, client id, product id‣ select single values or small ranges via index
– Create or update rows‣ Sold items on invoice, payments, order status‣ insert or update values
11
Thursday, April 18, 13
• A great way to mimic or measure OLTP performance is
SLOB
– Silly Little Oracle Benchmark
– Author: Kevin Closson
http://oaktable.net/articles/slob-silly-little-oracle-benchmark
12
Thursday, April 18, 13
SLOBIt can do reading:FOR i IN 1..5000 LOOP v_r := dbms_random.value(257, 10000) ; SELECT COUNT(c2) into x FROM cf1 where custid > v_r - 256 AND custid < v_r;END LOOP;
13
Thursday, April 18, 13
SLOBAnd writing:FOR i IN 1..500 LOOP v_r := dbms_random.value(257, 10000) ; UPDATE cf1 set c2 = 'AAAAAAAABBBBBBBBAAAAAAAABBBBBBBBAAAAAAAABBBBBBBBAAAAAAAABBBBBBBBAAAAAAAABBBBBBBBAAAAAAAABBBBBBBBAAAAAAAABBBBBBBBAAAAAAAABBBBBBBB', ....up to column 20 (c20).... where custid > v_r - 256 AND custid < v_r; COMMIT;END LOOP;
14
Thursday, April 18, 13
SLOB• Lets run SLOB with 0 writers and 1 reader at:• Single instance database, 10G SGA, 9G cache.
>> Cold cache <<
• Exadata V2 / Oracle 11.2.0.2 / HP – Half rack / 7 storage servers / 84 disks (15k rpm)
• Exadata X2 / Oracle 11.2.0.2 / HC – Quarter rack / 3 storage servers / 36 disks (7.2k rpm)
15
Thursday, April 18, 13
SLOB - 1 reader
16
0
2.5
5.0
7.5
10.0
V2 / Half rack X2 / Quarter rack -00
5.0
Response time
Thursday, April 18, 13
SLOB - 1 reader
17
0
2.5
5.0
7.5
10.0
V2 / Half rack X2 / Quarter rack -0
4.0
5.0
Response time
Thursday, April 18, 13
SLOB - 1 reader
18
0
2.5
5.0
7.5
10.0
V2 / Half rack X2 / Quarter rack ODA0
4.0
5.0
Response time
Thursday, April 18, 13
SLOB - 1 reader
19
0
2.5
5.0
7.5
10.0
V2 / Half rack X2 / Quarter rack ODA
4.04.0
5.0
Response time
Thursday, April 18, 13
SLOB - 1 reader
20
• V2 time: 5 sec - CPU: 84.3%– PIO: 10’768 (0.8%) -- IO time 0.8 sec– LIO: 1’299’493
• X2 time: 4 sec - CPU: 75.7%– PIO: 10’922 (0.8%) -- IO time 0.9 sec– LIO: 1’300’726
• ODA tm: 4 sec - CPU: 55.2% (20 disks 15k rpm)– PIO: 10’542 (0.8%) -- IO time 2.2 sec– LIO: 1’297’502
Thursday, April 18, 13
SLOB - 1 reader - conclusion• The time spend on PIO is 15% - 45%
• Majority of time spend on LIO/CPU time
• Because main portion is CPU, fastest CPU “wins”– Actually: fastest CPU, memory bus and memory.
21
Thursday, April 18, 13
SLOB - 1 reader - LIO• Let’s do a pre-warmed cache run
– Pre-warmed means: no PIO, data already in BC
– This means ONLY LIO speed is measured
22
Thursday, April 18, 13
SLOB - 1 reader - LIO• ODA: 1 sec
• X2: 2 sec
• V2: 3 sec
23
Thursday, April 18, 13
SLOB - 1 reader - LIO• Reason: LIO essentially means
– Reading memory– CPU processing
Use ‘dmidecode’ to look to the system’s components!
24
Thursday, April 18, 13
• V2– Intel Xeon E5540 @ 2.53GHz (2s8c16t)– L1:128kB, L2:1MB, L3:8MB– Memory: Type DDR3, speed 800 Mhz
• X2:– Intel Xeon X5670 @ 2.93GHz (2s12c24t)– L1:384kB, L2:1.5MB, L3:12MB– Memory: Type DDR3, speed 1333 Mhz
• ODA– Intel Xeon X5675 @ 3.07GHz (2s12c24t)– L1:384kB, L2:1.5MB, L3:12MB– Memory: Type DDR3, speed 1333 Mhz
25
Thursday, April 18, 13
SLOB - LIO - multiple readers• Let’s take the 1 reader figures, and scale up.
• What would ODA, Exadata V2 and Exadata X2 do?
26
Thursday, April 18, 13
SLOB - LIO - multiple readers
27
0
7.5
15.0
22.5
30.0
1 10 20 30 40 50 60 70 80
LIO read response time
oda x2 v2
Thursday, April 18, 13
SLOB - LIO - multiple readers
28
0
7.5
15.0
22.5
30.0
1 10 20 30 40 50 60 70 80
LIO read response time
oda x2 v2
Thursday, April 18, 13
SLOB - LIO - multiple readers
29
0
7.5
15.0
22.5
30.0
1 10 20 30 40 50 60 70 80
LIO read response time
oda x2 v2
Thursday, April 18, 13
SLOB - LIO - multiple readers
30
0
7.5
15.0
22.5
30.0
1 10 20 30 40 50 60 70 80
LIO read response time
oda x2 v2
Thursday, April 18, 13
SLOB - LIO - multiple readers
30
0
7.5
15.0
22.5
30.0
1 10 20 30 40 50 60 70 80
LIO read response time
oda x2 v2
Core difference and slower memory shows when # readers exceeds core count.
Thursday, April 18, 13
SLOB - LIO - multiple readers
30
0
7.5
15.0
22.5
30.0
1 10 20 30 40 50 60 70 80
LIO read response time
oda x2 v2
Core difference and slower memory shows when # readers exceeds core count.
Same memory speed: CPU speed matters less with more concurrency.
Thursday, April 18, 13
SLOB - LIO - multiple readers
31
0
7.5
15.0
22.5
30.0
1 10 20 30 40 50 60 70 80
LIO read response time
oda x2 v2
Thursday, April 18, 13
SLOB - LIO - multiple readers
31
0
7.5
15.0
22.5
30.0
1 10 20 30 40 50 60 70 80
LIO read response time
oda x2 v2
Lesser core’s and slower memory make LIO processing increasingly slower with more concurrency
Thursday, April 18, 13
SLOB - LIO - multiple readers
31
0
7.5
15.0
22.5
30.0
1 10 20 30 40 50 60 70 80
LIO read response time
oda x2 v2
Lesser core’s and slower memory make LIO processing increasingly slower with more concurrency
For LIO processing ODA (non-Exadata) and Exadata does not matter.
Thursday, April 18, 13
SLOB - LIO - multiple readers
32
• Conclusion:
– LIO performance is impacted by:
‣CPU speed‣Number of sockets and core’s‣ L1/2/3 cache sizes‣Memory speed
– Exadata does not matter here!
Thursday, April 18, 13
SLOB - LIO - multiple readers• When comparing entirely different systems also
consider:
– Oracle version – O/S and version (scheduling)– Hyper threading / CPU architecture– NUMA (Exadata/ODA: no NUMA!)
33
Thursday, April 18, 13
SLOB - PIO - 1 reader• But how about physical IO?
– Lower the buffercache to 4M – sga_max_size to 1g– cpu_count to 1– db_cache_size to 1M (results in 4M)
• Slob run with 1 reader
34
Thursday, April 18, 13
SLOB - PIO - 1 reader
35
0
28
55
83
110
1 10 20 30 40 50 60 70 80
PIO reader response time
oda x2 v2
Thursday, April 18, 13
SLOB - PIO - 1 reader
35
0
28
55
83
110
1 10 20 30 40 50 60 70 80
PIO reader response time
oda x2 v2
Surprise! ODA is the fastest here with 73 seconds.
Thursday, April 18, 13
SLOB - PIO - 1 reader
35
0
28
55
83
110
1 10 20 30 40 50 60 70 80
PIO reader response time
oda x2 v2
Surprise! ODA is the fastest here with 73 seconds.
The X2 is only a little slower with 76 seconds.
Thursday, April 18, 13
SLOB - PIO - 1 reader
35
0
28
55
83
110
1 10 20 30 40 50 60 70 80
PIO reader response time
oda x2 v2
Surprise! ODA is the fastest here with 73 seconds.
The X2 is only a little slower with 76 seconds.
The V2 is the slowest with 106 seconds.
Thursday, April 18, 13
SLOB - PIO - 1 reader
36
Total time (s) CPU time (s) IO time (s)
ODA 73 17 60
X2 76 33 55
V2 106 52 52
• IO ODA:– 60/1264355= 0.047 ms
• IO X2:– 55/1265602= 0.043 ms
• IO V2:– 52/1240941= 0.042 ms
Thursday, April 18, 13
SLOB - PIO - 1 reader• This is not random disk IO!
– Average latency of random IO 15k rpm disk ~ 5ms– Average latency of random IO 7.2k rpm disk ~ 8ms
• So this must come from a cache or is not random disk IO– Exadata has flashcache.– On ODA, data probably very nearby on disk.
37
Thursday, April 18, 13
SLOB - PIO - 1 reader
• Exadata IO takes (way) more CPU. • Roughly the same time is spend on doing IO’s.
38
Total time (s) CPU time (s) IO time (s)
ODA 73 17 60
X2 76 33 55
V2 106 52 52
Thursday, April 18, 13
10 readers
39
Thursday, April 18, 13
SLOB - PIO - 10 readers
40
0
1000
2000
3000
4000
1 10 20 30 40 50 60 70 80
PIO reader response time
oda x2 v2
Thursday, April 18, 13
SLOB - PIO - 10 readers
40
0
1000
2000
3000
4000
1 10 20 30 40 50 60 70 80
PIO reader response time
oda x2 v2
Now IO responsetime on ODA is way higher than Exadata (3008s)
Thursday, April 18, 13
SLOB - PIO - 10 readers
40
0
1000
2000
3000
4000
1 10 20 30 40 50 60 70 80
PIO reader response time
oda x2 v2
Now IO responsetime on ODA is way higher than Exadata (3008s)
Both Exadata’s perform alike: X2 581s, V2 588s.
Thursday, April 18, 13
SLOB - PIO - 10 readers
41
Total time (s) CPU time (s) IO time (s)
ODA 3008 600 29428
X2 581 848 5213
V2 588 1388 4866
• IO ODA:– 29428/13879603 = 2.120 ms
• IO X2:– 5213/14045812 = 0.371 ms
• IO V2:– 4866/14170303 = 0.343 ms
Thursday, April 18, 13
20 readers
42
Thursday, April 18, 13
SLOB - PIO - 20 readers
43
0
1250
2500
3750
5000
1 10 20 30 40 50 60 70 80
PIO reader response time
oda x2 v2
Thursday, April 18, 13
SLOB - PIO - 20 readers
44
• IO ODA:– 88756/28246604 =0.003142183039 = 3.142 ms
• IO X2:– 13010/28789330=0.0004519035351 = 0.452 ms
• IO V2:– 12405/28766804=0.0004312262148 = 0.431 ms
Total time (s) CPU time (s) IO time (s)
ODA 4503 1377 88756
X2 721 2069 13010
V2 747 3373 12405
Thursday, April 18, 13
SLOB - PIO - 30 readers
45
0
1500
3000
4500
6000
1 10 20 30 40 50 60 70 80
PIO reader response time
oda x2 v2
Thursday, April 18, 13
SLOB - PIO - 30 readers
45
0
1500
3000
4500
6000
1 10 20 30 40 50 60 70 80
PIO reader response time
oda x2 v2
ODA (20 x 15krpm HDD) disk capacity is saturated so response time increases with more readers.
Thursday, April 18, 13
SLOB - PIO - 30 readers
45
0
1500
3000
4500
6000
1 10 20 30 40 50 60 70 80
PIO reader response time
oda x2 v2
ODA (20 x 15krpm HDD) disk capacity is saturated so response time increases with more readers.
Flashcache is not saturated, so response time of IO of 10-20-30 readers increases very little.
Thursday, April 18, 13
...up to 80 readers
46
Thursday, April 18, 13
SLOB - PIO - 80 readers
47
0
2500
5000
7500
10000
1 10 20 30 40 50 60 70 80
PIO reader response time
oda x2 v2
Thursday, April 18, 13
SLOB - PIO - 80 readers
47
0
2500
5000
7500
10000
1 10 20 30 40 50 60 70 80
PIO reader response time
oda x2 v2
ODA response time more or less increases linear.
Thursday, April 18, 13
SLOB - PIO - 80 readers
47
0
2500
5000
7500
10000
1 10 20 30 40 50 60 70 80
PIO reader response time
oda x2 v2
ODA response time more or less increases linear.
X2 flashcache (3x96GB) is not saturated, so little increase in response time.
Thursday, April 18, 13
SLOB - PIO - 80 readers
47
0
2500
5000
7500
10000
1 10 20 30 40 50 60 70 80
PIO reader response time
oda x2 v2
ODA response time more or less increases linear.
X2 flashcache (3x96GB) is not saturated, so little increase in response time.
The V2 response time (with more flashcache!) starts increasing at 70 readers. A bottleneck is showing up! (7x96GB!!)
Thursday, April 18, 13
SLOB - PIO - 80 readers
48
0
37500
75000
112500
150000
1 10 20 30 40 50 60 70 80
PIO readers IOPS
oda x2 v2
IOPS view instead of responsetime
Thursday, April 18, 13
SLOB - PIO - 80 readers
48
0
37500
75000
112500
150000
1 10 20 30 40 50 60 70 80
PIO readers IOPS
oda x2 v2
IOPS view instead of responsetime
3x96GB Flashcache and IB can serve > 115’851 read IOPS!
Thursday, April 18, 13
SLOB - PIO - 80 readers
48
0
37500
75000
112500
150000
1 10 20 30 40 50 60 70 80
PIO readers IOPS
oda x2 v2
IOPS view instead of responsetime
3x96GB Flashcache and IB can serve > 115’851 read IOPS!
This V2 has more flashcache, so decline in read IOPS probably due to something else!
Thursday, April 18, 13
SLOB - PIO - 80 readers
48
0
37500
75000
112500
150000
1 10 20 30 40 50 60 70 80
PIO readers IOPS
oda x2 v2
IOPS view instead of responsetime
3x96GB Flashcache and IB can serve > 115’851 read IOPS!
This V2 has more flashcache, so decline in read IOPS probably due to something else!
ODA maxed out at ~ 11’200 read IOPS
Thursday, April 18, 13
SLOB - PIO - 80 readers
49
V2 top 5 timed events with 80 readers:Event Waits Time(s) (ms) time Wait Class------------------------------ ------------ ----------- ------ ------ ----------cell single block physical rea 102,345,354 56,614 1 47.1 User I/Olatch: cache buffers lru chain 27,187,317 33,471 1 27.8 Otherlatch: cache buffers chains 14,736,819 19,594 1 16.3 ConcurrencDB CPU 13,427 11.2wait list latch free 932,930 553 1 .5 Other
X2 top 5 timed events with 80 readers:Event Waits Time(s) (ms) time Wait Class------------------------------ ------------ ----------- ------ ------ ----------cell single block physical rea 102,899,953 68,209 1 87.9 User I/ODB CPU 9,297 12.0latch: cache buffers lru chain 10,917,303 1,585 0 2.0 Otherlatch: cache buffers chains 2,048,395 698 0 .9 Concurrenccell list of blocks physical r 368,795 522 1 .7 User I/O
Thursday, April 18, 13
SLOB - PIO - 80 readers
49
V2 top 5 timed events with 80 readers:Event Waits Time(s) (ms) time Wait Class------------------------------ ------------ ----------- ------ ------ ----------cell single block physical rea 102,345,354 56,614 1 47.1 User I/Olatch: cache buffers lru chain 27,187,317 33,471 1 27.8 Otherlatch: cache buffers chains 14,736,819 19,594 1 16.3 ConcurrencDB CPU 13,427 11.2wait list latch free 932,930 553 1 .5 Other
X2 top 5 timed events with 80 readers:Event Waits Time(s) (ms) time Wait Class------------------------------ ------------ ----------- ------ ------ ----------cell single block physical rea 102,899,953 68,209 1 87.9 User I/ODB CPU 9,297 12.0latch: cache buffers lru chain 10,917,303 1,585 0 2.0 Otherlatch: cache buffers chains 2,048,395 698 0 .9 Concurrenccell list of blocks physical r 368,795 522 1 .7 User I/O
44.1%
Thursday, April 18, 13
SLOB - PIO - 80 readers
49
V2 top 5 timed events with 80 readers:Event Waits Time(s) (ms) time Wait Class------------------------------ ------------ ----------- ------ ------ ----------cell single block physical rea 102,345,354 56,614 1 47.1 User I/Olatch: cache buffers lru chain 27,187,317 33,471 1 27.8 Otherlatch: cache buffers chains 14,736,819 19,594 1 16.3 ConcurrencDB CPU 13,427 11.2wait list latch free 932,930 553 1 .5 Other
X2 top 5 timed events with 80 readers:Event Waits Time(s) (ms) time Wait Class------------------------------ ------------ ----------- ------ ------ ----------cell single block physical rea 102,899,953 68,209 1 87.9 User I/ODB CPU 9,297 12.0latch: cache buffers lru chain 10,917,303 1,585 0 2.0 Otherlatch: cache buffers chains 2,048,395 698 0 .9 Concurrenccell list of blocks physical r 368,795 522 1 .7 User I/O
44.1%
2.9%
Thursday, April 18, 13
SLOB - PIO - 80 readers• On the V2, cache concurrency control throttles throughput• On the X2, this happens only very minimal- V2: CPU: Intel Xeon E5540 @ 2.53GHz (2s8c16t)- X2: CPU: Intel Xeon X5670 @ 2.93GHz (2s12c24t)
V2Event Waits <1ms <2ms <4ms <8ms <16ms <32ms <=1s >1s-------------------------- ----- ----- ----- ----- ----- ----- ----- ----- -----latch: cache buffers chain 14.7M 44.1 37.0 16.3 2.6 .0 .0latch: cache buffers lru c 27.2M 37.2 42.6 20.0 .2 .0 .0
X2latch: cache buffers chain 2048. 91.8 7.5 .6 .0 .0latch: cache buffers lru c 10.9M 97.4 2.3 .2 .0
50
Thursday, April 18, 13
SLOB - Read IO
51
1 LIO 80 LIO 1 PIO 80 PIO
ODA 1 11 73 9795
X2 (HC disks) 2 11 76 976
V2 (HP disks) 3 22 106 1518
Thursday, April 18, 13
SLOB - Read IO
51
1 LIO 80 LIO 1 PIO 80 PIO
ODA 1 11 73 9795
X2 (HC disks) 2 11 76 976
V2 (HP disks) 3 22 106 1518
1 PIO w/o flashcache 80 PIO w/o flashcache
ODA 73 9795
X2 167 ? (> 18000)
V2 118 5098
Thursday, April 18, 13
SLOB - Read IO• For scalability, OLTP needs buffered IO (LIO)
• Flashcache is EXTREMELY important physial IO scalability
• Probably should not let flash be used for something else– Unless you can always keep all your small reads in
cache
52
Thursday, April 18, 13
SLOB - Read IO• Flashcache mimics a SAN/NAS cache
– So nothing groundbreaking here, it does what current, normal infra should do too...
– The bandwidth needed to deliver the data to the database is provided by Infiniband
– 1 Gb ethernet = 120MB/s, 4 Gb fiber = 400MB/s
• Infiniband is generally available outside of Oracle
53
Thursday, April 18, 13
SLOB - Read IO• How much IOPS can a single cell do?
– According to https://blogs.oracle.com/mrbenchmark/entry/inside_the_sun_oracle_database
– A single cell can do 75’000 IOPS from flash (8kB)‣ Personal calculation: 60’000 IOPS with 8kB
• Flashcache cache– Caches small reads* (8kB and less) mostly– Large multiblock reads are not cached, unless
segment property ‘cell_flash_cache’ is set to ‘keep’.
54
Thursday, April 18, 13
SLOB - Read IO• Is Exadata a good idea for OLTP?
– From a strictly technical point of view, there is no benefit.
• But...– Exadata gives you IORM– Exadata gives you reasonably up to date hardware– Exadata gives a system engineered for performance– Exadata gives you dedicated disks – Exadata gives a validated combination of database,
clusterware, operating system, hardware, firmware.
55
Thursday, April 18, 13
Writing and Exadata• Exadata storage servers provide NO redundancy for data
– That’s a function of ASM
• Exadata is configured with either– Normal redundancy (mirroring) or– High redundancy (triple mirroring)
• to provide data redundancy.
56
Thursday, April 18, 13
Writing and Exadata• Reading has no problem with normal/high redundancy.
• During writes, all two or three AU’s need to be written.
• This means when you calculate write throughput, you need to double all physical writes if using normal redundancy.
57
Thursday, April 18, 13
Writing on Exadata• But we got flash! Right?
• Yes, you got flash. – But it probably doesn’t do what you think it does:
58
Thursday, April 18, 13
Writing on ExadataThis is on the half rack V2 HP:
[oracle@dm01db01 [] stuff]$ dcli -l celladmin -g cell_group cellcli -e "list metriccurrent where name like \'FL_.*_FIRST\'"dm01cel01: FL_DISK_FIRST FLASHLOG 316,563 IO requestsdm01cel01: FL_FLASH_FIRST FLASHLOG 9,143 IO requestsdm01cel02: FL_DISK_FIRST FLASHLOG 305,891 IO requestsdm01cel02: FL_FLASH_FIRST FLASHLOG 7,435 IO requestsdm01cel03: FL_DISK_FIRST FLASHLOG 307,634 IO requestsdm01cel03: FL_FLASH_FIRST FLASHLOG 10,577 IO requestsdm01cel04: FL_DISK_FIRST FLASHLOG 299,547 IO requestsdm01cel04: FL_FLASH_FIRST FLASHLOG 10,381 IO requestsdm01cel05: FL_DISK_FIRST FLASHLOG 311,978 IO requestsdm01cel05: FL_FLASH_FIRST FLASHLOG 10,888 IO requestsdm01cel06: FL_DISK_FIRST FLASHLOG 315,084 IO requestsdm01cel06: FL_FLASH_FIRST FLASHLOG 10,022 IO requestsdm01cel07: FL_DISK_FIRST FLASHLOG 323,454 IO requestsdm01cel07: FL_FLASH_FIRST FLASHLOG 8,807 IO requests
59
Thursday, April 18, 13
Writing on ExadataThis is on the quarter rack X2 HC:[root@xxxxdb01 ~]# dcli -l root -g cell_group cellcli -e "list metriccurrent where name like \'FL_.*_FIRST\'"xxxxcel01: FL_DISK_FIRST FLASHLOG 68,475,141 IO requestsxxxxcel01: FL_FLASH_FIRST FLASHLOG 9,109,142 IO requestsxxxxcel02: FL_DISK_FIRST FLASHLOG 68,640,951 IO requestsxxxxcel02: FL_FLASH_FIRST FLASHLOG 9,229,226 IO requestsxxxxcel03: FL_DISK_FIRST FLASHLOG 68,388,238 IO requestsxxxxcel03: FL_FLASH_FIRST FLASHLOG 9,072,660 IO requests
60
Thursday, April 18, 13
Writing on Exadata• Please mind these are cumulative numbers!
• The half-rack is a POC machine, no heavy usage between POC’s.
• The quarter-rack has had some load, but definately not heavy OLTP.
61
Thursday, April 18, 13
Writing on Exadata• I can imagine flashlog can prevent long write times if
disk IO’s queue.– A normal configured database on Exadata has online
redo in DATA and in RECO diskgroup– Normal redundancy means every log write must be
done 4 times
• Log writer wait times:– V2 min: 16ms (1 writer), max: 41ms (20 writers)– X2 min: 39ms (10 writers), max: 110ms (40 writers)
62
Thursday, April 18, 13
Writing on Exadata: conclusion• Log file write response time on Exadata is not in the
same range as reads.
• There’s the flashlog feature, but it might work different than the whitepaper explains
• Be careful with heavy writing on Exadata. – There’s no Exadata specific improvement for writes*.
63
Thursday, April 18, 13
Thank you for attending!
Questions?
64
Thursday, April 18, 13
Thanks, Links, etc.• VX Company• Klaas-Jan Jongsma• Martin Bach• Kevin Closson
65
Thursday, April 18, 13