heat map and automatic data optimization with oracle database 12c
DESCRIPTION
Présentation des nouveautés Oracle Database Data Optimization 12c. http://www.digicomp.ch/fr/oracleTRANSCRIPT
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 1
Heat Map and Automatic Data Optimization with Oracle Database 12c
Jean-Philippe Breysse
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 3
Program Agenda
Heat Map
Automatic Data Optimization
Benefits of Heat Map and ADO
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 4
Data Compression Reduce storage footprint, read compressed data faster
Hot Data
111010101010101001101010101011010001011011000110100101000001001110001010101101001011010010110001010010011111001001000010001010101101000
10101010111010100110101
11000010100010110111010
10100101001001000010001
01010110100101101001110
00010100100101000010010
00010001010101110011010
Warm Data
101010101110101001101011100001010001011011101010100101001001000010001010101101001011010011100001010010010100001001000010001010101101001
10101010111010100110101110000101000101
10111010101001010010010000100010101011
01001011010011100001010010010100001001
00001000101010111001101110011000111010
Archive Data
101010101110101001101011100001010001011011101010100101001001000010001010101101001011010011100001010010010100001001000010001010101101001
10101010111010100110101110000101000101101110101
01001010010010000100010101011010010110100111000
01010010010100001001000010001010101110011011100
3X Advanced Row Compression
10X
Columnar Query Compression
15X
Columnar Archive Compression
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 5
Oracle Advanced Compression Transparent, Smaller, Faster
100% Application Transparent
End-to-end Cost/Performance Benefits across CPU, DRAM, Flash,
Disk & Network
Runs Faster: OLTP Apps (Transactional & Analytics) & DW
Reduces Database Footprint
– CapEx & OpEx savings
– Increases Cloud ROI through Database Footprint reduction in
DRAM Memory
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 6
Details: Oracle Advanced Compression + New Features, New Feature Names
Ora
cle
A
dva
nc
ed
Co
mp
re
ssio
n
Oracle Database 11g Oracle Database 12c
OLTP Compression Advanced Row Compression
Secure Files Compression Advanced LOB Compression
Secure Files De-duplication Advanced LOB Deduplication
NEW Heat Map (Object and Row Level)
NEW Automatic Data Optimization
NEW Temporal Optimization
+
Hybrid Columnar Compression Hybrid Columnar Compression
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 8
Understanding Data Usage Patterns Database ‘heat map’
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 1 1
0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0
0 0 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 0 0 0 1 1 1 0 0 1 1
0 1 0 1 0 1 1 0 0 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 0 0 0 0 0 1 1 1 1 0 0 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 0 1 1 0 1 0 0 1 0 1 0 0 1 1
0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 0 0 0 1 1 1 0 0 1 1
0 1 0 1 0 1 1 0 0 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 0 0 0 0 0 1 1 1 1 0 0 1
0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 0 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 1 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 1 1 1 0 0 1 1 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 1 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 1 1 1 0 0 1 1 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 1
1 1
1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0
1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 0 0 0 1 1 1 0 0 1 1
0 1 0 1 0 1 1 0 0 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 0 0 0 0 0 1 1 1 1 0 0 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 0 1 1 0 1 0 0 1 0 1 0 0 1 1
0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 0 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 0 0 0 1 1 1 0 0 1 1
0 1 0 1 0 1 1 0 0 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0
0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 0 0 1 1
0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 0 0 0 1 1 1 0 0 1 1
0 1 0 1 0 1 1 0 0 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0
0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 0 0 1 1
0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 0 0 0 1 1 1 0 0 1 1
0 1 0 1 0 1 1 0 0 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 0 0 0 0 0 1 1 1 1 0 0 1
0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 1 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 1 1 1 0 0 1 1 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 1 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 1 1 1 0 0 1 1 0 1 1 1 0 0 1 1 1 1 0 0 0
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 9
Understanding Data Usage Patterns Database ‘heat map’
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 1 1
0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0
0 0 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 0 0 0 1 1 1 0 0 1 1
0 1 0 1 0 1 1 0 0 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 0 0 0 0 0 1 1 1 1 0 0 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 0 1 1 0 1 0 0 1 0 1 0 0 1 1
0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 0 0 0 1 1 1 0 0 1 1
0 1 0 1 0 1 1 0 0 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 0 0 0 0 0 1 1 1 1 0 0 1
0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 0 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 1 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 1 1 1 0 0 1 1 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 1 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 1 1 1 0 0 1 1 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 1
1 1
1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0
1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 0 0 0 1 1 1 0 0 1 1
0 1 0 1 0 1 1 0 0 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 0 0 0 0 0 1 1 1 1 0 0 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 0 1 1 0 1 0 0 1 0 1 0 0 1 1
0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 0 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 0 0 0 1 1 1 0 0 1 1
0 1 0 1 0 1 1 0 0 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0
0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 0 0 1 1
0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 0 0 0 1 1 1 0 0 1 1
0 1 0 1 0 1 1 0 0 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0
0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 0 0 1 1
0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 0 0 0 1 1 1 0 0 1 1
0 1 0 1 0 1 1 0 0 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 0 0 0 0 0 1 1 1 1 0 0 1
0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 1 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 1 1 1 0 0 1 1 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0
1 0 0 1 1 1 1 1 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 1 1 1 0 0 1 1 0 1 1 1 0 0 1 1 1 1 0 0 0
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 10
Heat Map What it tracks “Heat Map” tracking
– Access and modification times tracked for tables and
partitions
– Modification times tracked for database blocks
Comprehensive
– Segment level shows both reads and writes
– Distinguishes index lookups from full table scans
– Automatically excludes stats gathering, DDLs, table
redefinitions, etc
High Performance – Object level at no cost
– Block level < 5% cost
Active
Frequent
Access
Occasional
Access
Dormant
HOT
COLD
WARM
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 11
Heat Map Enterprise Manager
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 12
0101101110101010010100100100001000
1010101101001011010011100001010010
Archive Data
011100001010001011011
101010100101001001000
010001010101101001011
010101001010010010001
Automatic Data Optimization Usage Based Data Compression
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Confidential – Oracle Restricted 12
Hot Data
3X
Advanced Row Compression
Warm Data
1010101011101010011010111000010100
0101101110101010010100100100001000
1010101101001011010011100001010010
0101000010010000100010101011010010
10X
Columnar Query Compression
1000010100100101001010110111000010
101010101110101001101011100001010001011011
101010100101001001000010001010101101001011
010011100001010010010100001001000010001010
101010101110101001101011100001010001011011
15X
Columnar Archive Compression
01110101010010
10000100010101
01011100001010
10101010111010100110101
11000010100010110111010
10100101001001000010001
01010110100101101001110
00010100100101000010010
00010001010101110011010
10100101001001000010001
1110010100100101001010110111011010
101010101110101001101011100001011101011001
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 13
Automatic Data Optimization Simple Declarative SQL extension
ALTER TABLE sales ILM add
Active
Frequent
Access
Occasional
Access
Dormant
Advanced Row Compression (2-4x)
Affects ONLY candidate rows
Cached in DRAM & FLASH
row store compress advanced row
after 2 days of no update
Warehouse Compression(10x)
High Performance Storage
column store compress for query low
after 1 week of no update
Warehouse Compression(10x)
Low Cost Storage
tier to SATA Tablespace
Archive Compression(15-50X)
Archival Storage
column store compress for archive high
after 6 months of no access
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 14
Automatic Data Optimization
This Quarter This Year Prior Years
Row Store
for fast OLTP
Compressed Column Store for fast analytics
10x compressed 15x compressed As data cools
down, Automatic
Data Optimization
automatically
converts data to
columnar
compressed
Online
Archive Compressed Column Store for max compression
Reporting Compliance & Reporting OLTP
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 16
Up to 15x Smaller Footprint & Faster Queries
Both Columnar & Archive Compression now complement Advanced
Row Compression
Best Practice:
– Step 1: Use Advanced Row Compression for entire DB and then
– Step 2: ADO automatically converts into columnar compressed once
the updates cool down, and is used mainly for reporting
=> Query speed of Columnar & 10x smaller footprint
– Step 3: ADO automatically converts into archive compressed once
data cools down further and is no longer frequently queried
=> 15-50x smaller footprint
Automatic Data Optimization for OLTP
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 17
Optimizes Data Based on Heat Map Automatic Data Optimization for DW
Data generally comes in via Bulk Loading
Workload dominated by queries, even during loading
Step 1: Bulk Load directly into Columnar Compressed
– 10x smaller footprint, Query speed of Columnar
Step 2: ADO automatically converts to Archive Compressed
and moves to Lower Cost Storage once its queried infrequently
– Data remains online, with 15-50x smaller footprint, & lower
storage cost
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 18
Fast, Flexible Loads & Queries on Columnar Fastest Load with uncompressed & Fastest Queries with columnar
– Mixed workloads often use Java app or 3rd party tools to insert and update data
that does not use Bulk Loads, so cannot use Columnar
Step 1: Load into uncompressed, conventional inserts & updates
– Fast loading, & flexibility of using a regular OLTP app for loading
Step 2: ADO moves to Row Compressed or Columnar
Compressed or Low Cost Storage once updates cool down
– Faster Queries, 3-10x smaller footprint
Automatic Data Optimization – Mixed Use
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 19
Backup & Recovery Automatic Data Optimization
Read / Write
Tablespace Read-mostly data is moved to a READONLY Tablespace
10x compressed 15x compressed
READONLY
TBS
As data cools
down, Automatic
Data Optimization
automatically
moves it to a
READONLY TBS,
it’s backed up only
once after that
Reporting Compliance & Reporting OLTP
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 20
Summary Heat Map & Automatic Data Optimization
Heat Map
– Automatically tracks access
– Database-aware: maintenance jobs, backups, etc. don’t affect Heat Map
Automatic Data Optimization
– Declarative easy-to-use syntax to define data compression & movement
policies
– Extensible with business-specific logic
– Automates and simplifies ILM for Oracle Database data
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 21