„automate information lifecycle management with oracle database 12c” krzysztof marciniak, senior...
DESCRIPTION
Plug into the Cloud with Oracle Database12cTRANSCRIPT
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential Restricted1
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential Restricted2
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. Release timing for Oracle Database 12c is planned for Calendar Year 2013.
Automatic Data Optimization with Oracle Database 12cKrzysztof MarciniakSenior Sales Consultant
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential Restricted4
Agenda
Data Growth’s Impact
Heat Map Access Tracking
Automatic Data Optimization
ILM Benefits of 12c
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential Restricted5
Growth in Data Diversity and Usage1,800 Exabytes of Data in 2011, 20x Growth by 2020
Mobile#1 Internet access device in 2013
Big DataLarge customers top 50PB
Enterprise45% per year growth
in database data
Cloud80% of newapplications
and their data
Regulation300 exabytes in archives by 2015
Social Business $30B/year in commerce by 2015
Today’s Drivers Emerging Growth Factors
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential Restricted6
Storage Management Challenges
Compress data, without impacting
performance
Manage more data without incurring additional cost
Tier and compress data based on usage
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential Restricted7
Information Lifecycle ManagementManaging Data Over its Lifetime
“The policies, processes, practices, and tools used to align the business value of information with the most appropriate and cost effective IT infrastructure from the time information is conceived through its final disposition.”
Storage Networking Industry Association (SNIA) Data Management Forum
$$$Total Cost of Ownership (TCO)
$$$
HighValue
MediumValue
LowValue
Val
ue
at R
isk
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential Restricted8
Introducing Heat Map for Data CompressionReduce storage footprint, read compressed data faster
Hot Data
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Confidential – Oracle Restricted8
111010101010101001101010101011010001011011000110100101000001001110001010101101001011010010110001010010011111001001000010001010101101000
101010101110101001101011
10000101000101101110101
01001010010010000100010
101011010010110100111000
01010010010100001001000
010001010101110011010
Warm Data
101010101110101001101011100001010001011011101010100101001001000010001010101101001011010011100001010010010100001001000010001010101101001
101010101110101001101011100001010001011
011101010100101001001000010001010101101
001011010011100001010010010100001001000
01000101010111001101110011000111010
Archive Data
101010101110101001101011100001010001011011101010100101001001000010001010101101001011010011100001010010010100001001000010001010101101001
101010101110101001101011100001010001011011101010
10010100100100001000101010110100101101001110000
1010010010100001001000010001010101110011011100
3X Advanced Row Compression
10X Columnar Query Compression
15X Columnar Archive Compression
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential Restricted9
Oracle Advanced CompressionTransparent, Smaller, Faster
100% Application Transparent End-to-end Cost/Performance Benefits across CPU, DRAM, Flash,
Disk & Network Runs Faster: OLTP Apps (Transactional & Analytics) & DW Reduces Database Footprint
– Decreases CapEx & OpEx
– Increases Cloud ROI through Database Footprint reduction in DRAM Memory
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential Restricted10
Oracle Advanced CompressionNew Features, New Feature Names
Oracle Advanced Compression
Oracle Database 11g Oracle Database 12c
OLTP Compression Advanced Row Compression
Secure Files Compression Advanced LOB Compression
Secure Files De-duplication Advanced LOB Deduplication
Hybrid Columnar Compression Hybrid Columnar Compression
NEW Heat Map (Object and Row Level)
NEW Automatic Data Optimization
NEW Temporal (Advancements)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential Restricted11
Heat MapWhat it tracks “Heat Map” tracking
– Database level Heat Map shows which tables and partitions are being used
– Block level Heat Map shows last modification at the block level
Comprehensive– Segment level shows both reads and writes
– Distinguishes index lookups from full scans
– Automatically excludes stats gathering, DDLs or table redefinitions
High Performance– Object level at no cost– Block level < 5% cost
Active
Frequent Access
Occasional Access
Dormant
Actively updated
Infrequently updated, Frequently Queried
Infrequent access for query and updates
Long term analytics & compliance
HOT
COLD
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential Restricted12
Understanding Data Usage PatternsDatabase ‘heat map’
10 0111 10 10 0111 101 1
00 0101 10 10 0101 10
0 010 0111 10 10 0111 100 010 0111 10 00 0111 001 1
01 0101 10 00 0101 101 110 0100 11 10 0111 100 0
10 0111 10 10 0111 100 010 0111 00 00 0111 100 110 0100 11 10 0111 100 0
10 0101 10 10 0101 001 1
00 0101 10 10 0101 101 110 0100 11 10 0111 100 0
10 0110 11 10 0111 100 0
10 0111 10 00 0111 001 1
01 0101 10 00 0101 101 110 0100 11 10 0111 100 0
10 0111 10 10 0111 100 010 0111 00 00 0111 100 1
00 0101 10 10 0101 101 110 0100 11 10 0111 100 0
00 0101 10 10 0101 101 110 0111 10 10 0111 100 010 0101 10 10 0101 001 110 0100 11 10 0111 100 0
10 0111 11 10 0111 100 010 0111 10 10 0111 101 110 0110 11 10 0111 100 0
10 0100 11 10 0111 100 0
10 0111 11 10 0111 100 010 0111 10 10 0111 101 110 0110 11 10 0111 100 0
10 0111 10 10 0111 100 01 1
1 1
10 0100 11 10 0111 100 0
10 0100 11 10 0111 1010 0111 10 10 0111 10
10 0100 11 10 0111 100 0
10 0100 11 10 0111 100 0
10 0111 10 10 0111 100 010 0111 10 00 0111 001 1
01 0101 10 00 0101 101 110 0100 11 10 0111 100 0
10 0111 10 10 0111 100 010 0111 00 00 0111 100 110 0100 11 10 0111 100 0
10 0101 10 10 0101 001 1
00 0101 10 10 0101 101 110 0100 11 10 0111 100 0
00 0101 10 10 0101 101 110 0111 10 10 0111 100 010 0101 10 10 0101 001 110 0100 11 10 0111 100 0
10 0111 10 10 0111 100 010 0111 10 00 0111 001 1
01 0101 10 00 0101 101 110 0100 11 10 0111 100 0
10 0111 10 10 0111 100 0
00 0101 10 10 0101 101 110 0111 10 10 0111 100 010 0101 10 10 0101 001 1
00 0101 10 10 0101 101 110 0100 11 10 0111 100 0
10 0111 10 10 0111 100 010 0111 10 00 0111 001 1
01 0101 10 00 0101 101 110 0100 11 10 0111 100 0
10 0111 10 10 0111 100 0
00 0101 10 10 0101 101 110 0111 10 10 0111 100 010 0101 10 10 0101 001 1
00 0101 10 10 0101 101 110 0100 11 10 0111 100 0
10 0111 10 00 0111 001 1
01 0101 10 00 0101 101 110 0100 11 10 0111 100 0
10 0111 10 10 0111 100 010 0111 00 00 0111 100 1
00 0101 10 10 0101 101 110 0100 11 10 0111 100 0
10 0111 11 10 0111 100 010 0111 10 10 0111 101 110 0110 11 10 0111 100 0
10 0111 10 10 0111 100 010 0111 10 10 0111 100 0
10 0110 11 10 0111 100 0
10 0100 11 10 0111 100 0
10 0111 11 10 0111 100 010 0111 10 10 0111 101 110 0110 11 10 0111 100 0
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential Restricted13
Understanding Data Usage PatternsDatabase ‘heat map’
10 0111 10 10 0111 101 1
00 0101 10 10 0101 10
0 010 0111 10 10 0111 100 010 0111 10 00 0111 001 1
01 0101 10 00 0101 101 110 0100 11 10 0111 100 0
10 0111 10 10 0111 100 010 0111 00 00 0111 100 110 0100 11 10 0111 100 0
10 0101 10 10 0101 001 1
00 0101 10 10 0101 101 110 0100 11 10 0111 100 0
10 0110 11 10 0111 100 0
10 0111 10 00 0111 001 1
01 0101 10 00 0101 101 110 0100 11 10 0111 100 0
10 0111 10 10 0111 100 010 0111 00 00 0111 100 1
00 0101 10 10 0101 101 110 0100 11 10 0111 100 0
00 0101 10 10 0101 101 110 0111 10 10 0111 100 010 0101 10 10 0101 001 110 0100 11 10 0111 100 0
10 0111 11 10 0111 100 010 0111 10 10 0111 101 110 0110 11 10 0111 100 0
10 0100 11 10 0111 100 0
10 0111 11 10 0111 100 010 0111 10 10 0111 101 110 0110 11 10 0111 100 0
10 0111 10 10 0111 100 01 1
1 1
10 0100 11 10 0111 100 0
10 0100 11 10 0111 1010 0111 10 10 0111 10
10 0100 11 10 0111 100 0
10 0100 11 10 0111 100 0
10 0111 10 10 0111 100 010 0111 10 00 0111 001 1
01 0101 10 00 0101 101 110 0100 11 10 0111 100 0
10 0111 10 10 0111 100 010 0111 00 00 0111 100 110 0100 11 10 0111 100 0
10 0101 10 10 0101 001 1
00 0101 10 10 0101 101 110 0100 11 10 0111 100 0
00 0101 10 10 0101 101 110 0111 10 10 0111 100 010 0101 10 10 0101 001 110 0100 11 10 0111 100 0
10 0111 10 10 0111 100 010 0111 10 00 0111 001 1
01 0101 10 00 0101 101 110 0100 11 10 0111 100 0
10 0111 10 10 0111 100 0
00 0101 10 10 0101 101 110 0111 10 10 0111 100 010 0101 10 10 0101 001 1
00 0101 10 10 0101 101 110 0100 11 10 0111 100 0
10 0111 10 10 0111 100 010 0111 10 00 0111 001 1
01 0101 10 00 0101 101 110 0100 11 10 0111 100 0
10 0111 10 10 0111 100 0
00 0101 10 10 0101 101 110 0111 10 10 0111 100 010 0101 10 10 0101 001 1
00 0101 10 10 0101 101 110 0100 11 10 0111 100 0
10 0111 10 00 0111 001 1
01 0101 10 00 0101 101 110 0100 11 10 0111 100 0
10 0111 10 10 0111 100 010 0111 00 00 0111 100 1
00 0101 10 10 0101 101 110 0100 11 10 0111 100 0
10 0111 11 10 0111 100 010 0111 10 10 0111 101 110 0110 11 10 0111 100 0
10 0111 10 10 0111 100 010 0111 10 10 0111 100 0
10 0110 11 10 0111 100 0
10 0100 11 10 0111 100 0
10 0111 11 10 0111 100 010 0111 10 10 0111 101 110 0110 11 10 0111 100 0
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential Restricted14
Heat MapEnterprise Manager
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential Restricted15
Heat Map for Tables and Partitions“segment” level tracking
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential Restricted16
Heat Map for Blocks“row” level tracking
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential Restricted17
0101101110101010010100100100001000
1010101101001011010011100001010010
Archive Data
011100001010001011011
101010100101001001000
010001010101101001011
010101001010010010001
Automatic Data Optimization Usage Based Data Compression
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Confidential – Oracle Restricted17
Hot Data
3X Advanced Row Compression
Warm Data
10101010111010100110101110000101000
10110111010101001010010010000100010
10101101001011010011100001010010010
1000010010000100010101011010010
10X Columnar Query Compression
1000010100100101001010110111000010
1010101011101010011010111000010100010110111
010101001010010010000100010101011010010110
10011100001010010010100001001000010001010
101010101110101001101011100001010001011011
15X Columnar Archive Compression
011101010100101000010001010101011100001010
101010101110101001101011
10000101000101101110101
01001010010010000100010
101011010010110100111000
01010010010100001001000
01000101010111001101010100101001001000010001
1110010100100101001010110111011010
101010101110101001101011100001011101011001
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential Restricted18
Compress Rows Based on Usage “background row compression”
SQL> ALTER TABLE EMPLOYEE ILM ADD POLICY ROW STORE COMPRESS ADVANCED ROWAFTER 1 DAY OF NO MODIFICATION
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential Restricted19
Compress Partitions Based on UsageAutomatic Data Optimization with HCC
SQL> ALTER TABLE ORDERS ILM ADD POLICY COMPRESS FOR QUERY HIGH SEGMENT AFTER 30 DAYS OF NO MODIFICATION
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential Restricted20
Optimize Data Storage Based on UsageAutomatic Data Optimization with HCC
Optimized use of Row Store & Column Store within a single table
– Row Format - Insert / load data as fast as possible
– Fast analytics & reporting – data optimized in columnar format for analytics
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential Restricted21
Automatic Data Optimization
This Quarter This Year Prior Years
Row Store for fast OLTP
Compressed Column Storefor fast analytics
10x compressed15x compressed As data cools
down, Advanced Data Optimization automatically converts data to columnar compressed Online
Archive Compressed Column Storefor max compression
Reporting Compliance & ReportingOLTP
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential Restricted22
Up to 15x Smaller Footprint & Faster Queries
Both Columnar & Archive Compression now complement Advanced Row Compression
Best Practice:– Step 1: Use Advanced Row Compression for entire DB and then
– Step 2: ADO automatically converts into columnar compressed once the updates cool down, and is used mainly for reporting
=> Query speed of Columnar & 10x smaller footprint
– Step 3: ADO automatically converts into archive compressed once data cools down further and is no longer frequently queried
=> 15-50x smaller footprint
Automatic Data Optimization for OLTP
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential Restricted23
Optimizes Data Based on Heat MapAutomatic Data Optimization for DW
Data generally comes in via Bulk Loading Workload dominated by queries, even during loading
Step 1: Bulk Load directly into Columnar Compressed
– 10x smaller footprint, Query speed of Columnar
Step 2: ADO automatically converts to Archive Compressed and moves to Lower Cost Storage once its queried infrequently
– Data remains online, with 15-50x smaller footprint, & lower storage cost
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential Restricted24
Fast, Flexible Loads & Queries on Columnar
Fastest Load with uncompressed & Fastest Queries with columnar– Mixed workloads often use Java app or 3rd party tools to insert and update data
that does not use Bulk Loads, so cannot use Columnar
Step 1: Load into uncompressed, conventional inserts & updates– Fast loading, & flexibility of using a regular OLTP app for loading
Step 2: ADO moves to Row Compressed or Columnar Compressed or Low Cost Storage once updates cool down
– Faster Queries, 3-10x smaller footprint
Automatic Data Optimization – Mixed Use
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential Restricted25
Storage Tiering – How it Works?Automatic Data Optimization
1. Tables grow in size ADO policies compress data
2. Tablespace containing partitions reaches ADO tiering threshold
3. Partitions are moved to different tablespace on lower spec disk group
+DATA
(High Perf)
+DATA_LOW
(Low Cost)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential Restricted26
Powerful Policy Specification Automatic Data Optimization
Declarative Policy Specification: Condition Action– alter table sales ilm add policy compress for OLTP row after 3 days of no
modification;
– Conditions are time period after creation, access, modification of data
– Actions can be Compression Tiering or Tablespace Tiering
Policies are inherited from the tablespace or table– New tables inherit from tablespace; can also be applied to existing tables
– New partitions (including interval partitions) inherit from table
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential Restricted27
Simple Declarative SQL extensionAutomatic Data Optimization
ALTER TABLE sales ILM add
Active
Frequent Access
Occasional Access
Dormant
OLTP Compressed (2-4x) Affects ONLY Candidate Rows Cached in DRAM & FLASH
compress for OLTP row after 2 days of no update
Warehouse Compressed (10x) High Performance Storage
compress for query low after 1 week of no update
Warehouse Compressed (10x) Low Cost Storage
tier to SATA Tablespace
Archive Compressed (15-50X) Archival Storage
compress for archive high after 6 months no access
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential Restricted28
Storage Tiering - Policy Automatic Data Optimization
SQL> ALTER TABLE EMPLOYEE ILM ADD POLICY TIER TO LOW_COST_TABLESPACE
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential Restricted29
Scheduled Policy ExecutionAutomatic Data Optimization
Immediate and background policy execution
– Row level policies are executed periodically
(Users can configure the frequency of execution)
– Segment level policies are executed in maintenance windows
Policies can be extended to incorporate Business Rules
– Users can add custom conditions to control placement
(e.g. 3 months after the ship date of an order)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential Restricted30
Optimized Backups Automated READONLY data movement
1. Tables grow in size ADO policies compress data
2. Tablespace containing partitions reaches ADO tiering threshold
3. Partitions are moved to new read only tablespace on lower spec disk group
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential Restricted31
Optimized Backups Automated READONLY Data Movement
SQL> ALTER TABLE EMPLOYEE ILM ADD POLICY TIER TO DATA2 READ ONLYAFTER 180 DAYS OF NO MODIFICATION
Expected syntax for production..
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential Restricted32
Backup & Recovery Automatic Data Optimization
Read / WriteTablespace
Read-mostly data is moved to a READONLY Tablespace
10x compressed15x compressed
READONLYTBS
As data cools down, Automatic Data Optimization automatically moves it to a READONLY TBS, it’s backed up only once after that
Reporting Compliance & ReportingOLTP
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential Restricted33
More Compression FeaturesNew in Oracle Database 12c
Logminer and GoldenGate support Capture side changes in 11.1 logminer Apply side changes in 11.2
Faster queries on advanced row compression
Wide tables (>255 columns) for advanced row (OLTP) compression
Network Compression
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential Restricted34
SummaryHeat Map & Automatic Data Optimization
Heat Map
– Automatically tracks access
– Database-aware: maintenance jobs, backups, etc don’t affect Heat Map Automatic Data Optimization
– Declarative easy-to-use syntax to define data compression & movement policies
– Extensible with business-specific logic
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential Restricted35
Graphic Section Divider
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential Restricted36
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential Restricted37
Oracle Database 12c introduces Automatic Data Optimization capabilities which allow organizations to use policy-based management to automate data compression and movement with minimal administration. In this session you’ll learn how Heat Map provides deep insight into how your data is accessed by users and applications to trigger Automatic Data Optimization policies that reduce the costs associated with storing and managing more data, while improving database performance.
Abstract