oracle database in-memory meets oracle rac
TRANSCRIPT
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Oracle Database In-‐Memory Meets Oracle RAC
Markus Michalewicz Senior Director of Product Management Andy Rivenes Senior Principal Product Manager
#DBIM12c
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement The following is intended to outline our general product direcQon. It is intended for informaQon purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or funcQonality, and should not be relied upon in making purchasing decisions. The development, release, and Qming of any features or funcQonality described for Oracle’s products remains at the sole discreQon of Oracle.
3
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
What is Database In-‐Memory
4
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Oracle Database In-‐Memory: Dual Format Database
• BOTH row and column formats for same table
• Simultaneously acQve and transacQonally consistent
• AnalyQcs & reporQng use new in-‐memory column format
• OLTP uses proven row format
5
Normal Buffer Cache
New In-‐Memory Format
SALES SALES Row
Format Column Format
SALES
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Oracle In-‐Memory Columnar Technology • Pure in-‐memory column format
• Enable for subset of database
• Cheap to maintain – no logging or IO
• Allows efficient OLTP
• No change to disk format
• Built seamlessly into Oracle Database • Appears as a new storage type • Transparent to ApplicaQons • All Enterprise Features work ..
• Availability – RAC, Flashback, DataGuard, etc. • Security – EncrypQon, AudiQng, etc.
6
SALES
Pure In-‐Memory Columnar
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
PopulaQng the In-‐Memory Column Store
• New INMEMORY ATTRIBUTE
• Following segment types are eligible
• Tables
• (Sub)ParQQons
• Materialized views
• Following segment types not eligible
• IOTs • Hash clusters
• Out of line LOBs
ALTER TABLE sales INMEMORY; ALTER TABLE sales NO INMEMORY;
Pure OLTP Features
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Why is an In-‐Memory scan faster than the buffer cache?
SELECT COL4 FROM MYTABLE;
8
X X X X X
RESULT
Row Format
Buffer Cache
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Why is an In-‐Memory scan faster than the buffer cache?
SELECT COL4 FROM MYTABLE;
9
RESULT
Column Format
IM Column Store
RESULT
X X X X
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
How easy is it to get started
10
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Oracle In-Memory: Simple to Implement
1. Configure Memory Capacity • inmemory_size = XXX GB
2. Configure tables or partitions to be in memory • alter table | partition … inmemory;
3. Later drop analytic indexes to speed up OLTP
11
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Configuring: New StaQc Area within SGA
System Global Area SGA
Buffer Cache
Shared Pool
Log Buffer
Large Pool
In-‐Memory Area Other
12
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Configuring : In-‐Memory Column Store
• Controlled by INMEMORY_SIZE parameter
• Minimum size of 100MB
• SGA_TARGET must be large enough to accommodate • StaQc Pool
SELECT * FROM V$SGA; NAME VALUE ------------------ ---------- Fixed Size 2927176 Variable Size 570426808 Database Buffers 4634022912 Redo Buffers 13848576 In-Memory Area 1024483648
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Oracle In-Memory Requires Zero Application Changes
Full Func9onality -‐ No restricQons on SQL Easy to Implement -‐ No migraQon of data Fully Compa9ble -‐ All exisQng applicaQons run unchanged Fully Mul9tenant -‐ Oracle In-‐Memory is Cloud Ready
Uniquely Achieves All In-‐Memory Benefits With No Applica9on Changes
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
How does Database In-‐Memory work with RAC
15
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
The Setup is Basically the Same
1. Configure Memory Capacity • inmemory_size = XXX GB
2. Configure tables or partitions to be in memory • alter table | partition … inmemory;
3. Later drop analytic indexes to speed up OLTP
16
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Configuring IM Column Stores for RAC – Details Mamer
• If you can’t avoid spilling to temp do it efficiently - Increase size of temp writes from 200KB to 1MB - Set _smm_auto_max_io_size =1024
• Database In-‐Memory is not enabled by default • INMEMORY_SIZE must be set to 100MB or larger • SGA_TARGET ExisQng SGA_TARGET INMEMORY_SIZE
• PGA_TARGET PARALLEL_MAX_SERVERS 2GB
(Single Instance) • SGA_TARGET (ExisQng SGA_TARGET INMEMORY_SIZE) * 1.1 (RAC)
(See MOS Note 1903683.1)
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
In-‐Memory and DistribuQon of Data in RAC
• DistribuQon allows in memory segments larger than individual instance memory
• Policy is automaQc (Distribute AUTO) or user-‐specifiable
• Controlled by DISTRIBUTE subclause
• Distribute by rowid range • Distribute by parQQon • Distribute by subparQQon
18
ALTER TABLE sales INMEMORY DISTRIBUTE BY PARTITION; ALTER TABLE COSTS INMEMORY DISTRIBUTE ROWID RANGE;
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
ORDERS PARTITION BY HASH ON ORDER_ID
0
1
2
3
4 ….
Scale-‐Out: Distribute by ParQQon
19
• Distribute by ParQQon (top-‐level parQQon for composite parQQoned tables)
• Ideal for Hash ParQQons
• Also for other parQQon types if uniformly accessed
• Allows in-‐memory parQQon-‐wise joins
19
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
ORDERS PARTITION BY RANGE ON ORDER_DATE SUBPARTITION BY HASH ON ORDER_ID
Nov ‘13 1
Nov ‘13 2
Nov ’13 3
Nov ’13 4
Dec ‘13 1 ….
Scale-‐Out: Distribute by Sub-‐ParQQon
20
• For composite parQQons, can distribute by Sub-‐ParQQon
• Ideal for Hash Sub-‐ParQQons
• Also for other sub-‐parQQon types if uniformly accessed
• Allows in-‐memory parQQon-‐wise joins
20
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
ORDERS Rowid Ranges
1-‐105
106-‐201
202-‐310
311-‐421.
422-‐535
….
Scale-‐Out: Distribute by Rowid Range
21
• Distributes IMCUs by uniform hash on first rowid
• For non-‐parQQoned tables
• Also for parQQoned tables with skewed access across parQQons
• Ensures uniform distribuQon of load across instances
21
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
How to get data in and out
22
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
PopulaQon: Priority • Order in which objects are populated controlled by PRIORITY subclause
- CriQcal, high, medium, low
- Default – none (populate on first access)
- Does not control the speed of populaQon
23
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
PopulaQon: Compression • Objects compressed during populaQon
• Controlled by MEMCOMPRESS subclause
• 2x to 20x compression typical
• MulQple levels of compression - FOR DML
- FOR QUERY (Default)
- FOR CAPACITY
24
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
• Speed of memory • Scan and Filter only the needed Columns
• Vector InstrucQons
Improvements to All Aspects of an AnalyQc Query
25
Data Scans Ve
ctor Register
Load mulQple region values
Vector Compare all values an 1 cycle
CPU CA
CA CA
CA
In-‐Memory Aggrega9on
• Runs Reports Instantly • Create In-‐Memory Report Outline that is Populated during Fast Scan
Joins
• Convert Star Joins into 10X Faster Column Scans
• Search large table for values that match small table
HASH JOIN
Table A Table B
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
How does it work on RAC
26
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Querying Oracle Database In-‐Memory in RAC environment • Serial queries will only
access a fraction of the data from the IM column store on its node
• IMCUs not shipped across interconnect
• Rest of the data comes from disk
27
User
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Querying Oracle Database In-‐Memory in RAC environment • Parallel execution helps as it
starts multiple processes
• BUT we can’t control where the parallel server processes are started
• IMCUs not shipped across interconnect
• Remaining data comes from disk
28
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Enable Auto DOP • Set parallel_degree_policy = auto
– Enables Auto DOP, statement queuing, in-‐memory PX
• For Database In-‐Memory it is important, because it – Ensures that at least one parallel server slave is allocated for each database instance
– Ensures that the query coordinator is IMCU home locaQon aware
29
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Querying Oracle Database In-‐Memory in RAC environment • Shared nothing architecture
means Auto DOP must be used to access Data
• With Auto DOP query coordinator is aware of IMCU locations
• Query coordinator automatically starts parallel server processes on the correct nodes
30
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
What’s different on Engineered Systems
31
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
What’s Special on Engineered Systems?
32
1. Direct-to-wire InfiniBand protocol speeds messaging on Engineered Systems
2. Unique Fault Tolerance allows duplicaQng IMCUs on another node (mirroring)
ODA
Engineered
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Oracle Database In-‐Memory: Unique Fault Tolerance
• Similar to storage mirroring
• Duplicate in-memory columns on another node
- Enabled per table/partition
- Application transparent
• Downtime eliminated by using duplicate after failure
33
Only Available on Engineered Systems
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Oracle Database In-‐Memory: Unique Fault Tolerance
• Policy is user-‐specifiable
• Controlled by DUPLICATE subclause
- DUPLICATE - DUPLICATE ALL
34
ALTER TABLE sales INMEMORY DUPLICATE; ALTER TABLE COSTS INMEMORY DISTRIBUTE AUTO DUPLICATE ALL;
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
RAC-‐ special: Service-‐oriented Column Stores
35
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Services and IM Column Stores • Use services to run independent IM column stores on certain nodes
– Enables access to certain / subset of nodes – Allows to control IM column store populaQon
• Can be used to enforce applicaQon affinity
36
Service IM2 Service: IM3 Service IM1 Service
IM3
IMRACDB
IMRACD1 IMRACD2 IMRACD3 IMRACD4
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
How to Use Services in a IM-‐RAC DB • Services control access to database instances • RAC services should be created with srvctl (register them with the cluster) • Example to add and start services: srvctl add service –d IMRACDB –service IM1 –preferred IMRACDB1, IMRACDB2 srvctl add service –d IMRACDB –service IM2 –preferred IMRACDB3 srvctl add service –d IMRACDB –service IM3 –preferred IMRACDB4 srvctl start service –d IMRACDB –service “IM1,IM2,IM3”
• Corresponding TNS entry example: IM1 =
(DESCRIPTION= (ADDRESS=(PROTOCOL=TCP)(HOST=MySCAN)(PORT=1521)) (CONNECT_DATA= (SERVICE_NAME=IM1)))
37
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
How to set up Independent Column Stores
• PARALLEL_INSTANCE_GROUP – Restricts parallel query operaQons to a limited number of instances – Specify service_name or group_name (i.e. INSTANCE_GROUPS parameter)
• PARALLEL_FORCE_LOCAL – If true, then parallel server processes can only run on the local node
Oracle ConfidenQal – Internal/Restricted/Highly Restricted 38
init.ora Parameters
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Running Independent Column Stores …
• Use services & parallel_instance_group to restrict access – Both, connecQons and PQ must honor service connecQon
• Can be set up to allow priority populaQon, but will restrict all PQ to the service
… on a subset of nodes
Popula9on
srvctl add service –d IMRACDB –s IMPERF –r IMRACDB1, IMRACDB2, IMRACDB3 alter system set parallel_instance_group=IMPERF scope=both sid='*';
Service IMPERF
IMRACDB
IMRACD1 IMRACD2 IMRACD3 IMRACD4
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Running Independent Column Stores …
• User services & parallel_instance_group to restrict access – no overlapping of services – Both connecQons and PQ must honor service connecQon
• Must use priority of NONE and iniQate populaQon from the node (service)
… on individual nodes using mul9ple services (original example)
IMRACDB
IMRACD1 IMRACD2 IMRACD3 IMRACD4
srvctl add service –d IMRACDB –s IM2 –r IMRACDB3 alter system set parallel_instance_group=IM2 scope=both sid='IMRACDB3';
Service IM2 Service IM1 Service
IM3
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Using Independent Column Stores for …
• ApplicaQon access restricted to node(s) based on service name – All "in-‐memory" applicaQon data resides in node(s) IM column store – Parallel queries are restricted to the node(s) supporQng the service
• Allows applicaQons to co-‐exist in a RAC system independently and make use of Database In-‐Memory
• parallel_force_local – can be used to insure no inter-‐node parallelism
Oracle ConfidenQal – Internal/Restricted/Highly Restricted 41
… Applica9on Affinity
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Using Independent Column Stores for …
• On Engineered Systems running with DUPLICATE or DUPLICATE ALL
• A node can be brought down and not affect response Qme SLAs
• All IMCUs in the downed node will be available in another node
Oracle ConfidenQal – Internal/Restricted/Highly Restricted 42
… Rolling Patches and Upgrades
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Join the Conversa9on
43
hmps://twimer.com/TheInMemoryGuy hmps://blogs.oracle.com/in-‐memory/
Related White Papers • Oracle Database In-‐Memory White Paper • Oracle Database In-‐Memory AggregaQon Paper • When to use Oracle Database In-‐Memory • Oracle Database In-‐Memory Advisor
Related Database In-‐Memory Free Webcasts • Oracle Database In-‐Memory meets Data Warehousing
Related Videos • In-‐Memory YouTube Channel • Database Industry Experts Discuss Oracle Database In-‐Memory (11:10)
• Soxware on Silicon
Any Addi9onal Ques9ons • Oracle Database In-‐Memory Blog • My email: [email protected]
hmps://www.facebook.com/OracleDatabase
hmp://www.oracle.com/goto/dbim.html
AddiQonal Resources
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 44