Download - Partitioning Demystified
pr ligence Empowering Intelligence
Partitioning Demystified
Arup NandaProligence, Inc.
Norwalk, CT
pr ligence Empowering Intelligence
What’s It• Divide and Conquer• The Best Thing Since Sliced Bread
pr ligence Empowering Intelligence
Partition ViewsView Created as
SELECT some fieldsFROM Table1UNION ALLSELECT same fieldsFROM Table2
Table A
Table B
View
pr ligence Empowering Intelligence
ProblemWhile INSERTING
Table A
Table B
Row1 View
pr ligence Empowering Intelligence
Real Partitions• Came Out with Oracle 8.0• Range Partitioned Only• Partition Key Column
– VALUES LESS THAN (CONSTANT)
Partition A
Partition B
Main Table
pr ligence Empowering Intelligence
Hash Partitions• Where Range is Not Practical• Performance
Partition1
Partition2Row1 Hash
Func
pr ligence Empowering Intelligence
Composite Partitioning• Partitions are subdivided• Range-Hash• Range-List Partition1
Partition2
Subpart1Subpart2
Subpart1Subpart2
pr ligence Empowering Intelligence
List PartitioningSimilar to Range but Values are DiscretePARTITION BY LIST (STATE_CODE)(
PARTITION P1 VALUES (‘CT’,’NY’),PARTITION P2 VALUES (‘NJ’),PARTITION PM VALUES (DEFAULT)
)Multi-column not supported
pr ligence Empowering Intelligence
Local Index• Partitioning Schemes
of Index and Table are Same
• Easy to Administer• REBUILD part by part
Table
Index
pr ligence Empowering Intelligence
Global Indexes• Indexes That Span
Across All Rows of All Partitions
• Typically Used to Enforce Primary Keys
Index
pr ligence Empowering Intelligence
Equi-PartitioningIf a Table and another Table
are partitioned in exactly same way with the same Partitioning Key
E.g.SALES on ORDER_DATEORDERS on ORDER_DATE
TableB
TableA
pr ligence Empowering Intelligence
SALES
pr ligence Empowering Intelligence
QTR1 QTR2 QTR3 DEFAULT
SALESPARTITIONS
pr ligence Empowering Intelligence
DEFAULT
QTR1 QTR2 QTR3 DEFAULTCUST3
CUST2
CUST1
DEFAULT
CUST3
CUST2
CUST1
DEFAULT
CUST3
CUST2
CUST1
DEFAULT
CUST3
CUST2
CUST1
SALESPARTITIONS
SUBPARTITIONS
pr ligence Empowering Intelligence
DEFAULT
QTR1 QTR2 QTR3 DEFAULTCUST3CUST2CUST1
DEFAULTCUST3CUST2CUST1
DEFAULTCUST3CUST2CUST1
DEFAULTCUST3CUST2CUST1
TAB
ANALYZE
pr ligence Empowering Intelligence
DEFAULT
QTR1 QTR2 QTR3 DEFAULTCUST3CUST2
TAB
DEFAULTCUST3CUST2
DEFAULTCUST3CUST2CUST1
DEFAULTCUST3CUST2CUST1CUST1 CUST1
pr ligence Empowering Intelligence
DEFAULT
QTR1 QTR2 QTR3 DEFAULTCUST3CUST2CUST1
DEFAULTCUST3CUST2CUST1
DEFAULTCUST3CUST2CUST1
DEFAULTCUST3CUST2CUST1
DEFAULT
QTR1 QTR2 QTR3 DEFAULTCUST3CUST2CUST1
DEFAULTCUST3CUST2CUST1
DEFAULTCUST3CUST2CUST1
DEFAULTCUST3CUST2CUST1
TAB
IND
TABLE
INDEX
replace
replace
pr ligence Empowering Intelligence
DEFAULT
QTR1 QTR2 QTR3 DEFAULTCUST3CUST2
TAB
DEFAULTCUST3CUST2
DEFAULTCUST3CUST2
DEFAULTCUST3CUST2
DEFAULT
QTR1 QTR2 QTR3 DEFAULTCUST3CUST2
IND
DEFAULTCUST3CUST2
DEFAULTCUST3CUST2
DEFAULTCUST3CUST2
CUST1
CUST1
TABLE
INDEX
CUST1
CUST1
CUST1 CUST1
CUST1 CUST1
pr ligence Empowering Intelligence
SubpartitioningSelect from a partition by
SELECT … FROM TAB1 PARTITION (P1);Select from a subpartition by
SELECT … FROM TAB1 SUBPARTITION (SP1);
Works for Insert/Update/Delete, SQL*LoaderINTO TABLE TAB1 SUBPARTITION (SP1)And Export, too TABLE=MYTAB1:SP1
pr ligence Empowering Intelligence
Subpartitioning …DBA_SEGMENTS
SELECT * FROM DBA_SEGMENTSWHERE TABLE_NAME = ‘TAB1’AND PARTITION_NAME = ‘SP1’/
SEGMENT_TYPE
pr ligence Empowering Intelligence
SubpartitionsSubpartition Templates
PARTITION BY RANGE (COL1)SUBPARTITION BY HASH (COL2)SUBAPARTITION TEMPLATE(
SUBPARTITION SP1 TABLESPACE T1,SUBPARTITION SP2 TABLESPACE T2
)(
PARTITION P1 VALUES LESS THAN (101),PARTITION P2 VALUES LESS THAN (201),
… P1_SP1, P1_SP2, P2_SP1, P2_SP2
Can’t Specify Storage (8i)Can specify Storage (9i) DBA_SUBPARTITION_TEMPLATES
pr ligence Empowering Intelligence
Partition PruningPartition Based on Date – 1 partition per quarter
SELECT … FROM SALES WHERE ORDER_DATE = ‘1/1/2003’
Will search only the 2003 Q1 partion
ptest1 ins_ptest1; ptest2
pr ligence Empowering Intelligence
Plan_Table RevisitedRelvant Columns
PARTITION_STARTPARTITION_STOPPARTITION_ID
The step id that decided the partition start and stop
FILTER_PREDICATESThe exact condition used to evaluate partitions
pr ligence Empowering Intelligence
Dbms_xplan
dbms_xplan.display (‘plan_table’,/* plan table name */
NULL,/* statement id */
‘BASIC’,/* format */ ‘TYPICAL’,‘ALL’,‘SERIAL’ )
select * fromtable (
)
pr ligence Empowering Intelligence
Partition Wise JoinsEqui-partioned tables
SALES partitioned on SALES_DATEREVENUE partitioned on BOOKED_DATESELECT … FROM SALES S, REVENUE RWHERE S.SALES_DATE =
R.BOOKED_DATE REVENUESALESPartition1Partition2Partition3
Partition1Partition2Partition3
•Ptest3a ptest3b
pr ligence Empowering Intelligence
Partitionwise Join• Hash Partitioned Tables• Elimination will occur in Equality only, not in
range.
Ptest3ha ptest3hb
pr ligence Empowering Intelligence
Character Values in Range
CREATE TABLE EMPLOYEE (…………)PARTITION BY RANGE (LAST_NAME)(PARTITION P1 VALUES LESS THAN (‘D%’),PARTITION P2 VALUES LESS THAN (‘M%’),PARTITION P3 VALUES LESS THAN (‘T%’),PARTITION PM VALUES LESS THAN (MAXVALUE))
ptest4a, ptest4b, ins_ptest4
pr ligence Empowering Intelligence
Multi-column KeysCol1 col2--- ---- ----- 50 50 rec1150 150 rec2 50 150 rec3150 50 rec4101 50 rec5101 101 rec6101 150 rec7201 50 rec8201 101 rec9201 201 rec10
p1p2p2p2p2p2
partition by range (col1, col2)
(partition p1 values less than (101, 101),
partition p2 values less than (201, 201),
partition pm values less than (maxvalue, maxvalue))
ptab1
p2pmpmpm
pr ligence Empowering Intelligence
Multi-Column DecisionConsider
1st Column<
BoundaryValue
= Boundary
Value
1st Partition Consider 2nd Column
< Boundary
Value2nd Partition 3rd Partition
Y
N
Y
NY
pr ligence Empowering Intelligence
Converting to Partitioning
Oracle Recommended (MetaLink Note 1070693.6)
• Create the partitoned table and Insert• Create Table As Select (CTAS)• Create Small Tables and Exchange Partitions• PROBLEM: Space Requirement and Time
pr ligence Empowering Intelligence
Alternatives• Oracle 9i Online Redefinition
– Small Downtime– Space Needed– Oracle 9i
pr ligence Empowering Intelligence
Split-Split Method• Create partitioned table exactly same as source
table with one partition with MAXVALUE• Exchange source table with this partition• Split this partition at the lowest boundary• Split the maximum partition at the next boundary• Repeat till all partitions are created
pr ligence Empowering Intelligence
ExampleTable NOPART
COL1 NUMBERCOL2 VARCHAR2(10)COL3 CHAR(2)
Index IN_NOPART(COL2)
Constraint CK_NOPART(COL3 IS NOT NULL)
Partitioned BY RANGE (COL1)P1 LESS THAN (101)P2 LESS THAN (201)P3 LESS THAN (301)P4 LESS THAN (401)PM … (MAXVALUE)
Table PARTTable, Index, Constraint
Name Change
pr ligence Empowering Intelligence
Summary• Create table part …
• Exchange partition pMAX with table NOPART
• Split partition pMAX repeatedly
• Drop original table nopart
• Rename part to no part
• Rename index to in_nopart
• Rename constraint to ck_nopart
pr ligence Empowering Intelligence
Pros & Cons• Space Needs – Less• Redo Log Generation – Minimized• Time – Less• Concurrency – Higher• Time – Still High• Concurency – Still Low
pr ligence Empowering Intelligence
Exchanging PartitionsMain Table
Partition p1Subpartition sp1Subpartition sp2Subpartition sp3
),
…
Source Tablepartition p1partition p2partition p3
pr ligence Empowering Intelligence
Subpartition Statistics• DBMS_STATS.GATHER_TABLE_STATS
– tabname => ‘MYTABLE’– partname => ‘P1’– granularity =>
• DEFAULT• GLOBAL• PARTITION• SUBPARTITION• ALL
Spart1.sql
pr ligence Empowering Intelligence
Statistics Collection
Granularity Table Global
Partition Global
Partition Stats
Subpart Stats
GLOBAL YES NO NO NO
PARTITION NO YES YES NO
DEFAULT YES YES YES NO
SUBPARTITION NO NO YES YES
ALL YES YES YES YES
pr ligence Empowering Intelligence
Parallel Index Rebuilding
– One Query Server per Partition– Slows down processing– Can’t exploit the Parallel Processing Capabilities
q1 q2
QC
q3 q4
pr ligence Empowering Intelligence
Parallel Index Rebuilding
DBMS_PCLXUTIL.BUILD_PART_INDEXjobs_per_batch NUMBER DEFAULT 1procs_per_job NUMBER DEFAULT 1tab_name VARCHAR2idx_name VARCHAR2force_opt BOOLEAN DEFAULT FALSE
pr ligence Empowering Intelligence
The Rule Based Optimizer
• Invokes CBO• Makes Up Statistics• Don’t Use Partitioning if RBO is Used
pr ligence Empowering Intelligence
Coalesce –vs- Merge• Coalesce – Hash Partitions• Merge – Range and List
coalesce
merge
pr ligence Empowering Intelligence
A Real Life Case• Sales Datawarehouse• Several Customers• Several Quarters• Data Coming from Customers • Irregular Frequency• Data Comes Often Late
pr ligence Empowering Intelligence
Datawarehouse
Cust1
Cust2
Cust3
Cust4
Cust10
Cust9
Cust8
Cust7
Cust11 ?
Cust5 Cust6
DB3 DB6DB4 DB5
DB1 DB2
pr ligence Empowering Intelligence
DetailTable
SummaryTable
DetailTable
CUST2
Problem of Irregular Data
pr ligence Empowering Intelligence
CUST1
Summary tables
DE
TAIL TA
BLE
S
CUST2
CUST2CUST1
aggregation
pr ligence Empowering Intelligence
Problems• Data Coming from Customers Irregularly• Need to Refresh the Summary Tables when
New Data Arrives• Quarters are Added Continuosly• Archival Requirements Vary Across Customers• Quick Retrieval of Archival Needed
pr ligence Empowering Intelligence
Problems contd.• Summary on Summary Tables as Materialized
Views• Needed to be Refreshed whenever New Data
Arrived• Or When Data is Purged/Reinstated• Customers Added and Deleted Frequently
pr ligence Empowering Intelligence
Objective• To Minimize Downtime for Refreshes
– Incrementally Refresh– Partitioning Techniques
• To Add Customers Easily• To Add Quarters Easily• To Archive Off and Purge Easily and Atomically• To Restore Archives Quickly
pr ligence Empowering Intelligence
Design• Varying Dimensions –
– Customer – Quarter
• Composite Partitioning– Range (for Quarters)– List (for Customers)
• Local Indexes
pr ligence Empowering Intelligence
Design contd…• Mat Views Created As Tables First• Query Rewrite and Stale Tolerated• Same Partitioning as Parent Tables
– Partition-Wise Joins– Partition Pruning– Partition Independence
pr ligence Empowering Intelligence
cust1
SummaryTable
DW
View
Owned byCust
Schema
Temporary
Table
Index ofTempora
ryTable
MassagingAnalyzing
Filter:Where SALES_DATE
is in that quarter
INDEX
TABLE
For Customer cust1 and
Quarter Q1
pr ligence Empowering Intelligence
cust
DW
View
Old Sub Partition
Old Sub Partition
INDEX
TABLE
pr ligence Empowering Intelligence
DesignEach Subpartition – of Index or Table is kept in
seprate tablespaces named in the formatS<n>_<Cust>_Y<Year>Q<Qtr>E.g. S1_CUST1_Y02Q2
pr ligence Empowering Intelligence
subpa
rtition
spartitions
PARENT
MV1
MV2
TableSpace1 TableSpace2
pr ligence Empowering Intelligence
Design …MV_* Subpartitions are on the same tablespace as
the parents.
Subparts of MV_S1_0? are in the same TS as S1Subparts of MV_S2_0? in S2
pr ligence Empowering Intelligence
Qtr1 Qtr2 Qtr3 DEF
Cust1
Cust2
Cust3
DEF
pr ligence Empowering Intelligence
Qtr1 Qtr2 Qtr3 DEF
Cust1
Cust2
Cust3
DEF
pr ligence Empowering Intelligence
Qtr1 Qtr2 Qtr3 DEF
Cust1
Cust2
Cust3
DEF
Cust4
pr ligence Empowering Intelligence
Qtr1 Qtr2 Qtr3 DEF
Cust1
Cust2
Cust3
DEF
pr ligence Empowering Intelligence
Qtr1 Qtr2 Qtr3 Qtr4
Cust1
Cust2
Cust3
DEF
DEF
pr ligence Empowering Intelligence
Objectives Revisited• To Minimize Downtime for Refreshes
– Incrementally Refresh– Partitioning Techniques
• To Add Customers Easily• To Add Quarters Easily• To Archive Off and Purge Easily and Atomically• To Restore Archives Quickly
pr ligence Empowering Intelligence
Thank you!Thank you!
www.proligence.cowww.proligence.comm