partitioning demystified

63
pr ligence Empowering Intelligence Partitioning Demystified Arup Nanda Proligence, Inc. Norwalk, CT

Upload: aria

Post on 13-Feb-2016

40 views

Category:

Documents


0 download

DESCRIPTION

Partitioning Demystified. Arup Nanda Proligence, Inc. Norwalk, CT. What’s It. Divide and Conquer The Best Thing Since Sliced Bread. Partition Views. View Created as SELECT some fields FROM Table1 UNION ALL SELECT same fields FROM Table2. Table A. View. Table B. Problem. - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Partitioning Demystified

pr ligence Empowering Intelligence

Partitioning Demystified

Arup NandaProligence, Inc.

Norwalk, CT

Page 2: Partitioning Demystified

pr ligence Empowering Intelligence

What’s It• Divide and Conquer• The Best Thing Since Sliced Bread

Page 3: Partitioning Demystified

pr ligence Empowering Intelligence

Partition ViewsView Created as

SELECT some fieldsFROM Table1UNION ALLSELECT same fieldsFROM Table2

Table A

Table B

View

Page 4: Partitioning Demystified

pr ligence Empowering Intelligence

ProblemWhile INSERTING

Table A

Table B

Row1 View

Page 5: Partitioning Demystified

pr ligence Empowering Intelligence

Real Partitions• Came Out with Oracle 8.0• Range Partitioned Only• Partition Key Column

– VALUES LESS THAN (CONSTANT)

Partition A

Partition B

Main Table

Page 6: Partitioning Demystified

pr ligence Empowering Intelligence

Hash Partitions• Where Range is Not Practical• Performance

Partition1

Partition2Row1 Hash

Func

Page 7: Partitioning Demystified

pr ligence Empowering Intelligence

Composite Partitioning• Partitions are subdivided• Range-Hash• Range-List Partition1

Partition2

Subpart1Subpart2

Subpart1Subpart2

Page 8: Partitioning Demystified

pr ligence Empowering Intelligence

List PartitioningSimilar to Range but Values are DiscretePARTITION BY LIST (STATE_CODE)(

PARTITION P1 VALUES (‘CT’,’NY’),PARTITION P2 VALUES (‘NJ’),PARTITION PM VALUES (DEFAULT)

)Multi-column not supported

Page 9: Partitioning Demystified

pr ligence Empowering Intelligence

Local Index• Partitioning Schemes

of Index and Table are Same

• Easy to Administer• REBUILD part by part

Table

Index

Page 10: Partitioning Demystified

pr ligence Empowering Intelligence

Global Indexes• Indexes That Span

Across All Rows of All Partitions

• Typically Used to Enforce Primary Keys

Index

Page 11: Partitioning Demystified

pr ligence Empowering Intelligence

Equi-PartitioningIf a Table and another Table

are partitioned in exactly same way with the same Partitioning Key

E.g.SALES on ORDER_DATEORDERS on ORDER_DATE

TableB

TableA

Page 12: Partitioning Demystified

pr ligence Empowering Intelligence

SALES

Page 13: Partitioning Demystified

pr ligence Empowering Intelligence

QTR1 QTR2 QTR3 DEFAULT

SALESPARTITIONS

Page 14: Partitioning Demystified

pr ligence Empowering Intelligence

DEFAULT

QTR1 QTR2 QTR3 DEFAULTCUST3

CUST2

CUST1

DEFAULT

CUST3

CUST2

CUST1

DEFAULT

CUST3

CUST2

CUST1

DEFAULT

CUST3

CUST2

CUST1

SALESPARTITIONS

SUBPARTITIONS

Page 15: Partitioning Demystified

pr ligence Empowering Intelligence

DEFAULT

QTR1 QTR2 QTR3 DEFAULTCUST3CUST2CUST1

DEFAULTCUST3CUST2CUST1

DEFAULTCUST3CUST2CUST1

DEFAULTCUST3CUST2CUST1

TAB

ANALYZE

Page 16: Partitioning Demystified

pr ligence Empowering Intelligence

DEFAULT

QTR1 QTR2 QTR3 DEFAULTCUST3CUST2

TAB

DEFAULTCUST3CUST2

DEFAULTCUST3CUST2CUST1

DEFAULTCUST3CUST2CUST1CUST1 CUST1

Page 17: Partitioning Demystified

pr ligence Empowering Intelligence

DEFAULT

QTR1 QTR2 QTR3 DEFAULTCUST3CUST2CUST1

DEFAULTCUST3CUST2CUST1

DEFAULTCUST3CUST2CUST1

DEFAULTCUST3CUST2CUST1

DEFAULT

QTR1 QTR2 QTR3 DEFAULTCUST3CUST2CUST1

DEFAULTCUST3CUST2CUST1

DEFAULTCUST3CUST2CUST1

DEFAULTCUST3CUST2CUST1

TAB

IND

TABLE

INDEX

replace

replace

Page 18: Partitioning Demystified

pr ligence Empowering Intelligence

DEFAULT

QTR1 QTR2 QTR3 DEFAULTCUST3CUST2

TAB

DEFAULTCUST3CUST2

DEFAULTCUST3CUST2

DEFAULTCUST3CUST2

DEFAULT

QTR1 QTR2 QTR3 DEFAULTCUST3CUST2

IND

DEFAULTCUST3CUST2

DEFAULTCUST3CUST2

DEFAULTCUST3CUST2

CUST1

CUST1

TABLE

INDEX

CUST1

CUST1

CUST1 CUST1

CUST1 CUST1

Page 19: Partitioning Demystified

pr ligence Empowering Intelligence

SubpartitioningSelect from a partition by

SELECT … FROM TAB1 PARTITION (P1);Select from a subpartition by

SELECT … FROM TAB1 SUBPARTITION (SP1);

Works for Insert/Update/Delete, SQL*LoaderINTO TABLE TAB1 SUBPARTITION (SP1)And Export, too TABLE=MYTAB1:SP1

Page 20: Partitioning Demystified

pr ligence Empowering Intelligence

Subpartitioning …DBA_SEGMENTS

SELECT * FROM DBA_SEGMENTSWHERE TABLE_NAME = ‘TAB1’AND PARTITION_NAME = ‘SP1’/

SEGMENT_TYPE

Page 21: Partitioning Demystified

pr ligence Empowering Intelligence

SubpartitionsSubpartition Templates

PARTITION BY RANGE (COL1)SUBPARTITION BY HASH (COL2)SUBAPARTITION TEMPLATE(

SUBPARTITION SP1 TABLESPACE T1,SUBPARTITION SP2 TABLESPACE T2

)(

PARTITION P1 VALUES LESS THAN (101),PARTITION P2 VALUES LESS THAN (201),

… P1_SP1, P1_SP2, P2_SP1, P2_SP2

Can’t Specify Storage (8i)Can specify Storage (9i) DBA_SUBPARTITION_TEMPLATES

Page 22: Partitioning Demystified

pr ligence Empowering Intelligence

Partition PruningPartition Based on Date – 1 partition per quarter

SELECT … FROM SALES WHERE ORDER_DATE = ‘1/1/2003’

Will search only the 2003 Q1 partion

ptest1 ins_ptest1; ptest2

Page 23: Partitioning Demystified

pr ligence Empowering Intelligence

Plan_Table RevisitedRelvant Columns

PARTITION_STARTPARTITION_STOPPARTITION_ID

The step id that decided the partition start and stop

FILTER_PREDICATESThe exact condition used to evaluate partitions

Page 24: Partitioning Demystified

pr ligence Empowering Intelligence

Dbms_xplan

dbms_xplan.display (‘plan_table’,/* plan table name */

NULL,/* statement id */

‘BASIC’,/* format */ ‘TYPICAL’,‘ALL’,‘SERIAL’ )

select * fromtable (

)

Page 25: Partitioning Demystified

pr ligence Empowering Intelligence

Partition Wise JoinsEqui-partioned tables

SALES partitioned on SALES_DATEREVENUE partitioned on BOOKED_DATESELECT … FROM SALES S, REVENUE RWHERE S.SALES_DATE =

R.BOOKED_DATE REVENUESALESPartition1Partition2Partition3

Partition1Partition2Partition3

•Ptest3a ptest3b

Page 26: Partitioning Demystified

pr ligence Empowering Intelligence

Partitionwise Join• Hash Partitioned Tables• Elimination will occur in Equality only, not in

range.

Ptest3ha ptest3hb

Page 27: Partitioning Demystified

pr ligence Empowering Intelligence

Character Values in Range

CREATE TABLE EMPLOYEE (…………)PARTITION BY RANGE (LAST_NAME)(PARTITION P1 VALUES LESS THAN (‘D%’),PARTITION P2 VALUES LESS THAN (‘M%’),PARTITION P3 VALUES LESS THAN (‘T%’),PARTITION PM VALUES LESS THAN (MAXVALUE))

ptest4a, ptest4b, ins_ptest4

Page 28: Partitioning Demystified

pr ligence Empowering Intelligence

Multi-column KeysCol1 col2--- ---- ----- 50 50 rec1150 150 rec2 50 150 rec3150 50 rec4101 50 rec5101 101 rec6101 150 rec7201 50 rec8201 101 rec9201 201 rec10

p1p2p2p2p2p2

partition by range (col1, col2)

(partition p1 values less than (101, 101),

partition p2 values less than (201, 201),

partition pm values less than (maxvalue, maxvalue))

ptab1

p2pmpmpm

Page 29: Partitioning Demystified

pr ligence Empowering Intelligence

Multi-Column DecisionConsider

1st Column<

BoundaryValue

= Boundary

Value

1st Partition Consider 2nd Column

< Boundary

Value2nd Partition 3rd Partition

Y

N

Y

NY

Page 30: Partitioning Demystified

pr ligence Empowering Intelligence

Converting to Partitioning

Oracle Recommended (MetaLink Note 1070693.6)

• Create the partitoned table and Insert• Create Table As Select (CTAS)• Create Small Tables and Exchange Partitions• PROBLEM: Space Requirement and Time

Page 31: Partitioning Demystified

pr ligence Empowering Intelligence

Alternatives• Oracle 9i Online Redefinition

– Small Downtime– Space Needed– Oracle 9i

Page 32: Partitioning Demystified

pr ligence Empowering Intelligence

Split-Split Method• Create partitioned table exactly same as source

table with one partition with MAXVALUE• Exchange source table with this partition• Split this partition at the lowest boundary• Split the maximum partition at the next boundary• Repeat till all partitions are created

Page 33: Partitioning Demystified

pr ligence Empowering Intelligence

ExampleTable NOPART

COL1 NUMBERCOL2 VARCHAR2(10)COL3 CHAR(2)

Index IN_NOPART(COL2)

Constraint CK_NOPART(COL3 IS NOT NULL)

Partitioned BY RANGE (COL1)P1 LESS THAN (101)P2 LESS THAN (201)P3 LESS THAN (301)P4 LESS THAN (401)PM … (MAXVALUE)

Table PARTTable, Index, Constraint

Name Change

Page 34: Partitioning Demystified

pr ligence Empowering Intelligence

Summary• Create table part …

• Exchange partition pMAX with table NOPART

• Split partition pMAX repeatedly

• Drop original table nopart

• Rename part to no part

• Rename index to in_nopart

• Rename constraint to ck_nopart

Page 35: Partitioning Demystified

pr ligence Empowering Intelligence

Pros & Cons• Space Needs – Less• Redo Log Generation – Minimized• Time – Less• Concurrency – Higher• Time – Still High• Concurency – Still Low

Page 36: Partitioning Demystified

pr ligence Empowering Intelligence

Exchanging PartitionsMain Table

Partition p1Subpartition sp1Subpartition sp2Subpartition sp3

),

Source Tablepartition p1partition p2partition p3

Page 37: Partitioning Demystified

pr ligence Empowering Intelligence

Subpartition Statistics• DBMS_STATS.GATHER_TABLE_STATS

– tabname => ‘MYTABLE’– partname => ‘P1’– granularity =>

• DEFAULT• GLOBAL• PARTITION• SUBPARTITION• ALL

Spart1.sql

Page 38: Partitioning Demystified

pr ligence Empowering Intelligence

Statistics Collection

Granularity Table Global

Partition Global

Partition Stats

Subpart Stats

GLOBAL YES NO NO NO

PARTITION NO YES YES NO

DEFAULT YES YES YES NO

SUBPARTITION NO NO YES YES

ALL YES YES YES YES

Page 39: Partitioning Demystified

pr ligence Empowering Intelligence

Parallel Index Rebuilding

– One Query Server per Partition– Slows down processing– Can’t exploit the Parallel Processing Capabilities

q1 q2

QC

q3 q4

Page 40: Partitioning Demystified

pr ligence Empowering Intelligence

Parallel Index Rebuilding

DBMS_PCLXUTIL.BUILD_PART_INDEXjobs_per_batch NUMBER DEFAULT 1procs_per_job NUMBER DEFAULT 1tab_name VARCHAR2idx_name VARCHAR2force_opt BOOLEAN DEFAULT FALSE

Page 41: Partitioning Demystified

pr ligence Empowering Intelligence

The Rule Based Optimizer

• Invokes CBO• Makes Up Statistics• Don’t Use Partitioning if RBO is Used

Page 42: Partitioning Demystified

pr ligence Empowering Intelligence

Coalesce –vs- Merge• Coalesce – Hash Partitions• Merge – Range and List

coalesce

merge

Page 43: Partitioning Demystified

pr ligence Empowering Intelligence

A Real Life Case• Sales Datawarehouse• Several Customers• Several Quarters• Data Coming from Customers • Irregular Frequency• Data Comes Often Late

Page 44: Partitioning Demystified

pr ligence Empowering Intelligence

Datawarehouse

Cust1

Cust2

Cust3

Cust4

Cust10

Cust9

Cust8

Cust7

Cust11 ?

Cust5 Cust6

DB3 DB6DB4 DB5

DB1 DB2

Page 45: Partitioning Demystified

pr ligence Empowering Intelligence

DetailTable

SummaryTable

DetailTable

CUST2

Problem of Irregular Data

Page 46: Partitioning Demystified

pr ligence Empowering Intelligence

CUST1

Summary tables

DE

TAIL TA

BLE

S

CUST2

CUST2CUST1

aggregation

Page 47: Partitioning Demystified

pr ligence Empowering Intelligence

Problems• Data Coming from Customers Irregularly• Need to Refresh the Summary Tables when

New Data Arrives• Quarters are Added Continuosly• Archival Requirements Vary Across Customers• Quick Retrieval of Archival Needed

Page 48: Partitioning Demystified

pr ligence Empowering Intelligence

Problems contd.• Summary on Summary Tables as Materialized

Views• Needed to be Refreshed whenever New Data

Arrived• Or When Data is Purged/Reinstated• Customers Added and Deleted Frequently

Page 49: Partitioning Demystified

pr ligence Empowering Intelligence

Objective• To Minimize Downtime for Refreshes

– Incrementally Refresh– Partitioning Techniques

• To Add Customers Easily• To Add Quarters Easily• To Archive Off and Purge Easily and Atomically• To Restore Archives Quickly

Page 50: Partitioning Demystified

pr ligence Empowering Intelligence

Design• Varying Dimensions –

– Customer – Quarter

• Composite Partitioning– Range (for Quarters)– List (for Customers)

• Local Indexes

Page 51: Partitioning Demystified

pr ligence Empowering Intelligence

Design contd…• Mat Views Created As Tables First• Query Rewrite and Stale Tolerated• Same Partitioning as Parent Tables

– Partition-Wise Joins– Partition Pruning– Partition Independence

Page 52: Partitioning Demystified

pr ligence Empowering Intelligence

cust1

SummaryTable

DW

View

Owned byCust

Schema

Temporary

Table

Index ofTempora

ryTable

MassagingAnalyzing

Filter:Where SALES_DATE

is in that quarter

INDEX

TABLE

For Customer cust1 and

Quarter Q1

Page 53: Partitioning Demystified

pr ligence Empowering Intelligence

cust

DW

View

Old Sub Partition

Old Sub Partition

INDEX

TABLE

Page 54: Partitioning Demystified

pr ligence Empowering Intelligence

DesignEach Subpartition – of Index or Table is kept in

seprate tablespaces named in the formatS<n>_<Cust>_Y<Year>Q<Qtr>E.g. S1_CUST1_Y02Q2

Page 55: Partitioning Demystified

pr ligence Empowering Intelligence

subpa

rtition

spartitions

PARENT

MV1

MV2

TableSpace1 TableSpace2

Page 56: Partitioning Demystified

pr ligence Empowering Intelligence

Design …MV_* Subpartitions are on the same tablespace as

the parents.

Subparts of MV_S1_0? are in the same TS as S1Subparts of MV_S2_0? in S2

Page 57: Partitioning Demystified

pr ligence Empowering Intelligence

Qtr1 Qtr2 Qtr3 DEF

Cust1

Cust2

Cust3

DEF

Page 58: Partitioning Demystified

pr ligence Empowering Intelligence

Qtr1 Qtr2 Qtr3 DEF

Cust1

Cust2

Cust3

DEF

Page 59: Partitioning Demystified

pr ligence Empowering Intelligence

Qtr1 Qtr2 Qtr3 DEF

Cust1

Cust2

Cust3

DEF

Cust4

Page 60: Partitioning Demystified

pr ligence Empowering Intelligence

Qtr1 Qtr2 Qtr3 DEF

Cust1

Cust2

Cust3

DEF

Page 61: Partitioning Demystified

pr ligence Empowering Intelligence

Qtr1 Qtr2 Qtr3 Qtr4

Cust1

Cust2

Cust3

DEF

DEF

Page 62: Partitioning Demystified

pr ligence Empowering Intelligence

Objectives Revisited• To Minimize Downtime for Refreshes

– Incrementally Refresh– Partitioning Techniques

• To Add Customers Easily• To Add Quarters Easily• To Archive Off and Purge Easily and Atomically• To Restore Archives Quickly

Page 63: Partitioning Demystified

pr ligence Empowering Intelligence

Thank you!Thank you!

www.proligence.cowww.proligence.comm