flatcubes and inmemory option for sap bw · copyright © 2015, oracle and/or its affiliates. all...
Post on 10-Mar-2020
8 Views
Preview:
TRANSCRIPT
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
FlatCubes and InMemory Option for SAP BW Jörn Bartels Architect - SAP Development Oracle Deutschland
Oracle Confidential – Internal
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Agenda
Star Cube
Flat Cube
Bloom Filters
Group By and Aggregate push down
Group by Placement
Results
1
2
3
Oracle Confidential – Internal/Restricted/Highly Restricted 2
4
5
6
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Flat Tables
Oracle Confidential – Internal
Facts
Masterdata
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Star Cube (old Oracle - SAP BW Standard)
Oracle Confidential – Internal
Fact E/F
Masterdata
Dimensions
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Agenda
Star Cube
Flat Cube
Bloom Filters
Group By and Aggregate push down
Group By Placement
Results
1
2
3
Oracle Confidential – Internal/Restricted/Highly Restricted 5
4
5
6
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Flat Cube
Oracle Confidential – Internal
Facts
Masterdata
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Disksizes Star - Flat
0
5000
10000
15000
20000
star star comp flat flat comp
Table
Indexes
Dimensions
Sum
Oracle Confidential – Internal
8 Dimensisons 188 Values Comp – OLTP compression
MB
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Disksizes Star - Flat
0
1000
2000
3000
4000
5000
6000
star star comp flat flat comp
Table
Indexes
Dimensions
Sum
Oracle Confidential – Internal
14 Dimensions 6 Values Comp – OLTP compression
MB
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
F
Flat Cubes and Partitioning Old Implementation (Classic Star Schema)
9
F Table Partition 6
F Table Partition 5
F Table Partition 4
F Table Partition 3
F Table Partition 2
F Table Partition 1
F FACT TABLE
F
E FACT TABLE
E Table Partition 6
E Table Partition 5
E Table Partition 4
E Table Partition 3
E Table Partition 2
E Table Partition 1
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
• New Implementation (Non-Oracle Databases) – The F table goes away
– Data from F table now part of E table
• Major disadvantage – No partition pruning on the larges part of the data
possible for queries
Flat Cubes and Partitionierung
10
F Table Partition 6
F Table Partition 5
F Table Partition 4
F Table Partition 3
F Table Partition 2
F Table Partition 1
"E Table" (Partition 0)
E FACT TABLE
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
• New Implementation (Oracle Databases) – The F table goes away
– Data from F table now part of E table
• Oracle Advantage – Partition pruning possible for queries due to
subpartitioning
Flat Cubes and Partitionierung
11
F Table Partition 6
F Table Partition 5
F Table Partition 4
F Table Partition 3
F Table Partition 2
F Table Partition 1
E FACT TABLE
E Partition, Subpartition 6
E Partition, Subpartition 5
E Partition, Subpartition 4
E Partition, Subpartition 3
E Partition, Subpartition 2
E Partition, Subpartition 1
Partition 0
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Plan Startransformation
Oracle Confidential – Internal
---------------------------------------------------------------------------------
| Id | Operation | Name |
---------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | |
| 1 | SORT GROUP BY | |
|* 2 | HASH JOIN | |
|* 3 | TABLE ACCESS STORAGE FULL | /BI0/YVC_PROD2 |
|* 4 | HASH JOIN | |
| 5 | TABLE ACCESS STORAGE FULL | /BIC/DZMK_TCSC2 |
|* 6 | HASH JOIN | |
| 7 | TABLE ACCESS STORAGE FULL | /BIC/DZMK_TCSC4 |
|* 8 | HASH JOIN | |
| 9 | TABLE ACCESS STORAGE FULL | /BIC/DZMK_TCSCU |
|* 10 | HASH JOIN | |
| 11 | TABLE ACCESS BY INDEX ROWID BATCHED | /BIC/DZMK_TCSCP |
|* 12 | INDEX RANGE SCAN | /BIC/DZMK_TCSCP~01 |
|* 13 | HASH JOIN | |
| 14 | TABLE ACCESS STORAGE FULL | /BIC/DZMK_TCSC5 |
|* 15 | HASH JOIN | |
|* 16 | TABLE ACCESS STORAGE FULL | /BIC/DZMK_TCSCT |
| 17 | PARTITION RANGE ITERATOR | |
|* 18 | TABLE ACCESS BY LOCAL INDEX ROWID BATCHED| /BIC/EZMK_TCSC |
| 19 | BITMAP CONVERSION TO ROWIDS | |
| 20 | BITMAP AND | |
| 21 | BITMAP MERGE | |
| 22 | BITMAP KEY ITERATION | |
| 23 | BUFFER SORT | |
|* 24 | TABLE ACCESS STORAGE FULL | /BIC/DZMK_TCSCT |
|* 25 | BITMAP INDEX RANGE SCAN | /BIC/EZMK_TCSC~020 |
| 26 | BITMAP MERGE | |
| 27 | BITMAP KEY ITERATION | |
| 28 | BUFFER SORT | |
| 29 | INDEX STORAGE FAST FULL SCAN | /BIC/DZMK_TCSC4~0 |
|* 30 | BITMAP INDEX RANGE SCAN | /BIC/EZMK_TCSC~070 |
| 31 | BITMAP MERGE | |
| 32 | BITMAP KEY ITERATION | |
| 33 | BUFFER SORT | |
|* 34 | TABLE ACCESS STORAGE FULL | /BIC/DZMK_TCSC3 |
|* 35 | BITMAP INDEX RANGE SCAN | /BIC/EZMK_TCSC~060 |
---------------------------------------------------------------------------------
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Plan Flat Cube
Oracle Confidential – Internal
-----------------------------------------------------------------------
| Id | Operation | Name |
-----------------------------------------------------------------------
| 0 | SELECT STATEMENT | |
| 1 | SORT GROUP BY | |
|* 2 | HASH JOIN | |
| 3 | JOIN FILTER CREATE | :BF0001 |
| 4 | PART JOIN FILTER CREATE | :BF0000 |
| 5 | NESTED LOOPS | |
| 6 | TABLE ACCESS BY INDEX ROWID BATCHED| /BIC/DZMK_TCSCP |
|* 7 | INDEX RANGE SCAN | /BIC/DZMK_TCSCP~01 |
|* 8 | TABLE ACCESS STORAGE FULL | /BI0/YVC_PROD2 |
| 9 | JOIN FILTER USE | :BF0001 |
| 10 | PARTITION LIST JOIN-FILTER | |
| 11 | PARTITION RANGE ITERATOR | |
|* 12 | TABLE ACCESS INMEMORY FULL | /BIC/FZMK_TCFL |
-----------------------------------------------------------------------
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Agenda
Star Cube
Flat Cube
Bloom Filters
Group By and Aggregate push down
Group By Placement
Results
1
2
3
Oracle Confidential – Internal/Restricted/Highly Restricted 14
4
5
6
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Joining and Combining Data
• Converts joins of data in multiple tables into fast column scans
• Joins tables 10x faster
15
Example: Find total sales in outlet stores
Sales Stores
Sto
re ID
StoreID in 15, 38, 64
Type=‘Outlet’
Typ
e
Sum
Sto
re ID
Am
ou
nt
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 16
Bloom Filter
Hash Join
Bit vector
0 0 0 1 1 1 1 STORES
Bloom Filter Creation: A bit vector is created that has a bit set for each join column there is a value for & 0 where there is no value Bloom Filter
created
2
SALES
3. Table Scan: SALES table is scan and rows are filtered based on query predicates
4
Bloom Filter sent: BIT VECTOR is sent as an additional filter criteria to the scan of the sales table
3
Reduced row sent: Only rows that have a match in the bit vector get sent to the hash join
5
Hash Join: Join completed by probing into the hash table from the store_id to find actual matching rows
6
PGA
1. Table Scan: STORES table is scanned and matching rows sent to hash join
1
Typ
e
Sto
re ID
Sto
re ID
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Identifying : INMEMORY Joins • Bloom filters enable joins to be
converted into fast column scans
• Tried and true technology originally released in 10g
• Same technique used to offload joins on Exadata
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Multiple bloom filter
• Multi-table joins are common in analytic queries
• Possible to apply multiple bloom filters to the fact table due to sophisticated Optimizer
• Simultaneously joining to all of the dimension tables while scanning the fact table
Select d.d_year, s.s_nation,
sum(lo_revenue - lo_supplycost)
From LINEORDER l, DATE_DIM d,
PART p, SUPPLIER s
Where l.lo_orderdate = d.d_datekey
And l.lo_partkey = p.p_partkey
And l.lo_suppkey = s.s_suppkey
And p.p_mfgr= 'MFGR#12’
And s.s_region = 'AMERICA’
Group by d.d_year, s.s_nation
Order by d.d_year, s.s_nation;
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Multiple Bloom Filter
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Standard Left Deep Tree Plan: How Can Multiple Bloom Filters Be Possible?
20
PART
HASH JOIN
LINEORDER
HASH JOIN
SUPPLIER
HASH JOIN
DATEDIM
3
4
1
2
3
1
Bloom filter created
2 Bloom filter used
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Multiple Bloom Filters Possible Due To Swap Join Input Optimization and Right Deep Trees
21
PART
HASH JOIN
LINEORDER
HASH JOIN
SUPPLIER
HASH JOIN
DATEDIM
1
2
3
4
1
2
3
Why is this significant?
Applying multiple filters on initial scan of LINEORDER reduces the number of rows sent upstream in the plan
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Multiple Bloom Filter
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Agenda
Star Cube
Flat Cube
Bloom Filters
Group By and Aggregate push down
Group By Placement
Results
1
2
3
Oracle Confidential – Internal/Restricted/Highly Restricted 23
4
5
6
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Simple grouping Query
• SELECT "F"."SID_0MAT_SALES", SUM ("F"."/BIC/INV_QTY_T") FROM "/BIC/8LADFRGACT" "F" GROUP BY "F"."SID_0MAT_SALES“ -----------------------------------------------------------------
| Id | Operation | Name | Rows |
-----------------------------------------------------------------
| 0 | SELECT STATEMENT | | 46336 |
| 1 | HASH GROUP BY | | 46336 |
| 2 | TABLE ACCESS INMEMORY FULL | /BIC/8LADFRGACT | 93M|
-----------------------------------------------------------------
Oracle Confidential – Internal
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Inmemory Statistics
• NAME VALUE --------------------------------------- -----------
IM scan CUs memcompress for query low 227
IM scan rows 93541477
IM scan rows projected 1147304
Oracle Confidential – Internal
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Agenda
Star Cube
Flat Cube
Bloom Filters
Group By and Aggregate push down
Group By Placement
Results
1
2
3
Oracle Confidential – Internal/Restricted/Highly Restricted 26
4
5
6
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Group by placement
• SELECT "S21"."MAT_SALES", SUM ("F"."/BIC/INV_QTY_T") FROM "/BIC/8LADFRGACT" "F"
JOIN "/BI0/SMAT_SALES" "S21"
ON "F"."SID_0MAT_SALES" = "S21"."SID"
GROUP BY "S21"."MAT_SALES"
Oracle Confidential – Internal
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Group By Placement
• WITH "F" AS (SELECT /*+ no_merge */ "F"."SID_0MAT_SALES",
SUM ("F"."/BIC/INV_QTY_T")
AS "F"."/BIC/INV_QTY_T"
FROM "/BIC/8LADFRGACT" "F"
GROUP BY "F"."SID_0MAT_SALES")
SELECT "S21"."MAT_SALES", SUM ("F"."/BIC/INV_QTY_T")
FROM "F"
JOIN "/BI0/SMAT_SALES" "S21"
ON "F" . "SID_0MAT_SALES" = "S21" . "SID"
GROUP BY "S21"."MAT_SALES"
Oracle Confidential – Internal
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Group By Placement • ---------------------------------------------------------------------
| Id | Operation | Name | Rows |
---------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 46336 |
| 1 | HASH GROUP BY | | 46336 |
|* 2 | HASH JOIN | | 46336 |
| 3 | JOIN FILTER CREATE | :BF0000 | 46336 |
| 4 | VIEW | VW_GBC_5 | 46336 |
| 5 | HASH GROUP BY | | 46336 |
| 6 | TABLE ACCESS INMEMORY FULL | /BIC/8LADFRGACT | 93M|
| 7 | JOIN FILTER USE | :BF0000 | 2609K|
|* 8 | TABLE ACCESS INMEMORY FULL | /BI0/SMAT_SALES | 2609K|
---------------------------------------------------------------------
Oracle Confidential – Internal
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Agenda
Star Cube
Flat Cube
Bloom Filters
Group By and Aggregate push down
Group By Placement
Results
1
2
3
Oracle Confidential – Internal/Restricted/Highly Restricted 30
4
5
6
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Star Schema: 9 Dimensions / 61 Measurements / 2 Join Dims. Flat Schema: 40 Keyvalues / 61 Measurements / 1 Join Dim.
Oracle Confidential – Internal/Restricted/Highly Restricted 31
Simple Query 1.2 Million Rows
GB
Query:
1 Result column 1 Attribute table
Flat cube / Inmemory: Independent of table width Cheap access for low number of columns
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Star Schema: 8 Dimensions / 145Measurements / 8 Join Dims. Flat Schema: 37 Keyvalues / 145 Measurements / 1 Join Dim.
Oracle Confidential – Internal/Restricted/Highly Restricted 32
Medium Query 6.5 Million Rows
GB
Query:
1 Result column 2 Attribute tables
Flat cube / Inmemory: Independent of table width Cheap access for low number of columns
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Star Schema: 12 Dimensions / 129 Measurements / 11 Join Dims. Flat Schema: 46 Keyvalues / 129 Measurements / 1 Join Dim.
Oracle Confidential – Internal/Restricted/Highly Restricted 33
Complex Query 9 Million Rows
GB
Query:
many Result columns 3 Attribute tables
Flat cube / Inmemory: Many result columns Complex cross table predicates
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Agenda
Star Cube
Flat Cube
Bloom Filters Oracle 10g
Group By and Aggregate push down
Group By Placement Oracle 10g
Results
1
2
3
Oracle Confidential – Internal/Restricted/Highly Restricted 34
4
5
6
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Q & A
Fragen werden gerne gesehen!
Und es werden Pilot Kunden gesucht!
top related