flatcubes and inmemory option for sap bw · copyright © 2015, oracle and/or its affiliates. all...

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |

FlatCubes and InMemory Option for SAP BW Jörn Bartels Architect - SAP Development Oracle Deutschland

Oracle Confidential – Internal


Agenda

Star Cube

Flat Cube

Bloom Filters

Group By and Aggregate push down

Group by Placement

Results

1

2

3

Oracle Confidential – Internal/Restricted/Highly Restricted 2

4

5

6


Flat Tables


Facts

Masterdata


Star Cube (old Oracle - SAP BW Standard)


Fact E/F

Masterdata

Dimensions


Agenda

Star Cube

Flat Cube

Bloom Filters


Group By Placement

Results

1

2

3


4

5

6


Flat Cube


Facts

Masterdata


Disksizes Star - Flat

0

5000

10000

15000

20000

star star comp flat flat comp

Table

Indexes

Dimensions

Sum


8 Dimensisons 188 Values Comp – OLTP compression

MB


Disksizes Star - Flat

0

1000

2000

3000

4000

5000

6000

star star comp flat flat comp

Table

Indexes

Dimensions

Sum


14 Dimensions 6 Values Comp – OLTP compression

MB


F

Flat Cubes and Partitioning Old Implementation (Classic Star Schema)

9

F Table Partition 6

F Table Partition 5

F Table Partition 4

F Table Partition 3

F Table Partition 2

F Table Partition 1

F FACT TABLE

F

E FACT TABLE

E Table Partition 6

E Table Partition 5

E Table Partition 4

E Table Partition 3

E Table Partition 2

E Table Partition 1


• New Implementation (Non-Oracle Databases) – The F table goes away

– Data from F table now part of E table

• Major disadvantage – No partition pruning on the larges part of the data

possible for queries

Flat Cubes and Partitionierung

10

F Table Partition 6

F Table Partition 5

F Table Partition 4

F Table Partition 3

F Table Partition 2

F Table Partition 1

"E Table" (Partition 0)

E FACT TABLE


• New Implementation (Oracle Databases) – The F table goes away

– Data from F table now part of E table

• Oracle Advantage – Partition pruning possible for queries due to

subpartitioning

Flat Cubes and Partitionierung

11

F Table Partition 6

F Table Partition 5

F Table Partition 4

F Table Partition 3

F Table Partition 2

F Table Partition 1

E FACT TABLE

E Partition, Subpartition 6






Partition 0


Plan Startransformation


---------------------------------------------------------------------------------

| Id | Operation | Name |

---------------------------------------------------------------------------------

| 0 | SELECT STATEMENT | |

| 1 | SORT GROUP BY | |

|* 2 | HASH JOIN | |

|* 3 | TABLE ACCESS STORAGE FULL | /BI0/YVC_PROD2 |


| 5 | TABLE ACCESS STORAGE FULL | /BIC/DZMK_TCSC2 |




| 9 | TABLE ACCESS STORAGE FULL | /BIC/DZMK_TCSCU |

|* 10 | HASH JOIN | |

| 11 | TABLE ACCESS BY INDEX ROWID BATCHED | /BIC/DZMK_TCSCP |

|* 12 | INDEX RANGE SCAN | /BIC/DZMK_TCSCP~01 |

|* 13 | HASH JOIN | |


|* 15 | HASH JOIN | |

|* 16 | TABLE ACCESS STORAGE FULL | /BIC/DZMK_TCSCT |

| 17 | PARTITION RANGE ITERATOR | |

|* 18 | TABLE ACCESS BY LOCAL INDEX ROWID BATCHED| /BIC/EZMK_TCSC |

| 19 | BITMAP CONVERSION TO ROWIDS | |

| 20 | BITMAP AND | |

| 21 | BITMAP MERGE | |

| 22 | BITMAP KEY ITERATION | |

| 23 | BUFFER SORT | |

|* 24 | TABLE ACCESS STORAGE FULL | /BIC/DZMK_TCSCT |

|* 25 | BITMAP INDEX RANGE SCAN | /BIC/EZMK_TCSC~020 |




| 29 | INDEX STORAGE FAST FULL SCAN | /BIC/DZMK_TCSC4~0 |





|* 34 | TABLE ACCESS STORAGE FULL | /BIC/DZMK_TCSC3 |


---------------------------------------------------------------------------------


Plan Flat Cube


-----------------------------------------------------------------------

| Id | Operation | Name |

-----------------------------------------------------------------------

| 0 | SELECT STATEMENT | |

| 1 | SORT GROUP BY | |


| 3 | JOIN FILTER CREATE | :BF0001 |

| 4 | PART JOIN FILTER CREATE | :BF0000 |

| 5 | NESTED LOOPS | |

| 6 | TABLE ACCESS BY INDEX ROWID BATCHED| /BIC/DZMK_TCSCP |

|* 7 | INDEX RANGE SCAN | /BIC/DZMK_TCSCP~01 |

|* 8 | TABLE ACCESS STORAGE FULL | /BI0/YVC_PROD2 |

| 9 | JOIN FILTER USE | :BF0001 |

| 10 | PARTITION LIST JOIN-FILTER | |

| 11 | PARTITION RANGE ITERATOR | |

|* 12 | TABLE ACCESS INMEMORY FULL | /BIC/FZMK_TCFL |

-----------------------------------------------------------------------


Agenda

Star Cube

Flat Cube

Bloom Filters


Group By Placement

Results

1

2

3


4

5

6


Joining and Combining Data

• Converts joins of data in multiple tables into fast column scans

• Joins tables 10x faster

15

Example: Find total sales in outlet stores

Sales Stores

Sto

re ID

StoreID in 15, 38, 64

Type=‘Outlet’

Typ

e

Sum

Sto

re ID

Am

ou

nt

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 16

Bloom Filter

Hash Join

Bit vector

0 0 0 1 1 1 1 STORES

Bloom Filter Creation: A bit vector is created that has a bit set for each join column there is a value for & 0 where there is no value Bloom Filter

created

2

SALES

3. Table Scan: SALES table is scan and rows are filtered based on query predicates

4

Bloom Filter sent: BIT VECTOR is sent as an additional filter criteria to the scan of the sales table

3

Reduced row sent: Only rows that have a match in the bit vector get sent to the hash join

5

Hash Join: Join completed by probing into the hash table from the store_id to find actual matching rows

6

PGA

1. Table Scan: STORES table is scanned and matching rows sent to hash join

1

Typ

e

Sto

re ID

Sto

re ID


Identifying : INMEMORY Joins • Bloom filters enable joins to be

converted into fast column scans

• Tried and true technology originally released in 10g

• Same technique used to offload joins on Exadata


Multiple bloom filter

• Multi-table joins are common in analytic queries

• Possible to apply multiple bloom filters to the fact table due to sophisticated Optimizer

• Simultaneously joining to all of the dimension tables while scanning the fact table

Select d.d_year, s.s_nation,

sum(lo_revenue - lo_supplycost)

From LINEORDER l, DATE_DIM d,

PART p, SUPPLIER s

Where l.lo_orderdate = d.d_datekey

And l.lo_partkey = p.p_partkey

And l.lo_suppkey = s.s_suppkey

And p.p_mfgr= 'MFGR#12’

And s.s_region = 'AMERICA’

Group by d.d_year, s.s_nation

Order by d.d_year, s.s_nation;


Multiple Bloom Filter


Standard Left Deep Tree Plan: How Can Multiple Bloom Filters Be Possible?

20

PART

HASH JOIN

LINEORDER

HASH JOIN

SUPPLIER

HASH JOIN

DATEDIM

3

4

1

2

3

1

Bloom filter created

2 Bloom filter used


Multiple Bloom Filters Possible Due To Swap Join Input Optimization and Right Deep Trees

21

PART

HASH JOIN

LINEORDER

HASH JOIN

SUPPLIER

HASH JOIN

DATEDIM

1

2

3

4

1

2

3

Why is this significant?

Applying multiple filters on initial scan of LINEORDER reduces the number of rows sent upstream in the plan


Multiple Bloom Filter


Agenda

Star Cube

Flat Cube

Bloom Filters


Group By Placement

Results

1

2

3


4

5

6


Simple grouping Query

• SELECT "F"."SID_0MAT_SALES", SUM ("F"."/BIC/INV_QTY_T") FROM "/BIC/8LADFRGACT" "F" GROUP BY "F"."SID_0MAT_SALES“ -----------------------------------------------------------------

| Id | Operation | Name | Rows |

-----------------------------------------------------------------

| 0 | SELECT STATEMENT | | 46336 |

| 1 | HASH GROUP BY | | 46336 |

| 2 | TABLE ACCESS INMEMORY FULL | /BIC/8LADFRGACT | 93M|

-----------------------------------------------------------------



Inmemory Statistics

• NAME VALUE --------------------------------------- -----------

IM scan CUs memcompress for query low 227

IM scan rows 93541477

IM scan rows projected 1147304



Agenda

Star Cube

Flat Cube

Bloom Filters


Group By Placement

Results

1

2

3


4

5

6


Group by placement

• SELECT "S21"."MAT_SALES", SUM ("F"."/BIC/INV_QTY_T") FROM "/BIC/8LADFRGACT" "F"

JOIN "/BI0/SMAT_SALES" "S21"

ON "F"."SID_0MAT_SALES" = "S21"."SID"

GROUP BY "S21"."MAT_SALES"



Group By Placement

• WITH "F" AS (SELECT /*+ no_merge */ "F"."SID_0MAT_SALES",

SUM ("F"."/BIC/INV_QTY_T")

AS "F"."/BIC/INV_QTY_T"

FROM "/BIC/8LADFRGACT" "F"

GROUP BY "F"."SID_0MAT_SALES")

SELECT "S21"."MAT_SALES", SUM ("F"."/BIC/INV_QTY_T")

FROM "F"

JOIN "/BI0/SMAT_SALES" "S21"

ON "F" . "SID_0MAT_SALES" = "S21" . "SID"

GROUP BY "S21"."MAT_SALES"



Group By Placement • ---------------------------------------------------------------------

| Id | Operation | Name | Rows |

---------------------------------------------------------------------

| 0 | SELECT STATEMENT | | 46336 |

| 1 | HASH GROUP BY | | 46336 |

|* 2 | HASH JOIN | | 46336 |

| 3 | JOIN FILTER CREATE | :BF0000 | 46336 |

| 4 | VIEW | VW_GBC_5 | 46336 |

| 5 | HASH GROUP BY | | 46336 |

| 6 | TABLE ACCESS INMEMORY FULL | /BIC/8LADFRGACT | 93M|

| 7 | JOIN FILTER USE | :BF0000 | 2609K|

|* 8 | TABLE ACCESS INMEMORY FULL | /BI0/SMAT_SALES | 2609K|

---------------------------------------------------------------------



Agenda

Star Cube

Flat Cube

Bloom Filters


Group By Placement

Results

1

2

3


4

5

6


Star Schema: 9 Dimensions / 61 Measurements / 2 Join Dims. Flat Schema: 40 Keyvalues / 61 Measurements / 1 Join Dim.


Simple Query 1.2 Million Rows

GB

Query:

1 Result column 1 Attribute table

Flat cube / Inmemory: Independent of table width Cheap access for low number of columns


Star Schema: 8 Dimensions / 145Measurements / 8 Join Dims. Flat Schema: 37 Keyvalues / 145 Measurements / 1 Join Dim.


Medium Query 6.5 Million Rows

GB

Query:

1 Result column 2 Attribute tables

Flat cube / Inmemory: Independent of table width Cheap access for low number of columns


Star Schema: 12 Dimensions / 129 Measurements / 11 Join Dims. Flat Schema: 46 Keyvalues / 129 Measurements / 1 Join Dim.


Complex Query 9 Million Rows

GB

Query:

many Result columns 3 Attribute tables

Flat cube / Inmemory: Many result columns Complex cross table predicates


Agenda

Star Cube

Flat Cube

Bloom Filters Oracle 10g


Group By Placement Oracle 10g

Results

1

2

3


4

5

6


Q & A

Fragen werden gerne gesehen!

Und es werden Pilot Kunden gesucht!

flatcubes and inmemory option for sap bw · copyright © 2015, oracle and/or its affiliates. all...

Documents