synopsys: foundationsfor...

Michael Rudolf1, Hannes Voigt1, Christof Bornhoevd2,and Wolfgang Lehner1

SynopSys: Foundations forMultidimensional Graph AnalyticsBusiness Intelligence for the Real-Time Enterprise (BIRTE 2014)1Database Technology Group, Technische Universität Dresden2SAP Labs, LLC, Palo Alto

September 1, 2014

Motivation: Big (Graph) Data

Peak Performance

Nov. 26, 2012: 26.5M items (306/sec)Nov. 23, 2013: 36.8M items (426/sec)

645M users135 K new every day

58M tweets & 2.1 G searches / day

Motivation: Big (Graph) Data

Peak Performance

Nov. 26, 2012: 26.5M items (306/sec)Nov. 23, 2013: 36.8M items (426/sec)

645M users135 K new every day

58M tweets & 2.1 G searches / day

Intensional vs. Extensional� Schema & integrity constraints

� Created at design time bydomain experts

Once & forever

� Collect lots of data �rst

� Try to deduce the intension

� �The Fourth Paradigm� [Mic09]

Time© Michael Rudolf | SynopSys: Foundations for Multidimensional Graph Analytics | 2

The Property Graph Model

64GB�Apple iPadMC707LL/A�

�AppleiPhone 5�

3white

16 GB�Apple

iPhone 4�

4�ConsumerElectronics�

5�Phones�7�Tablets�

8�Freddy�

�Karl�DE

10�Mike�US

11�Steve�US

125/5stars

135/5 stars

4/5 stars

delivered 24/02/14

16ordered24/02/14

part ofpart of

authors

likeslikes

records

contains 1

contains 2

contains 1

� Provides directed, attributed multi-relational graphs

� Attributes on vertices and edges as key-value pairs(instance-level instead of class-level)

The Property Graph Model

�AppleiPhone 5�

3white

16 GB�Apple

iPhone 4�

8�Freddy�

�Karl�DE

10�Mike�US

11�Steve�US

125/5stars

135/5 stars

4/5 stars

delivered 24/02/14

16ordered24/02/14

part ofpart of

authors

likeslikes

records

contains 1

contains 2

contains 1

� Provides directed, attributed multi-relational graphs

� Attributes on vertices and edges as key-value pairs(instance-level instead of class-level)

Agenda

Analytical Scenario: From Graphs to Cubes

Operations: Roll-up, Drill-down, Slice & Dice

Challenges: Unbalanced Hierarchies & OLAP Anomalies

Graph Cube

�AppleiPhone 5�

3white

16 GB�Apple

iPhone 4�

8�Freddy�

�Karl�DE

10�Mike�US

11�Steve�US

125/5stars

135/5 stars

4/5 stars

delivered 24/02/14

16ordered24/02/14

part ofpart of

inauthors

authors

likeslikes

records

contains 1

contains 2

contains 1

1. Identify facts

2. Specify dimensions

3. De�ne measures

Graph Cube

�AppleiPhone 5�

3white

16 GB�Apple

iPhone 4�

8�Freddy�

�Karl�DE

10�Mike�US

11�Steve�US

125/5stars

135/5 stars

4/5 stars

delivered 24/02/14

16ordered24/02/14

part ofpart of

inauthors

authors

likeslikes

records

contains 1

contains 2

contains 1

1. Identify facts

3. De�ne measures

Graph Cube

�AppleiPhone 5�

3white

16 GB�Apple

iPhone 4�

8�Freddy�

�Karl�DE

10�Mike�US

11�Steve�US

125/5stars

135/5 stars

4/5 stars

delivered 24/02/14

16ordered24/02/14

part ofpart of

inauthors

authors

likeslikes

records

contains 1

contains 2

contains 1

1. Identify facts

3. De�ne measures

Graph Cube

�AppleiPhone 5�

3white

16 GB�Apple

iPhone 4�

8�Freddy�

�Karl�DE

10�Mike�US

11�Steve�US

125/5stars

135/5 stars

4/5 stars

delivered 24/02/14

16ordered24/02/14

part ofpart of

inauthors

authors

likeslikes

records

contains 1

contains 2

contains 1

1. Identify facts

3. De�ne measures

Depending on the use case, a (base) fact can be

� a vertex attribute, an edge attribute, or

� the presence of an edge.

in general: a subgraph

Ô Use pattern matching Ô graphical speci�cation instead of DSL

Example

authorsrates Match reviews of products and

their authors (vertex typesindicated via color)

� the presence of an edge.in general: a subgraph

Example

Dimensions

Dimensions can be

1. vertex or edge attributes

2. connectivity

�AppleiPhone 5�

3white

16 GB�Apple

iPhone 4�

8�Freddy�

�Karl�DE

10�Mike�US

11�Steve�US

125/5stars

135/5 stars

4/5 stars

delivered 24/02/14

16ordered24/02/14

part ofpart of

authors

authorsrates

likeslikes

records

contains 1

contains 2

contains 1

Structure in Dimensions� extrinsic: not contained in graph data,needs to be provided externally (e.g., GeoNames)

� intrinsic: embodied in graph data

explicit: captured as topological informationimplicit: has to be derived from attribute values

Dimensions

Dimensions can be

1. vertex or edge attributes

2. connectivity

�AppleiPhone 5�

3white

16 GB�Apple

iPhone 4�

8�Freddy�

�Karl�DE

10�Mike�US

11�Steve�US

125/5stars

135/5 stars

4/5 stars

delivered 24/02/14

16ordered24/02/14

part ofpart of

authors

authorsrates

likeslikes

records

contains 1

contains 2

contains 1

Structure in Dimensions� extrinsic: not contained in graph data,needs to be provided externally (e.g., GeoNames)

� intrinsic: embodied in graph data

explicit: captured as topological informationimplicit: has to be derived from attribute values

Intrinsic Dimensions

Explicit Dimensions� Can be speci�ed using path expressions� In general requires one path expression per level, e.g.

-[@type='belongsTo ']->[@type='state ']-[@type='partOf ']->[@type='country ']

Implicit Dimensions� Might require bucketization� In general requires one expression per level, e.g.

GetWeekOfYear(@ordered) and GetYear(@ordered)

alias @ attribute access of vertex or edge attribute

- [ edge predicate ] -> [ vertex predicate ] ( length )

paths (with optional recursion depth), optionally satisfying the predicates© Michael Rudolf | SynopSys: Foundations for Multidimensional Graph Analytics | 8

Dimension Speci�cation

ExampleName Seed Pattern Levels

Nationality $c $c@nationality

Category $p

Product category:

$p-[@type='in']->

Product group:

$p-[@type='in']->-[@type='part-of']->

Product area:

$p-[@type='in']->-[@type='part-of']->(2)

Seed Pattern� Connects facts to dimensions� Is matched against facts

Ô Has to be a super pattern of the fact pattern (i.e., more general)

Dimension Speci�cation

ExampleName Seed Pattern Levels

Nationality $c $c@nationality

Category $p

Product category:

$p-[@type='in']->

Product group:

Product area:

Seed Pattern� Connects facts to dimensions� Is matched against facts

Ô Has to be a super pattern of the fact pattern (i.e., more general)© Michael Rudolf | SynopSys: Foundations for Multidimensional Graph Analytics | 9

Properties of Dimensions

MonotonyLevels should be ordered such thatthe number of items decreases.

Level Name # Elements

1 Region 125

2 Country 30

3 Continent 3

HierarchyLevels should form hierarchies.If two facts map to the sameelement in li, they should map tothe same element in li+1 as well.Ô Functional dependency

Fact Level 1 Level 2 Level 3

A Saxony Germany Europe

B Saxony Germany Europe

C Bavaria Germany Europe

Properties of Dimensions

MonotonyLevels should be ordered such thatthe number of items decreases.

Level Name # Elements

1 Region 125

2 Country 30

3 Continent 3

HierarchyLevels should form hierarchies.If two facts map to the sameelement in li, they should map tothe same element in li+1 as well.Ô Functional dependency

Fact Level 1 Level 2 Level 3

A Saxony Germany Europe

B Saxony Germany Europe

C Bavaria Germany Europe

Measures

A measure is a derived fact

� combining several facts

� computed by a speci�ed function(e.g., scalar, aggregation).

Ô Annotate the fact patternÔ Introduce representative vertex

Example� Average product rating byproduct category

� Minimum age of customersby nationality

$r $p++

(Min. Age, $c@age,MIN)

(Avg. Rtg., $r@stars,AVG)

authors$a

rates$e

Measures

$r $p++

authors$a

rates$e

Measures

$r $p++

authors$a

rates$e

Operations: Roll-up, Drill-down, Slice & Dice

Roll-up/Drill-down

Granularity of the Cube� Represents the �grouping�: the current levels of interest

� Initially: the lowest level of each dimension

Roll-up� Reduces the granularity

� For dimension d, move up one level from li to li+1

Drill-down� Increases the granularity

� For dimension d, move down one level from li to li−1

Ô Introduce representative vertex for each groupÔ Expose computed values for measures as attributes

Roll-up/Drill-down

Slice & Dice

Function �lter transforms fact base of cube

� evaluates level-predicate pairs

� removes facts not matching the predicates

For a single predicate applied to one dimension Ô slice

Example

Slice product reviews by German customers from the cube c:filter(c, {(Nationality, λ = �DE�)}).

Slice & Dice

Function �lter transforms fact base of cube

� evaluates level-predicate pairs

� removes facts not matching the predicates

For a single predicate applied to one dimension Ô slice

Example

Slice product reviews by German customers from the cube c:filter(c, {(Nationality, λ = �DE�)}).

Challenges: Unbalanced Hierarchies & OLAP Anomalies

Unbalanced Hierarchies

Facts with di�erent granularities

Relative dimension speci�cation:

Product category:

$p-[@type='in']->

Product group:

Product area:

Ô Absolute instead of relativedimension speci�cation required

Example

Products in categories andgroups

15red 16 GB

�Google Nexus 5�16black

�SamsungE1200�

4�Cell Phones& Accessories�

5 �Phones�

�Computers &Accessories�

7�Tablets�

12�Smartphones�in in

part of

part ofpart of

Product category:

$p-[@type='in']->

Product group:

Product area:

Example

15red 16 GB

�SamsungE1200�

5 �Phones�

7�Tablets�

part of

part ofpart of

Product category:

$p-[@type='in']->

Product group:

Product area:

Example

15red 16 GB

�SamsungE1200�

5 �Phones�

7�Tablets�

part of

part ofpart of

Product category:

$p-[@type='in']->

Product group:

Product area:

Example

15red 16 GB

�SamsungE1200�

5 �Phones�

7�Tablets�

part of

part ofpart of

Product category:

$p-[@type='in']->

Product group:

Product area:

Example

15red 16 GB

�SamsungE1200�

5 �Phones�

7�Tablets�

part of

part ofpart of

Product category:

$p-[@type='in']->

Product group:

Product area:

Example

15red 16 GB

�SamsungE1200�

5 �Phones�

7�Tablets�

part of

part ofpart of

Solution: Pre-process the graph

Data Cleansing� Balance hierarchies

� Add missing root nodes

Tagging� Add attributes for absolutereferencing

15red 16 GB

�SamsungE1200�

5 �Phones�

7�Tablets�

part of

part ofpart of

15red 16 GB

�SamsungE1200�

5 �Phones�

7�Tablets�

12�Smartphones�in

part of

part ofpart of

13 �Dumbphones�

part of

part ofpart of

15red 16 GB

�SamsungE1200�

43 �Cell Phones& Accessories�

�Phones�

7 1�Tablets�

12 1�Smartphones�in in

part of

part ofpart of

OLAP Anomalies

It depends on the model:

� double counting can occur, if acardinality assumption isviolated (1:1 vs. 1:nrelationship)

7�Tablets�5 �Phones�

15128GB

�Apple iPad Air�

16 black

�SamsungE1200�

inin in

� incompleteness can occur, if aconnectivity assumption isviolated

15128GB

16 black

�SamsungE1200�

OLAP Anomalies

It depends on the model:

� double counting can occur, if acardinality assumption isviolated (1:1 vs. 1:nrelationship)

15128GB

16 black

�SamsungE1200�

inin in

� incompleteness can occur, if aconnectivity assumption isviolated

15128GB

16 black

�SamsungE1200�

Conclusion

Powerful Mapping of Multidimensional Analytics� Expose well-known concepts and operations

� Emphasize challenges posed by graph data

Ô Open up the graph world to Business Intelligence

Flexible Work�ow for the Big Graph Data Era� No up-front schema design

� Adapt to changing data and requirements

Ô What is a fact today can be a dimension tomorrow

1 Additional Material & References

References I

Chen Chen, Xifeng Yan, Feida Zhu, Jiawei Han, and Philip S. Yu.Graph OLAP: Towards Online Analytical Processing on Graphs.In Proceedings of the Eighth International Conference on Data Mining, pages 103�112, Pisa,Italy, December 2008. IEEE.

Microsoft Research.The Fourth Paradigm: Data-Intensive Scienti�c Discovery.Microsoft Press, 2009.

Marko A. Rodriguez and Peter Neubauer.Constructions from Dots and Lines.Bulletin of the American Society for Information Science and Technology, 36(6):35�41, 2010.

Yuanyuan Tian and Jignesh M. Patel.TALE: A Tool for Approximate Large Graph Matching.In 2008 IEEE 24th International Conference on Data Engineering, pages 963�972. IEEE, April2008.

Peixiang Zhao, Xiaolei Li, Dong Xin, and Jiawei Han.Graph Cube: On Warehousing and OLAP Multidimensional Networks.In Proceedings of the ACM SIGMOD International Conference on Management of Data, pages853�864, Athens, Greece, 2011. ACM.

References II

Ning Zhang, Yuanyuan Tian, and Jignesh M. Patel.Discovery-Driven Graph Summarization.In Proceedings of the 26th International Conference on Data Engineering, pages 880�891,Long Beach, CA, USA, 2010. IEEE.

synopsys: foundationsfor...

Documents