rainbow - bridging xml and relational databases: design, implementation, and evaluation

04-19-2001 1

Rainbow - Bridging XML and Relational Databases: Design, Implementation, and Evaluation

MQP Advisor:MQP Advisor:Prof. Elke A. Rundensteiner, PhDProf. Elke A. Rundensteiner, PhD

Sponsor:Sponsor:Verizon Laboratories IncorporatedVerizon Laboratories Incorporated

MQP Project Members:MQP Project Members:

Tien Vu, Tien Vu,

Mirek Cymer, Mirek Cymer,

John LeeJohn Lee

04-19-2001 2

HTML vs. XML

04-19-2001 3

XML Data Management by RDBMS Microsoft, IBM, Informix, Oracle,...Microsoft, IBM, Informix, Oracle,...

Advantages:Advantages: Matured database tools available.Matured database tools available. Efficient query and analysis tools.Efficient query and analysis tools. Easy integration with existing business databases.Easy integration with existing business databases.

Issues:Issues: Map between XML and Relational Model.Map between XML and Relational Model. Update Propagation.Update Propagation. Query Translation and Optimization.Query Translation and Optimization.

04-19-2001 4

XMLXMLDataData

SubSubsystemsystem

LegendLegend

Traditional System Architecture

XMLXML

XMLXMLQueryQuery

XMLXMLUserUser

XML Query EngineXML Query Engine

XML ManagerXML Manager

RDBMS

04-19-2001 5

Motivation for Flexible Mapping

CarCar

MakeMake

ModelModel

YearYear

car

Query Performance varies with respect to how data is mapped.Query Performance varies with respect to how data is mapped.

SELECT * FROM model;SELECT * FROM model;

SELECT model FROM car WHERE make = ‘Ford’;SELECT model FROM car WHERE make = ‘Ford’;

iidiid pidpid

11 00

…… ……

iidiid pidpid ValueValue

22 11 FordFord

…… …… ……

iidiid pidpid ValueValue

33 11 MustangMustang

…… …… …… iidiid pidpid ValueValue

33 11 20012001

…… …… ……

iidiid pidpid MakeMake ModelModel YearYear

11 00 FordFord MustangMustang 20012001

…… …… …… …… ……

04-19-2001 6

XMLXMLDataData

SubSubsystemsystem

LegendLegend

Rainbow Architecture

XMLXML

XMLXMLQueryQuery

XMLXMLUserUser

XML Query EngineXML Query Engine

XML ManagerXML Manager

RDBMSRestructuring SubsystemRestructuring Subsystem

DTDDTD

DTD ManagerDTD Manager

Flexible mapping = fixed Mapping + restructuringFlexible mapping = fixed Mapping + restructuring

04-19-2001 7

Goals of our MPQ Target:Target:

ImplementImplement and and evaluateevaluate restructuring subsystem within restructuring subsystem within the large-scale Rainbow system.the large-scale Rainbow system.

Methodology:Methodology: Learn about the database technologies and web tools.Learn about the database technologies and web tools. Translate research ideas to software system design.Translate research ideas to software system design. Practice software engineering techniques.Practice software engineering techniques. Design an experimental test plan and test bed.Design an experimental test plan and test bed. Conduct performance study and analysis.Conduct performance study and analysis.

04-19-2001 8

Rainbow Restructuring Subsystem

DTDDTD XMLXML

DTD ManagerDTD Manager XML ManagerXML Manager

SubSubsystemsystem

DataData

ProcessProcess

LegendLegend

XMLXMLQueryQuery

XMLXMLUserUser

XML Query EngineXML Query Engine

Restructuring SubsystemRestructuring Subsystem

04-19-2001 9

Rainbow Restructuring Subsystem

DTD XML

DTD Manager XML Manager

SubSubsystemsystem

DataData

ProcessProcess

LegendLegend

Res

truc

turi

ngR

estr

uctu

ring

Mapping

RestructureOperatorLibrary

RestructurerRestructurer

XMLQuery

XMLUser

XML Query Engine

04-19-2001 10

Restructuring Library

OptimizationOptimization Pushup/Pushdown AttributePushup/Pushdown Attribute Pushup/Pushdown NestingPushup/Pushdown Nesting

OrganizationOrganization Rename Item/AttributeRename Item/Attribute

CompletenessCompleteness SwitchNestingSwitchNesting Split/Merge NestingSplit/Merge Nesting Reference/DereferenceReference/Dereference

04-19-2001 11

Pushup Attribute Operator

CREATE VIEW new.A (CREATE VIEW new.A (<all-columns>, a) AS, a) ASSELECT A.SELECT A.<all_columns>, B.b, B.bFROM old.A, old.BFROM old.A, old.BWHERE B.pid = A.iidWHERE B.pid = A.iid

CREATE VIEW new.B (CREATE VIEW new.B (<all-columns-but-b>) AS) ASSELECT B.SELECT B.<all-columns-but-b>FROM old.BFROM old.B

A

B

A

B

Push-up

b

a

04-19-2001 12

Instantiated Restructuring Operators

Mapping = Operators + Parameters.Mapping = Operators + Parameters.

pushUpAttribute(‘Model’, ‘Value’, ‘Car’, ‘Model’);

CarCar

ModelModel

ValueValue ParentParent

MustangMustang

ValueValue ParentParent ValueValue ModelModel

CarCar

MustangMustang

ParentParent

04-19-2001 13

Rainbow Implementation Development ToolsDevelopment Tools

Java: Visual Café2, Java: Visual Café2, Javadocs, JAVA2Javadocs, JAVA2

Oracle 8i, XML 4J, Oracle 8i, XML 4J, JDBC1.2, SQL QueriesJDBC1.2, SQL Queries

Code FactsCode Facts 44 total system classes44 total system classes 17 classes created by 17 classes created by

RainbowRainbow 19 classes extended19 classes extended 8 classes reused8 classes reused

# of Classes

17

8

19

New Reused Extended

04-19-2001 14

Screen Shot of Rainbow

04-19-2001 15

Setup for Rainbow Evaluation ExperimentalExperimental

Database Server: Database Server: Oracle 8i on a PII 300MHz, 256MB, Microsoft NT ServerOracle 8i on a PII 300MHz, 256MB, Microsoft NT Server

Client: Client: Pentium 233MHz, 128MB, Microsoft NT WorkstationPentium 233MHz, 128MB, Microsoft NT Workstation

DataData Designed a DTDDesigned a DTD Generated XML using IBM’s Generated XML using IBM’s

XML-GeneratorXML-Generator

DTD CONTENT:DTD CONTENT:

<!ELEMENT one (two+)><!ELEMENT one (two+)>

<!ELEMENT two (three)><!ELEMENT two (three)>

<!ELEMENT three (four)><!ELEMENT three (four)>

<!ELEMENT four (five)><!ELEMENT four (five)>

<!ELEMENT five (six)><!ELEMENT five (six)>

<!ELEMENT six (seven)><!ELEMENT six (seven)>

<!ELEMENT seven EMPTY><!ELEMENT seven EMPTY>

<!ATTLIST seven attribute #REQUIRED><!ATTLIST seven attribute #REQUIRED>

04-19-2001 16

Query Performance Evaluation

Time of Join Query (data-size=22Mb)

0

10

20

30

40

0 1 2 3 4 5 6 7

# of PushUpAttribute

Que

ry T

ime

(s)

Time of Join Query

04-19-2001 17

Overhead Cost

Time of overhead (datasize=22Mb)

050

100150200250300350

0 1 2 3 4 5 6 7

# of pushUpAttribute

Res

truct

ure

Tim

e (s

)

overhead time

04-19-2001 18

MQP Accomplishments

Technical accomplishmentsTechnical accomplishments Implemented functional prototype systemImplemented functional prototype system Confirmed feasibility of Rainbow architectureConfirmed feasibility of Rainbow architecture Designed automated test bedDesigned automated test bed Conducted preliminary experimental studiesConducted preliminary experimental studies

Knowledge acquiredKnowledge acquired OO, Java, JDBC, SQL, RDBMS, XML, DTDOO, Java, JDBC, SQL, RDBMS, XML, DTD Teamwork & S/W Engineering & Software ReuseTeamwork & S/W Engineering & Software Reuse Logistics of setting up and running experimentsLogistics of setting up and running experiments

04-19-2001 19

Potential Future Work

XML query translation to SQLXML query translation to SQL Experiment with test plans and test beds to realize Experiment with test plans and test beds to realize

the full potential of the restructuring component.the full potential of the restructuring component.

04-19-2001 20

Special thanks to:Prof. Elke A. Rundensteiner

Ph.D. Xin Zhang

Visit Rainbow at http://davis.wpi.edu/dsrg/TJM/

Project Members:Tien Vu, Mirek Cymer, John Lee