query execution on nettraveler angel l. villalaín-garcía manuel rodríguez-martínez university of...

Query Execution on Query Execution on NeNetTtTravelerraveler

Angel L. Villalaín-GarcíaAngel L. Villalaín-GarcíaManuel RodrManuel Rodrííguez-Martguez-MartííneznezUniversity of Puerto Rico - Mayaguez University of Puerto Rico - Mayaguez CampusCampus

WALSAIP 04/21/23

2

ObjectivesObjectives

Develop a framework for Parallel and Develop a framework for Parallel and Distributed Query Optimization and Distributed Query Optimization and Execution on NetTravelerExecution on NetTraveler

Facilitate and Optimize the access of Facilitate and Optimize the access of data across WANsdata across WANs Transparent data accessTransparent data access Uniform access interfaceUniform access interface

Robust operation by exploiting Robust operation by exploiting replicationreplication

WALSAIP 04/21/23

3

Road MapRoad Map

ObjectivesObjectives Motivation Motivation Problem FormulationProblem Formulation Proposed SolutionProposed Solution Execution ExampleExecution Example ContributionsContributions Technical DetailsTechnical Details StatusStatus Next Steps Next Steps SummarySummary Questions & DemoQuestions & Demo

WALSAIP 04/21/23

4

MotivationMotivation

INTERNET

Mobile Devices

Relational DBMS(Oracle,

PostgreSQL, etc)

XML Repositories

Other Data Repositories

Wireless Sensors

LAN

WALSAIP 04/21/23

5

Problem FormulationProblem Formulation

INTERNET

Mobile Devices

Relational DBMS(Oracle,

PostgreSQL, etc)

XML Repositories

Other Data Repositories

Wireless Sensors

LAN

Dispersed and Heterogeneous data sources

No uniformity on WANs•Several limitations

•Bandwidth

•Memory

•Power

•Processing Capabilities

WALSAIP 04/21/23

6

Problem Formulation (cont.)Problem Formulation (cont.)

Traditional DBMS Plan

Parser

Semantic Analyzer

Logical Optimizer

Physical Optimizer

Query Plan Execution

Centralized Query Optimizer

Scan Relations

Select A.id, A.name from A,B where A.id = B.id and A.sage <30

WALSAIP 04/21/23

7

Proposed SolutionProposed Solution

Parser

Semantic Analyzer

Logical Optimizer

Physical Optimizer

Query Plan Execution

Physical Optimizer

Query Plan Execution

Decentralized Query OptimizerDistributed and Parallel DBMS Plan

Scan Replicated Relations

Select A.id, A.name from A,B where A.id = B.id and A.sage <30

WALSAIP 04/21/23

8

Execution ExampleExecution Example

Q

QSB1Knows:

QSB2, QSB3, QSB4

QSB2

QSB3

QSB4

QQ

IG2

IG1

R

R

R

R

IG3

WALSAIP 04/21/23

9

ReplicatesManagement

Manage Partitions

Pre Hashed On The Fly

Hashing MechanismParallel Ops

IG Level Operations

Memory Management

SchedulersMechanisms Hash Join Exchange Op

Parallel OpsQSB Level Operations

Site ManagementScheduling Management

Parallel & Distributed Ops

Physical OptimizerLogical Optimizer

Optimizer Level Operations

Technical DetailsTechnical Details

WALSAIP 04/21/23

10

ContributionsContributions

Facilitate Integration for scientific Facilitate Integration for scientific applicationsapplications Heterogeneous data sourcesHeterogeneous data sources Heterogeneous schemas Heterogeneous schemas

Load BalancingLoad Balancing Spread work to various nodesSpread work to various nodes

RobustnessRobustness Can get data from multiple sourcesCan get data from multiple sources

AsynchronousAsynchronous Dynamically replace nodes used for processingDynamically replace nodes used for processing

Decentralized Query OptimizationDecentralized Query Optimization

WALSAIP 04/21/23

11

StatusStatus

Manage Replicates

Manage Partitions

Pre Hashed On The Fly

Hashing MechanismParallel Ops

IG Level Operations

Memory Management

SchedulersMechanisms Hash Join Exchange Op

Parallel OpsQSB Level Operations

Site ManagementScheduling Management

Parallel & Distributed Ops

Physical OptimizerLogical Optimizer

Optimizer Level Operations

WALSAIP 04/21/23

12

Next StepsNext Steps

StudStudy y Scheduling Effect aScheduling Effect and nd improvementsimprovements

Hash Join operators and functionalitiesHash Join operators and functionalities

User interface for configuration and User interface for configuration and demonstration purposesdemonstration purposes

WALSAIP 04/21/23

13

Additional Areas of ResearchAdditional Areas of Research

Distributed Catalog ManagerDistributed Catalog Manager Oliver MorenoOliver Moreno

Server-Side Query Recovery MechanismServer-Side Query Recovery Mechanism Victor KarehVictor Kareh

NetTraveler System AdministrationNetTraveler System Administration Osvaldo FerreroOsvaldo Ferrero

WALSAIP 04/21/23

14

Summary Summary

Facilitate and Optimize the access of Facilitate and Optimize the access of data across WANsdata across WANs Query Parallelization and ExecutionQuery Parallelization and Execution Exploiting ReplicationExploiting Replication Response Time improvementResponse Time improvement

See website for API and user manualSee website for API and user manual http://amadeus.ece.uprm.edu/~s08413http://amadeus.ece.uprm.edu/~s08413

WALSAIP 04/21/23

15

Demo and QuestionsDemo and Questions