soror sahri [email protected] ceria.dauphine.fr/soror/soror.html

Soror SAHRI – June 13th, 2006 Design & Implementation of a Scalable Distributed Database System: SD-SQL Server

1\46 pages

Soror SAHRI

[email protected] [email protected]

http://ceria.dauphine.fr/soror/soror.html

Design and Implementation of a Scalable Distributed Database System: SD-SQL Server

Thesis Presentation – CERIA LaboratoryThesis Presentation – CERIA Laboratory


2\46 pages

1. Introduction

2. State of The Art

3. SD-SQL Server Architecture

7. Conclusion & Future Work

5. Implementation of SD-SQL Server

4. SD-SQL Server Application Interface

P

L

A

N6. Performance Measurements


3\46 pages

Facts

Objectve

Introduction

State of the Art

SD-SQL Server Architecture

SD-SQL Server Application Interface

Implementation of SD-SQL Server

Conclusion & Future Work

Facts

Most of DBSs have distributed/parallel versions SQL Server, Oracle, DB2…

DBSs do not provide dynamically scalable tables All require manual repartitioning when tables scale-up


4\46 pages

Facts

Objective

Objective

Scalable Distributed Partitioning of Relational Tables

Introduction

State of the Art





Scalable Distributed Database System

SD-SQL Server


5\46 pages




1. Introduction

2. State of the Art

P

L

A

N


6. Performance Measurements


6\46 pages

Parallel DBMSs

Parallel DBMSs

SDDSs

Introduction

State of the Art





Oracle 10g [LB05]

[LB05] K, Loney & B, Bryla. Oracle Database 10g, DBA Handbook


7\46 pages

Parallel DBMS

SDDSs

SDDSs

SDDSs provide many scalable distributed partitioning schemes LH*, RP*, k-RP*, LH*RS…

These schemes can serve as the basis for an SD-DBS architecture

Introduction

State of the Art





An SDDS is a new class of data structures Specific for multicomputers, P2P, Grids…

Why ?


8\46 pages




1. Introduction

2. State of the Art

P

L

A

N




9\46 pages

Overview

Nodes, NDBs & SDBs

Scalable Tables

Images

SD-SQL Server?

SD-SQL Server is a Scalable Distributed Database System (SD-DBS)

SD-SQL Server uses the reference architecture Proposed by Pr. Litwin, Pr. Schwartz & Pr. Risch

2nd Intl. Workshop on Cooperative Internet Computing, 2002

SD-SQL Server is based on the RP* SDDS principles SD-SQL Server runs on Microsoft SQL Server 2000

Introduction

State of the Art






10\46 pages

Gross Architecture

Introduction

State of the Art





Overview

Nodes, NDBs & SDBs

Scalable Tables

Images

LinkedSQL

Servers

N1 N2 Ni Ni+1

S S PC

User/Application

sd_alter_table

User/Application

sd_insert

Split

[Litwin & Sahri. WDAS 2004]

NDBs

SD-SQLserver

SD-SQLserver

SD-SQLclient

SD-SQL Server

Managers

SD-SQLpeer

The SD-SQL Server originality ?

The automatic extension of the scalable tables of their NDBs of their SD-SQL Server nodes


11\46 pages

The Nodes, NDBs & SDBs

Introduction

State of the Art





Overview

Nodes, NDBs & SDBs

Scalable Tables

Images

DB1NDB

Node1 Node2 Node3 Nodei

DB1NDB

……

DB1NDB

MDB

DB1 SDB DB2 SDB

DB2NDB

DB2NDB


12\46 pages

Scalable Tables

A scalable (distributed) table is a collection of segments Segments are SQL tables

A scalable table has, initially, only one primary segment At some server or peer NDB

All the segments of a scalable table have the same scheme

Introduction

State of the Art





Overview

Nodes, NDBs & SDBs

Scalable Tables

Images


13\46 pages

Scalable Tables: Meta-data

Each scalable table has meta-data: The segment capacity The actual partitioning of the scalable table The check constraint of each segment

Introduction

State of the Art





Overview

Nodes, NDBs & SDBs

Scalable Tables

Images

A A check constraintcheck constraint defines the defines the MinMin and and MaxMax for each for each segmentsegment

These meta-data are stored in the meta-tables excluding the check constraints


14\46 pages

…….

DB1 SDB

Scalable Tables: Meta-data

N1.DB1 N2.DB1 N3.DB1

T Scalable Table

Ni.DB1

1000Size

N1.DB1Primary

Ni.DB1 Nodes

N1.DB1

N2.DB1

N3.DB1

RP

Introduction

State of the Art





Overview

Nodes, NDBs & SDBs

Scalable Tables

Images

Meta-Tables


15\46 pages

Scalable Tables: Splitting

Introduction

State of the Art





Overview

Nodes, NDBs & SDBs

Scalable Tables

Images

The number of segments in a scalable table is variable

A segment that overflows splits A split occurs when an insert overflows the

segment capacity Every split produces one or more new

segments for a scalable table


16\46 pages


Introduction

State of the Art





Overview

Nodes, NDBs & SDBs

Scalable Tables

Images

S

b+1

S S1

pb+1-p

p=INT(b/2)

C( S)= { c: c < h = c (b+1-p)}

C( S1)={c: c > = c (b+1-p)}

Check Constraint?

b

SELECT TOP P * INTO Ni.Si FROM S ORDER BY C ASCSELECT TOP P * WITH TIES INTO Ni.S1 FROM S ORDER BY C ASC


17\46 pages

DB1 SDB DB1 SDB

T Scalable Table

sd_insert

N1 N2 N4N3

NDBDB1

NDBDB1

NDBDB1

sd_insert

NDBDB1

Ni

sd_create_node

sd_insert

N3

NDBDB1

sd_create_node_database

NDBDB1

…….

sd_create_node_database


Introduction

State of the Art





Overview

Nodes, NDBs & SDBs

Scalable Tables

Images

DB1 SDB


18\46 pages

Images

Introduction

State of the Art





Overview

Nodes, NDBs & SDBs

Scalable Tables

Images

An Image hides the scalable table partitioning An image is an SQL Server distributed

updateable partitioned view of the table An SQL Server Union-all view with check

constraints An image resides on client or peer NDBs


19\46 pages

Image Types

Primary imageResides at the creation node Has the name of the scalable table

Secondary images Reside at other client or peer NDBs of the SDB Have a specific name, other than that of the

table To avoid name conflict

Introduction

State of the Art





Overview

Nodes, NDBs & SDBs

Scalable Tables

Images


20\46 pages

Image Adjustment

Introduction

State of the Art





Overview

Nodes, NDBs & SDBs

Scalable Tables

Images

An image presents the actual partitioning of its scalable table Defines the partitioning as known to the client It do not address any new segments resulted from a split

Are dynamically adjustable by the client When a query to the image comes in

Image checking Image adjustment if necessary


21\46 pages

Image Adjustment

Introduction

State of the Art





Overview

Nodes, NDBs & SDBs

Scalable Tables

Images

Get the number of segments presented in the image, N1 Get the number of segments of the scalable table, N2 Compare N1 and N2: If N1<N2 then Image Adjustment

Alter the partitioned view definition


22\46 pages

Images: Example

N1.DB1 N2.DB1 N3.DB1

T Scalable Table

Introduction

State of the Art





Overview

Nodes, NDBs & SDBs

Scalable Tables

Images

CREATE VIEW T AS SELECT * FROM N2.DB1.SD._N1_TCREATE VIEW T AS SELECT * FROM N2.DB1.SD._N1_T UNION ALL SELECT * FROM N3.DB1.SD._N1_T UNION ALL SELECT * FROM N4.DB1.SD._N1_T

PrimaryImage

DB1 SDB

N4.DB1

T Image


23\46 pages




1. Introduction

2. State of the Art

P

L

A

N




24\46 pages

Principles

The application interface manipulates scalable tables through SD-SQL Server commands

The SD-SQL Server commands start with ‘sd_’ to distinguish from SQL Server commands for static tables

Introduction

State of the Art





Principles

Nodes Management

SDBs & NDBs Management

Scalable Tables & Images Management

Scalable Queries Management

INSERT sd_insert

CREATE TABLE sd_create_table

[Litwin, Schwartz & Sahri. IASTED-DBA 2006]


25\46 pages

Nodes Management

Node Creation sd_create_node ‘Dell1’ /* Server by default */ sd_create_node ‘Ceria’, ‘client’

Node Alteration sd_alter_node ‘Ceria’, ‘ADD server’ /* Becomes peer*/

Node Removal sd_drop_node ‘Ceria’

Introduction

State of the Art





Principles

Nodes Management





26\46 pages

SDB & NDB Management

SDB Creation sd_create_scalable_database

‘SkyServer’, ‘Dell1’, ‘Server’, 2

/* Creates the primary SkyServer NDB as well at Dell1*/

SDB Alteration sd_create_node_database

‘SkyServer’, ‘Ceria’, ‘Client’

SDB Removal sd_drop_scalable_database ‘SkyServer’

Introduction

State of the Art





Principles

Nodes Management





27\46 pages

Scalable Tables Management

Scalable Table Creation sd_create_table ‘PhotoObj (Objid BIGINT PRIMARY KEY..)’,

10000 Scalable Table Alteration

sd_alter_table ‘PhotoObj ADD t INT’, 1000 sd_create_index ‘run_index ON Photoobj (run)’ sd_drop_index ‘PhotoObj.run_index’

Scalable Table Removal sd_drop_table ‘PhotoObj’

Introduction

State of the Art





Principles

Nodes Management





28\46 pages

Images Management

Secondary Image Creation sd_create_image ‘Ceria’, ‘PhotoObj’

Secondary Image Removal sd_drop_image 'PhotoObj’

Introduction

State of the Art





Principles

Nodes Management





29\46 pages


USE SkyServer /* SQL Server command */

Scalable Update Queries sd_insert ‘INTO PhotoObj SELECT * FROM

Ceria.Skyserver-S.dbo.PhotoObj’ Scalable Search Queries

sd_select ‘* FROM PhotoObj’ sd_select ‘TOP 5000 * INTO PhotoObj1 FROM

PhotoObj’, 500

Introduction

State of the Art





Principles

Nodes Management





30\46 pages




1. Introduction

2. State of the Art

P

L

A

N




31\46 pages

Image Binding

Internal Processing

Concurrency

Experiments

Command Processing

Let Q a scalable query using the PhotoObj image: sd_select ‘COUNT (*) FROM PhotoObj’

Introduction

State of the Art





Find Images in Q

PhotoObj Image Adjustment

Execution of Q

[Litwin, Schwartz & Sahri. WDAS 2006]


32\46 pages

Concurrency

Introduction

State of the Art





Internal Processing

Concurrency

Experiments

SD-SQL Server processes every command as SQL distributed transaction at Repeatable Read isolation level Much less blocking than at Serializable Level

SD-SQL Server performs the split asynchronously with the insert that triggered it It launches the actual splitting as an asynchronous job

called splitter


33\46 pages

Concurrency

Introduction

State of the Art





Internal Processing

Concurrency

Experiments

Splits use exclusive locks on segments and on tuples in RP meta-table. Shared locks on other meta-tables: Primary, NDB

meta-tables

Scalable queries use basically shared locks on meta-tables and any other table involved


34\46 pages

Concurrency: example

Introduction

State of the Art





Internal Processing

Concurrency

Experiments

Splitter sd_alter_table

Dell1.SkyServer

Dell3

Dell2

Dell1

RP

PhotoObj

Exclusive Lock Waiting

Exclusive Lock

Shared Lock

Exclusive Lock

X

X


35\46 pages




1. Introduction

2. State of the Art

P

L

A

N




36\46 pages

Experimental Environment

Introduction

State of the Art





Internal Processing

Concurrency

Experiments

6 Machines Pentium IV 1.7 GHz RAM: 780 Mb & 1 Gb Operating System: Windows 2K Server Ethernet Network: max bandwidth of 1 Gb/s

Use of SQL Analyzer for editing queries Use of SQL Profiler to take measurements


37\46 pages

We use SkyServer database as benchmark Provided and installed at Ceria by Dr. Gray

SkyServer brings the entire database of the Sloan Digital Sky Survey, SDSS We use of the PhotoObj table as an example scalable table In our experiments, PhotoObj has 158,426 tuples (about 260

MB)

Originally, it has 14 M tuples

The SkyServer Benchmark

Introduction

State of the Art





Principles

Nodes Management





38\46 pages

Splitting Measurements

Introduction

State of the Art





Internal Processing

Concurrency

Experiments

21.40 26.4212.157.11

37.1232.0818.9310.54

60.7359.5938.8722.42

104.15104.01

56.7946.17

66.5377.86

146.57165.11

0

50

100

150

200

2 3 4 5

Number of Segments

Spl

it Ti

me

(sec

)

10000 tuples 20000 tuples 40000 tuples

80000 tuples 160000 tuples

[Litwin, Sahri & Schwartz. WDAS 2004]

Splitting of PhotoObj scalable table into 2, 3, 4 and 5 segments according to different capacities


39\46 pages

Image Adjustment

Introduction

State of the Art





Internal Processing

Concurrency

Experiments

(Q1): sd_select ‘TOP 10 objid FROM PhotoObj WHERE objid not in (SELECT objid FROM PhotoObj WHERE objid <= @objidMax’

Query (Q1) execution time

0,7796 0,812 0,8515

0,11 0,1480,32

0,096 0,1760,281

0

0,5

1

39500 79000 158000

PhotoObj Capacity

Exe

cuti

on

Tim

e (s

ec)

With PhotoObj AdjustmentWithout PhotoObj AdjustmentDirectly on SQL Server


40\46 pages

Comparison between SD-SQL Server and SQL Server

(Q2): sd_select ‘COUNT (*) FROM PhotoObj’

Introduction

State of the Art





Internal Processing

Concurrency

Experiments

Execution time of (Q2) on SQL Server and SD-SQL Server

[Litwin, Sahri & Schwartz. BNCOD 2006]

0.093 0.1560.22 0.25

0.326

0.1060.164

0.226 0.2560.3430.436

0.3560.283

0.2030.093

0.016 0.0760.123

0.203 0.22

0

0.10.2

0.30.4

0.5

1 2 3 4 5 6

Number of Segments

Exe

cuti

on

Tim

e (s

ec)

SQL Server Distr SD-SQL Server

SQL Server Centr SD-SQL Server w ith LSV


41\46 pages





1. Introduction

2. State of the Art

P

L

A

N6. Performance Measurements


42\46 pages

Conclusion

Future Work

Conclusion

Introduction

State of the Art





Scalable distributed databases with scalable tables are now a reality with SD-SQL Server No more manual repartitioning

Unlike in any other DBS we know about

The performance analysis proves Efficiency of our design Immediate utility of SD-SQL Server


43\46 pages

Conclusion

Future Work

Future Work

More performance measurements With the SDSS queries With the SkyServer benchmark of 80 Gb size

Error processing Management of fault tolerance

Use of the high availability methods

Introduction

State of the Art






44\46 pages

Conclusion

Future Work

Future Work

Application on other DBMSs Oracle, DB2, etc.

Use of the SD-SQL Server principles on P2P systems or Grid Computing

Use of SD-SQL Server as core component of a virtual repository of eGov documents

Introduction

State of the Art






45\46 pages

Thanking

Work partly supported by CEE Project eGov MS Research CEE Project ICONS

Conclusion

Future Work

Introduction

State of the Art






46\46 pages

Soror [email protected] [email protected]

http://ceria.dauphine.fr/soror/soror.html

Conclusion

Future Work

Introduction

State of the Art





soror sahri [email protected] ceria.dauphine.fr/soror/soror.html

Documents