outline - university of waterlootozsu/courses/cs742/course notes/1...outline !introduction ......
TRANSCRIPT
Page 1
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 1.1
Outline n Introduction & architectural issues
l What is a distributed DBMS l Problems l Current state-of-affairs
q Data distribution q Distributed query processing q Distributed query optimization q Distributed transactions & concurrency control q Distributed reliability q Database replication q Parallel database systems q Database integration & querying q Advanced topics
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 1.2
File Systems
program 1
data description 1
program 2
data description 2
program 3
data description 3
File 1
File 2
File 3
Page 2
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 1.3
Database Management
database
DBMS
Application program 1 (with data semantics)
Application program 2 (with data semantics)
Application program 3 (with data semantics)
description manipulation
control
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 1.4
Motivation
Database Technology
Computer Networks
integration distribution
integration
integration ≠ centralization
Distributed Database Systems
Page 3
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 1.5
Distributed Computing
n A number of autonomous processing elements (not necessarily homogeneous) that are interconnected by a computer network and that cooperate in performing their assigned tasks.
n What is being distributed? l Processing logic l Function l Data l Control
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 1.6
What is a Distributed Database System?
A distributed database (DDB) is a collection of multiple, logically interrelated databases distributed over a computer network.
A distributed database management system (D–DBMS) is the software that manages the DDB and provides an access mechanism that makes this distribution transparent to the users.
Distributed database system (DDBS) = DDB + D–DBMS
Page 4
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 1.7
What is not a DDBS?
n A timesharing computer system
n A loosely or tightly coupled multiprocessor system
n A database system which resides at one of the nodes of a network of computers - this is a centralized database on a network node
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 1.8
Centralized DBMS on a Network
Site 5
Site 1
Site 2
Site 3 Site 4
Communication Network
Page 5
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 1.9
Distributed DBMS Environment
Site 5
Site 1
Site 2
Site 3 Site 4
Communication Network
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 1.10
Implicit Assumptions
n Data stored at a number of sites à each site logically consists of a single processor.
n Processors at different sites are interconnected by a computer network à not a multiprocessor system l Parallel database systems
n Distributed database is a database, not a collection of files à data logically related as exhibited in the users’ access patterns l Relational data model
n D-DBMS is a full-fledged DBMS l Not remote file system, not a TP system
Page 6
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 1.11
Data Delivery Alternatives
n Delivery modes l Pull-only l Push-only l Hybrid
n Frequency l Periodic l Conditional l Ad-hoc or irregular
n Communication Methods l Unicast l One-to-many
n Note: not all combinations make sense
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 1.12
Distributed DBMS Promises
� Transparent management of distributed, fragmented, and replicated data
� Improved reliability/availability through distributed transactions
� Improved performance
� Easier and more economical system expansion
Ch.x/12
Page 7
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 1.13
Transparency
n Transparency is the separation of the higher level semantics of a system from the lower level implementation issues.
n Fundamental issue is to provide data independence
in the distributed environment
l Network (distribution) transparency
l Replication transparency
l Fragmentation transparency u horizontal fragmentation: selection u vertical fragmentation: projection u hybrid
Ch.x/13
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 1.14
Example
Page 8
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 1.15
Transparent Access
SELECT ENAME,SAL FROM EMP,ASG,PAY
WHERE DUR > 12
AND EMP.ENO = ASG.ENO
AND PAY.TITLE = EMP.TITLE Paris projects Paris employees Paris assignments Boston employees
Montreal projects Paris projects New York projects with budget > 200000 Montreal employees Montreal assignments
Boston
Communication Network
Montreal
Paris
New York
Boston projects Boston employees Boston assignments
Boston projects New York employees New York projects New York assignments
Tokyo
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 1.16
Distributed Database - User View
Distributed Database
Page 9
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 1.17
Distributed DBMS - Reality
Communication Subsystem
DBMS Software
User Application User
Query
DBMS Software
DBMS Software
DBMS Software
User Query
DBMS Software
User Query
User Application
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 1.18
Types of Transparency
n Data independence n Network transparency (or distribution
transparency) l Location transparency l Fragmentation transparency
n Replication transparency n Fragmentation transparency
Page 10
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 1.19
Reliability Through Transactions
n Replicated components and data should make distributed DBMS more reliable.
n Distributed transactions provide l Concurrency transparency l Failure atomicity
• Distributed transaction support requires implementation of l Distributed concurrency control protocols l Commit protocols
n Data replication l Great for read-intensive workloads, problematic for updates l Replication protocols
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 1.20
Potentially Improved Performance
n Proximity of data to its points of use
l Requires some support for fragmentation and replication
n Parallelism in execution
l Inter-query parallelism
l Intra-query parallelism
Page 11
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 1.21
Parallelism Requirements
n Have as much of the data required by each application at the site where the application executes
l Full replication
n How about updates?
l Mutual consistency
l Freshness of copies
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 1.22
System Expansion
n Issue is database scaling
n Emergence of microprocessor and workstation technologies
l Demise of Grosh's law
l Client-server model of computing
n Data communication cost vs telecommunication cost
Page 12
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 1.23
Distributed DBMS Issues
n Distributed Database Design l How to distribute the database
l Replicated & non-replicated database distribution
l A related problem in directory management
n Query Processing l Convert user transactions to data manipulation
instructions
l Optimization problem
u min{cost = data transmission + local processing}
l General formulation is NP-hard
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 1.24
Distributed DBMS Issues
n Concurrency Control l Synchronization of concurrent accesses
l Consistency and isolation of transactions' effects
l Deadlock management
n Reliability l How to make the system resilient to failures
l Atomicity and durability
Page 13
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 1.25
Directory Management
Relationship Between Issues
Reliability
Deadlock Management
Query Processing
Concurrency Control
Distribution Design
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 1.26
Related Issues
n Operating System Support l Operating system with proper support for database
operations l Dichotomy between general purpose processing
requirements and database processing requirements
n Open Systems and Interoperability l Distributed Multidatabase Systems l More probable scenario l Parallel issues
Page 14
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 1.27
Architecture
n Defines the structure of the system l components identified
l functions of each component defined
l interrelationships and interactions between components defined
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 1.28
ANSI/SPARC Architecture
External Schema
Conceptual Schema
Internal Schema
Internal view
Users
External view
Conceptual view
External view
External view
Page 15
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 1.29
Generic DBMS Architecture
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 1.30
DBMS Implementation Alternatives
Page 16
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 1.31
Dimensions of the Problem
n Distribution l Whether the components of the system are located on the same
machine or not
n Heterogeneity l Various levels (hardware, communications, operating system) l DBMS important one
u data model, query language,transaction management algorithms
n Autonomy l Not well understood and most troublesome l Various versions
u Design autonomy: Ability of a component DBMS to decide on issues related to its own design.
u Communication autonomy: Ability of a component DBMS to decide whether and how to communicate with other DBMSs.
u Execution autonomy: Ability of a component DBMS to execute local operations in any manner it wants to.
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 1.32
Client/Server Architecture
Page 17
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 1.33
Advantages of Client-Server Architectures
n More efficient division of labor
n Horizontal and vertical scaling of resources
n Better price/performance on client machines
n Ability to use familiar tools on client machines
n Client access to remote data (via standards)
n Full DBMS functionality provided to client workstations
n Overall better system price/performance
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 1.34
Database Server
Page 18
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 1.35
Distributed Database Servers
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 1.36
Datalogical Distributed DBMS Architecture
...
...
...
ES1 ES2 ESn
GCS
LCS1 LCS2 LCSn
LIS1 LIS2 LISn
Page 19
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 1.37
Peer-to-Peer Component Architecture
Database
DATA PROCESSOR USER PROCESSOR
USER
User requests
System responses
External Schema
Use
r In
terf
ace
Han
dle
r Global
Conceptual Schema
Sem
anti
c D
ata
Con
trol
ler
Glo
bal
E
xecu
tion
M
onit
or
System Log
Loc
al R
ecov
ery
Man
ager
Local Internal Schema
Ru
nti
me
Su
pp
ort
Pro
cess
or
Loc
al Q
uer
y P
roce
ssor
Local Conceptual
Schema
Glo
bal
Qu
ery
Op
tim
izer
GD/D
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 1.38
Datalogical Multi-DBMS Architecture
...
GCS … …
GES1
LCS2 LCSn …
… LIS2 LISn
LES11 LES1n LESn1 LESnm
GES2 GESn
LIS1
LCS1
Page 20
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 1.39
MDBS Components & Execution
Multi-DBMS Layer
DBMS1 DBMS3 DBMS2
Global User
Request
Local User
Request Global
Subrequest Global
Subrequest Global
Subrequest
Local User
Request
CS742 – Distributed & Parallel DBMS M. Tamer Özsu Page 1.40
Mediator/Wrapper Architecture