peerdirect distributed enterprise reach your technical peak

PeerDirect Distributed PeerDirect Distributed EnterpriseEnterprise

Reach Your Technical Peak

The Final Ascent

Session IV – Q4 2003

PeerDirect Distributed EnterprisePeerDirect Distributed Enterprise

■ Introduction to Database Replication

■ Architecture Overview

■ Configuration

■ Replication Rule Design

■ APIs and Event Callback

■ Administration

Agenda

■ Configuration

■ Administration

Agenda

Introduction to Database ReplicationIntroduction to Database Replication

■ Back-up■ Fail-over■ Latency avoidance■ Load balancing■ Reporting■ Distributed environments

Why Database Replication?

Define Distributed?

Data Center

Remote Offices

Remote Users

Goals of a Distributed Enterprise

■ Latency avoidance

■ Risk avoidance

■ Application functionality

What if…

Data Center

Remote Offices

Remote Users

■ Breaks dependency on centralized application architectures

■ Radically improves employee productivity with decentralized applications– Disconnected use

■ Creates an inherently resilient application architecture

■ Eliminates the latency and availability problems caused by the centralized model

PeerDirect Distributed Enterprise

■ Log based– Database activity logged for periodic ‘replay’ at all

locations■ Queue based

– Middleware intercepts application to database activity for periodic replay at all locations

■ Table based– Captures changes, queries source and duplicates

values at all locations

Approaches

■ Log based– Log size– Can not remain dormant long– Periodically need to re-synch– Inefficient use of bandwidth– Typically only hub and spoke topology

■ Queue or Message based– All of the above– Application must be modified

■ Table based– Not real-time– Increases database size

Drawbacks of each approach

■ Configuration

■ Administration

Agenda

Architecture OverviewArchitecture OverviewThe Basics

■ Table based– Vs Log or Queue based

■ System tables for storing metadata■ Engine appears as another user■ Access via SQL and ODBC

Architecture OverviewArchitecture OverviewFeatures and benefits

Feature Benefit■ Bi-directional update-

everywhere model■ Read-write data between

multiple databases

■ Replicate, synchronize, and distribute corporate data across multiple locations

Feature Benefit■ Scheduled

synchronization■ Net change compression■ Strong encryption■ Subset data using

‘slices’

■ Maximize network efficiency, reduce costs

■ Security– Controlled access to

Feature Benefit■ Replicates between

different database types– MS SQL Server and

Progress– Oracle and PostreSQL

■ Supported databases– Progress

– Oracle

– SQL Server

– DB2

■ Share data in mixed environments

■ Consolidated view of corporate data

Feature Benefit■ Multiple topologies

supported– Peer-to-peer– Hub and spoke– Load balance clustering

■ Provide new options for building scalable systems

■ Flexible configurations

Feature Benefit■ Auto-discovery of nodes

within the replication network

■ Improve quality of service and system availability

■ Improved system administration

■ Allows mobile workers who are disconnected or have low-bandwidth limitations access to enterprise applications

Feature Benefit■ Database level

configuration– Adds needed tables to

replicated database to manage replication

– Does not modify user-defined tables

■ Existing application does not necessarily need to be altered

Architecture OverviewArchitecture Overview

■ Dynamic Data Slicing Architecture (DDSA)– Table partitioning - schema– Work set partitioning - query– Column level partitioning - fragment– Dynamic data migration

■ Dynamic Bandwidth-Managed Partner Selection (DBP)– Auto-discovery, auto-balanced– Avoids overloading any one server– Backbone, server and workstation algorithms

■ Collision Avoidance Methodology (CAM)– Record fragment management – related columns– Consistent resolution contracts– Rich API for custom resolution

Patented replication technology

System Tables41 tables ofsetup data

Control Tables1 table per

database table

Data Table “M” Rows

Database

Application

Control Table “M” Rows

Table-based

■ Principal components

PeerDirect Distributed Enterprise

ReplicationEngine(PDRE)

ReplicationDesigner(PDRD)

ReplicationAdministrator

(PDRA)

PeerDirect Distributed Enterprise(PDDE)

■ Configuration

■ Administration

Agenda

ConfigurationConfiguration

■ Communicates via SQL through ODBC■ The closer the Replication Engine is to the

database, the better the performance– Supports Win32 and Linux

■ Configuration of InnerEdge and OuterEdge servers affect the way in which sites will be replicated

Replication Engine

ConfigurationConfigurationPDRE setup

Engine

ReplicateEngine

Replication

Engine

Replication

Engine

Replication

■ Site Types – 3 Types

■ Complete ■ High Availability■ Low Availability

– Based upon■ Data set■ Connectivity

– Determines replication partner selection

Site Types

■ Considerations– A Replication Engine should be on either

end of any WAN connection– PDRE traffic is compressed and encrypted

whereas ODBC traffic is not– On a LAN, consider fault tolerance and

performance vs. cost of hardware– Engine may reside on a different platform

■ Win32 and Linux

Engine placement

ConfigurationsConfigurations

■ Dynamic partner selection algorithms allow for load balancing and fault tolerance

■ Partner selection is affected by a site’s type

■ Available site types– Low Availability– High Availability– Complete

Site types

ConfigurationsConfigurations

■ Low Availability– Sites that are mostly or occasionally

disconnected– Compact

■ High Availability– Sites that are mostly connected but do not

contain all data– ‘Spine sites’

Site types

ConfigurationConfigurationSimple Topologies – Inner Edge

– High Availability sites– All or most data slices– Reliable connectivity

Engine

ReplicateEngine

Replication

Data Center

– High or Low availability sites– Pertinent data slices– Questionable connectivity

Engine

ReplicateEngine

Replication

Remote Office

Simple Topologies – Outer Edge

Engine

ReplicateEngine

Replication

Data Center

Engine

ReplicateEngine

Replication

Engine

ReplicateEngine

Replication

Remote Office

Engine

ReplicateEngine

Replication

Remote Office

Sample PDDE Topology

■ Aircraft Manufacturer– Aircraft maintenance application– Maintenance records were handled either on paper or in different

centralized databases– Maintenance issues cause commercial aircraft delays and military

readiness issues■ Solution: One PeerDirect InnerEdge Server and multiple OuterEdge

Workstations– Remote capabilities, work sets, and occasionally connected users– Aircraft maintenance data captured while aircraft is being maintained

resulting in significant cost savings

InnerEdge

Distributed Enterprise

OuterEdge

■ Medical Imaging– Patient test results contain images and complex data sets– Data only available at site where test performed– Collaboration with doctors at other locations nearly impossible

■ Solution: One PeerDirect InnerEdge Server and multiple OuterEdge Workstations– New test introduced into replication network– Can be viewed by physician at any location at anytime

InnerEdge

Distributed Enterprise

OuterEdge

■ Configuration

■ Administration

Agenda

Replication Rule DesignReplication Rule Design

■ Specify the application database■ Select the tables to replicate■ Define fragments to minimize the number of data

collisions■ Subset data into work sets■ Arrange tables in transaction sets and/or groups■ Verify and save rules■ Prepare to activate replication-enabled database

Basic steps to defining rules

Replication Rule DesignReplication Rule DesignAnalyze the schema Partial Sports2000

Schema

PK,U1 Ordernum

FK1,FK2,FK3,U1 CustNumI1 OrderDate

ShipDatePromiseDateCarrierInstructionsPOTerms

I3 SalesRepFK3 BillToIDFK2 ShipToIDI2 OrderStatus

WarehouseNumCreditcard

RefCall

PK,U2,U1 CallNum

FK1,U1 CustNumCallDateSalesRep

U2 ParentI1 Txt

BillTo

PK,FK1 CustNumPK BillToID

NameAddressAddress2City

FK2 StatePostalCodeContactPhone

Salesrep

PK SalesRep

RepNameRegionMonthQuota

OrderLine

PK,FK1 OrdernumPK Linenum

FK2,I1 ItemnumPriceQtyDiscountExtendedPrice

I2 OrderLineStatus

ShipTo

PK,FK1 CustNumPK ShipToID

ContactAddressAddress2City

FK2 StatePostalCodePhoneCommentsName

Customer

PK CustNum

I3 NameAddressAddress2City

FK2 StateI2 Country

PhoneContact

FK1,I4 SalesRepI1 Comments

CreditLimitBalanceTermsDiscount

I2 PostalCodeFaxEmailAddress

Invoice

PK Invoicenum

FK2,I1 CustNumI2 InvoiceDate

AmountTotalPaidAdjustment

FK1,I3 OrderNumShipCharge

SalesRep Work Set

■ Unit of replication is whole record■ Changes occur at multiple locations

– One user changes Addr– One user changes Limit

■ Can cause ‘False Collisions’

Generic replication

Name Addr City Zip Ctry Pymt Rating Limit

■ Unit of replication is each field■ Changes occur at multiple locations

– One user changes Addr and Limit– One user changes Addr and Zip

■ Can cause ‘Silent Data Corruption’

Common solution to generic replication

Name Addr City Zip Ctry Pymt Rating LimitName Addr City Zip Ctry Pymt Rating Limit

■ Unit of replication is a group of fields■ Changes occur at multiple locations

– One user changes Addr and Limit– One user changes Addr and Zip

■ Helps avoid ‘False Collisions’■ Optimizes the replication cycle

PeerDirect solution - fragments

Name Addr City Zip Ctry Pymt Rating Limit

■ PeerDirect allows you to define the business rules for sub-setting around base tables

Introduction to work sets

HistCreditAcct

ADetl Trans

HistCreditAcct

ADetl Trans

Off Prod

■ Define the ‘Ctry’ base table■ Subscribe the site to its country

Defining work sets

142 - Bouchard

217 - W. Gates

330 - A. Palmer

401 - L. Haney

84006239

93050403

93072677

96193414

97005567

1 - Canada

2 - USA

3 - Australia

Acct12 - New York

29 - Toronto

37 - Montreal

44 - Sydney

550 - N. Peart

98717696

99656643

99700174

■ Each site stores "all and only" the data it needs

■ Tables belonging to work sets are replicated based on a request, called a subscription

■ Nesting work sets allows for further sub-setting of data

Work sets

■ A set of insert, update or delete operations– A single unit of work– Must be completed as a whole

■ Offer the ability to enforce data integrity■ Two types of transactions supported

– Transaction sets– Transactions groups

Introduction to transactions

■ Maintains transaction integrity of dependant table during incomplete replication session

■ Updates all dependant records in a single unit of work

■ Must include a base table■ Creation process similar to work sets

Transaction sets

■ Configuration

■ Administration

Agenda

APIs and Event CallbackAPIs and Event Callback

■ Available for use by all third-party developers

■ Available in both Unicode and ANSI■ 32-bit C-style function calls (standard

calling convention)

APIs and Event CallbackAPIs and Event Callback

■ Errors and Logging■ Control■ Command■ Configuration■ Database Record Information■ Event Handling■ Miscellaneous

API categories

APIs and Event CallbackAPIs and Event CallbackEvent handler callbacks

■ Allows you to modify some behavior■ Receive various kinds of event notifications■ PDRE events include

– Replication monitoring– Custom conflict resolution– Enhanced logging

■ Write and register custom DLLs■ Sample code

– C:\Program Files\PeerDirect\pdre\Samples\EvntClbck

■ Configuration

■ Administration

Agenda

AdministrationAdministration

■ PDRE process listens for commands on port 2584– Network security infrastructure must be

configured to pass TCP data on this port■ ASCII protocol■ Provides interactive or programmatic

interface

Remote protocol

AdministrationAdministrationRemote protocol example

AdministrationAdministration

■ Administers the replication network– Maintenance of site information– Subscribing and unsubscribing to and

from slices– Setting up replication schedules

■ Allows for remote administration of all tasks

GUI Administrator

peerdirect distributed enterprise reach your technical peak

Documents

chicago title massachusetts agency reach the ultimate peak...

k peak 5 k 0...

commission of the european communities › regdata ›...

… view document - martinspeechdebate -...

peak oil - the coming crisis 1.understanding peak oil 2.peak...

horton peak horton peak - hiking...

reach, engage, collaborate target your audience · target...

mera peak & island peak (2/2)

java interface to objectstore...

underdamped unstable · time to reach first peak (undamped...

central idaho dark sky reserve · herd peak jerry peak...

ofvwg: gpudirect and peerdirect - openfabrics alliance ›...

tecnical report one · 30% of the peak domestic hot water...

1. 2 3 sir edmund hillary & tenzing norgay were the first to...

peak performance is within your reach · 2018-04-01 ·...

peak growth, peak qe, peak complacency? · peak growth,...

together we can reach the peak of your business potential

noise near peak field is increased peak width narrow peak...

janne winther christensen - slu.seseconds (guyton & hall,...

summerland 5279ft lakes paciﬁc crest trail owyhigh ... ·...