application development for mobile and ubiquitous computing 5. …ts2/admuc/lecture0910/8... ·...

Department of Computer Science Institute for System Architecture, Chair for Computer Networks

Application Development for Mobile and Ubiquitous Computing

8. Disconnected Operations

Dr. Ing. Thomas SpringerTechnische Universität Dresden

Chair of Computer Networks

Structure

Disconnected Operations• CODA filesystem• Mobile Database Systems• Rover system• Generalization of concepts

Ocassionally-Connected Operations• Concepts• Weakly-connection support in CODA

Autonomous Operations

Dr. Thomas Springer 2Application Development - 8. Disconnected Operations

Motivation

Challenge: disconnections due to mobility• foreseeable (due to energy saving, communication costs,

location changes)• unforeseeable (uncovered regions, unavailability of

servers, network congestions)

Goal:• Availability of application functionality and data also

during phases of disconnection• preparation of disconnections during connection phase• Transparency for programmer and user:

o Application perspective: hiding disconnections from applications

o User perspective: application usage without interuptions


Coda – Distributed File System

Distributes file system with disconnection supportgoal - high availability and scalabilitysupports a global namespace

replication and caching of whole files at client sideemulation of file server functionality duringdisconnectionsprefetching of files (hoarding) to improve fileavailabilityoptimistic consistency mechanismsautomated conflict detection and resolutionapplication dependent conflict resolution

Noble, Satyanarayanan (Carnegie Mellon University)Dr. Thomas Springer 4Application Development - 8. Disconnected Operations

Coda - Architecture

distributed and replicated file server on server sideproxy on client side (cache manager, server emulation)


Disconnection Handling in Coda Cache Manager

Hoarding

Emulation Reintegration

disc

onne

ctio

n

disconnection

physical reconnection

logical

reconnection

hoarding database, prefetching

server emulationfiles from cachelogging of updates

reply of logsautomated conflict resolutionapplication specific conflict resolver

Noble, Satyanarayanan (Carnegie Mellon University)


Coda – Phases

Preparation phase – Hoarding• direct access to remote file servers• client-side caching of whole files• prioritized caching algorithm

o balance between- short-term need of caching for performance and- long-term need of caching for availability (hoarding)

o Least recently used scheme (LRU) for cache managemento combined with per device prioritized file list (hoard

database)• user defined file list for prefetching

o hoard profile: script for specifying a set of files to be cachedo indication of their importance based on hoard priorityo tool support for creating hoard profiles

• periodic cache updates – hoard walkingo first phase – update file sets (e.g. for full path hoarding)o second phase – re-evaluation of all priorities in cache and

hoarding db, cache update


Coda – Sample Hoard Files


Coda – Phases

Disconnection phase - Emulation• optimistic locking strategy• local file changes immediately visible• error message for access to uncached files• modification record (change modify log, cml)

o log entry contains copy of the performed file operation and the version state of all involved files

o operations are reexecuted on server during reintegration (replay log)

• optimization of modification logo reduce required storage for change modify logo increases reintegration performanceo any operation which overwrites the effect of previous operation

cancel corresponding record logs (e.g. delete cancels create and write operations)

• versioning of files for conflict detection


Coda – Phases

Reintegration phase – Reintegration• logged actions send to server and replayed• conflict detection and automatic conflict resolution based on

version comparisiono same name for a newly created directoryo parallel change of directory attributes as client and servero file changed in parallel at client and server

• application specific conflict resolutiono application specific conflict resolvers provided by application

developer (includes knowledge about file content)o tool for manual conflict resolution


DISCONNECTEDOPERATIONS IN MOBILE DATABASE SOLUTIONS

Embedded Mobile Databases• Local lightweight database embedded into application• Examples: PointBase, SQLite, iAnywhere‘s UltraLite, etc.

Mobile Client/Server Databases• Lightweight database synchronized with database server• Examples: DB2 Everyplace, Oracle 10g lite, Tamino

Mobile Suite etc.

Classes of Mobile Database Systems


Integrated Mobile Databases

Pointbase Micro:Java-based relational databaseoptimized to run on the Java Micro Edition (CDC and CLDC/MIDP) and Java SE platforms. ultra-compact footprint (<45K for J2ME MIDP) embedded within a Java applicationsAccess via JDBCSubset of SQL92 Transaction support


SQLiteEmbedded SQL database engineServerless, reads and writes directly to ordinary disk filesFeatures• Support of local ACID

transactions including nested transactions

• Implements most of SQL92 • Self-contained (no external

dependencies)• Cross-platform support (db files

can be moved)• Small code footprint: < 300KB

fully configured or less than 180KB with optional features omitted.



DB File(one file per database)DB File

(one file per database)

Interface

SQL Command Processor

Tokenizer

Parser

Code Generator

Virtual Machine

Database Management

DB File(one file per database)


Local data access• Runs on local device, embedded into application• No dependency on established, permanent network

connection• Small footprint• Good performance

• DB in the scope of one application -> no sharing of databetween applications


Mobile Client/Server Databases - Overview

A mobile database system• consists of a set of components• components are distributed over mobile and stationary

device

Two architectures• Extended Client/Server system• Middleware system with replication server


Extended Client/Server system

Client runs lightweight replica containing a subset of themaster DBLightweight DB performs operations locally in disconnectedmode (server emulation)Resynchronisation not permanently but• triggered by client or• periodically performed by system

Mobile Client Database System

mobile Database (replica)

Lightweight DBMS

Synchronization Client

Master database

Server-DBMS

Synchronization Serverreplica selection

Synchronization

Figure according to „Mobile Datenbanken und Informationssysteme“ Hagen Höpfner, Can Türker, Birgitta König-Ries


Middleware system with replication server

Intermediary middleware server for replication providing• decoupling of Mobile DB and Master DB – multiple sources can be

integrated at the replication server (transparent for client andserver)

• replication server can disburden data sources• access control at replication server

Mobile Client


Lightweight DBMS


Mobile Client


Lightweight DBMS


Database System

Master database

Server-DBMS

Synchronization Server

Database System

Master database

Server-DBMS

Synchronization Server

... ...

Replication Server

Figure according to „Mobile Datenbanken und Informationssysteme“ Hagen Höpfner, Can Türker, Birgitta König-Ries


Replica selection

Explicit replica selection• Client or administrator define data to be replicated at

connection phase• Main difference to caching (implicit selection of cached

data)

Replica definition typically in two steps1. Design time of DBS: Definition of data for potential

replication (publication)2. Runtime of application: Selection of published data for

replication (subscription)


Example: DB2 Everyplace

Architecture: Middleware system with replication server

Figure according to „Mobile Datenbanken und Informationssysteme“ Hagen Höpfner, Can Türker, Birgitta König-RiesDr. Thomas Springer 20Application Development - 8. Disconnected Operations

Example: IBM DB2 Everyplace

DB2 Everyplace:• lightweigth DB system for mobile and embedded devices• supports subset of SQL language

o creation and manipulation of tableso not supported: stored procedures, user defined functions, trigger, views, sub-

queries

GUI for creation of DB queries• Query-by-Example (QBE)

Sync Client• triggers synchronisation• Communication with Sync Server• Synchronization based on database tables

Sync Server• User authentification• manages synchronisation between DB2 Everyplace and Master DB• propages all changes in DB2 Everyplace to Master DB and updates in

Master DB to DB2 Everyplace


Replication

Tools• Mobile Device Administration Center (MDAC)• XML Scripting tool

Allow definition of part of Master DB to be replicated• at granularity of tables• specification of horizontal or vertical filters to select

particular rows or columns• target table can be created out of data from multiple

tables (join, merge)• target tables can be read-only or read-write• for read-write restrictions

o no aggregations for row/column selectiono source table must not be created based on join or merge of

other tableso just tables replicated (no integrity definition nor trigger)


Ref: DB2 Everyplace Database Release 8.1: Architecture and Key Features, Thomas Fanghänel, Jonas S Karlsson, Cliff Leung

Architecture of the DB2 Everyplace engine

Application 1 Application n

CLI/ODBC/JDBC

SQL Runtime Environment

Parser

Semantics

Interpreter

OptimizerCatalog

Management

Data Manager Services DMS

Record Manager Index Manager

Buffer & Recovery Mangement

File System Interface

Cry

ptop

lugi

n

Cry

pto

Libr

ary

(Opt

iona

l)

Catalog TablesCatalog Indexes

Temporary TablesShadow files

User TablesUser indexes

•••••


Queries and Transactions

DB2 Everyplace supports subset of SQL• supports join, merge, sort• creation and removal of tables and indexes on mobile

client• tables can contain primary keys, foraign keys and check-

clauses• QBE supported on Palm devices

o for data view and manipulation only (no table creation)

• Changes of read-write tables performed within transactions

o local changes are tentative until synchronization is sucessfully performed with master DB


Synchronization

Initiated• manually by user• automatically by Sync Client

Steps for synchronizing local changes with master DB1. After initiation Sync Client sends request to Sync Server2. Sync Server authenticates request (request contains login and

password of user), if sucessful request is enqueued, client receives ack

3. Changes are sent to Sync Server and stored in Staging table, enables continuous work at client

4. Changes in Staging table performed on mirror database, conflict detection and resolution

5. Changes in mirror table are logged in change table, log is used to propagate changes on master DB


Steps for synchronizing local changes with master DB

Mobile Client

DB2 Everyplace DB

DB2 Everyplace table

Middleware

Mirror DB

Database Server

Master DB

DB2 Everyplace table input

queue

Admin Control DBsync request

Staging table

Mirrortable

Mirrortable

Staging table

copy updates, resolve conflicts

writeupdates

DB2Log

Mirrortable

Change Data Tables

Logging

Capture Apply


Conflict handling

Detection based on version numbers• For each table row version number maintained by Sync

Server and Sync Client• Differing version numbers mark conflicts

Conflict Handling• Default:

o changes rejected, o transaction rolled backo logging of conflicto mobile DB updated with data from mirror DB

• User-based conflict resolutiono based on program for log searcho application-specific conflict resolution implementable


ROVER - A DISTRIBUTED OBJECT-ORIENTED SYSTEM

Rover – Replication of Objects

distributed object-oriented systemsupport for disconnection handling and low bandwidth connections• dynamic placement of objects• queued remote method calls


Rover - Architecture

Client-side

Access Manager

ObjectCache

RPCLog

application application

NetworkScheduler

DCE

HTTP

TCP

SMTP

Access Manager

RPCLog

TCP

SMTP

DCE

HTTP

NetworkScheduler

application

application

system

network

Server-sideimport/invoke/export

bind/fetch/flush

HTTP/SMTP/TCP


Rover – Preparation and Disconnection

Preparation• replication of application objects (code and data)• primary object on home server• secondary objects (replicated objects) on client• caching and prefetching of objects (what objects? –

application, when? – network scheduler)

Disconnection• local log of method calls within objects• application dependent operations for log optimization

(maintained and performed by access manager)• local changes immediately visible (marked as tentative)


Rover – Reintegration

Reintegration of changes triggered by application• export method called on object• call log of object is transformed into remote calls• written to local Operation Log maintained by Network

Scheduler• Network Scheduler controls connection and remote call

transfer to home server


Rover – Reintegration (Cont.)

Conflict detection based on version vector for each objectConflict resolution per method• conflict resolution performed at home server• each method contains code for conflict resolution• user has to handle conflicts if no automated resolution is

possible changes are committed after successful execution on home serverseveral levels of consistency for objects (uncacheable, immutable, verify-before use, verify-if-service-is-accessable, expires-after-date, service-callback)


GENERAL PRINCIPLES OFDISCONNECTEDOPERATIONS

Disconnected Operations - Priciple

TransitionFunction

Proxy-Client

Proxy-Server

Dispatcher

Conn

ected

Ser

vice

Func

tions

Disc

onne

cted S

ervic

eFu

nctio

ns

Proxy-Client

Proxy-Server

Dispatcher

Conn

ected

Ser

vice

Func

tions

Disc

onne

cted S

ervic

eFu

nctio

ns

introduction of proxyimplementation of functionality for connections and disconnections (f.connected(), f.disconnected())transition function for proxy state between connection/disconnection

• monitoring of actions

• caching, prefetching

• change of data copies

• recording of actions

reintegration with server

local server emulation


Disconnected Operations – 3 Phases

preparation phase• ensure local availability of data and code

o replication of data and cachingo monitoring of user actions and prefetchingo static or dynamic installation of emulation code on client

disconnection phase• local server emulation

o changes of local data copieso recording of actions

reintegration phase• mediate local changes to server

o conflict detection and resolution


General Architecture

server-side Proxy

client-side Proxy

Application

Prefetching

Caching

read data

Synchroni-zation

Write Log

write data

transfer control Server

Synchroni-zation

OCCASIONALLY CONNECTED OPERATIONS

limits of disconnected operations• limited task continuity due to lack of access to remote data• local changes not visible to remote clients• probability of conflicts raises with disconnection time• local resources required for caching, logging of operations and

server emulation• bad performance and response times using low bandwidth links

weak connection support• mode of operation to hide network latencies• minimized disconnection times• lower resource requirements on mobile devices• increased consistency and task continuity• decreased probability of conflicts

low bandwidth links require explicit support

Occasionally connected operations


Support of weak connections

Mechanism for adaptive and dynamic use of connection resources• load of new data and cache updates• reintegration of local changes

Load of requested data has highest priority to support task continuity and short response timeCache updates and reintegration independent from response times • fast reintegration decreases size of local operation log

and conflict probability• slower reintegration enables optimization of local

operation log (e.g. eliminate … operations), reduced amount of data for network transfer


Partially Connected Operations

Extension of Andrew File System (AFS)All file system writes are performed locally and loggedThree prioritized classes of data transmissions for efficient network usage1. Interactive communication2. Read/Fetch data3. Write data for reintegrationClasses have different prioritiesOne send queue for each classSpecific send strategy to prioritize interactive communication and high availability of data in cache


Coda - Weakly Connected Operations

Extension of disconnected operations in Coda

implicitly 2 classes of data transmissions1. Read operations for missed and outdated files with high

priority2. Trickle reintegration in background with low priority only

if no read operations active

Calculation of tresh-hold for max. file size for reading new files• error message for larger files


Coda - Weakly Connected Operations

disc

onne

ctio

n strong

connection

weak

connection

resolving strong distinction between connected and disconnectedif small bandwidth network connection available• operation in disconnected mode• but performance of reintegration functionality


Coda – Trickle Reintegration

ongoing background reintegration with minimally impacton foreground activity• balance between

o effectiveness of log optimization ando conflict probability

• minimal dwell time (adjusted according currentconnectivity)

o replay operation remains defined time in modify log (agingwindow) (10 min. based on simulations)

optimizationreintegration


AUTONOMOUS OPERATIONS


Generalization of disconnected operations• local availability of data and code required• can be performed on mobile device or infrastructure host

o independent of established connection

• operation executed independently from user and network connection

• autonomous behavior (failure recovery, resource discovery)

• transmission of results to application or user if connection is re-established

use of dynamic code installation or mobile code



migration: transfer of code, data, statecollect intermediate resultslocal interactions with server objects and other resources transfer of the result after reconnection


Summary

Goal: work with mobile devices similar to stationary PCs

Combination of Mechanisms• Emulation of server on client-device• Caching/Prefetching• Replication• Logging• Adapted network usage

Mechanism are general, no application specific behaviour

Autonomous Operations• more general concept including disconnected operations


IBM DB2 Product overview“Mobile Datenbanksysteme” Bela Mutschler, Günther Specht.„Mobile Datenbanken und Informationssysteme“ Hagen Höpfner, Can Türker, Birgitta König-RiesThomas Fanghänel, Jonas S. Karlsson, Cliff Leung: “DB2 Everyplace Database Release 8.1: Architecture and Key Features”“The Ultra-Compact Database for Enterprise Mobility”Product Datasheet. http://www.pointbase.com/resourcecenter/pdfs/micro.pdf“UltraLite Database Management and ReferenceSybase iAnywhere”PointBase Micro : Product overview

References


application development for mobile and ubiquitous computing 5. …ts2/admuc/lecture0910/8... ·...

Documents