Ph.D. Dissertation Defense

On Peer-to-Peer Data Management in Pervasive Computing Environments

Filip Perich

May 3, 2004

agenda

MotivationThesis StatementCentral Challenges in Data Management for Pervasive Computing EnvironmentsConceptual Model and Implementation PrototypeConclusions

on the road, in the mall, at work, at home

enabling technologyDevices increasingly more {powerful ^ smaller ^ cheaper}

Daily interaction with dozens of computing devicesMany of them mobile

traditional mobile computingMobile devices = traditionally standaloneAll required information present locallySynchronization through cable connection

Wireless connectivity = new way for data exchangeCellular networks, satellites, LANs and short range networks

Mobile devices now able to connect to the InternetClient end-points in client/proxy/server modelInitiate actions Receive information from servers

traditional mobile computingtranscoding

ad-hoc networking technologiesAd-hoc networking technologies (e.g. Bluetooth)Main characteristics:Short rangeSpontaneous connectivityFree, at least for now

Mobile devicesAware of their neighborhoodCan discover others in their vicinityInteract with peers in their neighborhoodInter-operate and cooperate as needed and as desiredBoth information consumers and providers

peer-to-peer model for ad-hoc networks

pervasive environment paradigmPervasive Computing Environment

Ad-hoc mobile connectivitySpontaneous interaction among mobile devicesNo guarantee of infrastructure support

PeersService/Information consumers AND sources

Highly dynamic data intensive deeply networked environmentEveryone can exchange informationData-centric modelSome sources generate streams of data, e.g. sensors

problem description

problem description

People increasingly rely on the use of information in electronic form

Require instant and complete access to anythingAnytimeAnywhere

Using their mobile devicesI.e., mobile devices making it happen

problem description

Traditional data management modelActive usersPassive databases

problem description

Stonebrakers data management modelPassive usersActive databases

problem description

Pervasive computing data management modelActive usersActive databases

problem description

A device must be able to

Determine what information will a user require

Determine when will the user need it

Be able to obtain the required information

dissertation goal

Combine and cross-interact research onNetworkingData managementContext-awareness

thesis statementIn order to exploit data-intensive pervasive computing environments, mobile devices must act as semi-autonomous, self-describing, highly interactive and adaptive peers that employ cross-layer interaction between their data management and communication layers for inferring and expressing information they need, and for obtaining and storing such information by pro-actively interacting with other devices in their vicinity using available short-range ad-hoc networking technologies.

contributions

Conceptual model for data managementPro-active peer-based interaction modelSupport for data routing, caching, querying, transactions, and trust

Formal user profile languageExpressing and reasoning over user and data profiles

MoGATU - data management middleware

topics not covered in this dissertation

How a device learns current context?Assume context info provided externally

How to handle security?Assume communication integrity

How to handle privacy?Assume data privacy concerns

central challenges in DM for PCEFFW 32FFW 22

communication challenges

Ad-hoc networksIRIEEE 802.11b (ad-hoc mode)Bluetooth

RoutingTable vs. source-initiatedDSDV, DSR, AODV

discovery challenges

Device discovery

Data/resource discoveryNumeric, keyword, interface, semantic basedRegistry lookupJini, Salutation, UPnP, UDDI, SLPPeer-basedBluetooth SDP, ESDP, DReggie

location management challenges

LocationImportant contextual informationAbsolute or relative

TriangulationGPS, cellular telephony providersSignal strengthRADAR, IEEE 802.11a/b/g

data management challengesWireless data management

Wireless networksLimited bandwidthProne to frequent failureAsymmetric channels

Mobile devicesLimited computing resourcesLimited battery power

data management challenges

Wireless data managementBased onMobile DBMSMobile FS

Client/server modelautonomyheterogeneitymobilitydistribution

data management challenges

Wireless data management challenges

Query processing and optimizationCachingReplicationName resolutionTransaction management

data management challengesWireless data managementRelaxing properties of wired-network solution and ACID

Location aware and proxy-based query processingKottkamp & Zukunft, Cost function, 1998Dunham et al, GPS LDQ, 2000Bukhres et al, Proxy Mailbox, 1997

Local access via cachingLauzac & Chrysanthis, View holders, 1998Acharya et al, Broadcast disks, 1995Goodman et al, Infostations, 1997

Weak commit and split transaction managementChrysanthis et al, Pro-motion, Local C&V Nested-split T, 1997Dunham et al, Kangaroo, 1997Demers et al, Bayou, Reconciliation, 1994

data management challenges

New pervasive computing challenges

Spatio-temporal data variationLack of a global catalog / schemaNo guarantee of reconnectionNo guarantee of collaboration

transaction mgmt challenges

ACID model too restrictiveIn extreme, no property may be supported

Concurrency controlLock and time based locking techniquesNeed distributed recovery mechanismFFW 29

security + privacy challenges

Secure transmission medium

Lack of guaranteed integrity for stored data

Theft of users data and devices

additional challenges

Need to avoid humans in the loopContext-based predicting of data importance and utility

Declarative (or inferred) descriptions helpInformation needsInformation capabilityConstraintsResourcesDataAnswer fidelity Expressive profiles can capture such descriptionsFFW 32

additional challenges

Expressing profilesCherniak et al, 2002Profiles = data domains + fixed utility values Not targeted to pervasive environments, but applicablePROFILE TravelerDOMAINR = www.hertz.comS = +shuttle +logan +downtownD = +directions +logan +boston +downtownUTILITYU (S) = UPTO (1,2,0)U (R [#D > 0]) = 5

Ongoing research effort onData management in wireless domainDoes not interact with networking layerWireless networkingDoes not interact with data management layer

data management model for pervasive computing environments

model postulates

Postulate 1All devices in pervasive computing environments are peers

Postulate 2All devices are semi-autonomous, self-describing, interactive, and adaptive

Postulate 3All devices require cross-layer interaction between their data management and communication layers

abstract architecture

abstract architecture

abstract architecture

data representation

Device has a subset of globally available data

To support heterogeneityDevice only speaks a common language

Common languageWeb Ontology Language OWLOWL-S

metadata representation

To provide information aboutInformation providers and consumersData objectsQueries and answersTo describe relationshipsTo describe restrictionsTo reason over information

user, device and data profiles

User profileBDI preferences, schedule, requirements

Device profileConstraints, providers, consumers

Data profileOwnership, restriction, requirements, process model

user profile

http://mogatu.umbc.edu/bdiAgentActionPlanPolicyTimeBelief,Desire,Goal,IntentionTStatement,Cond- TStatementCondition

user profilefor Agentabout itsBeliefsAbout user or environment/contextStatementsUnconditional (Facts)ConditionalActionsPoliciesSystemPersonal (preferences)DesiresIntentions

user profileDevice reasons over BDI profiles Generates domains of interest and utility functionsChanges domains and utility functions based on contextPriorityFor ordering beliefs, desires, intentions and actions

query representation

Explicit queriesThose asked by human users(O, s, q, S, t)Set of used ontologies OSelection list sFiltering statement qCardinality STemporal constraints t

query representation

Implicit queriesInferred by devices from a users profileFrom beliefs and desires

Belief / Desire = (O, s, q, S, l, t, P)Location constraint lTemporal constraints tProbability of a user asking the query P

architecture modelInformation ProviderInformation ProviderInformation ConsumerInformation ConsumerCommunication InterfaceCommunication InterfaceCommunication InterfaceInformation ManagerRoutingDiscoveryJoin QueryingQueryingCachingTransactionTrust Reputation

application layer

Information ProviderHas a subset of world knowledgeRegisters and interacts with local Information Manager

Information ConsumerAccess to information through Information ManagerRegisters and interacts with Information Manager

communication layer

Communication InterfaceNetwork abstractionRouting / Discovery not concerned with underlying networkRegisters and interacts with local Information ManagerInformation Manager still aware of the network attributes

data management layerOne Information Manager (InforMa) per deviceVarious types based on device strength and willMost of data management functionalityData + Service DiscoveryCachingQuery RoutingQuery ProcessingJoin Query ProcessingTrust + ReputationTransaction

routing

Data-based table routingPromiscuous mode Exploits cached advertisement information

DEXA 2002informa_route_query(f, t, query) { if (local(t)) if (answer = cached_answer(query) && valid(answer)) return answer if (answer = contact_local_info_provider(f, t, query)) return answer else error(no_answer)

if (intercept_foreign_queries) if (answer = cached_answer(query) && valid(answer)) return answer;

if (willing_to_forward) if (nexthop = lookup(t)) return forward_to(nexthop) else if (local(o)) forward_to_random(o, d, query) else error(no_destination) else error(forwarding_denied) error(no_answer)}Data + Service DiscoveryCachingQuery RoutingQuery ProcessingJoin Query ProcessingTrust + ReputationTransaction

caching

Information Manager caches incoming messagesOWL encoded information in profiles and data objects

Cache replacement policies based onLast access timeReasoning over associated metadata

IEEE TKDE 2004Data + Service DiscoveryCachingQuery RoutingQuery ProcessingJoin Query ProcessingTrust + ReputationTransaction

cachingReplacement policiesTraditional LRU and MRU

cachingReplacement policiesLRU / MRU + profile-based space pre-allocation

cachingReplacement policiesSemantic-basedSpace pre-allocation based on context and profile knowledgeDynamic utility values for each cache entry based on context and profile

FFW 66

caching simulation settingsOne day activity of a personStarts at 8AM in AnnapolisTravels to Washington D.C. for 10AM meetingLunch at noonTravels to UMBC for 2PM meetingShopping from 4:30PM until 5:30PMDinner at 8PM in Annapolis

Her PDA has some profile knowledgeLimited information about the schedulePlus other information about preferences and requirements

caching simulation settingsInformation sourcesCars, street lights, buildings, subway and peopleSome sources available at every time instance

Available informationDifferent type (8)Directions, traffic, gas, parking, merchandise, dining, subway, and anything elseDifferent utility valueDynamically computed by each Information Manager based on context and profile knowledge

Ex1: cache allocationHow does profile-based pre-allocation compare to measured LRU cache allocation?

Recorded cache content at every minute of the 12-hour simulation period while using traditional LRU for cache replacementComputed how context and profile knowledge affects cache pre-allocationComputed how a prior omniscient knowledge would affect cache pre-allocation

Without using the additional knowledge, some important data were not cachedLRU did not cache any subway dataLRU kept on caching restaurant data after the lunch was over

Ex1: cache allocationFFW 66

Ex2: single queries with varying updateHow does each cache replacement algorithm perform given varying update periods and single queries only?

Update period from 1 to 128 minutesDevices preferred refresh rate to prolong battery lifeA rate at which information providers appear/disappear Most data has 10-minute lifetime

Person asks 1 to 4 unique queries during each activityWhile driving, one query about traffic, one for a gas station, etc.54 queries total

Ex2: single queries with varying updateFFW 66

Ex3: repeating queries with varying updateHow does each cache replacement algorithm perform given varying update periods and REPEATING queries?

Update period from 1 to 128 minutes

Person asks same queries once every 5-minute period during each activityWhile driving from 8AM until 8:45AM, the person asks for traffic update once every 5 minutes = 9 queries374 queries total

Ex3: repeating queries with varying updateFFW 66

Ex4: repeating queries with constant updateHow does each cache replacement algorithm perform given a constant update period and queries repeating at different intervals?

Update period is fixed 5 minutesMore realistic scenarioDevice can reflect context changes every 5 minutes to preserve resources

Person asks same queries once every N-minute period during each activityN from 1 to 128 minutes

Semantic-based approach stays in 90% range

Ex4: repeating queries with constant update

query processing

Information consumer asks InforMa for dataInformation Manager attempts to answer queryFrom cacheBy contacting local information providerBy contacting remote information managerData + Service DiscoveryCachingQuery RoutingQuery ProcessingJoin Query ProcessingTrust + ReputationTransaction

collaborative query processing .

Selection and joinsMultiple peers can interactEach device provides some data streamAnd computation resources

VLDB-J 2004Data + Service DiscoveryCachingQuery RoutingQuery ProcessingJoin Query ProcessingTrust + ReputationTransactionAAAAAABBBBBCCCD

collaborative query processing

Using Contract Net Protocol concepts

Does not require concurrent presence of all data sourcesDevice executing a join query, can pre-cache one data stream while waiting for second data source

collaborative query processingTimeContractorBidderContractor transmits its query + cardinality and time constraints for an answer Each bidder willing to collaborate with a sufficient answer responds with its bidContractor chooses a winner among successfully received bidsContractor and winner synchronize their streams by sending ACKs to one anotherContractor requests data from a winner and terminates when all data received or data timer expiredCall-for-queryBidBid AwardACKACKData / Next requestClose streamAward TimerACK TimerACK TimerData TimerBid Submission TimerFFW 82

cqp simulation settingsGloMoSim-based simulatorRealistic model environmentStreets and intersections south of 72nd St in Manhattan793 beacons, one per intersectionSource of location-dependent information for a given intersection10 distinct ontologies/schemas100 carsTaxi and Visitor-like mobility modelsProfiles describing driversCan query other cars and intersection beacons for informationSelection and join (over 2 or 3 stream) queries+ Predictable and organized movement behavior+ Drivers more likely to ask similar questions- Moving faster than pedestrians

cqp simulation settings (2)36 distinct queries per car1 to 16 conjunctive boolean predicates1/2 static (location independent) and 1/2 dynamic queries18 queries involved one stream12 queries involved two streams (join over 2 streams)6 queries involved three streamsProfileHints on when and where each query could be askedVarying accuracy from 0% to 100%

Ex1: Query Success Rate vs. Profile AccuracyHow does profile accuracy affect query success rate?Vary profile accuracy from 0% to 100% in steps of 20%Measure performance of queries over 1, 2 and 3 streams2 scenarios No car is willing to help vs. probability of a car helping is 75%

Profile knowledge helpsComplete knowledge performed twice better than no profile (base case)High mobility had negative effects interacting cars move away too quickly

Help of other cars was required to improve performance

Ex1: Query Success Rate vs. Profile Accuracy75% willingness to helpFFW 82

Ex2: Computing Cost vs. Profile AccuracyHow much work a car has to perform for itself?

Vary profile accuracy from 0% to 100% in steps of 20%Calculate computing cost by measuring the operations and time it required to execute the operationsConsider only those operations that a car performs while executing its implicit queries2 scenarios (no help and 75% probability of help)

Cost quickly increases becauseMore accurate profile has twice as more queries as previous levelFrom previous experiment, half of queries could not be satisfied

Ex2: Computing Cost vs. Profile Accuracy75% willingness to helpFFW 82

Ex3: Q Success Rate vs. Willingness to HelpHow the different levels of willingness to help improve the average query success rates?

Profile accuracy at 80%Vary willingness to help, car degree of collaboration, from 0% to 100%Measure performance of queries over 1, 2 and 3 streams

13.5% improvement of average query success rate from no to full help

For queries over 3 streams, 45.6% improvement

Ex3: Q Success Rate vs. Willingness to HelpFFW 82

Ex4: Comp & Net Cost vs. Will to HelpHow much of the total resources was, on average, allocated to helping others?

Profile accuracy at 80%Vary willingness to help from 0% to 100%Collect total amount of computing costs used by each car and the amount of computing done on behalf of others.

Cost for helping others increases both in terms of computing resources and network trafficComputing cost at most 1/3 of total resource consumptionTraffic cost at most 11% of total traffic

Ex4: Comp & Net Cost vs. Will to HelpFFW 82

Ex5: Success Rate/Comp Cost vs. Will to HelpHow different willingness to help levels affect the ratio of utility to cost?

Profile accuracy at 80%Vary willingness to help from 0% to 100%Collect average success rate for answering explicit queries and the average computing cost

The utility to cost ratio remains constantEven though each car executes more computations, it is also able to improve its overall query success rateIncreasing computing cost justified by improving rate

Ex5: Success Rate/Comp Cost vs. Will to Help

transaction model

Optimistic transaction model with emphasis onHigh-rate of successful transaction terminationMaintaining neighborhood-based consistencyNo global consistency guarantee

Using session betweenTransacting devicesNeighboring active witness devices

DEXA 2003Data + Service DiscoveryCachingQuery RoutingQuery ProcessingJoin Query ProcessingTrust + ReputationTransaction

nc-transaction modelRelies on the use of active witnessesNo need of infrastructure support and network reliabilityRedundancy - increased correctness probabilityCollective decision of independent entities

WitnessesSocial device to ensure fairness and correctness of an event involving two partiesAW2W1BW3W2W1AW3Btime

nc-transaction model

Transacting devices solicit neighbors to witness transaction

Witnesses vote on transaction outcome

Parameterized witness group size and voting quorum

nc-transaction modelThree steps

Traditional transaction

T = {S, {R/W}, C/A}

NC-Transaction

TNC = {negotiation, execution, termination}

nc-transaction modelStep ONE: negotiation

Devices wishing to transact negotiateQuery planExpected durationActive witness set

Using Contract Nets principles

Witness Selection ProtocolsO NC-T: One-hop neighborsR NC-T: En-route neighborsO+R NC-T: combination of above

nc-transaction modelStep TWO: execution

Each transacting deviceExecutes according to query planInforms as many witnesses as possible about its proposed actionCommit or abort

Each witnessing deviceCollects termination proposalsOnce enough proposals informs transacting devices about the final action

nc-transaction modelStep THREE: termination

Each transacting device

Commits only when commit commands from a quorum of active witnesses

Otherwise abortFFW 98

transaction - experimentsPrimary goal of NC-TransactionHigh rate of successful transaction terminationPreserve neighborhood consistency

WHAT: Performance of NC-Transaction Comparison to traditional mobile transaction modelRepresentative: Kangaroo++ transactionsEffects of different active witness group sizesEffects of different voting quorum sizes

HOW:MoGATU frameworkGloMoSim simulator

experimental environment

Spatio-temporal environment:200 x 200 m2 field100 nodes36 stationary nodesVariable infrastructure supportRequired by traditional mobile transactions64 mobile nodesRandom waypoint movement5s waiting periodAt speeds 1m/s, 3m/s or 9m/sAODV routing protocol50-minute simulation runs

experimental environment (2)Performance of 2 nodes engaged in transactionsOne minute intervals50 transactions per simulation run10-20s transaction execution period

Infrastructure support varied from none to full support0% - none, 100% - all 36 static nodes present

Active witness group size3 to 33 members

Voting quorum size1% to 100%

Ex1: Infrastructure Support vs. Transaction Success and ConsistencyCompare the performance of NC-Transaction against that of a traditional mobile transaction Differentiate among three versions of NC-Transaction

Traditional model represented by Kangaroo Transaction

Vary infrastructure support from 0 to 100%

Set witness group size, K, to 5 and voting quorum, Q, to 66%

NC-Transaction is betterMore commits and less inconsistenciesDoes not rely on infrastructureR NC-T could not gather enough witnesses

Ex1: Infrastructure Support vs. Transaction Success and ConsistencyFFW 98

Ex2: Witness Group Size vs. Transaction Execution

Measure effect of witness group size on transaction execution. Consider O NC-T only (one-hop peer selection policy)Voting quorum at 66%Vary number of witnesses, K, from 3 to 33

Optimal performance for K = 5For high K, not enough witnesses to start

Ex2: Witness Group Size vs. Transaction ExecutionFFW 98

Ex3: Voting Quorum vs. Successful Transaction ExecutionMeasure effect of different voting quorums on transaction execution. Consider O NC-T onlyWitness group size set to 9K=5 was optimal, but K=9 is better for this experimentVary required voting quorum, Q, from 1 vote to all votes

For K=9, optimal performance is 25%For high values, not enough votes to commit or abort

Ex3: Voting Quorum vs. Successful Transaction Execution

trust and reputation

ProblemsNot all sources and data are correct/accurate/reliableNo common sensePerson can evaluate a web site based on how it looks, a computer cannotNo centralized party that could verify peer reliability or reliability of its dataDevice is reliable, malicious, ignorant or uncooperativeData + Service DiscoveryCachingQuery RoutingQuery ProcessingJoin Query ProcessingTrust + ReputationTransaction

trust and reputation

SolutionNeed to depend on other peers

Evaluate integrity of peers and data based on peer distributed beliefDetect which peer and what data is accurateDetect malicious peersIncentive model: if A is malicious, it will be excluded from the network

Under review

trust and reputation

Device sends a query to multiple peersAsk its vicinity for reputation of untrusted peers that responded to the queryTrust a device only if trusted before or if enough of trusted peers trust itUse answers from (recommended to be) trusted peers to determine answerUpdate reputation/trust level for all devices that respondedA trust level increases for devices that responded according to final answerA trust level decreases for devices that responded in a conflicting way

Each device builds a ring of trust

trust and reputationDistributed Belief Model Functions (Jonker et al. 1999)

Initial Trust FunctionPositive, negative, undecided

Trust Learning Function ++, --, F+/S-, S+/F-, F+/F-, S+/S-, exponential

Trust Weighting FunctionMultiplication, cosine

Accuracy Merging FunctionMax, min, average

trust and reputationFunctionality / stepsInformation source discoveryInformation advertisementQuerying peersAnswering peersCollecting answersRecommendation requestRecommendation responseCalculating final answerUpdating trust beliefFFW 109

trust and reputation - experimentsSpatio-temporal environment:150 x 150 m2 field50 nodesRandom way-point mobilityAODVCache to hold 50% of global knowledgeTrust-based LRU50-minute simulation runs800 questions-tuplesEach device 100 random unique questionsEach device 100 random unique answers not matching its questionsEach device initially trusts 3-5 other devices

results

Answer Accuracy vs. Trust Learning Functions

Answer Accuracy vs. Accuracy Merging Functions

Distrust Convergence vs. Dishonesty Level

Answer Accuracy vs. Trust Learning FunctionsThe effects of trust learning functions with an initial optimistic trust for environments with varying level of dishonesty. The results are shown for ++, --, s, f, f+, f-, and exp learning functions.FFW 109

Answer Accuracy vs. Trust Learning Functions (2)The effects of trust learning functions with an initial pessimistic trust for environments with varying level of dishonesty. The results are shown for ++, --, s, f, f+, f-, and exp learning functions.FFW 109

Answer Accuracy vs. Accuracy Merging FunctionsThe effects of accuracy merging functions for environments with varying level of dishonesty. The results are shown for(a) MIN using only-one (OO) final answer approach(b) MIN using highest-one (HO) final answer approach(c) MAX + OO, (d) MAX + HO, (e) AVG + OO, and (f) AVG + HO.FFW 109

Distrust Convergence vs. Dishonesty LevelAverage distrust convergence period in seconds for environments with varying level of dishonesty.The results are shown for ++, --, s, and f trust learning functions with an initial optimal trust strategy and for the same functions using an undecided initial trust strategy for results (e-h), respectively.

research summaryData + Service DiscoveryCachingQuery RoutingQuery ProcessingJoin Query ProcessingTrust + ReputationTransaction

research summaryInformation ProviderInformation ProviderInformation ConsumerInformation ConsumerCommunication InterfaceCommunication InterfaceCommunication InterfaceInformation ManagerRoutingDiscoveryJoin QueryingQueryingCachingTransactionTrust Reputation

research summary

research summary

research summary

Pervasive Computing EnvironmentHighly dynamic, data intensive, deeply networked environmentSpontaneous connectivity and interaction among devicesNo guarantee of infrastructure supportEveryone can exchange informationNew data management modelActive users, active databases

research summary

Device must be able toDetermine what information will a user requireDetermine when will the user need itBe able to obtain the required information

research summary

Combine and cross-interact research onNetworkingData managementContext-awareness

thesis statementIn order to exploit data-intensive pervasive computing environments, mobile devices must act as semi-autonomous, self-describing, highly interactive and adaptive peers that employ cross-layer interaction between their data management and communication layers for inferring and expressing information they need, and for obtaining and storing such information by pro-actively interacting with other devices in their vicinity using available short-range ad-hoc networking technologies.

contributions

Conceptual model for data managementPro-active peer-based interaction modelSupport for data routing, caching, querying, transactions, and trust

Formal user profile languageExpressing and reasoning over user and data profiles

MoGATUData management middleware

Thank you!

Hide this, just say it out loudHide this, just say it out loudAdd stonbrakers Add stonbrakers Add stonbrakers There is a need to combine cross layer, hide itAdd example of other people who handle these problems change