dong xuan (cse/osu) / 2009 design and implementation of smartphone-based systems and networking dong...
TRANSCRIPT
Dong Xuan (CSE/OSU) / 2009
Design and Implementation of Smartphone-based Systems and Networking
Dong Xuan
Department of Computer Science and Engineering
The Ohio State University, USA
Dong Xuan (CSE/OSU) / 2010 2
Outline Smartphones Basics
Mobile Social Networks
E-Commerce
E-Health
Safety Monitoring
Future Research Directions
Dong Xuan (CSE/OSU) / 2010
A smartphone is a mobile phone offering advanced capabilities, often with PC-like functionality
Hardware (Apple iPhone 3GS as an example) CPU at 600MHz, 256MB of RAM 16GB or 32GB of flash ROM Wireless: 3G/2G, WiFi, Bluetooth Sensors: camera, acceleration, proximity, light
Functionalities Communication News & Information Socializing Gaming Schedule Management etc.
Smartphone Basics
3
Dong Xuan (CSE/OSU) / 2010
Smartphones are popular and will become more popular
Smartphone Popularity
4
Dong Xuan (CSE/OSU) / 2010 6
Smartphone Features
Communication/Sensing/Computation
Inseparable from our human life
Dong Xuan (CSE/OSU) / 2010
Our Smartphone Systems E-SmallTalker [IEEE ICDCS10]:
senses information published by Bluetooth to help potential friends find each other (written in Java)
E-Shadow [IEEE ICDCS11]: enables rich local social interactions with local profiles and mobile phone based local social networking tools
P3-Coupon [IEEE Percom11]: automatically distributes electronic coupons based on an probabilistic forwarding algorithm
7
Dong Xuan (CSE/OSU) / 2010
Our Smartphone Systems Drunk Driving Detection [Per-
Health10]: uses smartphone (Google G1) accelerometer and orientation sensor to detect
Stealthy Video Capturer [ACM WiSec09]: secretly senses its environment and records video via smartphone camera and sends it to a third party (Windows Mobile application)
Download & Run Video sent by Email Captured Video
8
Dong Xuan (CSE/OSU) / 2010
Exemplary System I:E-SmallTaker
Small Talk
A Naïve Approach
Challenges
System Design
Implementation and Experiments
Remarks
9
Dong Xuan (CSE/OSU) / 2010
Small Talk
People come into contact opportunistically Face-to-face interaction
Crucial to people's social networking Immediate non-verbal communication Helps people get to know each other Provides the best opportunity to expand social network
Small talk is an important social lubricant Difficult to identify significant topics Superficial
10
Dong Xuan (CSE/OSU) / 2010
A Naive Approach of Smartphone-based Small Talk Store all user’s information, including each user’s full contact
list User report either his own geo-location or a collection of
phone IDs in his physical proximity to the server using internet connection or SMS
Server performs profile matching, finds out small talk topics (mutual contact, common interests, etc.)
Results are pushed to or retrieved by users
11
Dong Xuan (CSE/OSU) / 2010
However……
Require costly data services (phone’s internet connection, SMS)
Require report and store sensitive personal information in 3rd party
Trusted server may not exist Server is a bottleneck, single point of failure, target of
attack
12
Dong Xuan (CSE/OSU) / 2010
E-SmallTalker – A Fully Distributed Approach No Internet connection required No trusted 3rd party No centralized server
Information stored locally on mobile phones Original personal data never leaves a user’s phone Communication only happens in physical proximity
13
Dong Xuan (CSE/OSU) / 2010
Two Challenges
How to exchange information without establishing a Bluetooth connection Available data communication channels on mobile phones
Cellular network (internet, SMS, MMS), Bluetooth, WiFi, IrDA Bluetooth is a natural choice
Bluetooth connection needs user’s interaction due to security reasons How to find out common topics while preserving users privacy
No pre-shared secret for strangers Bluetooth Service Discovery Protocol can only transfer limited service
information
14
Dong Xuan (CSE/OSU) / 2010
System Architecture Context exchange Context encoding and matching Context data store User Interface
15
Dong Xuan (CSE/OSU) / 2010
Context Exchange Exploit Bluetooth service discovery protocol
No Bluetooth connection needed Publish encoded contact data (non-service related) as (virtual) service
attributes Limited size and number( e.g. 128 bytes max each attribute)
16
Dong Xuan (CSE/OSU) / 2010
Context Encoding Example of Alice’s Bloom
filter Alice has multiple contacts,
such as Bob, Tom, etc. Encode contact strings,
Firstname.lastname@phone_number, such as “Bob.Johnson@5555555555” and
“Tom.Mattix@6141234567”
17
Dong Xuan (CSE/OSU) / 2010
Implementation
J2ME about 40 java classes, 127Kb jar file
On real phones Sony Ericsson (W810i), Nokia (5610xm, 6650, N70, N75,
N82)
18
Dong Xuan (CSE/OSU) / 2010
Experiments
Settings 6 phones, n=150, k=7, m=1024 bits, default distance=4m, average of
10 runs Performance Metrics
Discovery time: the period from the time of starting a search to the time of finding someone with common interest, if there is any
Discovery rate: percentage of successful discoveries among all attempts Power consumption
Factors Bluetooth search interval Number of users Distance
19
Dong Xuan (CSE/OSU) / 2010
Experiment Results
Minimum, average and maximum discovery time are 13.39, 20.04 and 59.11 seconds respectively
Always success if repeat searching, 90% overall if only search once
Nokia N82 last 29 hours when discovery interval is 60 seconds
20
Dong Xuan (CSE/OSU) / 2010
Related Work Social network applications on mobile phones
Social Serendipity Centralized, Bluetooth MAC and profile matching, SMS, strangers
PeopleTones, Hummingbird, Just-for-Us, MobiLuck, P3 Systems, Micro-Blog, and Loopt
Centralized, GPS location matching, Internet, existing friends Nokia Sensor and PeopleNet
Distributed, profile, Bluetooth / Wifi connection, existing friends Private matching and set intersection protocols
Homomorphic encryption based Too much computation and message overhead for mobile phone
Limitations Require costly data services (phone’s internet connection, SMS) Require report and store sensitive personal information Bottleneck, single point of failure, target of attack
21
Dong Xuan (CSE/OSU) / 2010
Remarks
Propose, design, implement and evaluate the E-SmallTalker system which helps strangers initialize a conversation Leveraged Bluetooth SDP to exchange these topics without
establishing a connection Customized service attributes to publish non-service related
information. Proposed a new iterative commonality discovery protocol based on
Bloom filters that encodes topics to fit in SDP attributes to achieve a low false positive rate
22
Dong Xuan (CSE/OSU) / 2010
Exemplary System II:E-Shadow
Concept
Application Scenario
Goals and Challenges
System Design
Implementation and Experiments
Remarks
23
Dong Xuan (CSE/OSU) / 2010
Concept
MotivationImportance of Face-to-Face InteractionPrevalence of mobile phones
Distributed mobile phone-based local social networking systemLocal profilesMobile phone based local social interaction tools
Dong Xuan (CSE/OSU) / 2010
Goals and Challenges
Design Goals Far-reaching and Unobtrusive Privacy and Security Auxiliary Support for Further Interactions Broad Adoption
Challenges Lack of Communication Support Power and Computation Limitation Non-pervasive Localization Service
Dong Xuan (CSE/OSU) / 2010
Layered Publishing
Spatial Layering WiFi SSID
at least 40-50 meters, 32 Bytes Bluetooth Device (BTD) Name
20 meters, 2k Bytes Bluetooth Service (BTS) Name
10 meters, 1k Bytes
Temporal Layering For people being together long or repeatedly Erasure Code
Dong Xuan (CSE/OSU) / 2010
E-Shadow Publishing Procedure
Valve Generator
Information
Filter
Database
Sensor
Feedback
Decide
Decide
User Maual Input
Online Data
Mining
Help
BTDevice
BTService
WiFi
Dong Xuan (CSE/OSU) / 2010
Matching E-Shadow with its Owner
Intuitive Approach: Localization However, imprecision beyond 20-25 meters
Dong Xuan (CSE/OSU) / 2010
Human Direction-driven Localization
Direction more important than distance Human observation
A new range-free localization technique RSSI comparison: Less prone to errors Space partitioning: Tailored for direction decision
Dong Xuan (CSE/OSU) / 2010
Walking Route and Localization We allow users to walk a distance
Triangular route: A->B->C in (a), for illustration purposes Semi-octogonal route: A->B->C->D->E in (c), more natural
Take measurements on turning points Calculate the direction through RSSI comparison and space
partitioning
Dong Xuan (CSE/OSU) / 2010
Implementation Information
Publishing Module Database Generator Buffers Control Valve Broadcasting
Interfaces
Retrieval & Matching Module Receivers Localization Decoding & Storage
Sensing Module User Interface
Dong Xuan (CSE/OSU) / 2010
Evaluations (1)-Time & Energy
E-Shadow Collection Time WiFi SSID: 2 seconds BTD: 12-18 seconds BTS: 25-35 seconds
E-Shadow Power Consumption 3 hours in full performance
operation >12 hours in typical situation
Dong Xuan (CSE/OSU) / 2010
Evaluations (2)-Localization
3 Outdoor Experiments:Open field campus
2 Indoor Experiments:Large classroom
Dong Xuan (CSE/OSU) / 2010
Evaluation (3)-Simulations
Large-Scale Simulations:Angle deviation CDFs
12 times of exemplary direction decisions
Dong Xuan (CSE/OSU) / 2010
Related Work Centralized mobile phones applications
Social Serendipity Centralized, Bluetooth MAC and profile matching, SMS, strangers
Decentralized mobile phone applications Nokia Sensor
Distributed, profile, Bluetooth / Wifi connection, existing friends E-Smalltalker
Distributed, no Bluetooth / Wifi connection, strangers Localization techniques for mobile phones applications
GPS Virtual Compass
peer-based relative positioning system using Wi-Fi and Bluetooth radios Limitations
Privacy compromise Unable to capture the dynamics of surroundings No mapping between electronic ID and human face Localization techniques either not pervasive or not accurate for long range
36
Dong Xuan (CSE/OSU) / 2010
Remarks
Propose, design, implement and evaluate the E-Shadow system which lubricates local social interactions E-Shadow concept Layered publishing to capture the dynamics of surroundings Human-assisted matching that works for mapping E-Shadow with its
owner in a fairly large distance Implementing and evaluating E-Shadow on real world mobile phones
37
Dong Xuan (CSE/OSU) / 2010
Exemplary System III:P3-Coupon
Coupon Distribution
A Naïve Approach
Challenges
System Design
Implementation and Experiments
Remarks
38
Dong Xuan (CSE/OSU) / 2010
Electronic Coupon Distribution
Electronic coupons Similar to paper coupons Can be stored on mobile phones
Two distribution methods Downloading from Internet websites
Need to define target group Limited coverage Hard to maintain dynamic preferences lists on central databases
Peer to Peer Distribution No special destination/target group More coverage More flexible user-maintained preferences list
39
Dong Xuan (CSE/OSU) / 2010
A Naive Approach of Peer-to-Peer Coupon Distribution A store periodically broadcast the coupon Users within broadcast range receive the coupon
User can decide whether to use, forward or discard the coupon
Users forward the coupon to others in physical proximity Forwarder’s IDs are recorded in a dynamically expanding list
The coupon is used by some user The store reward all users who have forwarded the coupon
40
Dong Xuan (CSE/OSU) / 2010
However……
Require manually establishing wireless connections Cumbersome Not prompt Not possible for coupon forwarding among strangers
Require recording the entire forwarding path Potential privacy leakage Discourage user’s forwarding incentives
41
Dong Xuan (CSE/OSU) / 2010
Challenge
How to design a prompt coupon distribution mechanism that Incentivize coupon forwarder appropriately for keeping the
coupons circulating Preserve the privacy of coupon forwarders
42
Dong Xuan (CSE/OSU) / 2010
P3-Coupon – A Probabilistic Coupon Forwarding Approach Probabilistic sampling on forwarding path
Keep only one forwarder for each coupon: NO privacy leakage Probabilistically flip ownership at each hop
Accurate approximation of coupon rewards plenty of chances of interpersonal encounters Accurate bonus distribution with 50 coupons and 5000 people
Adaptive to different promotion strategies Flip-once model Always-flip model
No manual connection establishment Connectionless information exchange via Bluetooth SDP
43
Dong Xuan (CSE/OSU) / 2010
System Architecture Store Side
A central server for broadcasting and redeeming coupons Client side
Coupon forwarding manager, coupon exchange, coupon data store, user interface
44
Dong Xuan (CSE/OSU) / 2010
Probabilistic Forwarding Algorithm Always-Flip Model
The coupon ownership keeps flipping with certain probability at each hop. Good at assigning relative bonuses affected by the whole path lengths
E.g. the parent forwarder receives k times the bonus given to children forwarders The flip probability can be calculated in advance by the store, once k is fixed, using
the following formula
45
Dong Xuan (CSE/OSU) / 2010
Probabilistic Forwarding Algorithm Extension: Flip-Once Model
Once flipped, a coupon’s ownership remain the same in a forwarding path. Good at assigning absolute bonuses irrelevant of the number of following
forwarders E.g. hop 1 user gets 10%, hop 2 user gets 5%, etc. The flip probability can be calculated in advance by the store using the following
formula
46
Dong Xuan (CSE/OSU) / 2010
Coupon Format
Coupon description Product description Discounts Coupon issuer Coupon code Start/end date
Coupon forwarder information The current owner
Digital signature Prevent forging fraud coupons
47
Dong Xuan (CSE/OSU) / 2010
Implementation
J2ME about 17 java classes, 1390Kb jar file
On real phones Samsung (SGH-i550), Nokia (N82, 6650, N71x)
48
Dong Xuan (CSE/OSU) / 2010
Experiments
Experimental evaluations Coupon forwarding time Power consumption
Simulation evaluation Number of Coupon holders vs. Time Distribution saturation time vs. Number of Seeds Coupon ownership distribution for probabilistic sampling Deviation between theoretical and actual bonus (Always-Flip, Flip-
Once) Factors
Number of coupons Number of users Number of initial coupon holders
49
Dong Xuan (CSE/OSU) / 2010
Experiment Results
Average coupon forwarding time is 33.52 seconds Nokia N82 last 25 hours with P3-Coupon running in
background One coupon could be delivered to 5000 people within 32 hours Very small deviation between theoretical and actual bonus
distribution with 50 coupons circulating among 5000 people
50
Dong Xuan (CSE/OSU) / 2010
Remarks
Propose, design, implement and evaluate the P3-Coupon system which helps prompt and privacy preserving coupon distribution Probabilistic one-ownership coupon forwarding algorithm Implement the system on various types of mobile phones Extensive experiments and evaluations show that our approach
accurately approximate the theoretical coupon distribution in which the whole forwarding path needs to be recorded
Practical for real-world deployment
51
Dong Xuan (CSE/OSU) / 2010
Exemplary System IV – Drunk Driving DetectionMotivationOur ContributionsDetection CriteriaOur SystemRelated WorkImplementation and EvaluationRemarks
52
52
Dong Xuan (CSE/OSU) / 2010
MotivationCrashes caused by alcohol-impaired driving pose a
serious danger to the general public safety and health13,041 and 11,773 driving fatalities happened in 2007 and
2008*32% of the total fatalities in these two years*
Drunk driving also imposes a heavy financial burden on the whole societyAnnual cost of alcohol-related crashes totals more than $51
billion** * Data from U.S. NHTSA (National Highway Traffic Safety Administration)
** Data from U.S. CDC (Central of Disease Control)
53
53
Dong Xuan (CSE/OSU) / 2010
Motivation
Detection of drunk driving so far still relies on visual observation by patrol officersDrunk drivers usually make certain types of dangerous maneuversNHTSA researchers identify cues of typical drunk driving behavior
Visual observation is insufficient to prevent drunk drivingThe number of patrol officers is far from enoughThe guidelines are only descriptive and qualitativeUsually, it is too late when drunk drivers are stopped by officers
It is essential to develop systems actively monitoring drunk driving and to prevent accidents
54
54
Dong Xuan (CSE/OSU) / 2010
Our Contributions
Propose utilizing mobile phones as a platform for active drunk driving detection system
Design a real-time algorithm for drunk driving detection system using mobile phonesSimple sensors required only
i.e., accelerometers and orientation sensors
Design and implement a mobile phone-based active drunk driving detection systemReliable, Non-intrusive, Lightweight and power efficient, and
No extra hardware and service cost
55
55
Dong Xuan (CSE/OSU) / 2010
Cues for Drunk Driving DetectionCues related to lane position maintenance problems
E.g., weaving, drifting, swerving and turning with a wide radius
Cues related to speed control problemsE.g., accelerating or decelerating suddenly, and braking erratically
Cues related to judgment and vigilance problemsE.g., driving with tires on lane marker, slow response to traffic signals
56
56
Dong Xuan (CSE/OSU) / 2010
Drunk Driving Detection CriteriaExtract fundamental detection criteria from these cuesCapture the acceleration featuresE.g., for the lane position maintenance problems
57
57
Dong Xuan (CSE/OSU) / 2010
Drunk Driving Detection CriteriaFocus on the first two categories of cues
They correspond to higher probabilities of drunk drivingMap them into patterns of acceleration
Probability of drunk driving detection goes higher while the number of observed cues increases
58
Driver’s problems in
maintaining lane position
Abnormal lateral movements
Patterns of lateral
acceleration of vehicles
Driver’s problems in controlling
speed
Abrupt speed variations
Patterns of longitudinal
acceleration of vehicles
58
Dong Xuan (CSE/OSU) / 2010
Implementation
Develop the prototype system on Android G1 phone with accelerometer and orientation sensor
Implement the prototype in Java, with Eclipse and Android 1.6 SDK
The whole prototype system can be divided into five major components ☆ User interface System configuration ☆ Monitoring daemon☆ ☆ Data processing Alert notification☆
60
60
Dong Xuan (CSE/OSU) / 2010
Evaluation - Testing Data Collection
Test data72 sets of data with simulated drunk driving related behaviors
- Weaving, swerving, turning with a wide radius- Changing speed erratically (accelerating or decelerating)
22 sets of data for regular driving- Each one for 5 to 10 minutes
Mobile phone positions in the vehicle
61
61
Dong Xuan (CSE/OSU) / 2010
Evaluation - Detection Performance
Study the accuracy of detecting drunk driving related behaviorsIn terms of false negative and false positive
Study performance in the special case, such as the phone slides in the vehicle during drivingSlides has obvious impacts on detection accuracyMay add additional calibration procedure to solve it (future work)
62
62
Dong Xuan (CSE/OSU) / 2010
Evaluation – Energy Efficiency
Curves of battery level states during mobile phone runningPhone runs without drunk driving detection systemMonitoring daemon of system keeps running, sensing and doing the pattern
matching on the monitoring results
63
63
Dong Xuan (CSE/OSU) / 2010
Related WorkDriver vigilance monitoring and driver fatigue prevention
Monitoring the visual cues of drivers to detect fatigue in driving× Installed cameras just in front of drivers are potential safety hazard
Monitoring through vehicle-human interfaceCapture fatigued or drunk driving through monitoring interactions× Low compatibility, vehicles need to couple with auxiliary add-ons
Detect abnormal driving through GPS and acceleration dataPattern matching with GPS and acceleration data× However, GPS data are not always available
64
64
Dong Xuan (CSE/OSU) / 2010
Remarks
First to propose utilizing mobile phones as a platform for developing active drunk driving detection system
Design and implement an efficient detection system based on mobile phone platforms
Experimental results show our system achieves good detection performance and power efficiency
In the future work, to improve the system with additional calibration procedure and by integrating all available sensing data on a mobile phone such as camera image
65
65
Dong Xuan (CSE/OSU) / 2010
Exemplary System V: Stealthy Video Capturer Background SVC Overview Challenges Our Approaches Experimental Evaluations Remarks
66
Dong Xuan (CSE/OSU) / 2010
Background
More and more private information is entrusted to our friend, the 3G Smartphone, which is getting more and more powerful in performance and diversified in functionality.
67
Dong Xuan (CSE/OSU) / 2010
SVC Overview
Almost every 3G Smartphone is equipped with a camera and the wireless options, such as 3G networks, BlueTooth, WiFi or IrDA.
These wireless connections are good enough to handle certain types of video transmission.
We turn 3G Smartphones into an online stealthy video-recorder.
68
Dong Xuan (CSE/OSU) / 2010
Challenges
Stealthily install SVC into 3G Smartphones Windows Hiding Infection Method
Collect the video information from 3G Smartphones DirectShow Controls Data Compressing
Send the video file to the SVC intender File Sending
70
Dong Xuan (CSE/OSU) / 2010
Infection Method
To embed SVC in a 3G Smartphone is called a infection process.
We employ Trojan horse for downloads as the infection approach.
Our experimental SVC is hidden in the game of ”tic-tac-toe” that we develop in Windows Mobile environment.
71
Dong Xuan (CSE/OSU) / 2010
Triggering Schemes
Triggering Algorithm is designed to determine when to turn on the video capture process and send the captured video to make SVC stealthier and get more useful information.
Three scenarios are under consideration. The first scenario is tracking. The second scenario is related with political or business
espionage. The third scenario is a hybrid one, where SVC is used for
much diversified everyday purposes.
73
Dong Xuan (CSE/OSU) / 2010
Remarks
The initial study exploited from SVC will draw wide attentions on 3G Smartphone’s privacy protection and open a new horizon on 3G Smartphones security research and applications.
We are currently investigating the modeling of smart spyware from the study of ”spear and shield”.
79
Dong Xuan (CSE/OSU) / 2010
Future Research Directions
Smartphone-based Systems and Networking Mobile social networking, e-commerce, e-health, safety
monitoring etc. Easy to start and exciting but too many competitors, lack of
scientific depth Smartphone Core Improvement
Multitasking, power management, efficient local communication protocol, accurate localization, security/privacy protection
Deep but hard to start
81