design and implementation of smartphone-based systems and networking

Dong Xuan (CSE/OSU) / 2009

Design and Implementation of Smartphone-based Systems and Networking

Dong Xuan

Department of Computer Science and Engineering

The Ohio State University, USA

Dong Xuan (CSE/OSU) / 2010 2

Outline Smartphones Basics

Mobile Social Networks

E-Commerce

E-Health

Safety Monitoring

Future Research Directions


A smartphone is a mobile phone offering advanced capabilities, often with PC-like functionality

Hardware (Apple iPhone 3GS as an example) CPU at 600MHz, 256MB of RAM 16GB or 32GB of flash ROM Wireless: 3G/2G, WiFi, Bluetooth Sensors: camera, acceleration, proximity, light

Functionalities Communication News & Information Socializing Gaming Schedule Management etc.

Smartphone Basics

3


Smartphones are popular and will become more popular

Smartphone Popularity

4


Smartphone Accessories

5

Dong Xuan (CSE/OSU) / 2010 6

Smartphone Features

Communication/Sensing/Computation

Inseparable from our human life


Our Smartphone Systems E-SmallTalker [IEEE ICDCS10]:

senses information published by Bluetooth to help potential friends find each other (written in Java)

E-Shadow [IEEE ICDCS11]: enables rich local social interactions with local profiles and mobile phone based local social networking tools

P3-Coupon [IEEE Percom11]: automatically distributes electronic coupons based on an probabilistic forwarding algorithm

7


Our Smartphone Systems Drunk Driving Detection [Per-

Health10]: uses smartphone (Google G1) accelerometer and orientation sensor to detect

Stealthy Video Capturer [ACM WiSec09]: secretly senses its environment and records video via smartphone camera and sends it to a third party (Windows Mobile application)

Download & Run Video sent by Email Captured Video

8


Exemplary System I:E-SmallTaker

Small Talk

A Naïve Approach

Challenges

System Design

Implementation and Experiments

Remarks

9


Small Talk

People come into contact opportunistically Face-to-face interaction

Crucial to people's social networking Immediate non-verbal communication Helps people get to know each other Provides the best opportunity to expand social network

Small talk is an important social lubricant Difficult to identify significant topics Superficial

10


A Naive Approach of Smartphone-based Small Talk Store all user’s information, including each user’s full contact

list User report either his own geo-location or a collection of

phone IDs in his physical proximity to the server using internet connection or SMS

Server performs profile matching, finds out small talk topics (mutual contact, common interests, etc.)

Results are pushed to or retrieved by users

11


However……

Require costly data services (phone’s internet connection, SMS)

Require report and store sensitive personal information in 3rd party

Trusted server may not exist Server is a bottleneck, single point of failure, target of

attack

12


E-SmallTalker – A Fully Distributed Approach No Internet connection required No trusted 3rd party No centralized server

Information stored locally on mobile phones Original personal data never leaves a user’s phone Communication only happens in physical proximity

13


Two Challenges

How to exchange information without establishing a Bluetooth connection Available data communication channels on mobile phones

Cellular network (internet, SMS, MMS), Bluetooth, WiFi, IrDA Bluetooth is a natural choice

Bluetooth connection needs user’s interaction due to security reasons How to find out common topics while preserving users privacy

No pre-shared secret for strangers Bluetooth Service Discovery Protocol can only transfer limited service

information

14


System Architecture Context exchange Context encoding and matching Context data store User Interface

15


Context Exchange Exploit Bluetooth service discovery protocol

No Bluetooth connection needed Publish encoded contact data (non-service related) as (virtual) service

attributes Limited size and number( e.g. 128 bytes max each attribute)

16


Context Encoding Example of Alice’s Bloom

filter Alice has multiple contacts,

such as Bob, Tom, etc. Encode contact strings,

Firstname.lastname@phone_number, such as “Bob.Johnson@5555555555” and

“Tom.Mattix@6141234567”

17


Implementation

J2ME about 40 java classes, 127Kb jar file

On real phones Sony Ericsson (W810i), Nokia (5610xm, 6650, N70, N75,

N82)

18


Experiments

Settings 6 phones, n=150, k=7, m=1024 bits, default distance=4m, average of

10 runs Performance Metrics

Discovery time: the period from the time of starting a search to the time of finding someone with common interest, if there is any

Discovery rate: percentage of successful discoveries among all attempts Power consumption

Factors Bluetooth search interval Number of users Distance

19


Experiment Results

Minimum, average and maximum discovery time are 13.39, 20.04 and 59.11 seconds respectively

Always success if repeat searching, 90% overall if only search once

Nokia N82 last 29 hours when discovery interval is 60 seconds

20


Related Work Social network applications on mobile phones

Social Serendipity Centralized, Bluetooth MAC and profile matching, SMS, strangers

PeopleTones, Hummingbird, Just-for-Us, MobiLuck, P3 Systems, Micro-Blog, and Loopt

Centralized, GPS location matching, Internet, existing friends Nokia Sensor and PeopleNet

Distributed, profile, Bluetooth / Wifi connection, existing friends Private matching and set intersection protocols

Homomorphic encryption based Too much computation and message overhead for mobile phone

Limitations Require costly data services (phone’s internet connection, SMS) Require report and store sensitive personal information Bottleneck, single point of failure, target of attack

21


Remarks

Propose, design, implement and evaluate the E-SmallTalker system which helps strangers initialize a conversation Leveraged Bluetooth SDP to exchange these topics without

establishing a connection Customized service attributes to publish non-service related

information. Proposed a new iterative commonality discovery protocol based on

Bloom filters that encodes topics to fit in SDP attributes to achieve a low false positive rate

22


Exemplary System II:E-Shadow

Concept

Application Scenario

Goals and Challenges

System Design


Remarks

23


Concept

MotivationImportance of Face-to-Face InteractionPrevalence of mobile phones

Distributed mobile phone-based local social networking systemLocal profilesMobile phone based local social interaction tools


Application Scenario: Conference


Goals and Challenges

Design Goals Far-reaching and Unobtrusive Privacy and Security Auxiliary Support for Further Interactions Broad Adoption

Challenges Lack of Communication Support Power and Computation Limitation Non-pervasive Localization Service


Layered Publishing

Spatial Layering WiFi SSID

at least 40-50 meters, 32 Bytes Bluetooth Device (BTD) Name

20 meters, 2k Bytes Bluetooth Service (BTS) Name

10 meters, 1k Bytes

Temporal Layering For people being together long or repeatedly Erasure Code


E-Shadow Publishing Procedure

Valve Generator

Information

Filter

Database

Sensor

Feedback

Decide

Decide

User Maual Input

Online Data

Mining

Help

BTDevice

BTService

WiFi


Matching E-Shadow with its Owner

Intuitive Approach: Localization However, imprecision beyond 20-25 meters


Human Direction-driven Localization

Direction more important than distance Human observation

A new range-free localization technique RSSI comparison: Less prone to errors Space partitioning: Tailored for direction decision


Walking Route and Localization We allow users to walk a distance

Triangular route: A->B->C in (a), for illustration purposes Semi-octogonal route: A->B->C->D->E in (c), more natural

Take measurements on turning points Calculate the direction through RSSI comparison and space

partitioning


Implementation Information

Publishing Module Database Generator Buffers Control Valve Broadcasting

Interfaces

Retrieval & Matching Module Receivers Localization Decoding & Storage

Sensing Module User Interface


Evaluations (1)-Time & Energy

E-Shadow Collection Time WiFi SSID: 2 seconds BTD: 12-18 seconds BTS: 25-35 seconds

E-Shadow Power Consumption 3 hours in full performance

operation >12 hours in typical situation


Evaluations (2)-Localization

3 Outdoor Experiments:Open field campus

2 Indoor Experiments:Large classroom


Evaluation (3)-Simulations

Large-Scale Simulations:Angle deviation CDFs

12 times of exemplary direction decisions


Related Work Centralized mobile phones applications

Social Serendipity Centralized, Bluetooth MAC and profile matching, SMS, strangers

Decentralized mobile phone applications Nokia Sensor

Distributed, profile, Bluetooth / Wifi connection, existing friends E-Smalltalker

Distributed, no Bluetooth / Wifi connection, strangers Localization techniques for mobile phones applications

GPS Virtual Compass

peer-based relative positioning system using Wi-Fi and Bluetooth radios Limitations

Privacy compromise Unable to capture the dynamics of surroundings No mapping between electronic ID and human face Localization techniques either not pervasive or not accurate for long range

36


Remarks

Propose, design, implement and evaluate the E-Shadow system which lubricates local social interactions E-Shadow concept Layered publishing to capture the dynamics of surroundings Human-assisted matching that works for mapping E-Shadow with its

owner in a fairly large distance Implementing and evaluating E-Shadow on real world mobile phones

37


Exemplary System III:P3-Coupon

Coupon Distribution

A Naïve Approach

Challenges

System Design


Remarks

38


Electronic Coupon Distribution

Electronic coupons Similar to paper coupons Can be stored on mobile phones

Two distribution methods Downloading from Internet websites

Need to define target group Limited coverage Hard to maintain dynamic preferences lists on central databases

Peer to Peer Distribution No special destination/target group More coverage More flexible user-maintained preferences list

39


A Naive Approach of Peer-to-Peer Coupon Distribution A store periodically broadcast the coupon Users within broadcast range receive the coupon

User can decide whether to use, forward or discard the coupon

Users forward the coupon to others in physical proximity Forwarder’s IDs are recorded in a dynamically expanding list

The coupon is used by some user The store reward all users who have forwarded the coupon

40


However……

Require manually establishing wireless connections Cumbersome Not prompt Not possible for coupon forwarding among strangers

Require recording the entire forwarding path Potential privacy leakage Discourage user’s forwarding incentives

41


Challenge

How to design a prompt coupon distribution mechanism that Incentivize coupon forwarder appropriately for keeping the

coupons circulating Preserve the privacy of coupon forwarders

42


P3-Coupon – A Probabilistic Coupon Forwarding Approach Probabilistic sampling on forwarding path

Keep only one forwarder for each coupon: NO privacy leakage Probabilistically flip ownership at each hop

Accurate approximation of coupon rewards plenty of chances of interpersonal encounters Accurate bonus distribution with 50 coupons and 5000 people

Adaptive to different promotion strategies Flip-once model Always-flip model

No manual connection establishment Connectionless information exchange via Bluetooth SDP

43


System Architecture Store Side

A central server for broadcasting and redeeming coupons Client side

Coupon forwarding manager, coupon exchange, coupon data store, user interface

44


Probabilistic Forwarding Algorithm Always-Flip Model

The coupon ownership keeps flipping with certain probability at each hop. Good at assigning relative bonuses affected by the whole path lengths

E.g. the parent forwarder receives k times the bonus given to children forwarders The flip probability can be calculated in advance by the store, once k is fixed, using

the following formula

45


Probabilistic Forwarding Algorithm Extension: Flip-Once Model

Once flipped, a coupon’s ownership remain the same in a forwarding path. Good at assigning absolute bonuses irrelevant of the number of following

forwarders E.g. hop 1 user gets 10%, hop 2 user gets 5%, etc. The flip probability can be calculated in advance by the store using the following

formula

46


Coupon Format

Coupon description Product description Discounts Coupon issuer Coupon code Start/end date

Coupon forwarder information The current owner

Digital signature Prevent forging fraud coupons

47


Implementation

J2ME about 17 java classes, 1390Kb jar file

On real phones Samsung (SGH-i550), Nokia (N82, 6650, N71x)

48


Experiments

Experimental evaluations Coupon forwarding time Power consumption

Simulation evaluation Number of Coupon holders vs. Time Distribution saturation time vs. Number of Seeds Coupon ownership distribution for probabilistic sampling Deviation between theoretical and actual bonus (Always-Flip, Flip-

Once) Factors

Number of coupons Number of users Number of initial coupon holders

49


Experiment Results

Average coupon forwarding time is 33.52 seconds Nokia N82 last 25 hours with P3-Coupon running in

background One coupon could be delivered to 5000 people within 32 hours Very small deviation between theoretical and actual bonus

distribution with 50 coupons circulating among 5000 people

50


Remarks

Propose, design, implement and evaluate the P3-Coupon system which helps prompt and privacy preserving coupon distribution Probabilistic one-ownership coupon forwarding algorithm Implement the system on various types of mobile phones Extensive experiments and evaluations show that our approach

accurately approximate the theoretical coupon distribution in which the whole forwarding path needs to be recorded

Practical for real-world deployment

51


Exemplary System IV – Drunk Driving DetectionMotivationOur ContributionsDetection CriteriaOur SystemRelated WorkImplementation and EvaluationRemarks

52

52


MotivationCrashes caused by alcohol-impaired driving pose a

serious danger to the general public safety and health13,041 and 11,773 driving fatalities happened in 2007 and

2008*32% of the total fatalities in these two years*

Drunk driving also imposes a heavy financial burden on the whole societyAnnual cost of alcohol-related crashes totals more than $51

billion** * Data from U.S. NHTSA (National Highway Traffic Safety Administration)

** Data from U.S. CDC (Central of Disease Control)

53

53


Motivation

Detection of drunk driving so far still relies on visual observation by patrol officersDrunk drivers usually make certain types of dangerous maneuversNHTSA researchers identify cues of typical drunk driving behavior

Visual observation is insufficient to prevent drunk drivingThe number of patrol officers is far from enoughThe guidelines are only descriptive and qualitativeUsually, it is too late when drunk drivers are stopped by officers

It is essential to develop systems actively monitoring drunk driving and to prevent accidents

54

54


Our Contributions

Propose utilizing mobile phones as a platform for active drunk driving detection system

Design a real-time algorithm for drunk driving detection system using mobile phonesSimple sensors required only

i.e., accelerometers and orientation sensors

Design and implement a mobile phone-based active drunk driving detection systemReliable, Non-intrusive, Lightweight and power efficient, and

No extra hardware and service cost

55

55


Cues for Drunk Driving DetectionCues related to lane position maintenance problems

E.g., weaving, drifting, swerving and turning with a wide radius

Cues related to speed control problemsE.g., accelerating or decelerating suddenly, and braking erratically

Cues related to judgment and vigilance problemsE.g., driving with tires on lane marker, slow response to traffic signals

56

56


Drunk Driving Detection CriteriaExtract fundamental detection criteria from these cuesCapture the acceleration featuresE.g., for the lane position maintenance problems

57

57


Drunk Driving Detection CriteriaFocus on the first two categories of cues

They correspond to higher probabilities of drunk drivingMap them into patterns of acceleration

Probability of drunk driving detection goes higher while the number of observed cues increases

58

Driver’s problems in

maintaining lane position

Abnormal lateral movements

Patterns of lateral

acceleration of vehicles

Driver’s problems in controlling

speed

Abrupt speed variations

Patterns of longitudinal

acceleration of vehicles

58


Our System

59

59


Implementation

Develop the prototype system on Android G1 phone with accelerometer and orientation sensor

Implement the prototype in Java, with Eclipse and Android 1.6 SDK

The whole prototype system can be divided into five major components ☆ User interface System configuration ☆ Monitoring daemon☆ ☆ Data processing Alert notification☆

60

60


Evaluation - Testing Data Collection

Test data72 sets of data with simulated drunk driving related behaviors

- Weaving, swerving, turning with a wide radius- Changing speed erratically (accelerating or decelerating)

22 sets of data for regular driving- Each one for 5 to 10 minutes

Mobile phone positions in the vehicle

61

61


Evaluation - Detection Performance

Study the accuracy of detecting drunk driving related behaviorsIn terms of false negative and false positive

Study performance in the special case, such as the phone slides in the vehicle during drivingSlides has obvious impacts on detection accuracyMay add additional calibration procedure to solve it (future work)

62

62


Evaluation – Energy Efficiency

Curves of battery level states during mobile phone runningPhone runs without drunk driving detection systemMonitoring daemon of system keeps running, sensing and doing the pattern

matching on the monitoring results

63

63


Related WorkDriver vigilance monitoring and driver fatigue prevention

Monitoring the visual cues of drivers to detect fatigue in driving× Installed cameras just in front of drivers are potential safety hazard

Monitoring through vehicle-human interfaceCapture fatigued or drunk driving through monitoring interactions× Low compatibility, vehicles need to couple with auxiliary add-ons

Detect abnormal driving through GPS and acceleration dataPattern matching with GPS and acceleration data× However, GPS data are not always available

64

64


Remarks

First to propose utilizing mobile phones as a platform for developing active drunk driving detection system

Design and implement an efficient detection system based on mobile phone platforms

Experimental results show our system achieves good detection performance and power efficiency

In the future work, to improve the system with additional calibration procedure and by integrating all available sensing data on a mobile phone such as camera image

65

65


Exemplary System V: Stealthy Video Capturer Background SVC Overview Challenges Our Approaches Experimental Evaluations Remarks

66


Background

More and more private information is entrusted to our friend, the 3G Smartphone, which is getting more and more powerful in performance and diversified in functionality.

67


SVC Overview

Almost every 3G Smartphone is equipped with a camera and the wireless options, such as 3G networks, BlueTooth, WiFi or IrDA.

These wireless connections are good enough to handle certain types of video transmission.

We turn 3G Smartphones into an online stealthy video-recorder.

68


System Architecture

69


Challenges

Stealthily install SVC into 3G Smartphones Windows Hiding Infection Method

Collect the video information from 3G Smartphones DirectShow Controls Data Compressing

Send the video file to the SVC intender File Sending

70


Infection Method

To embed SVC in a 3G Smartphone is called a infection process.

We employ Trojan horse for downloads as the infection approach.

Our experimental SVC is hidden in the game of ”tic-tac-toe” that we develop in Windows Mobile environment.

71


The Scenario of Tic-Tac-Toe

72


Triggering Schemes

Triggering Algorithm is designed to determine when to turn on the video capture process and send the captured video to make SVC stealthier and get more useful information.

Three scenarios are under consideration. The first scenario is tracking. The second scenario is related with political or business

espionage. The third scenario is a hybrid one, where SVC is used for

much diversified everyday purposes.

73


Applications Suspects tracking Kids care

74


Kids tracking

75


Implementation

76


Experimental Evaluations:Power Consumption Power curve

77


Experimental Evaluations:CPU and Memory Usage CPU and Memory

78


Remarks

The initial study exploited from SVC will draw wide attentions on 3G Smartphone’s privacy protection and open a new horizon on 3G Smartphones security research and applications.

We are currently investigating the modeling of smart spyware from the study of ”spear and shield”.

79


A Summary

80


Future Research Directions

Smartphone-based Systems and Networking Mobile social networking, e-commerce, e-health, safety

monitoring etc. Easy to start and exciting but too many competitors, lack of

scientific depth Smartphone Core Improvement

Multitasking, power management, efficient local communication protocol, accurate localization, security/privacy protection

Deep but hard to start

81


Final Remarks

Smartphones have brought significant impacts to our daily life.

We present five exemplary systems on mobile social networking, e-commerce, e-health and safety.

Research and development on smartphones will be hot.

82

design and implementation of smartphone-based systems and networking

Documents