WalkSafe: A Pedestrian Safety App

Presented by,Ramya Deepa Palle

CS 541

Paper Overview:

Summarizes a historical view on a conference about mobile sensors development

Discuss about path toward cognitive phones

Explains few phone applications based on mobile sensors

Discuss problems and solution while analyzing Big Sensor Data

Sensors in mobile:

Mobile phone or smartphone becomes the most important communication device in people lives.

A smartphone is a mobile phone offering advanced capabilities, often with PC-like functionality

Hardware (Apple iPhone 3GS as an example) CPU at 600MHz, 256MB of RAM

16GB or 32GB of flash ROM

Wireless: 3G/2G, WiFi, Bluetooth

Sensors: camera, acceleration, proximity, light

Functionalities

Communication

News & Information

Socializing

Gaming

Schedule Management etc.

Embedded sensors to phone as a sensor

The view moved away from traditional embedded sensors to phone as a sensor.

Small history:

In early 2007 Nokia released a phone with an embedded accelerometer.

They did not mention about this in N95 because it was only there for video stabilization and photo orientation

When Nokia’s peter Boda (the SensorPlanet leader) visited Dart-mouth college, he casually mentioned that the N(% included an embedded accelerometer.

This, along with the phone’s GPS sensors, led researchers to study and develop new phone-based sensing applications, such as the CenceMe app.

CenceMe

CenceMe, the first social sensing application running on mobile phones.

Sensor application Accelerometer

Microphone

GPS

Bluetooth

Track user information Physical activity

Sitting, walking, running or driving

Social interaction

Public conversation

Share using social networking applications Ex. Facebook

CenceMe App Analysis

Data pushed to servers for analysis

Reconstruct status of users

Phone Classifiers:

Audio - Fourier transform to determine talking pattern

Understand behavior, mood, and health

Activity - Accelerometer data to determine if user is sitting, standing, walking, or running

CenceMe App Analysis Cont…

Backend Classifiers: Conversation

combines multiple audio primitives to determine whether someone is in a conversation, which may contain pauses

Social Context

Neighborhood conditions - are there CenceMe friends around?

Social status - combination of classifiers

Partying 

Dancing 

Mobility Mode

GPS to determine if user is traveling in a vehicle or not

Am I Hot

Nerdy - often alone and spends time in locations such as libraries

Party animal - often at parties and often near others

Cultured - often visits theaters/museums

Healthy - often walking, jogging, cycling, or at the gym

Greeny - low environmental impact - not typically in a car and often walks, cycles, or runs

Smart phone sensing:

Today’s top-end smartphones come with 1.4-GHz quad-core processors and a growing set of inexpensive yet powerful embedded sensors.

They include:

An accelerometer, a digital compass, a gyroscope, a GPS, quad microphones, dual cameras, near-field communication, a barometer, and light, proximity, and temperature sensors.

They also have multiple radios for body, local, and wide area communications; 64 Gbytes of storage; and the touchscreen.

BeWell: A Mobile Health App

The BeWell app continuously tracks user behaviors along three key health dimensions without requiring any user input — the user simply downloads the app and uses the phone as usual.

Monitor and promote Physical and emotional well-being

Persuasive feed back techniques

Tracks along three dimensions Physical activity

Social interaction

Sleep duration

Weighted scorecard Between 0 to 100

Ex. 8 Hrs/day sleep is score 100

BeWell : Health App Cont…

Physical activity tracking

Walking, stationary or running

Daily metabolic equivalent of task value

Centers for Disease Control and Prevention guidelines

Sleep tracking

Over 24 hrs period

No phone interaction

Daily bed time activity

Recharging phone

Keeping at same place longer time

Guidelines by National Sleep Foundation

BeWell : Health App Cont…

Social Isolation tracking

Total speech time during day

Phone microphone usage

No recordings

Social apps presence on the phone and usage

Low score linking to depression

Work as stand-alone and interwork with cloud

BeWell : Health App Cont…

Persuasive feedback techniques

Animated aquatic ecosystem as wallpaper

Orange clown fish

Reflects user’s activity

Blue clown fish

Reflects user’s social interaction

Ocean’s ambient lighting

Reflects user’s sleep duration

Score history accessibility

Ambient Display:

BeWell+

Store data on clouds

Share with friends on social networks

Compare with others

Targeted messages

Challenges

Low energy consumption

More accurate classification

Population Diversity Problem

BeWell : Big Sensor Data

No raw data to server

Upload features, inferences, scores and usability data

20MB data traffic per day per user

Population Diversity Problem

Walking patterns changes for a child and an adult

Community Similarity Networks (CSN)

Personalized model per user

Measure similarity between people

WalkSafe : Pedestrian Safety App

Research in social science

Safety app for pedestrians crossing roads while on phone

Back camera to detect vehicles

Machine learning algos

Difficulty

Minimum battery utilization

During active calls only

Trained model

Decision tree

WalkSafe: Pedestrian Safety App

A mobile-phone user deep in conversation while crossing a street is generally more at risk than other pedestrians not engaged in such behavior.

WalkSafe uses the smartphone’s back camera to detect vehicles approaching the user, alerting the user of any potentially unsafe situations.

Algorithms used:

More specifically, WalkSafe uses machine-learning algorithms implemented on the phone to detect moving vehicles. It also exploits phone APIs to save energy by running the vehicle-detection algorithm only during active calls.

The WalkSafe app offers real-time detection of the front and back views of cars, noting when a car is approaching or moving away from the user, respectively . It alerts the user using sound and phone vibration.

Image recognition:

The core WalkSafe car-detection technology is based on image-recognition algorithms. Image recognition is a computationally intensive process that, if not carefully designed, can easily drain the smartphone’s computational resources and batteries.

Continued……

To address this, WalkSafe bases its vehicle recognition process on a model that’s first trained offine and then uploaded to the phone and used for online vehicle recognition, which runs automatically whenever there’s an ongoing phone call.

WalkSafe activates the smartphone’s camera and captures video of the surroundings. Each video frame is preprocessed to compensate for the phone tilt and illumination variations, and is then analyzed by the decision tree model built during the offline training phase. If the decision tree detects a car in the picture, it triggers an alert to warn the user of possible danger.

NeuralPhone: A Brain-to-Smartphone Interface

There’s a growing interest in new handsfree interfaces for smartphones based on voice and face recognition systems.

They developed the EyePhone, which lets the user select and activate applications with the blink of an eye.1 We then wondered if a thought could also drive a smartphone application—and it turns out it can.

NeuralPhone : A brain to smartphone interface

Hands free interfaces for smart phone

Thought driving smart phone app

EEG headsets

Brain controlled address book dialing

P300

Summary

Towards cognitive phones

Pushing intelligence towards phone

Robust classification techniques

Mobile sensors to improve life conditions

Queries?

If time permits:

A small video based on how accelerometer inside the cell phone are made and How a Smartphone Knows Up from Down.

http://www.youtube.com/watch?v=OjwLnAZoDFE