iFrame: Dynamic Indoor Map Construction through Automatic Mobile Sensing

Chen Qiu and Matt W. Mutka

Dept. of Computer Science and Engineering Michigan State University

Outdoor LBS Indoor LBS

Ultrasound

WSN

RFID

Smartphone

GPS

Location Based Services (LBS)

Indoor Map Construction

Google Indoor Map (>1000 buildings in US and Japan)

Image Processing Footprints Collection

Draw the map by Hand

References: JigSaw, SLAM References: MapGenie, CrowdInside

Indoor Map Construction

Limitations of traditional indoor map constructions:

Complex Image Processing

Static Indoor Floor Plan

Not Passive Pattern

Dead Reckoning + Radio

Dynamic Updating Layouts

Unattended Mode

WiFi Detection

Bluetooth Detection

Distance - RSSI Relation

space

obstacle

Dead Reckoning

Markov Calibration ��

Merge and Learn

Multi-device Combination

Crowd Noise Filter

Curve Fit Fusion

Parameter Selection

��

Space Max <-�-> Space Min

Temporary shadow

Initial Shadow Map Anchor Points Rebuilt Floor Plan ��

b) System Refinementa) Inertial Sensing on Smartphones

c) Indoor Floor Plan Construction

S1S2

SnSn−1a x X

Y

Z

ay

az

Overview of iFrame System

Formulate the Indoor Environment

Original Map

Map Matrix

Shadow Map

Shadow Rate: 1 - occupied by objects0 - empty grid

Samples Value

1

0.1

0.70.9

0

...

g

a x

a!

O X

Y

Z

ay

az

Dead Reckoning Approach

a!= (ax ,ay ,az − g) Sn

!"!− Sn−1! "!!

= 12an−1! "!!

t 2 + vn−1! "!!

tn

Uniformly*Accelerated*Mo2on!

S1S2

SnSn−1

a!

g

Drawback of Dead Reckoning

Smartphone’s Acceleration Not Equal to User’s body Acceleration

Error Accumulation

0.5 degree error of orientation sensor 308m error within 1 minute

S1S2

SnSn−1a x X

Y

Z

ay

az

Enhance Dead Reckoning

Markov Chain State Prediction

User predicts the grids that are his/her next targets

The transition probability of k steps:

By applying C-K equation:

Consider historical and current information of the user’s traces:

In time period t, motion trace does not include the predicted grids

acceleration values for this time period t will be replaced

current Info. historical Info.

Distance)and)RSSI)Traditional formula is not accurate •  various obstructions •  multipath effect •  other factors

Measure the distance and RSSI •  Train for different devices •  Store in Hashmap on a smartphone

Distance)(meter)) RSSI)(dBm)))1m# $40dBm#2m# $45dBm#

Distance)(meter)) RSSI)(dBm)))

1m# $42dBm#2m# $46dBm#

Device 1 <-> Device 2

… …

d = 10[(P0−Fm−Pr−10×n×log10 ( f )+30×n−32.44)/10×n]

Radio Detection - RSSI

Bluetooth Detection• Build the connection between mobile devices• Describe the interferences between the wireless link• Use the mapping relation between RSSI and Distance • Satisfy the relation —> marked element in matrix as 0

Time (seconds)0 5 10 15

WiF

i RSS

I (dB

m)

-58

-56

-54

-52

-50

-48

Alice

Bob

Alice

RSSI Abruption

P1 P2

WiFi Detection

• Build the connection between mobile devices (WiFi Direct)• Describe the interferences between the wireless link• Use the mapping relation between RSSI and Distance • Not Satisfy the relation —> marked element in matrix as 1

Sensing Data Fusion

Curve Fit Fusion (CFF)

Solution: For each sensing technique, the one contains less errors will be assigned more weights

Matrix Generated by Sensing Approach

Error of each Sensing Approach

Differential Shadow Rate

Extend One Room to Multiple Rooms/Hallways

• Three sensing detection techniques have own features.

• Assign a, b, c values: one for a crowded room, one for a normal room, one for the room with few objects.

• When a user enters a room or a hallway, we set the parameters a, b, c as 1/3.

Matrix Generated by Sensing Approach

Error of each Sensing Approach

a=b=c=1/3

• Run iFrame and computes the average shadow rate of each room. • If the shadow rate satisfies the low/normal/high shadow rate, for the next period:

Challenge: how to assign weight for building multiple rooms?

a=b=c=1/3 corresponding a,b,c in CFF formula

Multi-device Combination

iFrame is a crowd sourcing mechanism, all the users collect and upload sensing data

iFrame provides three types of organizations

Mdi denotes the matrix computed by device i Maximum Space - combination matrix that has the most spaceMinimum Space - combination matrix that remains the least spaceMean Value - between these two extremes

default valueIndoor Map

Crowd Noise Filter

“Temporary Shadows” : users who are stationary in one room or a hallway, the generated shadow map might include errors

once a user does not change his/her position within 5 minutes, iFrame will not use his/her data until he/she leaves

Filtering “Temporary Shadows”

Phenomenon 1:

Phenomenon 2:Human crowd might cause interferences for the smartphones’ radio signals

Only dead reckoning approach is acceptable for the matrices with high crowd noise level

k-means clustering

Extend Rooms to a Real Building

Anchor Points AnalysisAnchor Points: Initial position of dead reckoning & the joints of rooms/hallways

acceleration ranges

correlation between the acceleration values on different axes

variance of acceleration magnitude Recognize Anchor Points

Hallway Assembling Users cannot cross the wall <—> elements in M are set as 1RSSI-Distance relation is satisfied <—> elements in M are set as 0More Samples, Higher Accuracy

Evaluation of iFrame

Experiment Setting

• The size of each grid as 0.5m x 0.5m

• 1-4 volunteers carry Samsung Galaxy S5 or Google Nexus Tablet

• Employ on Bluetooth and WiFi Adapter to communicate

• Volunteers in the experimental environment walk freely

Metrics

Error of Block Value:

Android 4.4 Kitkat

Original Floor Plan 10 mins

Ground Truth of Shadow Map

7.5 minsConstruction Approaches

Estimated Floor Plan 5 mins

�������������������������������������� �����������������

��������������������� ��������������������������������������������

������������������� �������������������� ��������������������

Dead Reckoning

������������������������������������ ���

Bluetooth

Shadow Map Construction

Original Floor Plan Estimated Floor PlanGround Truth of Shadow Map

Trash Cans

TablesTrash Cans

TablesTables

5 mins 10 mins

Construction Approach

����������������������������������������������Layout Change

Compare different sensing techniques

Detect the changes of layouts

Shadow Map Construction

Floor plan case study for the rooms with low shadow rate and high shadow rate

Floor plan case study for the rooms in a longtime

Original Floor Plan

Ground Truth of Shadow Map

Low Shadow Rate

High Shadow Rate

Construction Approaches 5 mins 7.5 mins 10 mins

�����������������

����������������� �����������������

�������������� ��

����������������

����������������

Estimated Floor Plan

�����������������

�����������������

Original Floor Plan Ground Truth of Shadow Map

Construction Approaches 7 hours

Estimated Floor Plan 2 hours

!!!!!!!!!!"#$%&'&&&! !!!!!!!!!!"#$%&'&()!!!!!!!!!!!"#$%&')*+!

30 mins

!!!!!!!!!!"#$%&'&,,!

Anchor Points Detection

Entrance

Elevator Stairs

Trace 1

Trace 2

Anchor

Anchor

Anchor

Anchor Types of Anchors: Entrance, Door, Elevator, Stairs, etc.

Sensing Information:1. Acceleration Range2. Air Pressure 3. GPS disappear

Recognize

Evaluation of iFrameSingle Room Case Study

Extended Environment Case Study

Conclusion of Evaluation:1. Indoor shadow map can be generated within 5-10 minutes 2. The updated information is shown on the shadow map 3. Unattended Mode (More users, Higher Accuracy)

Summary• Measure RSSI values with other scanned mobile devices to help

construct the layout of indoor environments

• Sensor fusion approach to combine the indoor maps computed by Dead Reckoning, Bluetooth, and WiFi RSSI detections

• Crowd Noise Filter

interferences caused by human crowd“temporary shadows”

• Generate 2D shadow map of a room within 5-10 minutes, the updating information of map can be represented

• Extend Rooms to a Real Building

Hallway Assembling Anchor Points Analysis

Thank you !

Questions ?