using area-based presentations and metrics for localization systems in wireless lans

Using Area-based Presentations and Metrics for Localization Systems in Wireless LANs

Eiman Elnahrawy, Xiaoyan Li, and Richard Eiman Elnahrawy, Xiaoyan Li, and Richard MartinMartin

Dept. of Computer Science, Rutgers UniversityDept. of Computer Science, Rutgers University

WLN, November 16WLN, November 16thth 2004 2004

Dept. of Computer Science, Rutgers University

WLAN-Based Localization

Localization in indoor environments using Localization in indoor environments using 802.11 and Fingerprinting802.11 and Fingerprinting

Numerous useful applicationsNumerous useful applications

Dual use infrastructure: a huge advantageDual use infrastructure: a huge advantage


Background: Fingerprinting Localization

Classifiers/matching/learning Classifiers/matching/learning approachesapproaches

Offline phase:Offline phase:Collect training data Collect training data (fingerprints)(fingerprints)

Fingerprint vectors: [(x,y),SS]Fingerprint vectors: [(x,y),SS]

Online phase:Online phase:MatchMatch RSS to existing RSS to existing fingerprints fingerprints probabilistically or probabilistically or using a distance metricusing a distance metric

[-80,-67,-50]

RSS

(x?,y?)

[(x,y),s1,s2,s3]

[(x,y),s1,s2,s3][(x,y),s1,s2,s3]


Output:Output:

A single location: the closest/best matchA single location: the closest/best match

We call such approaches We call such approaches “Point-based “Point-based Localization”Localization”

Examples:Examples:

RADARRADAR

Probabilistic approachesProbabilistic approaches[Bahl00, Ladd02, Roos02, Smailagic02, Youssef03, Krishnan04]


Contributions: Area-based Localization

Returned answer is area/volume Returned answer is area/volume likely to contain the localized likely to contain the localized objectobject

Area is described by a set of Area is described by a set of tilestiles

Ability to describe uncertaintyAbility to describe uncertainty

Set of highly possible Set of highly possible locationslocations


Contributions: Area-based Localization

Show that it has critical advantages over point-Show that it has critical advantages over point-based localizationbased localization

Introduce new performance metricsIntroduce new performance metrics

Present two novel algorithms: SPM and ABP-cPresent two novel algorithms: SPM and ABP-c

Evaluate our algorithms and compare them against Evaluate our algorithms and compare them against traditional point-based approachestraditional point-based approaches

Related Work: different technologies/algorithms Related Work: different technologies/algorithms [Want92, Priyantha00, Doherty01, Niculescue01, Savvides01, Shang03, He03, Hazas03, Lorincz04]


Why Area-based?

Noise and systematic errors introduce position uncertainty Noise and systematic errors introduce position uncertainty

Areas improve system’s ability to giveAreas improve system’s ability to give meaningful meaningful alternativesalternatives

A tool for understanding the confidence A tool for understanding the confidence

Ability to trade Ability to trade PrecisionPrecision (area size)(area size) for for AccuracyAccuracy (distance the localized object is from the area)(distance the localized object is from the area)

Direct users in their searchDirect users in their search

Yields higher overall accuracyYields higher overall accuracy

Previous approaches that attempted to use areas only use Previous approaches that attempted to use areas only use them as intermediate result them as intermediate result output still a single location output still a single location


Area-based vs. Single-Location

Object can be in a single room or multiple rooms Object can be in a single room or multiple rooms

Point-based to areasPoint-based to areas

Enclosing circles --Enclosing circles -- much largermuch larger

Rectangle? no longer point-based!Rectangle? no longer point-based!

0 2000

10

20

30

40

50

60

70

0 200

80


Outline

Introduction, Motivations, and Related WorkIntroduction, Motivations, and Related Work

Area-based vs. Point-based localizationArea-based vs. Point-based localization

MetricsMetrics

Localization AlgorithmsLocalization AlgorithmsSimple Point Matching (SPM)Simple Point Matching (SPM)

Area-based Probability (ABP-c)Area-based Probability (ABP-c)

Interpolated Map Grid (IMG)Interpolated Map Grid (IMG)

Experimental EvaluationExperimental Evaluation

Conclusion, Ongoing and Future WorkConclusion, Ongoing and Future Work


Performance Metrics

Traditional: Traditional: Distance error between returned and true Distance error between returned and true positionposition

Return avg, 95Return avg, 95thth percentile, or full CDF percentile, or full CDF

Does not apply to area-based algorithms!Does not apply to area-based algorithms!

Does not show accuracy-precision tradeoffs! Does not show accuracy-precision tradeoffs!


New Metrics: Accuracy Vs. Precision

Tile AccuracyTile Accuracy % true tile is returned % true tile is returned

Distance AccuracyDistance Accuracy distance between distance between true tile and returned tiles (sort and true tile and returned tiles (sort and use percentiles to capture use percentiles to capture distribution)distribution)

PrecisionPrecision size of returned area (e.g., size of returned area (e.g., sq.ft.) or % floor sizesq.ft.) or % floor size


Room-Level Metrics

Applications usually operate at the level of roomsApplications usually operate at the level of rooms

Mapping: divide floor into rooms and map tiles Mapping: divide floor into rooms and map tiles

(Point (Point ↔↔ Room): easy Room): easy

(Area (Area ↔↔ Room): tricky Room): tricky

Metrics: accuracy-precision Metrics: accuracy-precision Room AccuracyRoom Accuracy % true room is the returned room % true room is the returned room

Top-n rooms AccuracyTop-n rooms Accuracy % true room is among the % true room is among the returned rooms returned rooms

Room PrecisionRoom Precision avg number of returned rooms avg number of returned rooms


1. Simple Point Matching (SPM)

Build a regular grid of tiles, tile Build a regular grid of tiles, tile ↔↔ expected fingerprint expected fingerprint

Find Find ∩ tiles which fall within a tiles which fall within a “threshold”“threshold” of RSS for each AP of RSS for each AP

Eager:Eager: start from low threshold (= start from low threshold (= δδ, 2 × , 2 × δδ, …), …)

Threshold is picked based on the standard deviation of the Threshold is picked based on the standard deviation of the received signalreceived signal

Similar to Maximum Likelihood EstimationSimilar to Maximum Likelihood Estimation

∩ ∩

=


2. Area-Based Probability (ABP-c)

Build a regular grid of tiles, tile Build a regular grid of tiles, tile ↔↔ expected expected fingerprintfingerprint

Using Using “Bayes’ rule”“Bayes’ rule” compute compute likelihood likelihood of an RSS matching of an RSS matching the fingerprint for each tilethe fingerprint for each tile

p(Ti|RSS) p(Ti|RSS) αα p(RSS|Ti) . p(Ti) p(RSS|Ti) . p(Ti)

Return top tiles bounded by an overall probability that the Return top tiles bounded by an overall probability that the object lies in the area object lies in the area (Confidence: user-defined)(Confidence: user-defined)

Confidence Confidence ↑↑ →→ Area size Area size ↑↑

Dept. of Computer Science, Rutgers University 40

45

50

55

60

65

70

75

80

85

90

x in feet

y in

fee

t

0 210

140

Measurement At Each Tile Is Expensive!

Interpolated Map Grid: Interpolated Map Grid: (Surface Fitting)(Surface Fitting)

Goal:Goal: Extends original training data to cover the entire floor Extends original training data to cover the entire floor by deriving an expected fingerprint in each tileby deriving an expected fingerprint in each tile

Triangle-based linear interpolation using Triangle-based linear interpolation using “Delaunay “Delaunay TriangulationTriangulation”

Advantages:Advantages:

Simple, fast, and efficientSimple, fast, and efficient

Insensitive to the tile sizeInsensitive to the tile size


Impact of Training on IMG

BothBoth locationlocation and and numbernumber of training samples of training samples impact accuracy of the map, and localization impact accuracy of the map, and localization performanceperformance

Number of samples has an impact, Number of samples has an impact, but not strongbut not strong!!Little difference going from 30-115, no difference Little difference going from 30-115, no difference using > 115 training samplesusing > 115 training samples

Different strategies Different strategies [Fixed spacing vs. Average [Fixed spacing vs. Average spacing]spacing]: as long as samples are : as long as samples are “uniformly “uniformly distributed”distributed” but not necessarily but not necessarily “uniformly spaced”“uniformly spaced” methodology has no measurable effectmethodology has no measurable effect


Experimental Setup

CoRECoRE

802.11 data: 286 fingerprints (rooms + hallways)802.11 data: 286 fingerprints (rooms + hallways)

50 rooms50 rooms

200x80 feet200x80 feet

4 Access Points4 Access Points


Area-based Approaches: Accuracy-Precision Tradeoffs

Improving Improving AccuracyAccuracy worsens worsens PrecisionPrecision (tradeoff)(tradeoff)

0 50 100 150 200 25060

65

70

75

80

85

90

95

100

Training data size

% a

ccu

racy

Average Overall Room Accuracy

SPM

ABP-50

ABP-75

ABP-95

0 50 100 150 200 2500

2

4

6

8

10

Training data size

% f

loo

r

Average Overall PrecisionSPM

ABP-50

ABP-75

ABP-95


0 20 40 60 80 1000

0.2

0.4

0.6

0.8

1

pro

ba

bil

ity

ABP-75: Percentiles' CDF

Minimum25 PercentileMedian75 PercentileMaximum

0 20 40 60 80 1000

0.2

0.4

0.6

0.8

1

pro

ba

bil

ity



0 20 40 60 80 1000

0.2

0.4

0.6

0.8

1

distance in feet

pro

ba

bil

ity

SPM: Percentiles' CDF


0 20 40 60 80 1000

0.2

0.4

0.6

0.8

1

distance in feet

pro

ba

bil

ity



A Deeper Look Into “Accuracy”


Sample Outputs

SPM

ABP-75ABP-50

ABP-95Area expands into the true roomArea expands into the true room

Areas illustrate bias across different dimensions (APs’ Areas illustrate bias across different dimensions (APs’ location) location)


Comparison With Point-based localization: Evaluated Algorithms

RADARRADARReturn the Return the “closest”“closest” fingerprint to the RSS in the fingerprint to the RSS in the training set using training set using “Euclidean Distance in signal space” “Euclidean Distance in signal space” (R1)(R1)

Averaged RADAR Averaged RADAR (R2)(R2), Gridded RADAR , Gridded RADAR (GR)(GR)

Highest ProbabilityHighest ProbabilitySimilar to ABP: a typical approach that uses Similar to ABP: a typical approach that uses “Bayes’ “Bayes’ rule”rule” but returns the but returns the “highest probability single “highest probability single location” location” (P1)(P1)

Averaged Highest Probability Averaged Highest Probability (P2)(P2), Gridded Highest , Gridded Highest Probability Probability (GP)(GP)


Comparison With Point-based

Localization: Performance Metrics

Traditional error along with percentiles CDF for Traditional error along with percentiles CDF for area-based algorithms (min, median, max)area-based algorithms (min, median, max)

Room-level accuracyRoom-level accuracy


CDFs for point-based algorithms fall in-between the min, CDFs for point-based algorithms fall in-between the min, max CDFs for area-based algorithmsmax CDFs for area-based algorithms

Point-based algorithms perform more or less the same, Point-based algorithms perform more or less the same, closely matching the median CDF of area-based algorithmsclosely matching the median CDF of area-based algorithms

Min

Max

Median


Similar top-room accuracySimilar top-room accuracy

Area-based algorithms are Area-based algorithms are superior at returning multiple superior at returning multiple rooms, yielding higher overall room accuracyrooms, yielding higher overall room accuracy

If the true room is missed in point-based algorithms the user If the true room is missed in point-based algorithms the user has no clue!has no clue!


Conclusion

Area-based algorithms present users a more intuitive Area-based algorithms present users a more intuitive way to reason about localization uncertaintyway to reason about localization uncertainty

Novel area-based algorithms and performance metricsNovel area-based algorithms and performance metrics

Evaluations showed that qualitatively all the algorithms Evaluations showed that qualitatively all the algorithms are quite similar in terms of their accuracyare quite similar in terms of their accuracy

Area-based approaches however direct users in their Area-based approaches however direct users in their search for the object by returning an ordered set of search for the object by returning an ordered set of likely rooms and illustrate confidencelikely rooms and illustrate confidence


Ongoing and Future Work

Eiman Elnahrawy, Xiaoyan Li, and Richard P. Martin, Eiman Elnahrawy, Xiaoyan Li, and Richard P. Martin, “The Limits “The Limits of Localization Using Signal Strength: A Comparative Study”,of Localization Using Signal Strength: A Comparative Study”, In In IEEE SECON 2004IEEE SECON 2004

All algorithms behave the same: there is a fundamental All algorithms behave the same: there is a fundamental uncertainty due to environmental effects!uncertainty due to environmental effects!

Point-based approaches find the peaks in the PDFsPoint-based approaches find the peaks in the PDFs

Area-based approaches explore more of the PDF but cannot Area-based approaches explore more of the PDF but cannot narrow it (cannot eliminate the uncertainty)narrow it (cannot eliminate the uncertainty)

Useful accuracy, different ways to describe it to usersUseful accuracy, different ways to describe it to users

Future workFuture work: : additional HW or complex models to improve additional HW or complex models to improve accuracyaccuracy


Thank YouThank You

using area-based presentations and metrics for localization systems in wireless lans

Documents

areabased algorithms

areabased localizationshow

areabased presentations

true room

single room

easy area room

precision area size

precisiontile accuracy