Download - Image Based Positioning System
![Page 1: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/1.jpg)
Image Based Positioning System
Ankit GuptaRahul Garg
Ryan Kaminsky
![Page 2: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/2.jpg)
Outline
• Motivation• System Implementation• Technical Overview• Evaluation• Future Work
![Page 3: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/3.jpg)
Motivation
![Page 4: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/4.jpg)
Motivation
![Page 5: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/5.jpg)
Motivation
![Page 6: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/6.jpg)
Motivation
![Page 7: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/7.jpg)
Motivation
Hey there, I‘m think I’m lost!I have no idea.
Where are you?What is around you?
Well, there is a building.
Ok great, describe it for me.
It’s made of brick and has many windows.
Umm, that doesn’t really help.
The bricks are red.
Anything else, Einstein?I need to get back to work.
There are some trees around here also.
![Page 8: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/8.jpg)
Motivation
![Page 9: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/9.jpg)
Problem Definition
• Given an input image, identify a location on a map by querying for similar images
![Page 10: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/10.jpg)
Demo
![Page 11: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/11.jpg)
Web Architecture
Feature Extraction
Feature Descriptors
(Each Feature)
QueryEngine Feature DB
LocationVoting
(Best Location Match)
Network
Network
Query System
![Page 12: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/12.jpg)
Query System Architecture
Query Image
Feature Extraction
Feature Descriptors
[a,b,c], [x,n,d]
Query Processor
[a,b,c] ≈ [a,b,c][x,a,d] ≈ [x,n,d]
OID Vector LocationID 00 [x,y,z] 1 01 [a,b,c] 2 …100 [x,a,d] 0
Feature DB(Each Feature)
LocationID Votes
0 2
1 0
2 120
… …
N 4
(Location)
1
23
![Page 13: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/13.jpg)
Outline
• Motivation• System Implementation• Technical Overview• Evaluation• Future Work
![Page 14: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/14.jpg)
Technical Overview
Two key aspects:
• Feature point extraction
• Nearest Neighbor matching for each query image feature
![Page 15: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/15.jpg)
Feature Point Extraction
• Interest Point Detector of Schmid et. al. CVPR’06
• Build feature vector encoding the visual appearance around the interest point [Lowe et. al, IJCV’04]
![Page 16: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/16.jpg)
Nearest Neighbor Search
• Exact Approaches – Linear Search, Local Polar Coordinate (LPC) based indexed NN search [Cha et. al. IEEE Transactions on multimedia]
• Approximate Approaches – kd-tree, priority search using kd-tree
![Page 17: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/17.jpg)
LPC-based Indexed NN search
Database of featuresDatabase of compact features
• Obtain a compact representation of features that allows for selection of candidates without using the full representation
Filtering StageQueryFeature
CandidatesFor NN Compute NN
among candidatesNN
![Page 18: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/18.jpg)
LPC: Deriving compact representation• Divide space into discrete cells, and calculate
local polar coordinates of each point in its cell
Compact representation = <c,r,θ>
![Page 19: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/19.jpg)
Accelerating the LPC filtering• Expensive calculation of dmin and dmax
• Can we get coarser estimate of dmin efficiently? - estimate by distance of the cell from the query point
![Page 20: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/20.jpg)
Approximate Nearest Neighbor Strategies
• Spatial division using KD-trees
• Standard ANN Search
• Priority based ANN Search
![Page 21: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/21.jpg)
KD-Trees [Freidman et al, 77]
![Page 22: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/22.jpg)
Standard ANN Search[Freidman et al, 77]
A
B C
D E
Pass 1
A
B C
B DE
![Page 23: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/23.jpg)
Standard ANN Search[Freidman et al, 77]
A
B C
D E
Pass 2
A
B
B
C
DE
![Page 24: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/24.jpg)
Standard ANN Search[Freidman et al, 77]
A
B C
D E
Pass 3
DE
C
A
B
B
![Page 25: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/25.jpg)
Standard ANN Search[Freidman et al, 77]
A
B C
D E
Pass 4
DE
C
A
B
B
![Page 26: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/26.jpg)
Standard ANN Search[Freidman et al, 77]
A
B C
D E
Pass 5
C
A
B
DE
B
![Page 27: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/27.jpg)
Optimization
D
EB
Not process E (outside the sphere of radius r)
q
p
s
t
r
![Page 28: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/28.jpg)
Approximation
D
EB
Not process B (outside the sphere of radius r/(1+Є)
q
p
t
r
r/(1+Є)
s
![Page 29: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/29.jpg)
Standard ANN Search[Freidman et al, `77]
• Need to parse all leaves !
• Can do better if look at cells in sorted order of distance from the query – Priority-based ANN Search [Arya et al, `93]
• Need to maintain a priority queue
![Page 30: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/30.jpg)
Outline
• Motivation• System Implementation• Technical Overview• Evaluation• Future Work
![Page 31: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/31.jpg)
Evaluation
• Training database of 66 images – 11 classes (buildings)
• Query database of 50 images– Internet– Shot around campus
![Page 32: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/32.jpg)
Evaluation: On-Disk storage
• We compare Linear Search, LPC, LPC-SStrategy Avg Time Per query
feature (ms)Avg Number of I/O Accesses per query feature
Linear Search 1703.36 101394207
LPC 265.94 3692842
LPC-S 247.88 3724397
• The standard LPC filters out 97.23% data points in first pass
• The sphere test filters out 50.30%
![Page 33: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/33.jpg)
Evaluation: In-Memory Storage
Search Type Avg Response Time Per Query Image (seconds)
Accuracy(%)
Linear Search 86.12 90.0
kd-Tree Exact 76.76 90.0
kd-Tree ANN (ε=2) 7.78 88.0
kd-Tree Priority ANN (ε=2) 7.61 88.0
![Page 34: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/34.jpg)
Evaluation: In-Memory Storage
0 1 2 3 4 5 6 7 8 90
102030405060708090
Hierarchical ANN
epsilon
Resp
onse
Tim
e (s
econ
ds)
0 10 20 30 40 50 60 7040
50
60
70
80
90
100
Hierarchical ANN Linear Search
epsilon
Accu
racy
(%)
As є increases,
![Page 35: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/35.jpg)
Outline
• Motivation• System Implementation• Technical Overview• Evaluation• Future Work
![Page 36: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/36.jpg)
Future Work - Databases
• Survey of Better spatial division structures– BD Trees [Arya et al, J. ACM, `98]
– MD Trees [Nakamura et al, ICPR`88], G-Trees [Kumar, `94]
• Hybrid Storage Strategy• Better dimension mapping techniques
![Page 37: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/37.jpg)
Future Work - Databases
• Better spatial division structures• Hybrid Storage Strategy– Disk: easy to update but hard to query– Memory: easy to query but hard to update
• Better dimension mapping techniques
DISKMEMORY
![Page 38: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/38.jpg)
Future Work - Databases
• Better spatial division structures• Hybrid Storage Strategy• Better dimension mapping techniques– Non linear dimension reduction [Vu et al, SIGMOD`06]
![Page 39: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/39.jpg)
Future Work – Computer Vision
• Better descriptors for robustness
• Better ANN algorithms
• Full 3D scene calibration
Geometric Blur [Berg et al, CVPR01], Local self similarities [Schectman et al, CVPR07]
![Page 40: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/40.jpg)
Future Work – Computer Vision
• Better descriptors for robustness
• Better ANN algorithms
• Full 3D scene calibration
Locality-sensitive Hashing [Indyk, Motwani, STOC `98]
![Page 41: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/41.jpg)
Future Work – Computer Vision
• Better descriptors for robustness
• Better ANN algorithms
• Full 3D scene calibration
Photo Tourism [Snavely et al, SIGGRAPH `06]
![Page 42: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/42.jpg)
Ultimate Visualization
• Dynamic hybrid storage system• People uploading and removing photographs• 3D scene calibration• Extensions to museums
![Page 43: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/43.jpg)
Thank You
![Page 44: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/44.jpg)
LPC: Filtering
• <c,r,θ> allows for calculation of bounds dmin and dmax on actual distance of data point from query
• ifdmin > current estimate of NN distance
Reject pointelse
Accept point
![Page 45: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/45.jpg)
Our System vs. GPS
• Advantages– Internet connectivity only– Not dependent on satellite signal strength– More detailed information
• Disadvantages– Accuracy– Speed– More universal
![Page 46: Image Based Positioning System](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56814579550346895db24a71/html5/thumbnails/46.jpg)
Motivation
Hey there, I‘m think I’m lost!I have no idea.
Where are you?What is around you?
Well, there is a building.
Ok great, send a picture of it to campusfind.com.
Good idea! See you in a bit.