geoapplications development · 2018. 11. 25. · binary search and db index 3 approach 1...
TRANSCRIPT
![Page 1: Geoapplications development · 2018. 11. 25. · Binary search and DB index 3 Approach 1 (straightforward): Sequential scan, O(N) Drawback: slow Index Data 1 34 2 3 3 12 4 55 5 21](https://reader033.vdocuments.mx/reader033/viewer/2022053123/60ab9fe0e58a421cde76b5e0/html5/thumbnails/1.jpg)
Geoapplications development
http://rgeo.wikience.org
Higher School of Economics, Moscow, www.cs.hse.ru
![Page 2: Geoapplications development · 2018. 11. 25. · Binary search and DB index 3 Approach 1 (straightforward): Sequential scan, O(N) Drawback: slow Index Data 1 34 2 3 3 12 4 55 5 21](https://reader033.vdocuments.mx/reader033/viewer/2022053123/60ab9fe0e58a421cde76b5e0/html5/thumbnails/2.jpg)
Agenda2
![Page 3: Geoapplications development · 2018. 11. 25. · Binary search and DB index 3 Approach 1 (straightforward): Sequential scan, O(N) Drawback: slow Index Data 1 34 2 3 3 12 4 55 5 21](https://reader033.vdocuments.mx/reader033/viewer/2022053123/60ab9fe0e58a421cde76b5e0/html5/thumbnails/3.jpg)
Binary search and DB index3
Approach 1 (straightforward): Sequential scan, O(N)Drawback: slow
Index Data
1 34
2 3
3 12
4 55
5 21
Index Data
2 3
3 12
5 21
1 34
4 55
Approach 2 (optimized): Binary search, O(log N)Drawback: need to maintain index
![Page 4: Geoapplications development · 2018. 11. 25. · Binary search and DB index 3 Approach 1 (straightforward): Sequential scan, O(N) Drawback: slow Index Data 1 34 2 3 3 12 4 55 5 21](https://reader033.vdocuments.mx/reader033/viewer/2022053123/60ab9fe0e58a421cde76b5e0/html5/thumbnails/4.jpg)
Intersection problem: approach 14
Approach 1 (straightforward):
for each pair (gi, gj) compute Egenhofer matrix, if it is not empty, gi and gj intersect
Drawback: slow, because operation is complex
![Page 5: Geoapplications development · 2018. 11. 25. · Binary search and DB index 3 Approach 1 (straightforward): Sequential scan, O(N) Drawback: slow Index Data 1 34 2 3 3 12 4 55 5 21](https://reader033.vdocuments.mx/reader033/viewer/2022053123/60ab9fe0e58a421cde76b5e0/html5/thumbnails/5.jpg)
Intersection problem: approach 25
Approach 2 (with MBR):
1. for each geometry gi find MBR_i 2. for each (MBR_i, MBR_j) test whether they intersect (fast) 3. if they do, compute Egenhofer matrix, check for intersection pattern (slower)
Significantly reduce the need to perform step 3
![Page 6: Geoapplications development · 2018. 11. 25. · Binary search and DB index 3 Approach 1 (straightforward): Sequential scan, O(N) Drawback: slow Index Data 1 34 2 3 3 12 4 55 5 21](https://reader033.vdocuments.mx/reader033/viewer/2022053123/60ab9fe0e58a421cde76b5e0/html5/thumbnails/6.jpg)
Minimum Bounding Rectangle (MBR)6
http://purl.oclc.org/coordinates/a2.htm
3D case – Minimum Bounding Box (MBB)
https://en.wikipedia.org/wiki/Minimum_bounding_boxhttps://en.wikipedia.org/wiki/Minimum_bounding_rectangle
http://www.scriptspot.com/files/u9133/tm_bbox.png
![Page 7: Geoapplications development · 2018. 11. 25. · Binary search and DB index 3 Approach 1 (straightforward): Sequential scan, O(N) Drawback: slow Index Data 1 34 2 3 3 12 4 55 5 21](https://reader033.vdocuments.mx/reader033/viewer/2022053123/60ab9fe0e58a421cde76b5e0/html5/thumbnails/7.jpg)
Java Topology Suite: Envelope/MBR Checks7
public boolean crosses(Geometry g) {// short-circuit testif (! getEnvelopeInternal().intersects(g.getEnvelopeInternal()))return false;
return relate(g).isCrosses(getDimension(), g.getDimension());}
public boolean intersects(Envelope other) {if (isNull() || other.isNull()) { return false; }
return !(other.minx > maxx ||other.maxx < minx ||other.miny > maxy ||other.maxy < miny);
}
package com.vividsolutions.jts.geom;
![Page 8: Geoapplications development · 2018. 11. 25. · Binary search and DB index 3 Approach 1 (straightforward): Sequential scan, O(N) Drawback: slow Index Data 1 34 2 3 3 12 4 55 5 21](https://reader033.vdocuments.mx/reader033/viewer/2022053123/60ab9fe0e58a421cde76b5e0/html5/thumbnails/8.jpg)
PostGIS Envelope8
http://postgis.net/docs/manual-2.2/ST_Envelope.html
geometry ST_Envelope(geometry g1);Implements:• OpenGIS Simple Features Implementation Specification for SQL 1.1.• SQL/MM specification. SQL-MM 3: 5.1.15
geometry ST_MakeEnvelope(double precision xmin, double precision ymin, double precision xmax, double precision ymax, integer srid=unknown);
Example:
SELECT ST_AsText(ST_Envelope('LINESTRING(0 0, 1 3)'::geometry)); POLYGON((0 0,0 3,1 3,1 0,0 0))
![Page 9: Geoapplications development · 2018. 11. 25. · Binary search and DB index 3 Approach 1 (straightforward): Sequential scan, O(N) Drawback: slow Index Data 1 34 2 3 3 12 4 55 5 21](https://reader033.vdocuments.mx/reader033/viewer/2022053123/60ab9fe0e58a421cde76b5e0/html5/thumbnails/9.jpg)
Subset problem9
Approach 1 (straightforward):
Compute intersection of points/geometries with the rectangle
Approach 2:
![Page 10: Geoapplications development · 2018. 11. 25. · Binary search and DB index 3 Approach 1 (straightforward): Sequential scan, O(N) Drawback: slow Index Data 1 34 2 3 3 12 4 55 5 21](https://reader033.vdocuments.mx/reader033/viewer/2022053123/60ab9fe0e58a421cde76b5e0/html5/thumbnails/10.jpg)
Geohash10
https://en.wikipedia.org/wiki/Geohash
![Page 11: Geoapplications development · 2018. 11. 25. · Binary search and DB index 3 Approach 1 (straightforward): Sequential scan, O(N) Drawback: slow Index Data 1 34 2 3 3 12 4 55 5 21](https://reader033.vdocuments.mx/reader033/viewer/2022053123/60ab9fe0e58a421cde76b5e0/html5/thumbnails/11.jpg)
Geohash: decode example, step 111
https://en.wikipedia.org/wiki/Geohash
Only 5 bits are valid (2^31 = 11111)
u = 26 (decimal) = 11010 (binary)
c = 11 = 01011
f = 14 = 01110
t = 25 = 11001
and so on . . . the resulting sequence of bits is
11010 01011 01110 11001 . . .
![Page 12: Geoapplications development · 2018. 11. 25. · Binary search and DB index 3 Approach 1 (straightforward): Sequential scan, O(N) Drawback: slow Index Data 1 34 2 3 3 12 4 55 5 21](https://reader033.vdocuments.mx/reader033/viewer/2022053123/60ab9fe0e58a421cde76b5e0/html5/thumbnails/12.jpg)
Geohash: decode example, step 212
01234 56789
11010 01011 01110 11001 . . .
Odd bits for lat, even bits for lon. Q: why?
left to right:
lat = 1100111101 . . .
lon = 1001101010 . . .
https://en.wikipedia.org/wiki/Geohash
![Page 13: Geoapplications development · 2018. 11. 25. · Binary search and DB index 3 Approach 1 (straightforward): Sequential scan, O(N) Drawback: slow Index Data 1 34 2 3 3 12 4 55 5 21](https://reader033.vdocuments.mx/reader033/viewer/2022053123/60ab9fe0e58a421cde76b5e0/html5/thumbnails/13.jpg)
Geohash: decode example, step 313
lat = 1100111101…, lon = 1001101010…
1100111101…
•
•
![Page 14: Geoapplications development · 2018. 11. 25. · Binary search and DB index 3 Approach 1 (straightforward): Sequential scan, O(N) Drawback: slow Index Data 1 34 2 3 3 12 4 55 5 21](https://reader033.vdocuments.mx/reader033/viewer/2022053123/60ab9fe0e58a421cde76b5e0/html5/thumbnails/14.jpg)
Geohash: step 214
01234 56789
11010 01011 01110 11001 . . .
Odd bits for lat, even bits for lon. Q: why?
ANSWER:
Lon range = 2 x lon range (twice larger)
https://en.wikipedia.org/wiki/Geohash
![Page 16: Geoapplications development · 2018. 11. 25. · Binary search and DB index 3 Approach 1 (straightforward): Sequential scan, O(N) Drawback: slow Index Data 1 34 2 3 3 12 4 55 5 21](https://reader033.vdocuments.mx/reader033/viewer/2022053123/60ab9fe0e58a421cde76b5e0/html5/thumbnails/16.jpg)
Geohash properties: neighborhood16
![Page 17: Geoapplications development · 2018. 11. 25. · Binary search and DB index 3 Approach 1 (straightforward): Sequential scan, O(N) Drawback: slow Index Data 1 34 2 3 3 12 4 55 5 21](https://reader033.vdocuments.mx/reader033/viewer/2022053123/60ab9fe0e58a421cde76b5e0/html5/thumbnails/17.jpg)
Geohash limitations17
Source
![Page 18: Geoapplications development · 2018. 11. 25. · Binary search and DB index 3 Approach 1 (straightforward): Sequential scan, O(N) Drawback: slow Index Data 1 34 2 3 3 12 4 55 5 21](https://reader033.vdocuments.mx/reader033/viewer/2022053123/60ab9fe0e58a421cde76b5e0/html5/thumbnails/18.jpg)
Geohash algorithms: filter by rectangle18
•
•
source
![Page 19: Geoapplications development · 2018. 11. 25. · Binary search and DB index 3 Approach 1 (straightforward): Sequential scan, O(N) Drawback: slow Index Data 1 34 2 3 3 12 4 55 5 21](https://reader033.vdocuments.mx/reader033/viewer/2022053123/60ab9fe0e58a421cde76b5e0/html5/thumbnails/19.jpg)
Geohash algorithms: Ngram Tree Traveral19
source
1. Determine a “good” grid cell size with reasonable overlap of grid to query shape
2. Get the list of overlapping grid cells <g1, g2, …, gN>3. For each grid cell gi:
• Seek all points with geohashes having prefix of gi gird cell
• For each point decode its geohash and intersect with query shape
![Page 20: Geoapplications development · 2018. 11. 25. · Binary search and DB index 3 Approach 1 (straightforward): Sequential scan, O(N) Drawback: slow Index Data 1 34 2 3 3 12 4 55 5 21](https://reader033.vdocuments.mx/reader033/viewer/2022053123/60ab9fe0e58a421cde76b5e0/html5/thumbnails/20.jpg)
Geohash algorithms: filter – fixed grid depth20
source
1. Index each point at every grid level :D, DR, DRT, DRT2, DRT2Y
2. Recursive loop across top grid cells:• If a cell is within the query shape, simply
add all assigned entities to the result• If a cell intersects the query shape,
recursive(cell.subcells)
This requires careful implementation
![Page 21: Geoapplications development · 2018. 11. 25. · Binary search and DB index 3 Approach 1 (straightforward): Sequential scan, O(N) Drawback: slow Index Data 1 34 2 3 3 12 4 55 5 21](https://reader033.vdocuments.mx/reader033/viewer/2022053123/60ab9fe0e58a421cde76b5e0/html5/thumbnails/21.jpg)
Geohash uses21
GeoHash Java implementations:https://github.com/kungfoo/geohash-javahttps://github.com/astrapi69/jgeohashhttps://github.com/davidmoten/geo
![Page 22: Geoapplications development · 2018. 11. 25. · Binary search and DB index 3 Approach 1 (straightforward): Sequential scan, O(N) Drawback: slow Index Data 1 34 2 3 3 12 4 55 5 21](https://reader033.vdocuments.mx/reader033/viewer/2022053123/60ab9fe0e58a421cde76b5e0/html5/thumbnails/22.jpg)
R-Tree22
•
•
•
•
•
Query not intersecting MBR of a given node, cannot intersect any of the contained geometries
![Page 23: Geoapplications development · 2018. 11. 25. · Binary search and DB index 3 Approach 1 (straightforward): Sequential scan, O(N) Drawback: slow Index Data 1 34 2 3 3 12 4 55 5 21](https://reader033.vdocuments.mx/reader033/viewer/2022053123/60ab9fe0e58a421cde76b5e0/html5/thumbnails/23.jpg)
R-Tree: challenges23
•
•
![Page 24: Geoapplications development · 2018. 11. 25. · Binary search and DB index 3 Approach 1 (straightforward): Sequential scan, O(N) Drawback: slow Index Data 1 34 2 3 3 12 4 55 5 21](https://reader033.vdocuments.mx/reader033/viewer/2022053123/60ab9fe0e58a421cde76b5e0/html5/thumbnails/24.jpg)
R-Tree: one more example24
https://en.wikipedia.org/wiki/R-tree
![Page 25: Geoapplications development · 2018. 11. 25. · Binary search and DB index 3 Approach 1 (straightforward): Sequential scan, O(N) Drawback: slow Index Data 1 34 2 3 3 12 4 55 5 21](https://reader033.vdocuments.mx/reader033/viewer/2022053123/60ab9fe0e58a421cde76b5e0/html5/thumbnails/25.jpg)
R-Tree operations: insert25
•
•
•
•
•
![Page 26: Geoapplications development · 2018. 11. 25. · Binary search and DB index 3 Approach 1 (straightforward): Sequential scan, O(N) Drawback: slow Index Data 1 34 2 3 3 12 4 55 5 21](https://reader033.vdocuments.mx/reader033/viewer/2022053123/60ab9fe0e58a421cde76b5e0/html5/thumbnails/26.jpg)
R*-Tree, R+Tree26
https://en.wikipedia.org/wiki/R%2B_treehttps://en.wikipedia.org/wiki/R*_tree
![Page 27: Geoapplications development · 2018. 11. 25. · Binary search and DB index 3 Approach 1 (straightforward): Sequential scan, O(N) Drawback: slow Index Data 1 34 2 3 3 12 4 55 5 21](https://reader033.vdocuments.mx/reader033/viewer/2022053123/60ab9fe0e58a421cde76b5e0/html5/thumbnails/27.jpg)
JTS and R-Tree27
•
•
![Page 28: Geoapplications development · 2018. 11. 25. · Binary search and DB index 3 Approach 1 (straightforward): Sequential scan, O(N) Drawback: slow Index Data 1 34 2 3 3 12 4 55 5 21](https://reader033.vdocuments.mx/reader033/viewer/2022053123/60ab9fe0e58a421cde76b5e0/html5/thumbnails/28.jpg)
PostGIS and R-Tree28
http://postgis.net/docs/manual-2.2/using_postgis_dbmanagement.html#gist_indexes
![Page 29: Geoapplications development · 2018. 11. 25. · Binary search and DB index 3 Approach 1 (straightforward): Sequential scan, O(N) Drawback: slow Index Data 1 34 2 3 3 12 4 55 5 21](https://reader033.vdocuments.mx/reader033/viewer/2022053123/60ab9fe0e58a421cde76b5e0/html5/thumbnails/29.jpg)
Problem: efficient read of a 2D slice from disk29
![Page 30: Geoapplications development · 2018. 11. 25. · Binary search and DB index 3 Approach 1 (straightforward): Sequential scan, O(N) Drawback: slow Index Data 1 34 2 3 3 12 4 55 5 21](https://reader033.vdocuments.mx/reader033/viewer/2022053123/60ab9fe0e58a421cde76b5e0/html5/thumbnails/30.jpg)
Hilbert Curve30
https://en.wikipedia.org/wiki/Hilbert_curve
![Page 31: Geoapplications development · 2018. 11. 25. · Binary search and DB index 3 Approach 1 (straightforward): Sequential scan, O(N) Drawback: slow Index Data 1 34 2 3 3 12 4 55 5 21](https://reader033.vdocuments.mx/reader033/viewer/2022053123/60ab9fe0e58a421cde76b5e0/html5/thumbnails/31.jpg)
Readings31
• https://en.wikipedia.org/wiki/Geohash
•
•
•
![Page 32: Geoapplications development · 2018. 11. 25. · Binary search and DB index 3 Approach 1 (straightforward): Sequential scan, O(N) Drawback: slow Index Data 1 34 2 3 3 12 4 55 5 21](https://reader033.vdocuments.mx/reader033/viewer/2022053123/60ab9fe0e58a421cde76b5e0/html5/thumbnails/32.jpg)
Practical task32
![Page 33: Geoapplications development · 2018. 11. 25. · Binary search and DB index 3 Approach 1 (straightforward): Sequential scan, O(N) Drawback: slow Index Data 1 34 2 3 3 12 4 55 5 21](https://reader033.vdocuments.mx/reader033/viewer/2022053123/60ab9fe0e58a421cde76b5e0/html5/thumbnails/33.jpg)