6350 spatio -temporal data processing course overview

Click here to load reader

Upload: verna

Post on 09-Jan-2016

42 views

Category:

Documents


1 download

DESCRIPTION

6350 Spatio -temporal Data Processing Course Overview. Yan Huang [email protected]. Basic Information. Instructor: Yan Huang ( huangyan at unt.edu) Meeting place and time: M 2:30-:520pm B157 Office hours: M 12:30-2:30pm. Basic Information. TA : Sasi Koneru ([email protected]) - PowerPoint PPT Presentation

TRANSCRIPT

Literature Search1
Meeting place and time: M 2:30-:520pm B157
Office hours: M 12:30-2:30pm
Evaluation
class participation 10%
project - 40%.
Paper Analysis I
Write short summaries for 3 (100-200 words)
Make a 15 minutes presentation on what you learn on this topic
The presentation will take an integrated approach where you introduce the motivation of the three papers, give a precise problem definition, compare and contrast the ways the 3 papers approach the problem and how they validate their results, present conclusions, and point to some future directions if you can identify
Paper Analysis II
Collect two questions from each group
Ask two questions yourself
May need to explore and refine during your search
Often you can find electronic version of the papers, especially for publications related to computer science
Author’s website
ACM digital library
You school typically subscribes to these publishers
Search from a computer with IP address belonging to your school
8
10
access to a melon along its rattan
Term Project
Team of up-to 2 person
March 03, 10 minutes presentation on algorithm design and cost analysis
Score is based on normalized grade you get from submission.
Term Paper
Two choices
Term paper
Survey paper
Term paper
Research oriented
Key components:
Related Work and Our Contributions
Proposed Approach
Conclusions and Future Work
Our Contributions (usually it is the categorization/classification of the research literature)
A classification of the papers related to the problem. Use a concept hierarchy, figures, and diagrams if necessary.
Summarize, classify, contrast, and compare the research literature according to your classification scheme
A summary of the trend and future work of this line of research.
Conclusion.
Persistence across failures
Allows concurrent access to data
Scalability to search queries on very large datasets which do not fit inside main memories of computers
Efficient for non-spatial queries, but not for spatial queries
Non-spatial queries:
List the names of all bookstore with more than ten thousand titles.
List the names of ten customers, in terms of sales, in the year 2001
Use an index to narrow down the search
Spatial Queries:
List the names of all bookstores with ten miles of Minneapolis
List all customers who live in Tennessee and its adjoining states
List all the customers who reside within fifty miles of the company headquarter
Value of SDBMS
Examples of Spatial data
Weather and Climate Data
Rivers, Farms, ecological impact
A phone book
A Product catalog
User, Application domains
Many important application domains have spatial data and queries. Some Examples follow:
Army Field Commander: Has there been any significant enemy troop movement since last night?
Insurance Risk Manager: Which homes are most likely to be affected in the next great flood on the Mississippi?
Medical Doctor: Based on this patient's MRI, have we treated somebody with a similar condition ?
Molecular Biologist:Is the topology of the amino acid biosynthesis gene in the genome found in any other sequence feature map in the database ?
Astronomer:Find all blue galaxies within 2 arcmin of quasars.
Exercise: List two ways you have used spatial data. Which software did you use to manipulate spatial data?
SDBMS
can work with an underlying DBMS
supports spatial data models, spatial abstract data types (ADTs) and a query language from which these ADTs are callable
supports spatial indexing, efficient algorithms for processing spatial operations, and domain specific rules for query optimization
Example: Oracle Spatial data cartridge, ESRI SDE
can work with Oracle DBMS
Has spatial data types (e.g. polygon), operations (e.g. overlap) callable from SQL3 query language
Has spatial indices, e.g. R-trees
IBM: Spatial Option
Informix: Spatial Datablade
SDDMB vs. GIS
GIS is a software to visualize and analyze spatial data using spatial analysis functions such as
Search Thematic search, search by region, (re-)classification
Location analysis Buffer, corridor, overlay
Terrain analysis Slope/aspect, catchment, drainage network
Flow analysis Connectivity, shortest path
Distribution Change detection, proximity, nearest neighbor
Spatial analysis/Statistics Pattern, centrality, autocorrelation, indices of similarity, topology: hole description
Measurements Distance, perimeter, shape, adjacency, direction
GIS uses SDBMS
SDBMS vs. GIS
SDBMS focuses on
Provides simpler set based query operations
Example operations: search by region, overlay, nearest neighbor, distance, adjacency, perimeter etc.
Uses spatial indices and query optimization to speedup queries over large spatial datasets.
SDBMS may be used by applications other than GIS
Astronomy, Genomics, Multimedia information systems, ...
Issues in SDBMS
Spatial data model
Transaction data
Network monitoring
Financial application
Most recent data are commonly queried in a one-pass fashion
Monitoring
Aggregation
Stream Application
Environmental monitoring
Patient monitoring
Traffic monitoring
Traffic jam: aggregated speed much below speed limit on a road segment for extended time
Accident: vehicle on unintended space, e.g. high way for longer than expected time
Click-streams
Find the school districts of the houses that the user browses the most.
Geo-streams
Spatial stream queries are common in
traffic monitoring
environment monitoring
Spatio-temporal Analytics
"Everything is related to everything else, but near things are more related than distant things."
The analysis of data with both spatial and temporal information
The data are spatially and/or temporally correlated
31
Why do we need spatio-temporal analytics
Analytics help us to describe what happened in the past, understand what is happening now, predict what will happen in the future, and make decisions.
The proliferation of sensor devices makes spatio-temporal information a fundamental component for almost every analytical applications
32
Visualization and exploratory analysis
Segmentation (classification and clustering)
Map querying task
Static query (one-time query using map tools available on the interface)
Dynamic query[36] (setup of event alert conditions)
Spatial constraints are expressed using the map, while temporal constraints are expressed as linear time moments[37]
Map animation[38]
Map iteration[40]
Existential changes[25]
35
Temporal extensions to spatial classification/ Spatial extension to temporal classification
Clustering[42]
Temporal clustering
Interactive spatio-temporal clustering: perform clustering spatially or temporally and then test whether the cluster exist in both dimensions (EMM Test[43])
Simultaneous spatio-temporal clustering: space-time scan[44]
36
Model-based clustering[46]
define a multivariate density distribution and look for a set of fitting parameters for the model.
Distance-based method
38
“spatial-temporal object whose thematic attribute values are signicantly dierent from those of other spatially and temporally referenced objects in its spatial or/and temporal neighborhoods”.
Methods[48]
Clustering-based approach
Co-Location Mining
Colocation mining finds subset of Boolean features located in spatial proximity
Methods[50]
42
Methods[49]
Bayesian networks
Hieratical approach
Trend discovery
Commercial
ArcGIS desktop and server provide most advanced and complete toolkit
Has many extensions for different domains
Can use APIs to develop extensions, web or desktop applications for customized needs. Many other commercial tools such as CUBE[9] are built on top of ArcGIS.
46
3D Extension (Desktop and Server)
Analyze terrain data, model subsurface features, view and analyze impact zones, determine optimum facility placement, share 3D views, create a 3D virtual city.
Geostatistical Extension (Desktop and Server)
Visualize, model, and predict spatial relationships.
Link data, graphs, and maps dynamically.
Perform deterministic and geostatistical interpolation.
Evaluate models and predictions probabilistically
47
Network Extension (Desktop and Server)
Dynamically model realistic network conditions and solve vehicle routing problems
Multipoint optimized routing, time-sensitive, turn-by-turn driving directions , allocation of service areas, determining the fastest fixed route to the closest facility 
Schematics Extension (Desktop and Server)
Rapid checking of network connectivity
Automatically generate schematics
Spatial extension (Desktop and Server)
Comprehensive, raster-based spatial modeling and analysis.
Survey Extension (Desktop)
Capture, edit, and leverage land records using proven survey methodologies
Tracking Extension (Desktop)
Create time series visualizations so you can analyze information relative to time and location
49
ESRI Business Analyst Online 
Web-based solution that combines GIS technology with extensive demographic, consumer spending, and business data for the entire United States to deliver on-demand, boardroom-ready reports and maps
Perform drive-time analysis
Analyze trade areas
50
ESRI Domain-Specific Solutions
ArcGIS Community Analyst  
Web-based solution that provides GIS capabilities to analyze data in a geographic context as granular as congressional district, block groups, census tracks, or ZIP Codes.
ArcLogistics
Create optimized routes and schedules based on multiple factors such as customer needs, business rules, vehicle traits, and street restrictions. 
Esri Situational Awareness
Provides a geospatial framework for immediate and long-term situational awareness needs.
Includes a powerful data fusion and analysis engine; a set of fully customizable clients for data visualization and analysis; and locally hosted, prerendered data.
51
Combines SQL Server spatial library with stream processing engine
Integrating SQL library within StreamInsight engine
Focuses on data stream event processing workflow
GIS Support relies on SQL Server (limited), and therefore need extensive customization for applications
52
Complete GIS Suite (similar to ArcGIS)
Cardcorp SIS[8],Geomedia[17], IDRISI[18] , Mapinfo[19]
Spatio-temporal analysis
STIS[23]
Network (traffic) analysis tools
ACCESSION GIS[3], AltaMap Suite[4], CUBE[9], DYNAMEQ[15], EMME[14]
Terrain analysis
ANUDEM[5]
CAD applications
Emergency and hazard modeling and analysis
CadnaA[10], Calpuff View[11],Caris[12],CATS[13],Floodworks[16]
Specialized analysis
ClusterSeer and BoundarySeer[7] (cluster and boundary analysis), Mathematica[20]
Mathematics toolkit
53
Descartes and CommonGIS[24]
An interactive java based GIS tool for visualization and exploratory analysis.
Functionalities
Basic queries (distance, difference…)
54
Functionalities
Support many formats
OGC standard compliant
55
GRASS[28]
Spatio-temporal analysis
Terrain analysis
Landserf[29]
Mathematics toolkit
The future
Big data (bigger due to spatio-temporal dimension)
Real time (not only historical spatio-temporal data, but also streaming data that requires optimization at all levels)
57
[31] http://mapserver.gis.umn.edu/
[32] http://postgis.refractions.net /
[33] http://cran.r-project.org/web/views/Spatial.html
[34] http://regionalanalysislab.org/index.php/Main/STARS
[35] P. Compieta, S. Di Martino, M. Bertolotto, F. Ferrucci, and T. Kechadi. 2007. Exploratory spatio-temporal data mining and visualization. J. Vis. Lang. Comput. 18, 3 (June 2007), 255-279. 
[36] C. Ahlberg, C. Williamson, B. Shneiderman, Dynamic queries for information exploration: an implementation and evaluation, in: Proceedings ACM CHI’92, ACM Press, New York, 1992, pp. 619–626.
[37] M. Harrower, A.M. MacEachren, A.L. Griffin, Developing a geographic visualization tool to support earth science learning, Cartography and Geographic Information Science 27 (4) (2000) 279–293.
[38] W.L. Hibbard, B.E. Paul, D.A. Santek, C.R. Dyer, A.L. Battaiola, M.-F. Voidrot-Martinez, Interactive visualization of earth and space science computations, Computer. 27 (7) (1994) 65–72.
[39] A. Buja, J.A. McDonald, J. Michalak, W. Stuetzle, Interactive data visualization using focusing and linking, in: Proceedings IEEE Visualization’91, IEEE Computer Society Press, Washington, 1991, pp. 156–163.
[40] D. Stojanovic, S. Djordjevic-Kajan, A. Mitrovic, Z. Stojanovic, Cartographic visualization and animation of the dynamic geographic processes and phenomena, in: Proceedings of 19th International Cartographic Conference, Ottawa, Canada, Vol. 1, 1999, pp. 739–746.
59
References (III)
[41] Kumar, M.; Bhatt, G.; Beeson, P.; Duffy, C. Automated Detection and Spatio-Temporal Classification of Channel Reaches in Semi-arid Southwestern US Using ASTER. American Geophysical Union, 2006 Joint Assembly.
[42] Tim E. Carpenter, Methods to investigate spatial and temporal clustering in veterinary epidemiology, Preventive Veterinary Medicine, Volume 48, Issue 4, 29 March 2001, Pages 303-320.
[43] Fosgate, G.T., Carpenter, T.E., Case, J.T., Chomel, B.B., 2000. Time±spatial clustering of human cases of brucellosis: California, 1973±1992. In: Proceedings of the Ninth International Society on Veterinary Epidemiology and Economics, Breckenridge, CO
[44] McKenzie, J.S., Pfeiffer, D.U., Morris, R.S., 2000. Spatial and temporal patterns of vector-borne tuberculosis infection in beef breeding cattle in New Zealand. In: Proceedings of the Ninth International Society on Veterinary Epidemiology and Economics, Breckenridge, CO
[45] Chudova D, Gaffney S, Mjolsness E, Smyth P (2003) Translation-invariant mixture models for curve clustering. In: KDD ’03: Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining, ACM, New York, NY, USA, pp 79–88
[46] Ankerst M, Breunig MM, Kriegel HP, Sander J (1999) Optics: ordering points to identify the clustering structure. SIGMOD Rec 28(2):49–60
[47] Kalnis P, Mamoulis N, Bakiras S (2005) On discovering moving clusters in spatio-temporal data. Advances in Spatial and Temporal Databases pp 364–381
[48] Birant, D.;   Kut, A.. Spatio-temporal outlier detection in large databases. 28th International Conference on Information Technology Interfaces, 2006.
[49] Jeremy Mennis, Jun Wei Liu. Mining Association Rules in Spatio-Temporal Data: An Analysis of Urban Socioeconomic and Land Cover Change. http:// onlinelibrary.wiley.com/doi/10.1111/j.1467-9671.2005.00202.x/abstract .
[50] Y. Huang, S. Shekhar, and H. Xiong, “Discovering colocation patterns from spatial datasets: A general approach.,” IEEE Transactions on Knowledge and Data Engineering, vol. 16, no. 12, pp. 1472–1485, 2004
[51] Feng Qian ; Liang Yin ; Qinming He ; Jiangfeng He ;. Mining spatio-temporal co-location patterns with weighted sliding window. IEEE International Conference on Intelligent Computing and Intelligent Systems, 2009. ICIS 2009.
60