lecture 05 - database and pgdbdaisy.ccny.cuny.edu/~michael/courses/engrg59910/lecture... ·...

1

ENGRG 59910Introduction to

GISMichael Piasecki

October 5, 2014

Lecture 05: GIS and Databases Basics

•Basic geographic concepts• Introduction to GIS, coordinate system, projection, datum

•Data: Acquisition, Input and Management• Data model: vector vs. raster• Data source: map, attribute data (geocoding), GPS, remote sensing• Data input: digitizing• Data quality and meta data• Data management: database

•Analysis

•Output: map design

Where are we now?

October 5, 2014 ENGRG 59910 Intro to GIS 2

2

Context of what we are learning


Satellite Images

Aerial Photographs

Maps

Digitizing

GPS (later)

Non‐spatial data(Attribute Data)

Spatial Database

Geo‐Reference

Join, Relate, Geocoding

•a System ‐ a group of connected entities and activities

•an Information System ‐ a set of procedures, executed on raw data, to produce information for decision making

•a Geographic Information System ‐ an information system that uses geographically referenced data

GIS Definition 2: Break Down Words


3

• Introduction to Database Management System (DBMS)•Recognize four database types

•Relational Database Basics•Basic understanding of database theory

•GeoDatabase Overview

•Demo:•Common GIS database operation• Join/Relate• Spatial Join

Today’s Outline


Evolution of GIS Environments


4

• Flat Files• Flat files are easier to understand• Difficult to manage and manipulate• Large file size

• Database• Data is organized or structured using a database model• Reduce data redundancy• Data integrity is improved

• Can be “queried”‐‐many databases use the same language, SQL (Structured Query Language), for formulating queries.

Flat File vs. Database


DATABASE = Data file(s) + data organization + processing ready

•Goal for any DBMS: efficient searching and linking of tabular data.

•GIS DBMS Goal: efficient manipulation (including search and linking) of spatial objects (points, lines, polygons, polylines), relationships between objects and tabular data (i.e., topology, attributes).

Database Management Systems


Data Software Hardware

Database Management System

DBMS

5

•Field: One item of information per object (column)

Forest Trail Feature

Nantahala Bryson’s Knob Vista

Cherokee Slickrock Falls Ogrth

Pisgah Chimney Rock Wlife

Field vs. Column; Record vs. Row


•Record: Information items about one object (row)

Typically you operate on a field or column and select records or rows. A map object lights up when a row is selected

• There are four basic database structures:• Traditional

• Hierarchical• Network• Relational

• Recent development • Object Oriented (O‐O)

•Relational database is most widely used.

Ways of Organizing Information


6

Evolution of DBMS Technology


File System

Hierarchical DBMS Network DBMS

Relational DBMS Object-Oriented

System (OODBMS)

Object-Relational ORDBMS

Example of Data Organization


7

Example of Data Organization


From Chang’s book P161

The Basic of Relational Database


8

Database includes multiple tables

Tables are joined by relationships

•Relational model is grounded in mathematics: relational algebra defines the mathematical rules by which tables are manipulated.

•Any kind of attribute search (lateral, vertical) is possible.

Examples of relational database programsMicrosoft Access, Microsoft® SQL Server™, Oracle, DB2, FoxPro,MySQL, postgrSQL

Relational Database


Eliminate duplicate information

Assist in querying data

Simpler to manipulate data

Reduce disk space

Relational model has been most successful within GIS (and within the database world in general)

Why Use a Relational Database?


9

Key Fields (2 keys)

Relationships (3 relationships)

Referential Integrity

Database Normalization

Relational Databases: Terminology


• Keys – used to create uniqueness and link tables together• Primary Key: Uniqueness, eliminate Redundancy• Foreign Key: Linking tables, establishes relationships between tables

Key fields


10

Primary Key


• Primary keys uniquely identify each record in a table.• Primary keys become the foreign key in another table

Foreign Key


11

Data relationships: Cardinalities


ENGRG59910

ENG Courses

Students

One‐to‐Many (1:M)

Many students attend this class

Students

CCNY ID

One‐to‐One (1:1)

Each CCNY student has a unique ID number (i.e., functional redundancy)

Students Classes

Many‐to‐Many (M:M)

Many students are enrolled in many classes

Each student is taking manyclasses

Each class has many students

One‐tone Relationships


•Only one matching record•Uses primary key for both tables•Use to limit access or isolate information

12

One‐to‐Many Relationships


• Most common type of relationship

• Related between primary and foreign keys

Many‐to‐Many Relationships


• Not directly supported between tables• Use a junction table to relate• One order, many products• One product, many orders

13



PK

FK

• Maintain data accuracy

• Prevents orphan records

• Keeps relationships intact



14

•Reduce Duplication

• Improve Accuracy

•Data Maintenance

Form Analysis: Normalization


Form Analysis: Normal Forms


•Notice the redundancies in this table

• Put database into the First Normal Form

15

•Basically speaking is a rearrangement of data

•Redundant columns are removed

• Functional dependencies are rampant• E.g., Tship_ID Tship_name, Thall_add

• Tship_ID = 12 always is named “Birch” and is located at latitude 15W

•Allows you to identify groups!

First Normal Form


Parcels

Parcel_ID

Alderman

Tship_ID

Thall_add

Tship_name

Own_ID

Own_name

Own_add

Fields in theparcel dataset

First Normal Form


16

Second Normal Form


Parcel_ID

Alderman

Tship_ID

Tship_name

Own_ID

Thall_add

Own_name

Own_add

Functional dependency reduced by splitting data into like groups, possibly establishing a table to define the relationship between the first two

Third Normal Form


17

Third Normal Form


Parcel_ID

Alderman Tship_ID

Tship_ID

Tship_name Thall_add

Own_ID

Own_name Own_add

Transitive functional dependencies are removed

Parcel_ID Own_ID

•Most GIS packages still keep using hybrid solution: spatial data + attribute data (Arc+Info)

• The emergence of spatial database changes the way. Now many DBMS support spatial database: Oracle, DB2, MS SQL Server (commercial), and MySql, PostGreSql (open‐source)

Summary of Databases and GIS


Spatial Data

DBMSGeometric

DBMSAttribute

Geometric data

•Usually hierarchical

• Invisible to the user

Attribute data

•Almost entirely relational

•Manipulated by the user

18

•Arc/Info: Hybrid name, the history of ArcGIS.

•Without attribute data, spatial data will be of limited use.

Attribute Data and Spatial Data


•What is Geodatabase?

• Type of Geodatabase

•Geodatabase objects

Geodatabase


19

What is geodatabase ?


A geodatabase (short for geographic database) is a physical store of geographic information (spatial, attribute, metadata, and relationships) inside a relational database management system (RDBMS).

• Personal Geodatabase for Microsoft Access

• File Geodatabase (new since V9.2)•Workgroup Geodatabase (new since V9.2)

• SQL Server Express

• Enterprise Geodatabase: • 5 supported DBMSs:DB2, Informix, Oracle, MS SQl Server, PostgreSQL

Geodatabase Types Since Ver9.2


Increasing size and functionality

20

What does a Geodatabase look like?


What does a Geodatabase look like?


21

Personal GeoDatabase (Access)


Geodatabase (file‐based)


22

Geodatabase objects


• basic objects: ‐ feature classes,

‐ feature datasets,

‐ nonspatial tables.

• complex objects building on the basic objects:

‐ topology,

‐ relationship classes,

‐ geometric networks

•A feature class is a geographic feature include points, lines, polygons, and annotation feature class.

• Feature classes may exist independently in a geodatabase as stand‐alone feature classes or you can group them into feature datasets

Feature classes


The SouthAmerica geodatabase contains four stand-alone feature classes:a point feature class of cities, a dimension feature class of distances between cities, a polygon feature class of countries, and an annotation feature class of country names

Source: www.esri.com

23

Feature datasets


•A feature dataset is composed of feature classes that have been grouped together so they can participate in topological relationships with each other. All the feature classes in a feature dataset must share the same spatial reference (or coordinate system)

• Edits you make to one feature class may result in edits being made automatically to some or all of the other feature classes in the feature dataset

In the CityWater geodatabase, three point feature classes and one line feature class were groupedinto the PublicWater feature datasetto create a geometric network called WaterNet.


• Feature class tables and nonspatial attribute tables.

•Both types of tables are created and managed in ArcCatalog and edited in ArcMap. Both display in the traditional row‐and‐column format. The difference is that feature class tables have one or more columns that store feature geometry.

•Nonspatial tables contain only attribute data (no feature geometry) and display in ArcCatalog with the table icon . They can exist in a geodatabase as stand‐alone tables, or they can be related to other tables or feature classes.

Tables


The cfcc_desc table in the SantaBarbara geodatabase contains attribute data for the Roads feature class (stored inside the Roads feature dataset).


24

• Feature: A geographic representation of a spatial object

• Features: One row in a table represents one feature

• Feature Classes: one table or more than one table

• Feature Dataset: a set of feature classes

Organizing Geographic Features


GeoDatabase Elements


Feature class

Geometric network

Annotation class

Geodatabase

Relationship class

Table

Feature data set

25

• In a GIS, spatial relationships among feature classes in a feature dataset are defined by topology. You can choose whether to create topology for features.

• The primary spatial relationships that you can model using topology are adjacency, coincidence, and connectivity

• There are three types of topology available in the geodatabase: geodatabase topology (over 20 topology rules), map topology, and geometric network topology. Each type of topology is created from feature classes that are stored within a feature dataset. A feature class can participate in only one topology at a time

Topology


Example of Topology in a Geodatabase


26

Geometric Networks


• In the real world, examples of networks abound: streams joining together to form larger streams, pipes carrying water to homes and businesses throughout a city, and power lines carrying electricity.

• In a geodatabase, you can model each of these real‐world networks with a geometric network. Starting with simple point and line feature classes, you use ArcCatalog to create a geometric network that will enable you to answer questions such as: Which streams will be affected by a proposed dam? Which areas will be affected by a water main repair? What is the quickest route between two points in the network? Source: www.esri.com

Geometric Network example


Lateral

Service

Main

Feed

ValveFeature Classes

Source: ESRI European User Conference

Geometric Network

27

Relationship Classes


• In a geodatabase, relationship classes provide a way to model real‐world relationships that exist between objects such as parcels and buildings or streams and water sample data. By using relationship classes, you can make your GIS database more accurately reflect the real world and facilitate data maintenance.

The relationships stored in a relationship class can be between two feature classes (such as buildings and parcels, top) or between a feature class and a nonspatial attribute table (such as streams and water quality sampling data, bottom).


• Provided by ESRI http://support.esri.com/index.cfm?fa=downloads.dataModels.gateway

•Goal: provide a practical template for implementing GIS projects

• Start to think about your final project now

•Great start point for your GIS project

ESRI data models

54October 5, 2014 ENGRG 59910 Intro to GIS

28

• Address • Agriculture • Archiving • Atmospheric • Basemap• Biodiversity • Census‐Administrative Boundaries • Defense‐Intel • Energy Utilities • Energy Utilities ‐MultiSpeak TM • Environmental Regulated Facilities • Forestry• Geology• GIS for the nation• Groundwater

Industry‐specific Data models


• Health • Historic Preservation and Archaeology • Homeland Security

• Hydro • International Hydrographic Organization (IHO) S‐57 for ENC

• Land Parcels • Local Government

• Marine

• National Cadastre• Petroleum

• Pipeline • Raster • Telecommunications

• Transportation • Water Utilities

From: http://support.esri.com/index.cfm?fa=downloads.dataModels.gateway

Data model: national GIS

56

You can download it from ESRI website directly

October 5, 2014 ENGRG 59910 Intro to GIS

29

•Definition• Query is the action or result of selecting a subset of records based on specific attribute values

•General Categories:• Attribute (Tabular) • Spatial

• Two main methods:• Boolean operators (AND, OR, NOT)• SQL operators (< > + = …)\

• Structured querying language (SQL)• The mathematical basis of relational databases led to a standard languages for querying data (SQL) that uses simple mathematical operators

•Relational databases allow the user to “nest” operations for complex queries

Queries


Set Operators

= Equal< > Not Equal< Less Than> Greater Than<= Less Than or Equal>= Greater Than or Equal

Relational Operators UnionIntersectionDifferenceProduct

Database Use: Structured Query Language (SQL)


Aggregate Functions: “Summarize”

• Sum of values for all rows for a given column.

•Average of given column•Column Maximum•Column Minimum•Number of Rows (Count) that Satisfy a Condition

30

• Two Statements:•Females / Pop1990 < 0.55, then •Pop1990 > 100000 and Pop1990 < 200000

• Or, one statement:(Females/Pop1990 < 0.55) and ((Pop1990 > 100000) and (Pop1990 < 200000))

Compute all instances where the % of females in the 1990 population isless than 55%Then identify all population centers for the 1990 census where this trueif these are larger than 100,000 and less than 200,000

A Nested SQL Statement


• Point Queries• what is at a particular location?

•Range Queries• what is in a particular area?

•Nearest Neighbor Queries• where is the nearest object to a particular location?

• Spatial Join Queries• where are the areas that have water supply and power supply?

• Spatial Aggregate Queries• where is the most populated region?

Spatial Queries


31

•Queries – selection operations that produce data subsets

• Join and Relate – bringing data together

(one table with non‐spatial attribute, one table with features)

Common Attribute Operations


• General Categories:• Tabular – based on some

information within the attribute tables, e.g., a common field

• Spatial – based on location: nearest, within or aggregate

• Geodatabase Relationship class

• Strategies• Join• Relate

Relations


CountyPerson

Age

Polygon_id = 157

Gpsid = 29LC =

Agriculture

32

Forest-ID ForestName

1 Nantahala

2 Cherokee

Join vs. Relate


• Join• Appends fields from second table —with data for each record where a key field match is found (empty, otherwise)

• For 1:1 or M:1 only• In 1:M or M:M, it stops with first hit (can’t add rows/records for additional relationships)

•Relate• Allows automatic access to a related table’s records; keep tables physically separate

• For 1:M or M:M• Doesn’t add records to layer’s table, so not limited by initial table’s size

Forest-ID Trail_Name Features Trailhead

1 Bryson's Knob Vista X1, Y1

2 Slickrock Falls Ogrth X2, Y2

1 North Fork Wfall X3, Y3

2 Cade's Cave Wlife X4, Y4

1 Appalachian Cmp X5, Y5

•Related two attribute tables via a shared location in space, rather than common field

•Concept: A type of join operation in which fields from one layer’s attribute table are appended to another layer’s attribute table based on the relative locations of the features in the two layers.

•Examples• Finding the nearest feature • Finding what's inside a polygon • Finding what intersects a feature•Where is there an incompatibility between zoning and current land use? •How many Toxic Release Inventory sites are there in each county and what are the total releases per county?

Spatial Join


33

Spatial Join


URL: http://webhelp.esri.com/arcgisdesktop/9.2/index.cfm?TopicName=Learn%20more%20about%20spatial%20relationships

• Input layers: Population in US cities; US states;•Object: the city population in each state; how many cities in each state;

• Solutions:• Count/summarize manually?• Simple Query in DBMS (very easy but limited by data organization in many cases): select state, sum(pop) as cityPop , count(city) as totalcityfrom cities group by state

• Spatial join

Spatial Join: An example


34

Spatial Join Example


• Introduction to Database Management System (DBMS)• 4 types DBMS

•Relational Database Basics• 2 keys• 3 relations• Data Normalization Form analysis

• Spatial Database Overview and Operation• Join vs relate• Spatial join

What did we learn today?


lecture 05 - database and pgdbdaisy.ccny.cuny.edu/~michael/courses/engrg59910/lecture... ·...

Documents