4 spatial process · • g is a general, unspeciﬁed functional form ... luc created date:...

Copyright © 2017 by Luc Anselin, All Rights Reserved

Luc Anselin

Spatial Regression4. Spatial Process Models

http://spatial.uchicago.edu


• spatial covariance

• simultaneous spatial process models

• conditional spatial process models


Spatial Covariance


Concepts


• Structure of Covariance

• Cov[yi, yj] = E[yi.yj] - E[yi]E[yj] ≠ 0

• which pairs yi, yj have non-zero covariance

• if the non-zero covariances show a spatial pattern, then it is “spatial” covariance

• need a way to embed spatial structure


• How to Embed Spatial Structure

• need to respect location, distance, network structure

• need to respect Tobler’s law (distance decay)

• variance-covariance matrix must be positive definite


• Three Approaches

• direct representation

• non-parametric function

• spatial process model


• Direct Representation

• covariance is a specified function of distance

• Cov[yi,yj] = f (θ, dij)

• f = function, e.g., negative exponential

• θ = parameter vector

• dij = distance metric


• Non-Parametric Function

• spatial autocorrelation is an unspecified function of distance

• ρij = g(dij)

• g is a general, unspecified functional form

• how to fit the function? e.g., use kernel estimator


• Spatial Process Model

• spatial stochastic process or spatial random field

• { Z(s): s ∈ D }

• s ∈ Rd: location (e.g., lat-lon)

• D ∈ Rd: index set = possible locations

• Z(s): random variable at location s

• covariance structure follows from the specification of the process, i.e., how a random variable at a location depends on random variables at other locations


Direct Representation


• Requirements for Distance Function

• distance decay: as d ↑ Cov ↓

• covariance symmetric (distance has to be symmetric)

• variance-covariance matrix must be positive definite for all parameter values


• Illustration

• observations on a line5 points • ⎯ • ⎯ • ⎯ • ⎯ •

• x = 0, 1, 2, 3, 4

• distance matrix = 0 1 2 3 4 1 0 1 2 3 2 1 0 1 2 3 2 1 0 1 4 3 2 1 0


• Negative Exponential Distance Function

• Cov [yi,yj] = exp(-0.2 x dij)

• compute covariance for each distance pair

• d01 = 1 → Cov(y0,y1) = e-0.2 = 0.82


• Spatial Covariance Matrix

Cov =

⎡

⎢

⎢

⎢

⎢

⎣

1 0.82 0.67 0.55 0.45

0.82 1 0.82 0.67 0.55

0.67 0.82 1 0.82 0.67

0.55 0.67 0.82 1 0.82

0.45 0.55 0.67 0.82 1

⎤

⎥

⎥

⎥

⎥

⎦


• Properties

• smooth decay of covariance with distance

• potential issues for d = 0, sometimes requires ad hoc adjustments, e.g., for 1/d

• scale sensitive: magnitude of covariance depends on distance metric

• for d too large, function may become zero

• e.g., distance in meters vs km

• covariance matrix not necessarily positive definite


Nonparametric Representation


• Nonparametric Kernel Estimator

• Hall-Patil (1994)

• ρ(d) = Σi Σj K(dij/h)(zi.zj) / Σi Σj K(dij/h)

• zi.zj is the covariance between i and j

• K is the kernel function (many choices), with h as the bandwidth

• results depends critically on h, less so on K


• Spline Nonparametric Covariance Function

• use cubic B-spline kernel

• use bootstrap envelope for inference

• resample tuple (x, y, z) with replacement

• confidence interval around spline kernel


spline spatial covariance function with bootstrap envelope


• Interpretation

• range of significant spatial autocorrelation

• alternative to specifying an explicit distance function

• sensitive to kernel fit

• may violate Tobler’s law

• possible non-positive definite covariance matrix


Simultaneous Spatial Process Models


Concept


• Stochastic Process in Space

• random variable at i depends on random variable at “neighboring” locations

• spatial autoregressive process

• random variable is a weighted sum of a random innovation at the location and an average of random innovations at neighboring locations

• spatial moving average process


• Simultaneous vs Conditional Processes

• simultaneous = model for the complete pattern

• values at all locations are explained (as functions of exogenous variables)

• conditional = model for prediction

• values at unobserved locations are predicted by a model fit on observed locations

• alternative use: a spatial prior for parameters in a Bayesian hierarchical model


• Prediction

• in simultaneous models

• y = f(x)

• no y on right hand side (reduced form)

• in conditional models

• yi = f( yj* ), with yj* as values at other locations


Spatial Autoregressive Process


• Spatial Autoregressive Process (SAR)

• analog to time series setup

• example, first order autoregressive process

• yt = ρyt-1 + ut

• by consecutive substitution, this becomes a moving average process in the error terms


• Covariance Structure

• follows from the specification of the process

• in the time series case


• Spatial Autoregressive Process on a Line


• Fundamental Difference = Feedback

• substitution approach breaks down

• yi is present on the right hand side

• alternative: write full system of simultaneous equations using a spatial weights matrix


• SAR Process Covariance Matrix

• follows from the reduced form


• Example: Unconnected Line

• ρ = 0.2, unit variance


• Example: Connected Line (circle, torus)

• ρ = 0.2, unit variance


• Properties of SAR Covariance

• depends on weights matrix (e.g., connected vs unconnected) but DIFFERENT from the weights matrix structure

• every location covaries with every other location

• covariance decreases with order of contiguity, BUT same order of contiguity does not yield same covariance

• heteroskedastic when non-constant number of neighbors (not in torus case)

• strange covariances in torus structure


Spatial Moving Average Process


• Spatial Moving Average Process (SMA)

• smoothing + innovation

• in matrix form


• SMA Covariance

• follows directly from expression

• local covariance

• only first and second order neighbors


• SAR vs SMA Covariance

• example:

• irregular layout, ρ = 0.2, unit variance


• SAR-SMA Comparison

• SAR is global, SMA is local

• SMA has zero covariance beyond second order neighbor

• both heteroskedastict, but SAR has larger variance

• SAR covariance stronger for same order of contiguity


Conditional Spatial Process Models


Conceptual Framework


• Factorization

• joint distribution as a product of a conditional and a marginal

• iterative application yields factorization in conditionals

• example: time series


• Markov Property in Time

• only previous time period matters in conditional

• p[ yt | y1, y2, …, yt-1 ] = p[ yt | yt-1 ]

• simplifies factorization

• expression in conditionals yields proper joint distribution


• Factorization in Space

• factorization of joint distribution in terms of full conditionals

• p(y1, y2, …, yn) = Πi p(yi | y-i)

• where y-i consists of all yj with j ≠ i

• in space, factorization is not unique

• building a joint distribution from conditionals does not necessarily work

• individual p(yi | y-i) are incompatible

• resulting joint distribution is improper


• Brook’s Lemma

• Brook (1964)

• establishes link between full conditionals and a joint distribution

• joint may be proper or improper, but is specified up to a constant of proportionality


Markov Random Field (MRF)


• Conditional Perspective

• specify distribution conditional on neighbors

• specification for yi | yj*

• with j* as the set of neighbors for i

• make sure there is a proper joint distribution that results from the conditional ones


• Markov Random Field (MRF)

• full conditionals from Brook’s lemma quickly become unwieldy

• alternative = local specification, a MRF

• p(yi | y-i) = p(yi | yj) with j ∈ ∂i

• and ∂i as the set of neighbors for i

• under what conditions does the local specification uniquely specify a proper joint distribution


• Hammersley-Clifford Theorem

• connection between MRF and joint distribution

• if a Markov Random Field (defined as conditionals only) defines a unique joint distribution, that distribution is a Gibbs distribution

• implies restrictions on the neighbor structure

• Gibbs sampling

• reverse (Geman and Geman 1984)

• sampling from MRF is equivalent to sampling from its Gibbs distribution

• basis for the term Gibbs sampling


• Theoretical Building Blocks

• cliques

• set of observations, such that each observation is a neighbor of every other observation

• e.g., all pairs of observations i, j

• potential function

• exchangeable in elements of cliques

• examples: yiyj, (yi - yj)2

• Gibbs distribution

• only a function of potential functions


• Hammersley-Clifford in Practice

• leads to auto-models

• restrictions on weights (symmetric) and parameter space

• conditional specifications that yield a proper joint distribution for exponential family

• only for auto-normal models is the joint distribution also normal

• for other auto-models the joint distribution is not a multivariate version of the conditionals


Conditional Autoregressive (CAR) Model


• Autonormal (CAR) Model

• Normal Markov Random Field (NMRF)

• conditional is normal

• joint density D is diag( "i2 )

• symmetry requirementD-1(I - B) must be symmetric


• Intrinsically Autoregressive Model (IAR)

• introducing neighbor weights

• W a symmetric binary spatial weights matrix

• wi+ = Σj wij as the row sum

• bij = wij and "i2 = "2 / wi+

• joint density is improper

• D = diag(wi+) s.t. no variance exists (D - W singular)

• not a model for actual data

• use as a prior with centering s.t. Σi yi = 0


• Proper CAR Model

• introduce spatial autoregressive parameter

• conditional specification

• yi | yj for j ≠ i ~ N( ρ Σj wij yj /wi+, "2 / wi+)

• inverse variance in joint distribution is now proper

• D - ρW for ρ in the proper parameter space


• CAR as a Regression Model

• conditional linear specification

• spatial covariance

• restrictions

• weights matrix must be symmetric

• restrictive parameter space


CAR vs SAR


• CAR vs SAR

• SAR model has a CAR representation

• compare process covariance matrix

• CAR: (I - C)-1

• SAR: [(I - S)’(I - S)]-1 = [I - (S + S’) + S’S]-1

• set C = (S + S’) - S’S to obtain CAR representation of the SAR model: NOT the same weights structure

• SAR representation of CAR model not useful

• requires Cholesky decomposition of I - C


• CAR vs SAR Spatial Covariance

• using same example as before with ρ = 0.2, and unit variance


• CAR vs SAR Covariance (continued)

• same overall pattern

• SAR variances larger

• SAR covariances larger

• faster decay for CAR

• for same parameter and same weights SAR reflects stronger pattern of spatial covariance than CAR


Auto Models


• Non-Normal Auto-Models

• for exponential family, Hammersley-Clifford ensures existence of joint distribution

• general form


• Auto-Logistic

• for binary random variables

• also referred to as (conditional) spatial logit


• Auto-Poisson

• for count variables

• parameter restrictions due to Hammersley-Clifford

• only negative spatial autocorrelation!

4 spatial process · • g is a general, unspeciﬁed functional form ... luc created date:...

Documents