Copyright © 2017 by Luc Anselin, All Rights Reserved
Luc Anselin
Spatial Regression6. Specification Spatial Heterogeneity
http://spatial.uchicago.edu
1
Copyright © 2017 by Luc Anselin, All Rights Reserved
• homogeneity and heterogeneity
• spatial regimes
• spatially varying coefficients
• spatial random effects
2
Copyright © 2017 by Luc Anselin, All Rights Reserved
Homogeneity and Heterogeneity
3
Copyright © 2017 by Luc Anselin, All Rights Reserved
• Global Perspective
• single equilibrium - stationarity
• functional form fixed
• coefficients fixed
4
Copyright © 2017 by Luc Anselin, All Rights Reserved
• Local Perspective
• multiple equilibria
• non-stationarity
• functional and/or parameter variability
5
Copyright © 2017 by Luc Anselin, All Rights Reserved
• Extreme Homogeneity
• model same everywhere
• parameters same everywhere
• yi = xiβ + εi
• β constant across i
• εi ∼ i.i.d. with Var[εi] = σ2 for all i
6
Copyright © 2017 by Luc Anselin, All Rights Reserved
• Extreme Heterogeneity
• every observation is different
• yi = xiβi + εi
• a different parameters βi for each observation i
• εi ∼ i.n.i.d. with Var[εi] = σi2
• possible different functional forms for i
7
Copyright © 2017 by Luc Anselin, All Rights Reserved
• Incidental Parameter Problem
• number of unknown parameters increases with sample size
• no consistent estimation of individual parameters βi, σi2
8
Copyright © 2017 by Luc Anselin, All Rights Reserved
• Solutions
• imposing structure
• discrete variation - finite subsets of the data
• continuous variation - parameter surface
• heterogeneity parameters
• fixed effects
• random effects
• spatial heterogeneity may be complicated by spatial autocorrelation
9
Copyright © 2017 by Luc Anselin, All Rights Reserved
Spatial Regimes
10
Copyright © 2017 by Luc Anselin, All Rights Reserved
Discrete Heterogeneity
11
Copyright © 2017 by Luc Anselin, All Rights Reserved
• Spatial Regimes
• systematic discrete spatial subsets of the data
• different coefficient values in each subset
• corrects for heterogeneity, but does not explain
12
Copyright © 2017 by Luc Anselin, All Rights Reserved
• Spatial Regime Specifications
• varying intercepts
• spatial ANOVA
• spatial fixed effects
• full spatial regimes
13
Copyright © 2017 by Luc Anselin, All Rights Reserved
Varying Intercepts
14
Copyright © 2017 by Luc Anselin, All Rights Reserved
• ANOVA - Difference in Means
• approach is standard, regimes are spatial
• E[y1] = μ1 ∀ i ∈ R1 (R1 is region 1)
• E[y2] = μ2 ∀ i ∈ R2 (R2 is region 2)
• H0: μ1 = μ2
15
Copyright © 2017 by Luc Anselin, All Rights Reserved
• Dummy Variable Regression - Variant 1
• no constant term
• indicator variable for each regime
• yi = β1d1i + β2d2i + εi
• d1(2)i = 1 ∀ i ∈ R1(2), 0 elsewhere
• H0: β1 = β2
16
Copyright © 2017 by Luc Anselin, All Rights Reserved
• Dummy Variable Regression - Variant 2
• constant term for overall mean
• yi = α + βdi + εi
• di = 1 ∀ i ∈ R1 , 0 elsewhere
• H0: β = 0, difference from reference mean α
17
Copyright © 2017 by Luc Anselin, All Rights Reserved
• Spatial Fixed Effects
• reference mean and difference by regime
• fixed effects multi-level specification
18
Copyright © 2017 by Luc Anselin, All Rights Reserved
• Spatial Fixed Effects and Spatial Autocorrelation (Anselin and Arribas-Bel 2013)
• common misconception that spatial fixed effects “fix” spatial autocorrelation
• only in special case of group weights
• each observation has all other observations as neighbors
• so-called “Case” weights (Case 1992)
19
Copyright © 2017 by Luc Anselin, All Rights Reserved
Full Spatial Regimes
20
Copyright © 2013 by Luc Anselin, All Rights Reserved
• Spatial Regimes - Full Specification
all coefficients (intercept, slope, variance) vary by regimeequivalent to separate regression by regime
21
Copyright © 2017 by Luc Anselin, All Rights Reserved
Testing for Spatial Heterogeneity
22
Copyright © 2017 by Luc Anselin, All Rights Reserved
• Test on Spatial Homogeneity
• null hypothesis
• equal intercepts, equal slopes
• alternative hypothesis
• different intercepts
• different slopes
• both
23
Copyright © 2017 by Luc Anselin, All Rights Reserved
• Chow Test
• test on structural stability
• based on residual sum of squares in constrained (all coefficients equal - R) and unconstrained (coefficients different - U) regressions
• classic form C =
e′ReR − e′
UeU
k/
e′U
eU
N − 2k∼ F (k, N − 2k)
24
Copyright © 2017 by Luc Anselin, All Rights Reserved
• General Test on Coefficient Stability
• as a set of linear constraints on the coefficients in a pooled regression
• can be readily extended to spatial models G = (J - 1)K
J regimesK coefficients
V = variance
25
Copyright © 2017 by Luc Anselin, All Rights Reserved
Spatial Regimes with Spatial Dependence
26
Copyright © 2017 by Luc Anselin, All Rights Reserved
• Spatial Lag and Spatial Error Models
• allow varying coefficients by regime
• fixed spatial coefficient
• same spatial process throughout
• varying spatial coefficient
• different spatial process for each regime
• difficult assumption - needs to be based on a strong foundation
27
Copyright © 2013 by Luc Anselin, All Rights Reserved
• Spatial Regimes - Spatial Lag Model
fixed spatial autoregressive coefficient
varying spatial autoregressive coefficient
28
Copyright © 2013 by Luc Anselin, All Rights Reserved
• Spatial Regimes - Spatial Error Model
fixed spatial autoregressive coefficient
varying spatial autoregressive coefficient
29
Copyright © 2017 by Luc Anselin, All Rights Reserved
• Spatial Weights Specification
• necessary to construct spatially lagged variables
• neighbors spill over across regimes
• neighbors constrained to be within each regime
• weights truncated, possible isolates
30
Copyright © 2017 by Luc Anselin, All Rights Reserved
• Spatial Chow Test
• use general form of the test with V as coefficient variance matrix in pooled model
31
Copyright © 2017 by Luc Anselin, All Rights Reserved
Spatially Varying Coefficients
32
Copyright © 2017 by Luc Anselin, All Rights Reserved
• Spatially Varying Coefficients
• systematic variation with covariates
• coefficient as a function of other variables (including as a trend surface)
• spatial expansion method
• local estimation over space
• coefficients obtained from a subset (kernel) of nearby data points
• geographically weighted regression (GWR)
33
Copyright © 2017 by Luc Anselin, All Rights Reserved
Expansion Method
34
Copyright © 2017 by Luc Anselin, All Rights Reserved
• Casetti’s Expansion Method
• special case of varying coefficients
• each coefficient is a function of other covariates
• creates interaction effects
• similar in form to multi-level models
35
Copyright © 2017 by Luc Anselin, All Rights Reserved
• Sequential Modeling Strategy
• initial model
• yi = α + xiβi + εi
• expansion equation
• βi = γ0 + zi1γ1 + zi2γ2
• final model
• yi = α + xi (γ0 + zi1γ1 + zi2γ2) + εi
• yi = α + xiγ0 + (zi1xi)γ1 + (zi2xi)γ2 + εi
36
Copyright © 2017 by Luc Anselin, All Rights Reserved
• Implementation Issues
• multicollinearity
• t-test values unreliable
• various fixes
• principal components (orthogonal expansion)
• danger of overfitting
37
Copyright © 2017 by Luc Anselin, All Rights Reserved
• Random Expansion Model
• random error in expansion equation
• βi = γ0 + zi1γ1 + zi2γ2 + ψi
• error term in final model is heteroskedastic
• yi = α + xi (γ0 + zi1γ1 + zi2γ2 + ψi) + εi
• νi = xiψi + εi
• Var[νi] = xi2 σ2ψ + σ2ε
• similar to random coefficient model and multilevel models
38
Copyright © 2017 by Luc Anselin, All Rights Reserved
Geographically Weighted Regression
39
Copyright © 2017 by Luc Anselin, All Rights Reserved
• Geographically Weighted Regression
• local regression
• a different set of parameter values for each location
• parameter values obtained from a subset of observations using kernel regression
40
Copyright © 2017 by Luc Anselin, All Rights Reserved
• Local Regression
• non-parametric specification
• simple bivariate regression
• yi = m(xi) + ui
• functional form of m is unspecified
• m(xi) yields the conditional expectation of y | x
41
Copyright © 2017 by Luc Anselin, All Rights Reserved
• Local Average
• what is the expected value of yi given x, E[yi | x]
• special case: for a given x0 with multiple yi
• example: two values for PATIO dummy, house price
• solution:
• take m(x0) as the average of yi for x0 =0 and x0 =1
42
Copyright © 2017 by Luc Anselin, All Rights Reserved
local averagepredictor of PRICE for two values of PATIO
43
Copyright © 2017 by Luc Anselin, All Rights Reserved
• Locally Weighted Average
• expand the estimate of m(x0) to include values of yi observed for values of x “close” to x0
• compute a locally weighted average
• weights sum to one
• weights larger as x closer to x0 (for h ↓)
• m(x0) = ∑i wi0,h yi
44
Copyright © 2017 by Luc Anselin, All Rights Reserved
locally weighted average (lowess) of PRICE for LOTSZ
45
Copyright © 2017 by Luc Anselin, All Rights Reserved
• Kernel Regression
• special case of locally weighted average
• use kernel function as the weights
• m(x0) = ∑i K [(xi - x0)/h ]yi
• K is kernel function
• h is bandwidth s.t. K = 0 for xi - x0 > h
46
Copyright © 2017 by Luc Anselin, All Rights Reserved
• Kernel Functions with Finite Bandwidth
• Epanechnikov
• K(z) = 1 - z2
• Bisquare
• K(z) = (1 - z2)2
• with z = (xi - x0) / h
47
Copyright © 2017 by Luc Anselin, All Rights Reserved
• Gaussian Kernel
• asymptotic bandwidth
• specified in function of standard error or variance
• K(z) = exp(- z2/2)
48
Copyright © 2017 by Luc Anselin, All Rights Reserved
• GWR Estimation
• local estimation based on nearby locations
• not just yi but x-y pairs at nearby locations
• kernel regression yields a different coefficient for each location
• specify kernel function and bandwidth
49
Copyright © 2017 by Luc Anselin, All Rights Reserved
• GWR Kernel Regression
• location-specific kernel weights
• W(ui, vi) diagonal elements are weights
• b(ui,vi) = [X’W(ui,vi)X]-1X’W(ui,vi)y
• fixed kernel vs adaptive kernel
50
Copyright © 2017 by Luc Anselin, All Rights Reserved
fixed bandwidth kernel adaptive kernel
Source: Fotheringham et al (2002)
51
Copyright © 2017 by Luc Anselin, All Rights Reserved
• GWR - Practical Issues
• choice of bandwidth
• use cross-validation
• parameter inference
• still several theoretical loose ends
• visualizing parameter heterogeneity
52
Copyright © 2017 by Luc Anselin, All Rights Reserved
Spatial Random Effects
53
Copyright © 2017 by Luc Anselin, All Rights Reserved
Random Coefficients
54
Copyright © 2017 by Luc Anselin, All Rights Reserved
• Random Coefficient Regression
• extreme heterogeneity, but variability in βi driven by a random process - no space
• βi = β + ψi with E[ψi]=0 and Var[ψi]=σ2
• heteroskedastic regression for mean effect
• yi = α + xiβ + νi , var[νi] =σ2ψxi2 + σ2ε
55
Copyright © 2017 by Luc Anselin, All Rights Reserved
• Mixed Linear Models
• both fixed and random coefficients
• y = Xβ + Zψ + ε
• Z a “design matrix”, could be same as X
• ψ random coefficients with mean zero and variance Σψ
• ε random error vector with variance Σε
56
Copyright © 2017 by Luc Anselin, All Rights Reserved
• Spatial Random Coefficients
• introduce spatial dependence structure in random variation of coefficient
• βi - β = ρ Σj wij (βj - β) + ψi - SAR model
• βi = β + λ Σj wij ψj + ψi - SMA model
• complex covariance structures
57
Copyright © 2017 by Luc Anselin, All Rights Reserved
Spatial Random Effects
58
Copyright © 2017 by Luc Anselin, All Rights Reserved
• Spatial Random Effects
• βi = β + ψi with spatial effects introduced through random effect ψi
• typically a CAR process
• Bayesian hierarchical model - BYM model
• βi = β + ψi + νi
• spatial dependence in ψi, heterogeneity in νi
• not identified in Gaussian (linear regression) model
59
Copyright © 2017 by Luc Anselin, All Rights Reserved
• Example - Poisson Regression
• spatial autocorrelation needs to be introduced indirectly
• auto-Poisson model only allows negative spatial autocorrelation
• random effects model
60
Copyright © 2017 by Luc Anselin, All Rights Reserved
• Poisson Regression
• P[Y = y] = e-μ μy / y!
• μ is the mean
• μ as a function of regressors to model heterogeneity
• μi = exp(xi’β) no error term
• random effects
• μi = exp(xi’β + ψi + νi)
• spatial effects through ψi, e.g, CAR model
• non-spatial heterogeneity through νi
61