methods of validating maps of deforestation and selective logging carlos souza jr....

Methods of Validating Maps of Deforestation and Selective Logging

Carlos Souza [email protected]

Instituto do Homem e Meio Ambiente da Amazônia—ImazonCaixa Postal 5101, Belém, PA, Brasil CEP 66613-397

Department of Geography – University of CaliforniaSanta Barbara, California 93106, USA

[becky,matzke,carlos,mclark,numata,dar]@geog.ucsb.edu

Workshop Objectives

to propose protocols for ground-based accuracy assessments of remotely sensed estimates of deforestation and selective logged forests.

to address means of establishing error bars and accuracy estimates pertaining to deforestation and selective logging products.

Protocol for Accuracy Assessment of RS Products

Response DesignProtocol

Sampling DesignProtocol

Estimation andAnalysis Protocol

AccuracyAssessment

Stephen V. StehmanStephen V. StehmanRaymond L. CzaplewskiRaymond L. Czaplewski

RSE, 1998. 64:331-344RSE, 1998. 64:331-344

Sampling Protocol

Sample unit is a location (point) or

an area (pixel or polygon) used to assess the accuracy of a map.

links the map and the reference data. Pixel sampling

unit

Reference data (Ikonos)




AccuracyAssessment


RSE, 1998. 64:331-344RSE, 1998. 64:331-344

Sampling Protocol

Sampling method Requires an unbiased sample

Simple random Stratified random Cluster

Primary (psu) Secondary (ssu)

Polygon

psu

ssu

5x5 pixelblock

ssu

psu

““A scientifically defensible accuracy A scientifically defensible accuracy assessment requires a assessment requires a probability sampling”probability sampling”


RSE, 1998. 64:331-344RSE, 1998. 64:331-344




AccuracyAssessment

Response Design

It is a classification scheme applied to the sampling unit. Labeling protocol

Assigns a land-cover classification to the sampling unit based on the evaluation protocol:

Land cover proportion Predominant land-cover Centroid Multiple land cover classification Fuzzy classification

Evaluation:• 50% vegetation• 40% bare soil• 10% shrubs

Labeling:• Pasture

Evaluation:• pixel centerLabeling:

• Forest


RSE, 1998. 64:331-344RSE, 1998. 64:331-344




AccuracyAssessment

Reference Data

D – DecidiousC – ConiferAG – AgricultureSB - Shrub

43414111510375

10490374

1151985110

10085816

1152422465

D C AG SB Row Total

D

C

AG

SB

ColumnTotal

Overall Accuracy =(65+81+85+90)/434 = 74%

Cla

ssif

ied

Dat

a

D = 65/75 = 87%C = 81/103 = 79%AG = 85/115 = 74%SB = 90/141 = 64%

Producers Accuracy

D = 65/115 = 57%C = 81/100 = 81%AG = 85/115 = 74%SB = 90/104 = 87%

Users Accuracy

Source: Congalton and Green (1999)

Legend


RSE, 1998. 64:331-344RSE, 1998. 64:331-344




AccuracyAssessment

Estimation andAnalysis

Problems with the Standard Protocol

Study Area

Sampling Design

Flight-line randomly sampled by time-code Cluster samples centered on flight-line Reference sample unit = 30 m x 30 m area Five reference samples recorded per mosaic

Evaluation Protocol: Class Definitions Primary forest Pasture trees >20 m height herbaceous contiguous canopy shrubs < 2 m Second-growth forest annual crops shrubs > 2 m Urban/soil trees < 20 m human structures perennial crops roads Water bare soil

Response Design

Response Design Labeling protocol: dominant land-cover

34

1 2

0

Evaluation:100% Pasture

Label:“Pasture”

Evaluation:100% SecondaryForest

Label:“Secondary Forest”

Evaluation:55% Pasture45% Secondary Forest

Label:“Pasture”

Evaluation:100% Primary Forest

Label:“Primary Forest”

Accuracy AssessmentMethods and Analysis

Correct geolocation errors between video mosaics and map

Randomly select image frames from video based on time code

Mosaic and georectify video and overlay cluster samples

Group (i.e. five interpreters) re-interprets video to generate ‘gold’ reference data set

Five individuals independently interpret video to produce five reference data sets

Remove mixed samples and edge pixels; identify change

Parameters estimated from error matrices: overall accuracy, producer’s accuracy (omission errors), user’s accuracy (commission errors), Kappa (KHAT) statistics, Kappa variance (for statistical comparisons)

Interpreter vs. interpreter accuracyInterpreter vs. map accuracy

‘Gold’ reference vs. map accuracy

Influence of reference data corrections on map accuracy*statistically different Kappa coefficients (95% confidence level)

Version Corrections # Samples Total Agreement Kappa coefficient1 none 790 75.4% 0.60232 geocorrection 790 83.2% 0.7269

3geocorrection; remove change pixels 771 85.3% 0.7595

4geocorrection; remove mixed reference samples 684 85.4% 0.7437

5geocorrection; remove map edge pixels 605 88.1% 0.7914

6geocorrection; remove mixed reference and map edge pixels 532 91.0% 0.8283

7geocorrection; remove mixed, edge, and change pixels 523 92.5% 0.8557

Area estimation accuracy assessment

Gross Deforestation in the State of Acre 1999 a 2002

0.0

2000.0

4000.0

6000.0

8000.0

10000.0

12000.0

14000.0

16000.0

18000.0

1999 2000 20001 20002

Area (

km

2 )

Imazon-Imac

INPE

33%21%

23%17%

Average = 23.50%

Stdev = 6.81%

Deforestation Estimates in Acre

Area of Forest Fragments:

2864.9 km2

Are the Fragments “Capoeiras”?Deforestation Age

Forest

Deforestation until 2000

Water

Forest – Prodes digital

Landsat Image Classification (Roberts et al., 2003)

FragmentsFragments

Classification (INPE - Prodes Digital)

Scale Effect on Mapping Deforestation

1:50.000

1:250.000

9 km

9 km

1:50.000

1:250.000

Scale Effect on Mapping Deforestation

Deforestation by Classes of Size

Desmatamento - 2003

9720

14139

7348

4272

29582022

44083481

1727598 858

0%

20%

40%

60%

80%

100%

0.36-5 5-10 10-15 15-20 20-25 25-30 30-50 50-100 100-200

200-300

>300Área (ha)

Pe

rce

ntu

al

de

po

líg

on

o

de

sm

ata

do

0

3000

6000

9000

12000

15000

N°

de

po

lig

on

o

de

sm

ata

do

Desmatamento 2002

8794

11550

6306

3710

25611959

42503753

2030

737 9860%

20%

40%

60%

80%

100%

0.36-5 5-10 10-15 15-20 20-25 25-30 30-50 50-100 100-200

200-300

>300

Área (ha)

Pe

rce

ntu

al

de

po

lig

on

o

de

sm

ata

do

0

3000

6000

9000

12000

15000

N°

de

po

lig

on

o d

es

ma

tad

o

Contribição do polígono desmatado brutoContribuição cumulativa do polígono desmatadoNúmero de polígonos desmatado

Co

ntr

ibu

içã

o p

/ o

D

es

ma

tam

en

to (

%)

Co

ntr

ibu

içã

o p

/ o

D

es

ma

tam

en

to (

%)

Nú

me

ro d

e P

olí

go

no

sN

úm

ero

de

Po

líg

on

os

Classification Errors in 2002

Old deforestation

Forest

Increment

Incremento 2002 - INPE

Classification Imazon/Imac Classification - Prodes

Accuracy estimates of deforestation and selective logging products

Reference DataHigh resolution satellite image

Ikonos

Field data Several sources

Ikonos Data for LBA-Ecology Study Sites

Hurtt et al., (2003)

Proposal Map accuracy

Sample design: random cluster samplingResponse design: ?Estimation design: ?Correct reference data errorsSeveral research groups interpreting the

reference data Include area estimation accuracy

methods of validating maps of deforestation and selective logging carlos souza jr....

Documents