A Forest Cover Change Study Gone Bad

Lessons Learned(?) Measuring Changes in Forest Cover in

Madagascar

Ned HorningCenter for Biodiversity and Conservation

American Museum of Natural History

(horning@amnh.com)

Overview of the project

Goal:

Use available data to determine the rates of forest loss in the periphery of six protected areas in Madagascar to see if USAID interventions were having an impact.

Datasets:

• 1950 Forest maps (1:100,000) based on photography acquired in the late 1940s

• 1991/1992 black and white aerial photography 1:40,000

• 1993/1994 Landsat TM images

The issues

1. Land cover classes ill defined

2. Post-classification overlay used to derive results

3. Inappropriate data sets

4. Incorrect equations

5. Results indiscriminately modified

6. Accuracy not assessed

The land cover classes were not well defined

• Three classes were interpreted (primary forest, secondary forest, and other) and none were defined

• Primary and secondary forest classes can be very difficult to differentiate using satellite imagery

• Variations within the secondary class were severe since the image analysis was performed by several different groups without significant training

Typical land cover accuracy figures

• Forest/nonforest, water/no water, soil/vegetated: accuracies in the high 90%

• Conifer/hardwood: 80-90%

• Genus: 60-70%

• Species: 40-60%

• Bottom line: The greater the detail (precision) the lower the per class accuracy

Note: If including a Digital Elevation Model (DEM) in the classification accuracy typically improves by up to 10%

Recommendations

• Use the fewest number of classes that are practical - forest/non-forest in this case

• Clearly describe land cover classes before the analysis phase (an interpretation key can be helpful)

• Define classes that can be reliably interpreted using the available data

• Provide sufficient training

Used post-classification overlay to derive results

• Post-classification is the most common change detection method and is rarely the best choice, especially when the change-class of interest comprises a small percent of the entire area

• The errors from the individual layers are present in the final change image

• Error estimates for individual layers (dates) were not known

• Geometric registration errors were compounded since different data sets were compared using automated methods

The data sets and interpretation methods were not appropriate

• Different data set types were used for each time period and season of image acquisition varied significantly

• The original photos used to create the forest cover maps were available

• The 1991/1992 aerial photos were not orthorectified and were interpreted manually without a stereoscope or other suitable instrument

• Landsat TM data are not well suited for monitoring change in small areas, especially when the time interval is short and the terrain variation is significant

• The Landsat TM images were converted to 255-color index images

8 bits (0-255) 6 bits (0–63)

3 bits (0-7)

1 bit (0-1)

Recommendations

• Use aerial photos from the 1940’s and 1950’s in place of the forest cover maps

• Interpret aerial photos using appropriate techniques and if possible work with orthophotos

• If Landsat TM data are to be used the area of interest should be enlarged and/or the time interval between images/photos should be lengthened

• If possible select imagery from similar seasons

• Do not reduce the quantization of the Landsat TM images

• Take into account the desired accuracy and precision of the results when selecting data sets

The equation was incorrect

• The original formula for the 1991/92 – 1993/94 period was:

Change%=((forest T1-forest T2) * 100/(forest T2 * 3))

• The last part of the formula should have been (forest T1 * 3).

• The time period should not have been static (3).

Recommendations

• Calculate forest cover change like this:

For each target zone calculate:

• %forest loss = forest T1 – forest T2 / forest T1 * 100

• Annual forest loss = %forest loss / ((2nd date-1st date) / 365)

• Weighted %annual loss = annual forest loss * (forest T1 / sum of forest T1 for all zones)

Average forest loss for all areas:

average forest loss = sum of weighted % annual loss for all zones

The results were indiscriminately modified

• The non-forest to primary forest change class was often modified or eliminated

• The modifications were haphazard but biased

• The results happened to match the client’s perceived rate of deforestation

Recommendations

• Do not bias the results simply because they don’t appear to be correct

• Procedures for compiling data must be very clear and an effort should be made to verify the procedures are being followed

An accuracy assessment was not carried out

• The results were reviewed by several people but there was no attempt to assess the accuracy

• The project was rushed in order to produce results to meet a deadline

Recommendations

• Budget enough money to carry out an accuracy assessment

• Plan ahead to avoid the last minute rush

Consequences

• The rate of deforestation was overestimated by an order of magnitude

• The bottom line was that the results could not be used

• The client had to drop the deforestation indicator and reassessed the process used to monitor changes in forest cover

Lessons learned?

Extracted from a 2003 USAID Program Data Sheet:

“From 1993 - 2001, the rate of forest loss was 2.6% and 3.5% in USAID zones, compared to 6.7% loss in comparable non-intervention zones. “