using simulation to assess the accuracy of a forest ...€¦ · using simulation to assess the...
TRANSCRIPT
Using Simulation to Assess the Accuracy of a Forest Inventory System
Steve Fairweather, Ph.D., ACFFairweather Biometrics, LLCPO Box 1023Welches, Oregon 97067
© 2019 Fairweather Biometrics, LLC Western Forest Economists, 6/4/2019
© 2019 Fairweather Biometrics, LLC Western Forest Economists, 6/4/2019
Using Simulation to Assess the Accuracy of a Forest Inventory System
I. The Nature of a Stand-Based Inventory
II. The Problem
III. Why We’re Interested
IV. Basic Steps
V. Example Simulations for a Single Stand
VI. Example for a Small Group of Stands
VII. Example for a Stand Over Time
VIII.Large Ownership Example
© 2019 Fairweather Biometrics, LLC
I. The Nature of a Stand-Based
Inventory…
Inherited summary
data in 2004;
“updated” to 2019
Cruised 2010;
grown to 2019
Cruised Jan. 2019
Never cruised; stratum
average from 2015,
grown to 2019
The estimate of
the total inventory
is the sum of the
individual stand
estimates.
Using Simulation to Assess the Accuracy of a Forest Inventory System
Western Forest Economists, 6/4/2019
© 2019 Fairweather Biometrics, LLC
II. The Problem ….If all the stand estimates are based on new
cruises, we can assess the reliability of the
total inventory estimate using conventional
stratified sampling formulas; we can
construct a confidence interval for the total
inventory estimate.
But not when the individual stand
estimates are from a mix of sources.
Using Simulation to Assess the Accuracy of a Forest Inventory System
Western Forest Economists, 6/4/2019
© 2019 Fairweather Biometrics, LLC
III. Why We’re
Interested… • 2001 -------------------------
Western Forest Economists, 6/4/2019
• 2002 – Sarbanes-Oxley Act
• 2004 – How does a forest products
company CEO sign off on the annual
report stating the total forest
inventory?
Using Simulation to Assess the Accuracy of a Forest Inventory System
© 2019 Fairweather Biometrics, LLC
III. Why We’re
Interested…
Western Forest Economists, 6/4/2019
• In timberland acquisition due diligence,
inventory verification is usually a key component.
• Field-based checking is expensive and
has to be done in a short amount of time.
• An office audit of inventory practices is cheaper
and quicker, but not particularly convincing.
Using Simulation to Assess the Accuracy of a Forest Inventory System
• Accuracy assessment by simulation provides a
quick, low cost, and understandable alternative.
© 2019 Fairweather Biometrics, LLC
III. Why We’re
Interested…
Western Forest Economists, 6/4/2019
Using Simulation to Assess the Accuracy of a Forest Inventory System
• Accuracy Assessment by
Simulation – given the
characteristics of the ownership, and
given the inventory practices and
protocols in place, state the
probability of the inventory estimate
being within a specified error
tolerance; both for the total inventory,
and for individual stands.
Generate
a Model
Forest
Describe the
inventory
system/protocols
being used
In many
replications, apply
the protocols to
the model forest
and note how the
protocols perform
in each replication
Use the
distribution of
replication results
to assess the
average
(expected)
accuracy of the
inventory system.
IV. Basic Steps …
© 2019 Fairweather Biometrics, LLC
Using Simulation to Assess the Accuracy of a Forest Inventory System
Western Forest Economists, 6/4/2019
© 2019 Fairweather Biometrics, LLC
V. Example Simulations for a Single Stand…
Stand No. 102 –
• “Perfect Knowledge”
• 31 acres, age 28
• 12,877 bf/a
• mixed conifer
Cruise –
• n = 21 plots
• CV = 70%
Using Simulation to Assess the Accuracy of a Forest Inventory System
Western Forest Economists, 6/4/2019
© 2019 Fairweather Biometrics, LLC
V. Example Simulations for a Single Stand…
Stand No. 102 –
• “Perfect Knowledge”
• 31 acres, age 28
• 12,877 bf/a
• mixed conifer
Every time we cruise
this stand, we can get
a different estimate of
the volume per acre
Cruise –
• n = 21 plots
• CV = 70%
Western Forest Economists, 6/4/2019
Using Simulation to Assess the Accuracy of a Forest Inventory System
© 2019 Fairweather Biometrics, LLC
V. Example Simulations for a Single Stand…
Western Forest Economists, 6/4/2019
Using Simulation to Assess the Accuracy of a Forest Inventory System
© 2019 Fairweather Biometrics, LLC
V. Example Simulations for a Single Stand…
Western Forest Economists, 6/4/2019
Using Simulation to Assess the Accuracy of a Forest Inventory System
© 2019 Fairweather Biometrics, LLC
V. Example Simulations for a Single Stand…
Western Forest Economists, 6/4/2019
Using Simulation to Assess the Accuracy of a Forest Inventory System
© 2019 Fairweather Biometrics, LLC
V. Example Simulations for a Single Stand…
Western Forest Economists, 6/4/2019
Using Simulation to Assess the Accuracy of a Forest Inventory System
© 2019 Fairweather Biometrics, LLC
V. Example Simulations for a Single Stand…
Western Forest Economists, 6/4/2019
Using Simulation to Assess the Accuracy of a Forest Inventory System
© 2019 Fairweather Biometrics, LLC
V. Example Simulations for a Single Stand…
Western Forest Economists, 6/4/2019
Using Simulation to Assess the Accuracy of a Forest Inventory System
© 2019 Fairweather Biometrics, LLC
V. Example Simulations for a Single Stand…
Western Forest Economists, 6/4/2019
Using Simulation to Assess the Accuracy of a Forest Inventory System
© 2019 Fairweather Biometrics, LLC
V. Example Simulations for a Single Stand…
Western Forest Economists, 6/4/2019
Using Simulation to Assess the Accuracy of a Forest Inventory System
© 2019 Fairweather Biometrics, LLC
V. Example Simulations for a Single Stand…
Western Forest Economists, 6/4/2019
Using Simulation to Assess the Accuracy of a Forest Inventory System
© 2019 Fairweather Biometrics, LLC
V. Example Simulations for a Single Stand…
Western Forest Economists, 6/4/2019
Using Simulation to Assess the Accuracy of a Forest Inventory System
© 2019 Fairweather Biometrics, LLC
V. Example Simulations for a Single Stand…
101 cruises (25%)
were within +/- 5%
of true volume/a
Western Forest Economists, 6/4/2019
Using Simulation to Assess the Accuracy of a Forest Inventory System
© 2019 Fairweather Biometrics, LLC
V. Example Simulations for a Single Stand…
194 cruises (48%)
were within +/- 10%
of true volume/a
Western Forest Economists, 6/4/2019
Using Simulation to Assess the Accuracy of a Forest Inventory System
© 2019 Fairweather Biometrics, LLC
VI. Example for a Small Group of Stands…
Suppose we have a small ownership of 3 stands. We can simulate cruising in each stand and combine the results for the total ownership.
Using Simulation to Assess the Accuracy of a Forest Inventory System
Western Forest Economists, 6/4/2019
© 2019 Fairweather Biometrics, LLC
VI. Example for a Small Group of Stands…
Suppose we have a small ownership of 3 stands. We can simulate cruising in each stand and combine the results for the total ownership.
How reliable will our estimate of the total inventory be?
Could a low cruise in one stand offset a high cruise in another stand?
Using Simulation to Assess the Accuracy of a Forest Inventory System
Western Forest Economists, 6/4/2019
© 2019 Fairweather Biometrics, LLC
VI. Example for a Small Group of Stands…
Using Simulation to Assess the Accuracy of a Forest Inventory System
Western Forest Economists, 6/4/2019
© 2019 Fairweather Biometrics, LLC
VI. Example for a Small Group of Stands…
This sounds like a
confidence interval, but it
is not. It is a new metric
for describing accuracy.
Using Simulation to Assess the Accuracy of a Forest Inventory System
Western Forest Economists, 6/4/2019
© 2019 Fairweather Biometrics, LLC
VII. Example for a Stand Over Time…
Most of our current stand-based inventory estimates are the result of annual updates that try to account for growth and removals.
Using Simulation to Assess the Accuracy of a Forest Inventory System
Western Forest Economists, 6/4/2019
© 2019 Fairweather Biometrics, LLC
VII. Example for a Stand Over Time…
Most of our current stand-based inventory estimates are the result of annual updates that try to account for growth and removals.
Therefore our simulation has to account for past cruise estimates, growth, and the use of a growth model, with its own inherent error.
Using Simulation to Assess the Accuracy of a Forest Inventory System
Western Forest Economists, 6/4/2019
© 2019 Fairweather Biometrics, LLC
VII. Example for a Stand Over Time…
Using Simulation to Assess the Accuracy of a Forest Inventory System
Western Forest Economists, 6/4/2019
© 2019 Fairweather Biometrics, LLC
VII. Example for a Stand Over Time…
Specifying the possible error for predicted volume by the growth model …
Using Simulation to Assess the Accuracy of a Forest Inventory System
Western Forest Economists, 6/4/2019
© 2019 Fairweather Biometrics, LLC
VII. Example for a Stand Over Time…
Specifying the possible error for predicted volume by the growth model …
Could a bad initial cruise be offset by a bad growth prediction?
Using Simulation to Assess the Accuracy of a Forest Inventory System
Western Forest Economists, 6/4/2019
© 2019 Fairweather Biometrics, LLC
VII. Example for a Stand Over Time…
Using Simulation to Assess the Accuracy of a Forest Inventory System
Western Forest Economists, 6/4/2019
© 2019 Fairweather Biometrics, LLC
VII. Example for a Stand Over Time…
Using Simulation to Assess the Accuracy of a Forest Inventory System
Western Forest Economists, 6/4/2019
© 2019 Fairweather Biometrics, LLC
VII. Example for a Stand Over Time…
Using Simulation to Assess the Accuracy of a Forest Inventory System
Western Forest Economists, 6/4/2019
Generate
a Model
Forest
Describe the
inventory
system/protocols
being used
In many
replications, apply
the protocols to
the model forest
and note how the
protocols perform
in each replication
Use the
distribution of
replication results
to assess the
average
(expected)
accuracy of the
inventory system.
© 2019 Fairweather Biometrics, LLC
Using Simulation to Assess the Accuracy of a Forest Inventory System
VIII. Large Ownership Example…
Western Forest Economists, 6/4/2019
Generate a
Model Forest
that (1) resembles
the forest of
interest, and (2)
about which we
have perfect
knowledge.
VIII. Large Ownership Example…
© 2019 Fairweather Biometrics, LLC
Using Simulation to Assess the Accuracy of a Forest Inventory System
Western Forest Economists, 6/4/2019
Generate a Model Forest
© 2019 Fairweather Biometrics, LLC
Using Simulation to Assess the Accuracy of a Forest Inventory System
VIII. Large Ownership Example…
Western Forest Economists, 6/4/2019
Describe the
inventory
system/protocols
being used
For example -
• Cruise stands on a 10-year cycle
• Usually, n=14 plots in a cruised stand
• Do cruise new residual stands
• Don’t cruise new clearcuts
• Don’t cruise stands less than 10 acres
• Assign stratum average to uncruised
stands in first year of simulation
• Grow initial stand estimates forward until
the stand is cruised or cut
© 2019 Fairweather Biometrics, LLC
Using Simulation to Assess the Accuracy of a Forest Inventory System
VIII. Large Ownership Example…
Western Forest Economists, 6/4/2019
Describe the
inventory
system/protocols
being used
Assumptions (and Sources of Error) -
• Stands to be cruised each year are selected
randomly
• Underlying CV = 70% for volume
• Zero bias in our cruise procedure
• Observed capability of our growth model
over a 10-year period ranges from 75% to
130% of the true volume
• All stand acreage figures are correct
• All stands were assigned to the correct
stratum at the start
© 2019 Fairweather Biometrics, LLC
Using Simulation to Assess the Accuracy of a Forest Inventory System
VIII. Large Ownership Example…
Western Forest Economists, 6/4/2019
Generate
a Model
Forest
Describe the
inventory
system/protocols
being used
In many
replications, apply
the protocols to
the model forest
and note how the
protocols perform
in each replication
Use the
distribution of
replication results
to assess the
average
(expected)
accuracy of the
inventory system.
© 2019 Fairweather Biometrics, LLC
Using Simulation to Assess the Accuracy of a Forest Inventory System
VIII. Large Ownership Example…
Western Forest Economists, 6/4/2019
Estimating the
total inventory
(i.e., sum of all
the stand
estimates).
© 2019 Fairweather Biometrics, LLC
Using Simulation to Assess the Accuracy of a Forest Inventory System
VIII. Large Ownership Example…
Western Forest Economists, 6/4/2019
Estimating the
total inventory
(i.e., sum of all
the stand
estimates).
© 2019 Fairweather Biometrics, LLC
Using Simulation to Assess the Accuracy of a Forest Inventory System
VIII. Large Ownership Example…
Western Forest Economists, 6/4/2019
Using Simulation to Assess the Accuracy of a Forest Inventory System
Steve Fairweather, Ph.D., ACFFairweather Biometrics, LLCPO Box 1023Welches, Oregon 97067
© 2019 Fairweather Biometrics, LLC Western Forest Economists, 6/4/2019