nr 422 quality control jim graham spring 2009. staircase of knowledge increasing subjectivity human...
TRANSCRIPT
![Page 1: NR 422 Quality Control Jim Graham Spring 2009. Staircase of Knowledge Increasing Subjectivity Human value added Observation And Measurement Data Information](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649e7a5503460f94b7a55b/html5/thumbnails/1.jpg)
NR 422Quality Control
Jim Graham
Spring 2009
![Page 2: NR 422 Quality Control Jim Graham Spring 2009. Staircase of Knowledge Increasing Subjectivity Human value added Observation And Measurement Data Information](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649e7a5503460f94b7a55b/html5/thumbnails/2.jpg)
Staircase of Knowledge
Increasing Subjectivity
Hum
an v
alue
add
ed
ObservationAnd
Measurement
Data
Information
Knowledge
Understanding
Wisdom
OrganizationInterpretation
Verification
SelectionTesting
ComprehensionIntegration
Judgment
Environmental Monitoring and Characterization, Aritola, Pepper, and Brusseau
![Page 3: NR 422 Quality Control Jim Graham Spring 2009. Staircase of Knowledge Increasing Subjectivity Human value added Observation And Measurement Data Information](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649e7a5503460f94b7a55b/html5/thumbnails/3.jpg)
Error
• Data does not match reality (ever)
• Gross errors
• Accuracy (bias): distance from truth– | Measurement mean – Truth |
• Precision: variance within the data– Standard Deviation (stddev)
• Measurement Limits
![Page 4: NR 422 Quality Control Jim Graham Spring 2009. Staircase of Knowledge Increasing Subjectivity Human value added Observation And Measurement Data Information](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649e7a5503460f94b7a55b/html5/thumbnails/4.jpg)
Accuracy and Precision
High AccuracyLow Precision
http://en.wikipedia.org/wiki/Accuracy_and_precision
Low AccuracyHigh Precision
![Page 5: NR 422 Quality Control Jim Graham Spring 2009. Staircase of Knowledge Increasing Subjectivity Human value added Observation And Measurement Data Information](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649e7a5503460f94b7a55b/html5/thumbnails/5.jpg)
Bias (Accuracy)
• Bias = Distance from truth
Truth Mean
Bias
![Page 6: NR 422 Quality Control Jim Graham Spring 2009. Staircase of Knowledge Increasing Subjectivity Human value added Observation And Measurement Data Information](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649e7a5503460f94b7a55b/html5/thumbnails/6.jpg)
Standard Deviation (Precision)
Each band represents one standard deviationSource: Wikipedia
![Page 7: NR 422 Quality Control Jim Graham Spring 2009. Staircase of Knowledge Increasing Subjectivity Human value added Observation And Measurement Data Information](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649e7a5503460f94b7a55b/html5/thumbnails/7.jpg)
Other Approaches
• Confidence Intervals
• +- Some range (suspect)
![Page 8: NR 422 Quality Control Jim Graham Spring 2009. Staircase of Knowledge Increasing Subjectivity Human value added Observation And Measurement Data Information](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649e7a5503460f94b7a55b/html5/thumbnails/8.jpg)
Sources of Error• Measurement Error
– Protocol– User– Instrument
• Processing Errors– Procedure– User – Instrument
• Data Errors– Age– Metadata/Documentation
![Page 9: NR 422 Quality Control Jim Graham Spring 2009. Staircase of Knowledge Increasing Subjectivity Human value added Observation And Measurement Data Information](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649e7a5503460f94b7a55b/html5/thumbnails/9.jpg)
Protocol
• Rule #1: Have one!
• Step by step instructions on how to collect the data– Calibration– Equipment required– Training required– Steps– QAQC
• See Globe Protocols:– http://www.globe.gov/sda/tg00/aerosol.pdf
![Page 10: NR 422 Quality Control Jim Graham Spring 2009. Staircase of Knowledge Increasing Subjectivity Human value added Observation And Measurement Data Information](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649e7a5503460f94b7a55b/html5/thumbnails/10.jpg)
Protocol Error
• Is there a protocol?
• What is being measured?
• Is it complete: How large? How small?
• Unexpected circumstances (illness, weather, accidents, equipment failures, changing ecosystems)
![Page 11: NR 422 Quality Control Jim Graham Spring 2009. Staircase of Knowledge Increasing Subjectivity Human value added Observation And Measurement Data Information](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649e7a5503460f94b7a55b/html5/thumbnails/11.jpg)
User Measurement Errors
• Wrong Datum
• Data in wrong field/attribute
• Missing data
• Gross errors
• Precision and Accuracy
• Observer error: expertise and “drift”
![Page 12: NR 422 Quality Control Jim Graham Spring 2009. Staircase of Knowledge Increasing Subjectivity Human value added Observation And Measurement Data Information](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649e7a5503460f94b7a55b/html5/thumbnails/12.jpg)
Instrument Errors
• Calibration
• Drift
• Humans as instruments:– DBH– Weight– Humans are almost always involved!– Fortunately we can be calibrated and have
our drift measured
![Page 13: NR 422 Quality Control Jim Graham Spring 2009. Staircase of Knowledge Increasing Subjectivity Human value added Observation And Measurement Data Information](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649e7a5503460f94b7a55b/html5/thumbnails/13.jpg)
Calibration
• Sample a portion of the study area repeatedly and/or with higher precision– GPS: benchmarks, higher resolution– Measurements: lasers, known distances– Identifications: experts, known samples
• Use bias and stddev throughout study
• Also provides an estimate for min/max
![Page 14: NR 422 Quality Control Jim Graham Spring 2009. Staircase of Knowledge Increasing Subjectivity Human value added Observation And Measurement Data Information](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649e7a5503460f94b7a55b/html5/thumbnails/14.jpg)
Flow of error
• Capture error during data collection
• Determine error of other datasets– If unavailable, estimate the error
• Maintain error throughout processing– Error will increase
• Document final error in reports and metadata
![Page 15: NR 422 Quality Control Jim Graham Spring 2009. Staircase of Knowledge Increasing Subjectivity Human value added Observation And Measurement Data Information](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649e7a5503460f94b7a55b/html5/thumbnails/15.jpg)
Processing Error
• Error changes with processing
• The change depends on the operation and the type of error:– Min/Max– Average Error– Standard Error of the Mean– Standard Deviation– Confidence Intervals
![Page 16: NR 422 Quality Control Jim Graham Spring 2009. Staircase of Knowledge Increasing Subjectivity Human value added Observation And Measurement Data Information](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649e7a5503460f94b7a55b/html5/thumbnails/16.jpg)
Combing Bias
• Add/Subtraction:– Bias (Bias1+Bias2)=
• T- (Mean1*Num1+Mean2*Num2)/(Num1*Num2)
• Simplified: (|Bias1|+|Bias2|)/2
• Multiply Divide:– Bias (Bias1*Bias2)=
• T- (Mean1*Mean2)• Simplified: |Bias1|*|Bias2|
Derived by Jim Graham
![Page 17: NR 422 Quality Control Jim Graham Spring 2009. Staircase of Knowledge Increasing Subjectivity Human value added Observation And Measurement Data Information](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649e7a5503460f94b7a55b/html5/thumbnails/17.jpg)
Combining Standard Deviation
• Add/Subtract:– StdDev=sqrt(StdDev1^2+StdDev2^2)
• Multiply/Divide:– StdDev=
• sqrt((StdDev1/Mean1)^2+(StdDev2/Mean2)^2)
http://www.rit.edu/cos/uphysics/uncertainties/Uncertaintiespart2.html
![Page 18: NR 422 Quality Control Jim Graham Spring 2009. Staircase of Knowledge Increasing Subjectivity Human value added Observation And Measurement Data Information](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649e7a5503460f94b7a55b/html5/thumbnails/18.jpg)
Exact numbers
• Adding/Subtracting:– Error does not change
• Multiplying:– Multiply the error by the same number
![Page 19: NR 422 Quality Control Jim Graham Spring 2009. Staircase of Knowledge Increasing Subjectivity Human value added Observation And Measurement Data Information](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649e7a5503460f94b7a55b/html5/thumbnails/19.jpg)
Significant Digits (Figures)
• How many significant digits are in:– 12– 12.00– 12.001– 12000– 0.0001– 0.00012– 123456789
• Only applies to measured values, not exact values (i.e. 2 oranges)
![Page 20: NR 422 Quality Control Jim Graham Spring 2009. Staircase of Knowledge Increasing Subjectivity Human value added Observation And Measurement Data Information](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649e7a5503460f94b7a55b/html5/thumbnails/20.jpg)
Significant Digits
• Cannot create precision:– 1.0 * 2.0 = 2.0– 12 * 11 = 130 (not 131)– 12.0 * 11 = 130 (still not 131)– 12.0 * 11.0 = 131
• Can keep digits for calculations, report with appropriate significant digits
![Page 21: NR 422 Quality Control Jim Graham Spring 2009. Staircase of Knowledge Increasing Subjectivity Human value added Observation And Measurement Data Information](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649e7a5503460f94b7a55b/html5/thumbnails/21.jpg)
Rounding
• If you have 2 significant digits:– 1.11 -> ?– 1.19 -> ?– 1.14 -> ?– 1.16 -> ?– 1.15 -> ?– 1.99 -> ?– 1.155 -> ?
![Page 22: NR 422 Quality Control Jim Graham Spring 2009. Staircase of Knowledge Increasing Subjectivity Human value added Observation And Measurement Data Information](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649e7a5503460f94b7a55b/html5/thumbnails/22.jpg)
Quality Control/Assurance
• Calibrate “Instruments”
• Perform random checks on data
• Watch for “drift”
• Document all errors in Metadata!
![Page 23: NR 422 Quality Control Jim Graham Spring 2009. Staircase of Knowledge Increasing Subjectivity Human value added Observation And Measurement Data Information](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649e7a5503460f94b7a55b/html5/thumbnails/23.jpg)
Design of Sampling
• Random
• Stratified random
• Clustered
• Systematic
• Iterative
![Page 24: NR 422 Quality Control Jim Graham Spring 2009. Staircase of Knowledge Increasing Subjectivity Human value added Observation And Measurement Data Information](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649e7a5503460f94b7a55b/html5/thumbnails/24.jpg)
Number of Samples
• 30?
• Figure 2.7 from Environmental Monitoring and Characterization
![Page 25: NR 422 Quality Control Jim Graham Spring 2009. Staircase of Knowledge Increasing Subjectivity Human value added Observation And Measurement Data Information](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649e7a5503460f94b7a55b/html5/thumbnails/25.jpg)
Statistical Studies
• Is the sampling really random or uniform?– Bias– “Most data is collect near a road, a porta-
poty, and a restaurant!” – Tom Stohlgren
![Page 28: NR 422 Quality Control Jim Graham Spring 2009. Staircase of Knowledge Increasing Subjectivity Human value added Observation And Measurement Data Information](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649e7a5503460f94b7a55b/html5/thumbnails/28.jpg)
Spatial Autocorrelation
• Used to determine type of sampling
![Page 29: NR 422 Quality Control Jim Graham Spring 2009. Staircase of Knowledge Increasing Subjectivity Human value added Observation And Measurement Data Information](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649e7a5503460f94b7a55b/html5/thumbnails/29.jpg)
Rounding
• If you have 2 significant digits:– 1.11 -> 1.1– 1.19 -> 1.2– 1.14 -> 1.1– 1.16 -> 1.2– 1.15 -> 1.1– 1.99 -> 2.0– 1.155 -> 1.5