integrity overview todd walter stanford university bruce decleene faa todd walter stanford...

32
Integrity Overview Todd Walter Stanford University Bruce DeCleene FAA http://waas.stanford.edu

Upload: natalie-cunningham

Post on 16-Jan-2016

216 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: Integrity Overview Todd Walter Stanford University Bruce DeCleene FAA  Todd Walter Stanford University Bruce DeCleene FAA

Integrity Overview

Todd Walter

Stanford University

Bruce DeCleene

FAA

http://waas.stanford.edu

Todd Walter

Stanford University

Bruce DeCleene

FAA

http://waas.stanford.edu

Page 2: Integrity Overview Todd Walter Stanford University Bruce DeCleene FAA  Todd Walter Stanford University Bruce DeCleene FAA

2

Purpose

Provide a description of the process the FAA used to approve the design of the safety monitors in WAAS

Page 3: Integrity Overview Todd Walter Stanford University Bruce DeCleene FAA  Todd Walter Stanford University Bruce DeCleene FAA

3

Overall Philosophy

Traditional Differential GPS Systems Rely on Lack of DisproofAnecdotal evidence of no problems

10-7 Integrity Requires Active ProofAnalysis, Simulation, and Data Must

Each Support Each OtherNone sufficient by themselves

Clear Documentation of Safety Rationale is Essential

Page 4: Integrity Overview Todd Walter Stanford University Bruce DeCleene FAA  Todd Walter Stanford University Bruce DeCleene FAA

4

Integrity Triad

Data cannot prove 10-7

Theory may miss “real-world” effectsSimulation only tests specific scenarios

All 3 together!Data supports theoryTheory extends dataSimulation validates implementation

Page 5: Integrity Overview Todd Walter Stanford University Bruce DeCleene FAA  Todd Walter Stanford University Bruce DeCleene FAA

5

Lessons Learned through WIPP Process

Integrity Requirement of 10-7 Applies to Each and Every Approach

Threat Models Required to Judge System Performance and Safety

System Must Be Proven SafeRationale/evidence for safety claim

Small Probabilities Are Not IntuitiveProbabilities should be calculated before

being dismissed as sufficiently remote

Page 6: Integrity Overview Todd Walter Stanford University Bruce DeCleene FAA  Todd Walter Stanford University Bruce DeCleene FAA

6

Interpretation of “Probability of HMI < 10-7 Per Approach”Possible Interpretations

Ensemble Average of All Approaches

Over Space and TimeEnsemble Average of All Approaches

Over Time for the Worst LocationPrevious Plus No Discernable Pattern

(Rare & No Correlation With User Behavior)

Worst Time and Location

Page 7: Integrity Overview Todd Walter Stanford University Bruce DeCleene FAA  Todd Walter Stanford University Bruce DeCleene FAA

7

HMI Vs. Time & Space

Page 8: Integrity Overview Todd Walter Stanford University Bruce DeCleene FAA  Todd Walter Stanford University Bruce DeCleene FAA

8

WIPP Interpretation

Events Handled Case by CaseEvents That Are Rare and Random

May Take Advantage of an A PrioriDeterministic Events Must Be

Monitored or Treated as Worst-CaseEvents That Are Observable Must Be

Detected (If PHMI > 10-7)Must Account for Worst-Case

Undetected Events

Page 9: Integrity Overview Todd Walter Stanford University Bruce DeCleene FAA  Todd Walter Stanford University Bruce DeCleene FAA

9

Ionospheric Example

Ionospheric Storms Will OccurOnset of Storm Is Rare and Random

A priori used in storm detector

Ionospheric Storm is ObservableMust have a monitorBefore detector trips, assume

ionosphere is in a near storm stateAfter detector trips, assume worst case

ionospheric state

Page 10: Integrity Overview Todd Walter Stanford University Bruce DeCleene FAA  Todd Walter Stanford University Bruce DeCleene FAA

10

Threat Models

Not Fully Defined for Prior to WIPPLimit Extent of Threats

Provide description and likelihood

Basis for Judging CompletenessMonitors must be shown to mitigate full

threat space to sufficient probability

Models Must Be ComprehensiveCollectively full set must span all feared

events

Page 11: Integrity Overview Todd Walter Stanford University Bruce DeCleene FAA  Todd Walter Stanford University Bruce DeCleene FAA

11

SBAS Threats

Satellite Clock/Ephemeris Error Signal Deformation

Nominal Faulted

Code Carrier Incoherency

Ionosphere Local Non-Planar Behavior

Well-sampled Undersampled

Troposphere Receiver

Multipath Thermal Noise Antenna Bias Survey Errors Receiver Errors

Master Station SV Clock/Ephemeris Estimate

Errors Ionospheric Estimation Errors SV Tgd Estimate Errors Receiver IFB Estimate Errors WRS Clock Estimate Errors Communication Errors Broadcast Errors

User Errors

List is not comprehensive

Page 12: Integrity Overview Todd Walter Stanford University Bruce DeCleene FAA  Todd Walter Stanford University Bruce DeCleene FAA

12

CONUS Ionosphere Threat

Not Well-Modeled by Local Planar FitIonosphere well-sampled1

Ionosphere poorly sampled2

Ionosphere Changes Over the Lifetime of the Correction

User Interpolation Introduces Error

1. “Robust Detection of Ionospheric Irregularities,” Walter et al. ION GPS 2000

2. “The WAAS Ionospheric Threat Model,” Sparks et al. Beacon Symposium 2001

Page 13: Integrity Overview Todd Walter Stanford University Bruce DeCleene FAA  Todd Walter Stanford University Bruce DeCleene FAA

13

Rationale/Evidence for Safety

System must be proven safeEach threat requires a mitigation

Must be agreed to by the entire groupMust be fully and clearly documented for

future reference and modification

Small probabilities must be calculatedProduct of a priori and missed detection rate

must meet allocation

Sum of all threats must meet total 10-7 per approach requirement

Page 14: Integrity Overview Todd Walter Stanford University Bruce DeCleene FAA  Todd Walter Stanford University Bruce DeCleene FAA

14

Fault Tree

PHMI is calculated using a fault treeTwo main algorithm branches

Single fault dominatesMultiple faults convolve together

Cases are mutually exclusiveSingle fault worries about the tails of

the distributionMultiple fault worries about the meansSeparate monitors and analyses

Page 15: Integrity Overview Todd Walter Stanford University Bruce DeCleene FAA  Todd Walter Stanford University Bruce DeCleene FAA

15

Data Analysis

Large amount of data is essential for verifying performanceUnder many different conditions

Data must be sliced to look for systematic errorsSlices partition data into logical subsetsMultipath data may be sliced by elevation

angleIono data may me sliced by time of day

Errors are not stationary

Page 16: Integrity Overview Todd Walter Stanford University Bruce DeCleene FAA  Todd Walter Stanford University Bruce DeCleene FAA

16

Overbound

True distributions are messyRequire an analytically tractable

distribution (gaussians are good)Analytic distribution must predict

probability of large errors at least as great (or greater than) true distribution

Page 17: Integrity Overview Todd Walter Stanford University Bruce DeCleene FAA  Todd Walter Stanford University Bruce DeCleene FAA

17

Gaussian Tail Behavior

Zero-Mean, Unit-Variance Gaussian

Probability that |X| > K is shown in plot

Probability of Error Falls Off Rapidly As Mag-nitude Increases

Page 18: Integrity Overview Todd Walter Stanford University Bruce DeCleene FAA  Todd Walter Stanford University Bruce DeCleene FAA

18

Overbounding Tools

CDF Bounding - (DeCleene 2000) Real distributions must be zero-mean, symmetric, and unimodal

Gaussian Bounding - (Raytheon 2002) Allows for non-zero means, but still requires symmetry and

unimodality

Paired Bounding - (Rife 2004) Allows asymmetry, multiple modes, and non-zero means

Excess-Mass Bounding - (Rife 2004) Similar to paired bounding

Core Bounding - (Rife 2004) Advantages as above, but allows for uncertain tails

Moment Bounding - (Raytheon 2002) Unlike others, operates in moment domain Allows asymmetry, multiple modes, and non-zero means Less intuitive, but works well with real data

Page 19: Integrity Overview Todd Walter Stanford University Bruce DeCleene FAA  Todd Walter Stanford University Bruce DeCleene FAA

19

Overbounding Tools

Single CDF Paired CDF Moment Bounding

Excess-Mass CDF Excess-Mass PDF

-20 -10 0 10 206.2e-16

9.9e-10

3.2e-5

2.3e-2

5.0e-1

9.8e-1

1.0e+0

1.0e+0

1.0e+0GC Bound (M=4)

x/σ

CDF

NIG GC

-20 -10 0 10 206.2 -16e

9.9 -10e

3.2 -5e

2.3 -2e

5.0 -1e

9.8 -1e

1.0 +0e

1.0 +0e

1.0 +0e

Core Bounding

Page 20: Integrity Overview Todd Walter Stanford University Bruce DeCleene FAA  Todd Walter Stanford University Bruce DeCleene FAA

20

Overbound Validation

Key characteristic: All of these techniques enable overbound

validationIf overbound conservatively represents

correction domain error, it will conservatively represent derived errors (i.e UDREs, GIVEs, Protection Levels)

Overbound applicationDifferent techniques have different

strengths and weaknessesNeed to find method that yields best

performance

Page 21: Integrity Overview Todd Walter Stanford University Bruce DeCleene FAA  Todd Walter Stanford University Bruce DeCleene FAA

21

Mixing and Clipping

System Is Not StationaryCertain events lead to larger errorsIf not recognized, sampled distribution mixes

errors with different propertiesObserved tails worse than Gaussian

Reasonability Checks and Redundancy Remove Large ErrorsBoth faulted and large fault-free errorsObserved tails better than Gaussian

Mixing Dominates Over Clipping

Page 22: Integrity Overview Todd Walter Stanford University Bruce DeCleene FAA  Todd Walter Stanford University Bruce DeCleene FAA

22

Mixed Gaussian Distribution

Page 23: Integrity Overview Todd Walter Stanford University Bruce DeCleene FAA  Todd Walter Stanford University Bruce DeCleene FAA

23

Monitor Observability

Noise on the measurements limits the ability to detect errors

Only errors above a certain limit can be detected with a certain probability

Smaller errors are assumed presentGIVEs and UDREs must increase as

system noise increasesLarger errors may escape detection

Page 24: Integrity Overview Todd Walter Stanford University Bruce DeCleene FAA  Todd Walter Stanford University Bruce DeCleene FAA

24

Errors Below the Detection Threshold

Page 25: Integrity Overview Todd Walter Stanford University Bruce DeCleene FAA  Todd Walter Stanford University Bruce DeCleene FAA

25

Failure Rates and Monitors

Page 26: Integrity Overview Todd Walter Stanford University Bruce DeCleene FAA  Todd Walter Stanford University Bruce DeCleene FAA

26

Range Domain Vs. Position Domain

Position Domain is IntuitiveCommon errors fall in clockOnly tests specific geometry

Range/Correction Domain Generally More ConservativeProtects each individual correctionApplicable to all geometriesSmall correlated errors may cause HMI

Approaches Complement Each Other

Page 27: Integrity Overview Todd Walter Stanford University Bruce DeCleene FAA  Todd Walter Stanford University Bruce DeCleene FAA

27

Example User Geometry

WAAS user near Florida

8 satellites in view

25 m bias on PRN 6 creates a 4 m error for all-in-view, and a 50 m error without PRN 8

Weak vertical dependenceon PRN 6 for all-in-view

Strong dependence if PRN 8 is missing

Page 28: Integrity Overview Todd Walter Stanford University Bruce DeCleene FAA  Todd Walter Stanford University Bruce DeCleene FAA

28

Vertical Performance

Each User Has a Single Biased SVChange in Geometry Exposes Error

Page 29: Integrity Overview Todd Walter Stanford University Bruce DeCleene FAA  Todd Walter Stanford University Bruce DeCleene FAA

29

Conclusions

Probability of HMI Applies to Worst Predictable Condition

Data, Theory, and Simulation must Support each other

Threat Models Essential for Validating Implementation

Errors Below Detection Threshold Must Be Treated Conservatively

Position and Range Domain Needed to Protect All Geometries

Page 30: Integrity Overview Todd Walter Stanford University Bruce DeCleene FAA  Todd Walter Stanford University Bruce DeCleene FAA

30

Resources (1 of 3) Cabler, H. and DeCleene, B., “LPV: New, Improved WAAS Instrument Approach,” in Proceedings of the

ION GPS meeting, Portland, OR, 2002.

Walter, T., “WAAS MOPS: Practical Examples,” in Proceedings of the National Technical Meeting of the

Institute of Navigation, San Diego, January 1999.

Walter, T., Enge, P., and DeCleene, B., “Integrity Lessons from the WIPP,” in Proceedings of the National

Technical Meeting of the Institute of Navigation, Anaheim, January 2003.

Shallberg, K., Shloss, P., Altshuler, E., and Tahmazyan,. L., “WAAS Measurement Processing, Reducing

the Effects of Multipath,” in Proceedings of the ION GPS meeting, Salt Lake City, UT, 2001.

Hansen, A., Blanch, J., Walter, T., and Enge, P., “Ionospheric Correlation Analysis for WAAS: Quiet and

Stormy,” in Proceedings of the ION GPS meeting, Salt Lake City, UT, 2000.

Rajagopal, S., Walter, T., Datta-Barua, S., Blanch, J., and Sakai, T., “Correlation Structure of the Equatorial

Ionosphere,” Proceedings of the National Technical Meeting of the Institute of Navigation, San Diego,

January 2004.

Walter, T., Hansen, A., Blanch, J., Enge, P., Mannucci, T., Pi, X., Sparks, L., Iijima, B., El-Arini, B., Lejeune,

R., Hagen, M., Altshuler, E., Fries, R., and Chu, A., Robust Detection of Ionospheric Irregularities,

Navigation: Journal of The Institute of Navigation, Vol. 48, No. 2, Summer 2001.

Blanch, J., Using Kriging to bound Satellite Ranging Errors due to the Ionosphere, Stanford University

Thesis, December, 2003, available through http://waas.stanford.edu/pubs/phd_pubs.html.

Sparks, L., Pi, X., Mannucci, A.J., Walter, T. Blanch, J., Hansen, A., Enge, P., Altshuler, E., and Fries, R.,

“The WAAS Ionospheric Threat Model,” in Proceedings of the Beacon Satellite Symposium, Boston, MA,

June 2001.

Page 31: Integrity Overview Todd Walter Stanford University Bruce DeCleene FAA  Todd Walter Stanford University Bruce DeCleene FAA

31

Resources (2 of 3)

Datta-Barua, S., “Ionospheric Threats to Space-Based Augmentation System Development,” in

Proceedings of the ION GPS meeting, Long Beach, CA, 2004.

Walter, T., Rajagopal, S., Datta-Barua, S., and Blanch, J., “Protecting Against Unsampled Ionospheric

Threats,” in Proceedings of the Beacon Satellite Symposium, Trieste, Italy, October 2004.

Wu, T. and Peck, S., “An Analysis of Satellite Integrity Monitoring Improvement for WAAS,” in Proceedings

of the ION GPS meeting, Portland, OR, 2002.

Walter, T. Hansen, A., and Enge, P., “Message Type 28,” in Proceedings of the National Technical

Meeting of the Institute of Navigation, Long Beach, CA, January 2001.

FAA/William J. Hughes Technical Center, “Wide-Area Augmentation System Performance Analysis

Reports, available at http://www.nstb.tc.faa.gov/.

Jan, S. S., Chan, W., Walter, T., and Enge, P., “Matlab Simulation Toolset for SBAS Availability Analysis,”

in Proceedings of the ION GPS meeting, Salt Lake City, UT, 2001.

Walter, T. and Enge, P., “Modernizing WAAS,” in Proceedings of the ION GPS meeting, Long Beach, CA,

2004.

DeCleene, B., “Defining Psuedorange Integrity - Overbounding,” in Proceedings of the ION GPS meeting,

Salt Lake City, UT, 2000.

Schempp, T. R. and Rubin, A. L., “An Application of Gaussian Bounding for the WAAS Fault-Free Error

Analysis,” n Proceedings of the ION GPS meeting, Portland, OR, 2002. Rife J, Pullen S, Pervan B, and Enge P., “Paired overbounding and application to GPS

augmentation,” Proceedings IEEE Position, Location and Navigation Symposium, Monterey, California, April 2004.

Page 32: Integrity Overview Todd Walter Stanford University Bruce DeCleene FAA  Todd Walter Stanford University Bruce DeCleene FAA

32

Resources (3 of 3) Rife J, Pullen S, Pervan B, and Enge P., “Core overbounding and its implications for LAAS

integrity,” Proceedings of ION GNSS 2004, Long Beach, California, September, 2004.

Rife J, Walter T, and Blanch J, “Overbounding SBAS and GBAS error distributions with excess-mass functions,” Proceedings of the 2004 International Symposium on GPS/GNSS, Sydney, Australia, 6-8 December, 2004.

Walter, T., Blanch, J., and Rife, J., “Treatment of Biased error distributions in SBAS,” Proceedings of the 2004 International Symposium on GPS/GNSS, Sydney, Australia, 6-8 December, 2004.

Most Stanford Publications available at: http://waas.stanford.edu/pubs/index.htm

Email: Todd Walter: [email protected]

Jason Rife: [email protected]

Juan Blanch: [email protected]