modeling issues for validation, verification, and certification (vv&c) paul miner nasa langley...
TRANSCRIPT
![Page 1: Modeling Issues for Validation, Verification, and Certification (VV&C) Paul Miner NASA Langley Research Center p.s.miner@nasa.gov 22 September 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062519/5697bff61a28abf838cbdff2/html5/thumbnails/1.jpg)
Modeling Issues for Validation, Verification, and Certification
(VV&C)
Paul MinerNASA Langley Research Center
22 September 2015
![Page 2: Modeling Issues for Validation, Verification, and Certification (VV&C) Paul Miner NASA Langley Research Center p.s.miner@nasa.gov 22 September 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062519/5697bff61a28abf838cbdff2/html5/thumbnails/2.jpg)
• “Essentially, all models are wrong, but some are useful”– George Box
22 September 2015 Modeling for V&V 2
![Page 3: Modeling Issues for Validation, Verification, and Certification (VV&C) Paul Miner NASA Langley Research Center p.s.miner@nasa.gov 22 September 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062519/5697bff61a28abf838cbdff2/html5/thumbnails/3.jpg)
Example of Useful Models
• Canonical models for designing and analyzing digital hardware introduced in the mid-1950s
– Huffman, D.A., The synthesis of sequential switching circuits, The Journal of the Franklin Institute, 257(3):161-190, 1954
– Mealy, G.H., A method for synthesizing sequential circuits, Bell System Technical Journal, 34:1045-1079, September 1955
– Moore, E.F. Gedanken Experiments on Sequential Machines, in C. Shannon and J. McCarthy, editors, Automata Studies, Princeton University Press, 1956
• These modeling abstractions underpin the digital revolution– But, ..
• “There is no such thing as digital circuitry. There is only analog circuitry driven to extremes.”
– Unknown – quoted by Kevin Driscoll• https://c3.nasa.gov/dashlink/static/media/other/ObservedFailures6.html
• For VV&C, need to consider impact when modeling abstractions no longer hold
22 September 2015 Modeling for V&V 3
![Page 4: Modeling Issues for Validation, Verification, and Certification (VV&C) Paul Miner NASA Langley Research Center p.s.miner@nasa.gov 22 September 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062519/5697bff61a28abf838cbdff2/html5/thumbnails/4.jpg)
Role of Models in VV& C
Benefits• Explore system behavior earlier
in lifecycle• Ability to verify properties that
cannot be effectively demonstrated by test– E.g. Robust partitioning for
Integrated Modular Avionics– No memory leaks, buffer
overflows, etc.
• …
Risks• Invalid assumptions• Unstated assumptions• Tendency to conflate model with
reality• Maintaining consistency between
multiple models (with different underlying abstractions)
• Incompatibility between models– Especially design models vs.
failure models• …
22 September 2015 Modeling for V&V 4
![Page 5: Modeling Issues for Validation, Verification, and Certification (VV&C) Paul Miner NASA Langley Research Center p.s.miner@nasa.gov 22 September 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062519/5697bff61a28abf838cbdff2/html5/thumbnails/5.jpg)
Models for Design vs. VV&C
Design• Focus on functional correctness,
desired properties, and performance
• Emphasis on average case behavior (e.g., for performance)
• Intended interactions between components & environment– Presumption that the only
interaction is through defined interfaces
VV&C• Focus on non-functional
requirements – Safety, Security, etc.
• Emphasis on worst-case behavior• Preclude adverse interaction
between components & environment
– In addition to failure propagation through defined interfaces, must also consider “out-of-band” failure modes
22 September 2015 Modeling for V&V 5
![Page 6: Modeling Issues for Validation, Verification, and Certification (VV&C) Paul Miner NASA Langley Research Center p.s.miner@nasa.gov 22 September 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062519/5697bff61a28abf838cbdff2/html5/thumbnails/6.jpg)
Example “out-of-band” failure mode
https://xkcd.com/538/
22 September 2015
Modeling for V&V 6
![Page 7: Modeling Issues for Validation, Verification, and Certification (VV&C) Paul Miner NASA Langley Research Center p.s.miner@nasa.gov 22 September 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062519/5697bff61a28abf838cbdff2/html5/thumbnails/7.jpg)
Questions?
Downloaded from http://xkcd.com/246/
22 September 2015 Modeling for V&V 7
![Page 8: Modeling Issues for Validation, Verification, and Certification (VV&C) Paul Miner NASA Langley Research Center p.s.miner@nasa.gov 22 September 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062519/5697bff61a28abf838cbdff2/html5/thumbnails/8.jpg)
Backup Slides
22 September 2015 8Modeling for V&V
![Page 9: Modeling Issues for Validation, Verification, and Certification (VV&C) Paul Miner NASA Langley Research Center p.s.miner@nasa.gov 22 September 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062519/5697bff61a28abf838cbdff2/html5/thumbnails/9.jpg)
9
• Assumed importance order- Assumed/known fault hypothesis violated
exhaustion of resources (known fault hypothesis)
- Single point of failure unknown fault hypothesis forgotten failure modeunderestimated probability of occurrence
- Fault propagation = domino effect (fault containment)
• Real occurrence frequency order- Chain or domino effect (missing fault containment)
E.g. TTP membership; shown to be a fault propagation path [Ademaj, Sivencrona]
- Single point of failure (unknown fault hypothesis) E.g. quad-redundant control system (termination of bus)[ 2003]
- Exhaustion of resources (known fault hypothesis)
"How Systems Fail"
![Page 10: Modeling Issues for Validation, Verification, and Certification (VV&C) Paul Miner NASA Langley Research Center p.s.miner@nasa.gov 22 September 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062519/5697bff61a28abf838cbdff2/html5/thumbnails/10.jpg)
An assumption will remain valid only until you come to depend on it*.
22 September 2015 Modeling for V&V 10
* http://www.ece.mtu.edu/faculty/rmkieckh/Kieckhafer-top-ten.htm (version 9.1; law 4.2)