ontology evolution and regression analysis insights into ontology regression testing maria copeland...
TRANSCRIPT
![Page 1: Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing Maria Copeland Rafael Goncalvez Robert Stevens Bijan Parsia Uli Sattler](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ee45503460f94bf2e82/html5/thumbnails/1.jpg)
Ontology Evolution and Regression Analysis
Insights into Ontology Regression Testing
Maria CopelandRafael GoncalvezRobert Stevens
Bijan ParsiaUli Sattler
![Page 2: Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing Maria Copeland Rafael Goncalvez Robert Stevens Bijan Parsia Uli Sattler](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ee45503460f94bf2e82/html5/thumbnails/2.jpg)
Motivation
Current studies of Ontology Evaluation tend to:
• Focus on individual ontology versions
• Focus on shifts in the gross statistics
In either case we don’t get objective and systematic evaluations of the life span of the ontology
![Page 3: Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing Maria Copeland Rafael Goncalvez Robert Stevens Bijan Parsia Uli Sattler](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ee45503460f94bf2e82/html5/thumbnails/3.jpg)
Our goal is to extract insightful and useful information out of all the existing versions of an ontology
![Page 4: Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing Maria Copeland Rafael Goncalvez Robert Stevens Bijan Parsia Uli Sattler](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ee45503460f94bf2e82/html5/thumbnails/4.jpg)
Ontology Testing Challenge
– How do we systematically identify test areas?
– How do we systematically analyse change impacts to the ontology?
How can we effectively minimise testing efforts and cost
and still achieve adequate testing coverage
![Page 5: Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing Maria Copeland Rafael Goncalvez Robert Stevens Bijan Parsia Uli Sattler](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ee45503460f94bf2e82/html5/thumbnails/5.jpg)
Software Testing
![Page 6: Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing Maria Copeland Rafael Goncalvez Robert Stevens Bijan Parsia Uli Sattler](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ee45503460f94bf2e82/html5/thumbnails/6.jpg)
Software Regression TestingWhat is it?
• It is a test activity to systematicallyre-test existing components after software changes
• It test against current and updated requirements
![Page 7: Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing Maria Copeland Rafael Goncalvez Robert Stevens Bijan Parsia Uli Sattler](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ee45503460f94bf2e82/html5/thumbnails/7.jpg)
Software Regression TestingTesting Aspects
• Testing at the functional requirements– Unit Level– System Level
• Testing at the non-functional requirements
![Page 8: Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing Maria Copeland Rafael Goncalvez Robert Stevens Bijan Parsia Uli Sattler](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ee45503460f94bf2e82/html5/thumbnails/8.jpg)
Software Regression TestingTesting Plan
• Defines testing criteria
• Identifies test area or components
• Test execution strategy
• Test evaluation strategy
• Updates test and other relevant documentation
![Page 9: Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing Maria Copeland Rafael Goncalvez Robert Stevens Bijan Parsia Uli Sattler](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ee45503460f94bf2e82/html5/thumbnails/9.jpg)
Software Regression TestingProcess
Change DetectionChange Detection
Impact AnalysisImpact Analysis
Define Test Plan
Define Test Plan
Build Test Suite
Build Test Suite
Run TestRun Test
Evaluate ResultsEvaluate Results
Report ResultsReport Results
![Page 10: Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing Maria Copeland Rafael Goncalvez Robert Stevens Bijan Parsia Uli Sattler](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ee45503460f94bf2e82/html5/thumbnails/10.jpg)
Ontology Regression Testing?
![Page 11: Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing Maria Copeland Rafael Goncalvez Robert Stevens Bijan Parsia Uli Sattler](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ee45503460f94bf2e82/html5/thumbnails/11.jpg)
Change Detection
• Explicit Changes– Asserted logical and annotation axioms– Properties– Classes
• Implicit Changes?– Subsumption changes– Entailment changes
![Page 12: Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing Maria Copeland Rafael Goncalvez Robert Stevens Bijan Parsia Uli Sattler](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ee45503460f94bf2e82/html5/thumbnails/12.jpg)
Impact Analysis
• Previous Version or all versions?– Intentional Difference analysis?– Justifications analysis?
• Information Content?– Asserted content?– Entailed content?
• Requirements Impact?– Functional and Non-functional?– Which ones do we test?
![Page 13: Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing Maria Copeland Rafael Goncalvez Robert Stevens Bijan Parsia Uli Sattler](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ee45503460f94bf2e82/html5/thumbnails/13.jpg)
Define Test Plan
• Test criteria?
• Test area? Do we have test areas?
• How can the test be systematically run?
• Can results be interpreted?
![Page 14: Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing Maria Copeland Rafael Goncalvez Robert Stevens Bijan Parsia Uli Sattler](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ee45503460f94bf2e82/html5/thumbnails/14.jpg)
Can we systematically:
• Build test suites?
• Run tests?
• Evaluate results?
• Re-run tests if necessary?
![Page 15: Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing Maria Copeland Rafael Goncalvez Robert Stevens Bijan Parsia Uli Sattler](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ee45503460f94bf2e82/html5/thumbnails/15.jpg)
Manual vs. Automated Test Suites
• Manually test cases– Check against a methodology– Eyeballing
• Automated test cases– Satisfiability– Inconsistency
![Page 16: Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing Maria Copeland Rafael Goncalvez Robert Stevens Bijan Parsia Uli Sattler](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ee45503460f94bf2e82/html5/thumbnails/16.jpg)
Manual vs. Automated Test Suites
• Manually test cases– Time consuming– Subjective– Unsystematic
• Automated test cases– Reasoner based– Limited in scope
![Page 17: Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing Maria Copeland Rafael Goncalvez Robert Stevens Bijan Parsia Uli Sattler](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ee45503460f94bf2e82/html5/thumbnails/17.jpg)
Can we expand the range of automatic test suites?
![Page 18: Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing Maria Copeland Rafael Goncalvez Robert Stevens Bijan Parsia Uli Sattler](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ee45503460f94bf2e82/html5/thumbnails/18.jpg)
YES
By Analysing Ontology Dynamics
![Page 19: Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing Maria Copeland Rafael Goncalvez Robert Stevens Bijan Parsia Uli Sattler](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ee45503460f94bf2e82/html5/thumbnails/19.jpg)
What are Ontology Dynamics?
• Periods of growth, decline, and stability
• Axioms presence
• Types of axioms presence (e.g. continual, interrupted)
• Sequence editing types and patterns
![Page 20: Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing Maria Copeland Rafael Goncalvez Robert Stevens Bijan Parsia Uli Sattler](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ee45503460f94bf2e82/html5/thumbnails/20.jpg)
Axiom Life Span - We expect: Axioms with Constant Unchanged Presence
Ontology Versions
O1 Oi
αi
![Page 21: Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing Maria Copeland Rafael Goncalvez Robert Stevens Bijan Parsia Uli Sattler](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ee45503460f94bf2e82/html5/thumbnails/21.jpg)
Axiom Life Span - We expect: Axioms that are Modified
Ontology Versions
Split or Merge
O1
αi
Oi Oi+n
![Page 22: Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing Maria Copeland Rafael Goncalvez Robert Stevens Bijan Parsia Uli Sattler](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ee45503460f94bf2e82/html5/thumbnails/22.jpg)
Axiom Life Span - We expect: Axioms that Enter and Leave the Ontology
Ontology Versions
O1
αi
Oi Oi+n
![Page 23: Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing Maria Copeland Rafael Goncalvez Robert Stevens Bijan Parsia Uli Sattler](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ee45503460f94bf2e82/html5/thumbnails/23.jpg)
NCIt Ontology Dynamics
![Page 24: Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing Maria Copeland Rafael Goncalvez Robert Stevens Bijan Parsia Uli Sattler](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ee45503460f94bf2e82/html5/thumbnails/24.jpg)
National Cancer Institute Thesaurus (NCIt)
• The National Cancer Institute (NCI) is a U.S. government funded organisation for the research of causes, treatment, and prevention of cancer
• The NCIt is an ontology written in the Web Ontology Language (OWL) which supports the development and maintenance of a controlled vocabulary about cancer research
• Multiple publications about process, quality control, usage, and critiques
• Publicly available monthly releases and concept change logs
Rich source of ontology evolution data
![Page 25: Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing Maria Copeland Rafael Goncalvez Robert Stevens Bijan Parsia Uli Sattler](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ee45503460f94bf2e82/html5/thumbnails/25.jpg)
NCIt Dynamics – Axioms Life Span Analysis
![Page 26: Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing Maria Copeland Rafael Goncalvez Robert Stevens Bijan Parsia Uli Sattler](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ee45503460f94bf2e82/html5/thumbnails/26.jpg)
NCIt Change Dynamics – Detailed View of Axiom Life Span
Top Ten Frequency Distributions
![Page 27: Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing Maria Copeland Rafael Goncalvez Robert Stevens Bijan Parsia Uli Sattler](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ee45503460f94bf2e82/html5/thumbnails/27.jpg)
NCIt Axiom Life Span: Axioms with Constant Unchanged Presence
NCIt 2003 – 2012 103 Versions
O1
αi
O103
6.73%
![Page 28: Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing Maria Copeland Rafael Goncalvez Robert Stevens Bijan Parsia Uli Sattler](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ee45503460f94bf2e82/html5/thumbnails/28.jpg)
NCIt Axiom Life Span: Axioms that Enter and Leave the Ontology
NCIt 2003 – 2012 103 Versions
O1
αi
Oi Oi+n
![Page 29: Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing Maria Copeland Rafael Goncalvez Robert Stevens Bijan Parsia Uli Sattler](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ee45503460f94bf2e82/html5/thumbnails/29.jpg)
NCIt Axiom Life Span: Axioms with Gaps between Unchanged Presence
O1 Oi Oi+n
NCIt 2003 – 2012 103 Versions
Effectually Added
αi
Effectually Removed
O I≠ αi
Effectually Added
αi
Effectually Removed
O I≠ αi
Ok Ok+n
![Page 30: Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing Maria Copeland Rafael Goncalvez Robert Stevens Bijan Parsia Uli Sattler](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ee45503460f94bf2e82/html5/thumbnails/30.jpg)
NCIt Axiom Life Span: Axioms with Gaps between Unchanged Presence
O1 Oi Oi+n
NCIt 2003 – 2012 103 Versions
Effectually Added
αi
Ineffectually Removed
O I= αi
Effectually Removed
αi
Effectually Added
Ok Ok+n
O I≠ αi
![Page 31: Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing Maria Copeland Rafael Goncalvez Robert Stevens Bijan Parsia Uli Sattler](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ee45503460f94bf2e82/html5/thumbnails/31.jpg)
NCIt Axiom Life Span: Axioms with Gaps between Unchanged Presence
O1 Oi Oi+n
NCIt 2003 – 2012 103 Versions
Ineffectually Added
αi
IneffectuallyRemoved
O I= αi
Ineffectually Added
αi
Ineffectual Removed
Ok Ok+n
O I= αi
![Page 32: Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing Maria Copeland Rafael Goncalvez Robert Stevens Bijan Parsia Uli Sattler](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ee45503460f94bf2e82/html5/thumbnails/32.jpg)
NCIt Dynamics – Editing Patterns
![Page 33: Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing Maria Copeland Rafael Goncalvez Robert Stevens Bijan Parsia Uli Sattler](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ee45503460f94bf2e82/html5/thumbnails/33.jpg)
NCIt Regression Analysis – Main Finding
This means that we are able to identify ‘bugs’, the sequence pattern of these bugs, and their location!
![Page 34: Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing Maria Copeland Rafael Goncalvez Robert Stevens Bijan Parsia Uli Sattler](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ee45503460f94bf2e82/html5/thumbnails/34.jpg)
NCIt Regression Analysis: Indicative of Faults In Sequence of Changes
O1 Oi Oi+n
NCIt 2003 – 2012 103 Versions
Effectually Added
αi
Effectually Removed
O I≠ αi
Effectually Added
αi
Effectually Removed
O I≠ αi
Ok Ok+n
O1 Oi Oi+n
NCIt 2003 – 2012 103 Versions
Effectually Added
αi
Ineffectually Removed
O I= αi
Effectually Removed
αi
Effectually Added
Ok Ok+n
✗✓?
✗ ? ?✓
?
![Page 35: Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing Maria Copeland Rafael Goncalvez Robert Stevens Bijan Parsia Uli Sattler](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ee45503460f94bf2e82/html5/thumbnails/35.jpg)
NCIt Regression Analysis: Suggestive of Faults In Sequence of Changes
O1
NCIt 2003 – 2012 103 Versions
O I= αi
Effectually Added
αi
Ineffectually Removed
Ok Ok+n
Refactoring
![Page 36: Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing Maria Copeland Rafael Goncalvez Robert Stevens Bijan Parsia Uli Sattler](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ee45503460f94bf2e82/html5/thumbnails/36.jpg)
NCIt Regression Analysis: Suggestive of Faults In Sequence of Changes
O1
NCIt 2003 – 2012 103 Versions
O I= αi
IneffectuallyAddedαi
Ineffectually Removed
Ok Ok+n
O1
NCIt 2003 – 2012 103 Versions
O I= αi
Effectually Added
αi
Ineffectually Removed
Oi Oi+n
IneffectuallyAdded
Ok
?
![Page 37: Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing Maria Copeland Rafael Goncalvez Robert Stevens Bijan Parsia Uli Sattler](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ee45503460f94bf2e82/html5/thumbnails/37.jpg)
From Change Dynamics to Ontology Regression Testing
![Page 38: Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing Maria Copeland Rafael Goncalvez Robert Stevens Bijan Parsia Uli Sattler](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ee45503460f94bf2e82/html5/thumbnails/38.jpg)
Systematically Build Test Suites
Indicative of Faults In Sequence of Changes
![Page 39: Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing Maria Copeland Rafael Goncalvez Robert Stevens Bijan Parsia Uli Sattler](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ee45503460f94bf2e82/html5/thumbnails/39.jpg)
Automated Test Suites - Fault Detection
• It provides systematic regression test for all version of the ontology
• It conclusively identifies content regression and content refactoring
• It suggests other faults based on regression sequence
It is efficient and cheap to run
![Page 40: Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing Maria Copeland Rafael Goncalvez Robert Stevens Bijan Parsia Uli Sattler](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ee45503460f94bf2e82/html5/thumbnails/40.jpg)
And there are still more potential benefits …
![Page 41: Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing Maria Copeland Rafael Goncalvez Robert Stevens Bijan Parsia Uli Sattler](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ee45503460f94bf2e82/html5/thumbnails/41.jpg)
Automated Test Suites - Fault Detection
• Entailment Set Studies
• Sub Domain Dynamics
• Ontology Classes Dynamics
…
![Page 42: Ontology Evolution and Regression Analysis Insights into Ontology Regression Testing Maria Copeland Rafael Goncalvez Robert Stevens Bijan Parsia Uli Sattler](https://reader030.vdocuments.mx/reader030/viewer/2022032414/56649ee45503460f94bf2e82/html5/thumbnails/42.jpg)
Thanks