![Page 1: 11:00 Self-Introductions 11:15 Report on ontology-based data integration work in DCGS-A --- Goals and methodology --- Practical experience and results](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56649e905503460f94b9422e/html5/thumbnails/1.jpg)
11:00 Self-Introductions11:15 Report on ontology-based data integration work in DCGS-A--- Goals and methodology--- Practical experience and results so far--- Risks and problems12:15 Lunch (Brown Bag Lunch?)13:00 Discussion of NSA work and goals (to be expanded)--- ontology alignment / management--- OMaaS (Object Management as a Service)14:00-15:30 Discussion of how collaborative ontology effort (e.g. between I2WD and NSA) will work in practice--- how to ensure consistency (architectural and content)--- how to address governance--- business development and work flow
![Page 2: 11:00 Self-Introductions 11:15 Report on ontology-based data integration work in DCGS-A --- Goals and methodology --- Practical experience and results](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56649e905503460f94b9422e/html5/thumbnails/2.jpg)
DSC Ontology Work
1. Goals and methodology2. Practical experience and results so far3. Challenges
02/05/13
![Page 3: 11:00 Self-Introductions 11:15 Report on ontology-based data integration work in DCGS-A --- Goals and methodology --- Practical experience and results](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56649e905503460f94b9422e/html5/thumbnails/3.jpg)
Goal: To realize Horizontal Integration(HI) of intelligence data
HI =Def. the ability to exploit multiple data sources as if they are one Problem: the data coming onstream are out of our
control Any strategy for HI must be agile in the sense that
it can be quickly extended to new zones of emerging data according to need
Ontology can provide the needed agility and (incremental approach to) comprehensiveness
![Page 4: 11:00 Self-Introductions 11:15 Report on ontology-based data integration work in DCGS-A --- Goals and methodology --- Practical experience and results](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56649e905503460f94b9422e/html5/thumbnails/4.jpg)
The Business Case
• Huge resources are wasted as multiple different agencies create lexicons, glossaries, data models, messaging and exchange standards with the same or closely overlapping coverage.
• Additional resources are wasted in creating mappings between these artifacts, and in maintaining them in light of new needs and challenges. These mappings always fail.
• A sensible solution must incorporate an evolutionary process which will ensure that artifacts used to manage data converge over time.
![Page 5: 11:00 Self-Introductions 11:15 Report on ontology-based data integration work in DCGS-A --- Goals and methodology --- Practical experience and results](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56649e905503460f94b9422e/html5/thumbnails/5.jpg)
Methodology: Create an agile strategy for building ontologies within a Shared Semantic Resource (SSR) and apply and extend these ontologies
to annotate new source data as they come onstream
⁻ Strategy pioneered in biomedical and other scientific fields: leaves data as they are, and incrementally tags data sources with terms from a growing, consistent, non-redundant set of ontologies
⁻ Problem: Given the immense and growing variety of data sources, the development methodology must be applied by multiple different groups
⁻ How to manage collaboration? This is what this meeting is about
![Page 6: 11:00 Self-Introductions 11:15 Report on ontology-based data integration work in DCGS-A --- Goals and methodology --- Practical experience and results](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56649e905503460f94b9422e/html5/thumbnails/6.jpg)
Current State2010-2011
– Architectural implementation (DRIF) to create the Dataspace (a cloud of intelligence data) = lossless representation of sources with their native semantics
– Initiated Semantic Enhancement (SE) = using ontologies to annotate these native semantics
– Demonstration of use of SE to index and query the content of the Dataspace
2012– Created methodology and architecture for ontology development– Initiated Shared Semantic Resource (SSR); created suite of prototype ontologies
enabling SE also outside the Dataspace– CUBRC: demonstrated ability to leverage SE for analytics – Priming potential users of SE (in DoD CIO, NGA, JIEDDO, TRADOC …)
2013-– Milportal– Event Reporting Application– Initial negotiations with other agencies and groups
![Page 7: 11:00 Self-Introductions 11:15 Report on ontology-based data integration work in DCGS-A --- Goals and methodology --- Practical experience and results](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56649e905503460f94b9422e/html5/thumbnails/7.jpg)
Ontologies We Have (Feb. 2013)• Physical Artifact, including Infrastructure, Facility,
Vehicle, Weapon
• Information Artifact, including Report, Image, Map
• Event, including Military, Criminal, Economic, Political, Religious, Social
• Human Physical Characteristics• Agent, including Person, Organization, Social Network
• Geospatial • Time
![Page 8: 11:00 Self-Introductions 11:15 Report on ontology-based data integration work in DCGS-A --- Goals and methodology --- Practical experience and results](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56649e905503460f94b9422e/html5/thumbnails/8.jpg)
MilPortalhttp://milportal.ncor.buffalo.edu
![Page 9: 11:00 Self-Introductions 11:15 Report on ontology-based data integration work in DCGS-A --- Goals and methodology --- Practical experience and results](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56649e905503460f94b9422e/html5/thumbnails/9.jpg)
![Page 10: 11:00 Self-Introductions 11:15 Report on ontology-based data integration work in DCGS-A --- Goals and methodology --- Practical experience and results](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56649e905503460f94b9422e/html5/thumbnails/10.jpg)
![Page 11: 11:00 Self-Introductions 11:15 Report on ontology-based data integration work in DCGS-A --- Goals and methodology --- Practical experience and results](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56649e905503460f94b9422e/html5/thumbnails/11.jpg)
![Page 12: 11:00 Self-Introductions 11:15 Report on ontology-based data integration work in DCGS-A --- Goals and methodology --- Practical experience and results](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56649e905503460f94b9422e/html5/thumbnails/12.jpg)
slide from Margaret Storey
Next step
![Page 13: 11:00 Self-Introductions 11:15 Report on ontology-based data integration work in DCGS-A --- Goals and methodology --- Practical experience and results](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56649e905503460f94b9422e/html5/thumbnails/13.jpg)
slide from Margaret Storey
![Page 14: 11:00 Self-Introductions 11:15 Report on ontology-based data integration work in DCGS-A --- Goals and methodology --- Practical experience and results](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56649e905503460f94b9422e/html5/thumbnails/14.jpg)
Work plans 1: For DSC Cloud (on-going)
• Perform needs analysis: Review DCGS-A Logical Data Model and schemas of other DSCG-A data sets; in each case, examine content and establish what terms are needed to ensure sufficient coverage for SE;
• Where these terms already exist within the SSR, check that definitions exist and that these definitions are adequate
• Where the terms do not exist within the SSR, create new terms (or new ontologies), with appropriate definitions as necessary to fill gaps.
• Use the terms in 2. and 3. to annotate the corresponding entries in the data models to effect horizontal integration;
• With each new expansion in scope of DSGS-A data sets, iterate the above as needed.
• In addition, we are engaging in documentation of the methodology as here described, and in dissemination and training.
![Page 15: 11:00 Self-Introductions 11:15 Report on ontology-based data integration work in DCGS-A --- Goals and methodology --- Practical experience and results](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56649e905503460f94b9422e/html5/thumbnails/15.jpg)
Work plans 2: For interagency collaboration
Step 1: Initiating interagency collaboration in the service of horizontal integration of intelligence dataIdentify candidate teams / agenciesEstablish collaboration with one or more specific teams
– Formulate and ratify agreements as appropriate– Create work plan and identify funding needs– Perform risk assessment
Step 2: Establishment of the inter-agency ontology development process Examples of types of work to be performed would include: • Create governance infrastructure• Establish needed technological support• Implement workflow• Conduct training in methodology where needed • Explore opportunities for inter-agency HI of data • Begin application to relevant agency data models of the SE strategy• Dissemination of results in a form which will allow improved systems to perform
enhanced analytics exploiting semantic interoperability.
![Page 16: 11:00 Self-Introductions 11:15 Report on ontology-based data integration work in DCGS-A --- Goals and methodology --- Practical experience and results](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56649e905503460f94b9422e/html5/thumbnails/16.jpg)
The SSR methodology and governance is neutral as to how SSR-annotations are used• Currently SE of DSC enhancement only integration
through improved indexing/search capability• On CUBRC project, we have much more, including
ontology-based reasoning. • In the future we will have for DSC enhancement
applied – to multiple models such as LDM– to unstructured text
• see the various methodology documents provided so far
![Page 17: 11:00 Self-Introductions 11:15 Report on ontology-based data integration work in DCGS-A --- Goals and methodology --- Practical experience and results](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56649e905503460f94b9422e/html5/thumbnails/17.jpg)
Event Reporting Application (EvR)
• Aggregates content from various report generating applications (CPOF, TIGR, TIGR with MAPHT extension)
• The underlying data model contains nearly 500 terms (e.g. ReportName, EventName, DeclassificationEvent)
• The semantics of the data model seems similar to that of a relational model
![Page 18: 11:00 Self-Introductions 11:15 Report on ontology-based data integration work in DCGS-A --- Goals and methodology --- Practical experience and results](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56649e905503460f94b9422e/html5/thumbnails/18.jpg)
EvR Data Model as Relational Database• Hierarchies of events and units are referenced by EventType and UnitEchelon
columns, but this alone provides no capability for traversing these hierarchies• A report is related to both a report name and a report unit by the inclusion of
appropriately named columns, but what is the difference between these relationships?
• Our approach shows how to express the difference, e.g. between report-to-event and event-to unit relationships
Report
ReportId
ReportName
ReportUnit
ContainedEventIds
…
Event
ID
UnitIdentificationCode
Location
EventType
…
UnitInformation
UIC
UnitName
UnitAffiliation
UnitEchelon
…
![Page 19: 11:00 Self-Introductions 11:15 Report on ontology-based data integration work in DCGS-A --- Goals and methodology --- Practical experience and results](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56649e905503460f94b9422e/html5/thumbnails/19.jpg)
Enhancing the vocabulary of the EvR Data Model
• Semantic enhancement (SE) amounts to associating a database field to a whole knowledge system enabling machines to process data … – “vertically” e.g. by traversing the echelon command hierarchy
and – “horizontally” e.g. by following specified relations between
units and events. • SE separates semantics from structure, reducing
maintenance costs as source databases no longer have to be modified each time we improve our understanding of reality
• The ontology tags move with the data …
![Page 20: 11:00 Self-Introductions 11:15 Report on ontology-based data integration work in DCGS-A --- Goals and methodology --- Practical experience and results](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56649e905503460f94b9422e/html5/thumbnails/20.jpg)
Enhancing the vocabulary of the EvR Data Model
Progress to Date• Two techniques of SE are available
– Partial enhancement (now being used in the DSC) associating ontology label terms with EvR terms
– Full enhancement (not yet implemented in the DSC) aligning terms from the EvR to assertions using terms and relations from SSR ontologies
• At present the current ontologies… – provide 70% coverage for partial enhancement– provide 38% coverage for full enhancement– plan for extending the ontologies to raise the coverage for the
EvR to 90% and 60% respectively by end of February
![Page 21: 11:00 Self-Introductions 11:15 Report on ontology-based data integration work in DCGS-A --- Goals and methodology --- Practical experience and results](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56649e905503460f94b9422e/html5/thumbnails/21.jpg)
Enhancing the vocabulary of the EvR Data Model
Event Reporting Term
Partial Enhancement via Ontology Label
Full Enhancement via Ontology Assertion(s)
![Page 22: 11:00 Self-Introductions 11:15 Report on ontology-based data integration work in DCGS-A --- Goals and methodology --- Practical experience and results](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56649e905503460f94b9422e/html5/thumbnails/22.jpg)
Enhanced EvR Data as a Graph
Act of Reporting
Event
EventID
Report
ReportName
Report ID
Event Type
Unit Identification
Code
Unit
Command Unit
Service
designatespa
rt_of
describes
designates participates_in
agent_in
contro
ls
affiliated_w
ith
has_output
is_subcla
ss_of
UnitName
Report Unit
designates
designates
designates
shows how full annotation provides the semantics missing from the EvR relational model described above
![Page 23: 11:00 Self-Introductions 11:15 Report on ontology-based data integration work in DCGS-A --- Goals and methodology --- Practical experience and results](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56649e905503460f94b9422e/html5/thumbnails/23.jpg)
Act of Reporting
Event
EventID
Report
ReportName
Report ID
Event Type
Unit Identification
Code
Unit
Command Unit
Service
designatespa
rt_of
describes
designates participates_in
agent_in
contro
ls
affiliated_w
ith
has_output
is_subcla
ss_of
UnitName
Report Unit
designates
designates
designates
relationships between • event type and echelon • report name and reporting unit
are made distinct and machine processable
![Page 24: 11:00 Self-Introductions 11:15 Report on ontology-based data integration work in DCGS-A --- Goals and methodology --- Practical experience and results](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56649e905503460f94b9422e/html5/thumbnails/24.jpg)
![Page 25: 11:00 Self-Introductions 11:15 Report on ontology-based data integration work in DCGS-A --- Goals and methodology --- Practical experience and results](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56649e905503460f94b9422e/html5/thumbnails/25.jpg)
Challenges • Challenges to Horizontal Integration in general
– Too many lexicons – The scope of the domain: signal, sensor, image, … intelligence
about the whole world• Our solution
– Incremental extraction and sanitization, and creation of content– Distributed collaborative development – Strong methodology
• Challenges for our solution– Governance and management of ontology development to
ensure consistent evolution– Lack of expertise
• For ontology development and management• For annotation
– Success will breed failure
![Page 26: 11:00 Self-Introductions 11:15 Report on ontology-based data integration work in DCGS-A --- Goals and methodology --- Practical experience and results](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56649e905503460f94b9422e/html5/thumbnails/26.jpg)
![Page 27: 11:00 Self-Introductions 11:15 Report on ontology-based data integration work in DCGS-A --- Goals and methodology --- Practical experience and results](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56649e905503460f94b9422e/html5/thumbnails/27.jpg)
We have these precursor ontologies
• The prototype ontologies in the existing SE (Malyuta-Salmen) helped indexing
• The ontologies we now have are much better and more than these prototypes
• How should we implement them re Event Reporting Ontology, global graph, etc. …?
• Next steps with I2WD
![Page 28: 11:00 Self-Introductions 11:15 Report on ontology-based data integration work in DCGS-A --- Goals and methodology --- Practical experience and results](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56649e905503460f94b9422e/html5/thumbnails/28.jpg)
Our ideas are being heard
• 2 classes of collaborator: observers and partners
• companies such as DataTactics are realizing that in case of success this provides huge potential business benefits.
• (DT invited our team to talk at the Ontology Summit they have called on Feb. 12, devoted to the development of shared semantics)
![Page 29: 11:00 Self-Introductions 11:15 Report on ontology-based data integration work in DCGS-A --- Goals and methodology --- Practical experience and results](https://reader035.vdocuments.mx/reader035/viewer/2022081516/56649e905503460f94b9422e/html5/thumbnails/29.jpg)
Peter Morosoff and Bill Mandrick
• How to create a strategy?