peter doolan, gvp & chief technologist, oracle public sector david webber, information...
TRANSCRIPT
<Insert Picture Here>
Peter Doolan, GVP & Chief Technologist, Oracle Public Sector
David Webber, Information Architect, Oracle Public Sector
IJIS Industry Summer Briefing – July 2011
NIEMTest Model Data
Deploy Requirements
Build Exchange
GenerateDictionary
Exchange Development
Using NIEM EIEM Dictionaries forBetter Information Alignment
Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation
Overview
• The NIEM NTAC has introduced the use of Enterprise
Information Exchange Models (EIEM) and dictionaries
• How can your organization leverage these, what are the
technical advantages, business benefits and challenges?
• Understand lessons learned from existing experience
creating EIEM dictionaries and models and what are the
resources and assets needed to do this for an enterprise.
• Creating IEPD exchanges using EIEM assets can enable
rapid leveraging of NIEM for organizations; however,
managing the process and stakeholders for the EIEM then
provides additional needs.
Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation
Introduction
• What is the big picture of EIEM for federal enterprises?
• How does government acquire and manage these assets?
• How can you leverage your existing infrastructure?
• What roles do semantics and rules play?
• EIEM collaboration and alignment
• Lessons learned and examples
• What tools are available?
• Initial support provided in GTRI dictionary server
• CAM toolkit dictionary support
Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation
EIEM – THE BIG PICTURE
Transitioning from NIEM past/present to NIEM future
Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation
NIEM Past and Present Challenges
• Vertical domain vocabulary for DHS and DOJ
• One dozen domain dictionaries
• Technology limited - built using XSD schema
• Core components highly contextual to DHS/DOJ
• Surprisingly – no actual logical models of information!
• Gap with semantic technologies integration
• Limited data content rules and code lists
• Mapping automation support missing
• Multi-year development life cycles
Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation
NIEM Future?
• Abstract dictionary representations with strong
semantics, rules and code lists support
• Schema, models, documentation are generated from
dictionary automatically
• Contextual mechanisms allow linear number of true
core components – not exponential growth
• Easier for practitioners to leverage with more
predictable and repeatable results
• Enable domain use horizontally across government
• ETL mapping automation support
• Continuous collaboration environment
Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation
EIEM – EXPLORED
Understanding EIEM today
Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation
Why do we need EIEM dictionaries?
• Allows domains to manage their components libraries• Provides consistency for project development teams
• Sets of NIEM consistent XML exchange components• Aligned to enterprise data stores• Optimized for reuse and interoperability
• Save time and effort across the enterprise• Perennial question for developers – when should I use NIEM
components, and when our own local ones?• Are there components already available for that purpose?
• Provide formal mechanisms and procedures to share components and collaborate across SDLC process
• Provide external parties consistent data views
Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation
How to create EIEM dictionaries
• Top down - directly from information models
• Enterprise modelling tools
• SQL database physical table definitions
• Bottom up
• Harvesting from existing data assets
• Harvesting from XML exchange structures and schema
• Using neutral abstract standards based representations
Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation
Conceptual Information Model
Child(BBIE)Item
Child(BBIE)Item
Parent (ABIE)Item
Parent (ABIE)Item
Follows
Naming and Design Rule (NDR) principles and guidelines
Canonical Components
DictionaryXML
Canonical Components
DictionaryXML
ebXML CCTS terms (ABIE, BBIE, ASBIE)
Parent = Aggregate Business Information Entity
Child = Basic Business Information Entity
Attribute = Association Business Information Entity
Parent (ABIE)Item
Parent (ABIE)Item
Parent (ABIE)Item
Parent (ABIE)Item
Parent (ABIE)Item
Parent (ABIE)Item . . . . .
Child(BBIE)Item
Child(BBIE)Item
Child(BBIE)Item
Child(BBIE)Item
Child(BBIE)Item
Child(BBIE)Item
Attribute(ASBIE)
Attribute(ASBIE)
Attribute(ASBIE)
Attribute(ASBIE)
* CCTS – Core Components Technical Specification
Attribute(ASBIE)
Attribute(ASBIE)
Attribute(ASBIE)
Attribute(ASBIE)
Each compound component
Each atomic component
Optional attributes of component
Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation
Example – Person Name
• Person Name (ABIE)• Verified Details? (ASBIE)
• First Name (BBIE)• Middle Name (BBIE)• Last Name (BBIE)
• Previous Name? (ASBIE)• Language Code (BBIE)
Language Code may exist independently of Person Name
Verified Details and Previous Name are flags that denote additional information about the entity they are associated with
There are three component items aspects:
structure relationships; content rules; definitions
Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation
Top Down - ICE EIEM dictionary generation
ICE Enterprise Data Model (EDM)
OASIS CAM
template
OASIS CAM
template
Model Components
XSD schema
Model Components
XSD schema
ERwin Refactoring,Compress,
Renaming ToolNIEM NDR
Evaluations
ERwin Refactoring,Compress,
Renaming ToolNIEM NDR
Evaluations
GenerateStandard
Components Dictionary
XML
GenerateStandard
Components Dictionary
XML
55
Canonical Dictionary
of ICE exchange
components
XML
+
XSD
XML
+
XSD
33
Analyst Review
ebXML CCTS compatible (ABIE, BBIE, ASBIE)
22
EleEle
DefDef
DDLDDL
Production Rules
44
11
Completed CAM
template
Completed CAM
template
Excel Spreadsheet
66
77
BENEFITS: Will reduce ICE development time and effort
Pre-built NIEM consistent XML components
XML consistent with ICE enterprise SQL data
Removes scope for definition errors
Maximizes reuse potential across ICE
Open public standards based approach
Open technology infrastructure
Code
Lists
Code
ListsCode
Lists
Code
Lists
ERwinClient
ERwin model
Write
ApplyImportExport
Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation
Renaming Tool
• Expands abbreviations (has built-in lists with context)• Corrects common typos and spelling errors (200+)• Adjusts names to camel case with no separators (_)• Retains acronyms• Adds name suffix according to NIEM representation terms
and type of content, e.g. date, number, code, etc• Optionally refactor rules by their type for interoperability;
make nillable optional; relax DB length restrictions• Can optionally insert namespace prefix• Can be configured by editing the lists in: word-utils.xsl
Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation
Bottom Up – OASIS/NIST EML
Option 2 – Derive from existing exchange XSD schemaImport each XSD and merge into CAM dictionary
Exchange XSD
schema
Exchange XSD
schemaExchange XSD
schema
Exchange XSD
schemaExchange XSD
schema
Exchange XSD
schema
CAM template
CAM templateCAM
templateCAM
templateOASIS CAM
template
OASIS CAM
template
NDR Evaluation,Refactor,Renaming
Tool
NDR Evaluation,Refactor,Renaming
Tool
33
Merge & Generate
DictionaryXML
Merge & Generate
DictionaryXML
44
XMLXMLImport
Import
ImportDictionary of
canonical components
22
Analyst Review
ebXML CCTS compatible (ABIE, BBIE, ASBIE)
11
EML Core XSD
schema
EML Core XSD
schema
OASIS CIQ schema
OASIS CIQ schema
OASIS GIS schema
OASIS GIS schema
EML – Election Markup Language
State and Local voting management systems data standard
Generate model
diagramsUML/XMI
Mind maps
Generate model
diagramsUML/XMI
Mind maps
55
Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation
N-DEx Dictionary and Model
(Harvested from N-DEx schema – ndexia.xsd and ndexibp2.xsd)
Freemind Interactive ModelDictionary viewer with N-Dex components
Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation
Leveraging EIEM dictionaries
• Where can we use EIEM assets?
• What formats do we need to use?
• Are there applicable standards / guidelines?
Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation
Visual Designer with Dictionary Collection
Drag n’ Drop Tool
Drag n’ Drop Tool
Industry dictionaryIndustry dictionaryDomain dictionaryDomain dictionary
Component Definitions
44
Component Definitions
Search
Tools
Search
Tools
22
Exchange
Designer
Exchange
Designer
11
Insert
Dictionary
Parent
Components
Insert
Dictionary
Parent
Components
33
Completed Exchange Template
Completed Exchange Template
55
Collection
Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation
Alien Transfer Manifest – Mock Up
Technical Note: Inline child elements for ice: components shown for illustrative purposes only; actually are type reuse references (paperclip icon on parent node).
dictionariesdictionaries
XMLXML
XMLXML
XMLXML
• ICE EDM dictionary
• NIEM Immigration dictionary
• EID extensions dictionary
• ICE EDM dictionary
• NIEM Immigration dictionary
• EID extensions dictionary
ICE components referenced
EID pre-defined collections
based on NIEM + ICE + new
Immigration component
Namespaces of dictionary components
Definitions stored as syntax
neutral canonical XML
NIEM components
Collection
Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation
Challenges and Opportunities
• Learning curve for application development teams
• Teams used to manually crafting XML Schema and Java code
in developer XML IDE tools
• Paradigm shift
• Integration into SDLC and IEPD processes
• Integration with existing SOA stack environments
• Currently working with SAIC and State Department
• Automatic generation of WSDL and Java code assets
• Unit test XML instance generation
• XML test framework and validation services
Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation
Vision: Top Down Exchange Assembly
Canonical XML Components DictionaryCanonical XML Components Dictionary11
Component Associations and CouplingsComponent Associations and Couplings22
Exchange Templates and RulesExchange Templates and Rules33
W3C Schema and Model RepresentationsW3C Schema and Model Representations44
Delivery Control, Messaging, SecurityDelivery Control, Messaging, Security55
Presentation
Collaboration S
ervices
Artefact relationships
Implementation Artifacts and ExamplesImplementation Artifacts and Examples66
Navigation and Query
Tools
Domains Reuse Library
Components
Automatic Rendering Tools
Deployment Environments and Middleware
XML
User Inserts
Relationship Lookups
TemplateCatalog
Testing Workbench, Rules Engine, Data Samples, Integration ETL
SchemaXML
Models Components
Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation
Exchange Delivery Lifecycle
Monitor
Deploy
Design
Author
Test
Component Dictionary
CAM
Template
Compiled
Templates
XSD Schema
XML Samples
XMI / UML Models
Production
Log Results
Template
Updates
Documentation
Drag and Drop Visual Editor
Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation
htt
p:/
/ww
w.c
am
ed
ito
r.o
rg
CAM Toolkit
Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation
CAM V2.0 Release Features
• Dictionary “Drag and Drop” of components to build exchanges
• Enhanced visual editor and performance - 3x faster, smaller
memory footprint
• Cross-platform deploy - Windows, Ubuntu/LINUX, Mac; 32/64bit
• Mind map models – integration with Freemind tool
• UML/XMI export – import to UML tools / ArgoUML
• Code list table lookups – rule based and selectable
• This allows people to fully integrate their exchange development from dictionaries through to working deployed exchange schema.
23 NIEM Team - Public Sector Sales
Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation
Available XML Dictionaries
• LEXS 3.1.4 dictionary• OASIS EDXL dictionary• OASIS EML dictionary
• NIEM 2.1 dictionaries• CBRN dictionary• Emergency dictionary• Family dictionary• Immigration dictionary• Infrastructure dictionary• Intelligence dictionary• Justice dictionary• Maritime dictionary• Screening dictionary• Trade dictionary• Immigration blueprint• NIEM core dictionary
Available from download site
direct link:http://sourceforge.net/projects/camprocessor/files
XMLXMLXMLXML
XMLXMLXMLXML
XMLXMLXMLXML
+ includes spreadsheets and sample models
Packaged with CAM editor
see dictionary folder of install
+ spreadsheet
+ sample control files
XMLXMLXMLXML
XMLXML
Note: Those marked in bold are model style dictionaries with recursive components.
Copyright ©2011, Oracle. All rights reserved. Oracle Draft Materials – Limited Circulation
EIEM Next Steps
• Align CAM dictionary tools with GTRI dictionary server
• Integrate to other modelling tools – not just ERwin• Provide extended reporting tools for analysis across
dictionary of components and between dictionary collections – report on • Inconsistency• Common components• Extensions