soa for e-government conference mclean, va may 24, 2006 service-oriented information sharing:...

SOA for E-Government Conference

McLean, VAMay 24, 2006

Service-Oriented Information Sharing: Leveraging the FEA Data Reference Model (DRM) 2.0

Joseph M. ChiusanoBooz Allen Hamilton

The Information Sharing Challenge

What is “Service-Oriented Information Sharing”?

Introduction to FEA Data Reference Model (DRM) 2.0

How DRM 2.0 Facilitates Service-Oriented Information Sharing

Questions

Table of Contents

Information Sharing is not a means in itself, it is one mechanism to help ensure mission success

What is information sharing?

Information sharing is more than data exchange—it represents a means to achieve the organization’s mission

Information sharing is about migrating from a ‘need-to-know’ mindset—however it’s not clear that a ubiquitous ‘need-to-share’ environment can be realized due to legitimate societal, political, and legal reasons not to share information

Information sharing is not necessarily a new problem driven by new threats—in fact, what’s new are the characteristics of the information being shared

Information sharing is about creating and sustaining existing or new working relationships that promote information sharing—for example, the integrated Information Sharing Environment that facilitates the sharing of terrorism information

Information sharing reflects the need to address the changing threat environment, the realities of shared missions that defy traditional definitions and jurisdictions, and the evolving nature of information itself

It is also critical that the sharing of information be balanced with protecting the privacy and/or civil liberties of all Americans

Government leaders across all branches of government are discussing the benefits and challenges of information sharing

“Information-sharing works only when both parties demonstrate value or benefits that outweigh the risk of sharing and losing control over your information.”

Michael Assante, former US Navy intelligence officer, 7/01/05

"The holy grail of information sharing is the ability to move any piece of information, anytime, to anyone—and do so securely and cheaply."

Daniel G. Wolf, Director of the Information Assurance (IA) Directorate at the National Security Agency (NSA), 1/28/06

“In our digital age, law enforcement officers at every level must be able to make timely and effective use of records and reports from sister agencies - otherwise, we fight crime and terrorism wearing blinders.”

Deputy Attorney General James B. Comey, Department of Justice (DoJ), 8/12/2005

Various entities have produced a remarkably consistent set of findings and recommendations, reinforcing the validity and priority of solving the information sharing problem

Reduce the effect of need-to-know cultures, which not only prevent sharing but do not allow experts and analysts at the “edge” to have the information they need to begin to ask the right questions and make the right decisions

Recognize that information sharing within and external to an organization – including across Federal, State, local, and private entities – often need to overcome antiquated legal and policy obstacles

Create a distributed, decentralized, and trusted information network

Recognize that information sharing occurs in both vertical and horizontal dimensions, linking analysts to decision-makers, all levels of government, and the public and private sectors

Update agency incentive structures, which directly oppose information sharing, so that they lower the risks (criminal, civil, internal administrative penalties) and increase the rewards of sharing

Environment RulesSupporting Capabilities

Trusted Relationships

Incentives

“Ultimately, what is required is an ‘enduring institutional change to address our current threat environment’”

Weapons of Mass Destruction Commission

Numerous challenges and solution factors across the critical building blocks demand a right-sized, tailored approach for successful information sharing

Complex New Legislation

Competing Roles

Uncertain Roles

Balancing Privacy and Civil Liberty Protections

Strategy and Policy Development

IT Policies, Directives, and Guidance

Infrastructure Protection and Information Assurance Policy Development

Complex New Legislation

Competing Roles

Uncertain Roles

Balancing Privacy and Civil Liberty Protections

Strategy and Policy Development

IT Policies, Directives, and Guidance

Infrastructure Protection and Information Assurance Policy Development

Lack of Trust

Lack of Clear Stewardship

Lack of Accountability

Risk Management

Performance Management

Executive Dashboards

Partner Management (MOUs/SLAs/Charters)

Decision Framework

Risk Management Plan and Tools

Integration with Nationwide Efforts

Lack of Trust

Lack of Clear Stewardship

Lack of Accountability

Risk Management

Performance Management

Executive Dashboards

Partner Management (MOUs/SLAs/Charters)

Decision Framework

Risk Management Plan and Tools

Integration with Nationwide Efforts

Interoperability

Ensuring Information

Discovery/Visibility

Maintaining Adaptability

and Agility

Scalability

Integration

Standards

Systems

Architecture

Security

Functional/Services

Infrastructure

Interoperability

Ensuring Information

Discovery/Visibility

Maintaining Adaptability

and Agility

Scalability

Integration

Standards

Systems

Architecture

Security

Functional/Services

Infrastructure

Trust

Lack of Cross-Agency Relationships

Desire for Autonomy

Resistance to Change

“Turf Wars”

Organization and Strategic Leadership

Relationship Mapping

Strategic Alliances Analysis

Strategic Planning

Change Management

Incentive Structures

Trust

Lack of Cross-Agency Relationships

Desire for Autonomy

Resistance to Change

“Turf Wars”

Organization and Strategic Leadership

Relationship Mapping

Strategic Alliances Analysis

Strategic Planning

Change Management

Incentive Structures

Lack of Clear Measures

Limited Monitoring

Basic Modeling

Limited Value-identification

Dated Incentive Models

Business Decision Analysis

Business Case Analysis

Cost/Benefit Analysis

Cost Modeling and Analysis

Procurement Strategic Planning and Execution

Lack of Clear Measures

Limited Monitoring

Basic Modeling

Limited Value-identification

Dated Incentive Models

Business Decision Analysis

Business Case Analysis

Cost/Benefit Analysis

Cost Modeling and Analysis

Procurement Strategic Planning and Execution

Training and Education(E-learning, War Games and Simulations, Workshops)

Enterprise Architecture (Performance, Business, Service Component, Data, Technical Reference)

SOLUTIONFACTORS

andSERVICES

SOLUTIONFACTORS

andSERVICES

COMMONCHALLENGES

Today’s focus





Questions

Table of Contents

What is “Service-Oriented Information Sharing”? Service-Oriented Information Sharing is an approach to electronic information sharing, based on Service-

Oriented Architecture (SOA) principles, involving the provisioning of data services (and the services that support them), for the purpose of providing agile and scalable access to information in a timely and efficient manner

Data services provide access to data and data sources, typically for a specific type of data

– Example: A “Criminal Incidents” data service that provides information regarding criminal incidents aggregated from among multiple data sources between states/provinces of a country, further analyzed using business intelligence (BI) to look for patterns/trends among the data (e.g. similarities between victims)

– Technologies such as Enterprise Information Integration (EII) are often used to enable such capabilities, often known as “federated queries”

Data services may leverage supporting services, such as:

– Security services: Authenticate and authorize the data service to data sources as well as other services

– Discovery services: Dynamically discover WSDL descriptions for Web service-accessible data sources (i.e. other data services)

– Semantic Mediation services: Bridge semantic inconsistencies (also known as “semantic conflicts”) between controlled vocabularies of data sources

“Data consumers” and “data providers” are often part of the same or different Communities of Interest (COIs)

Time Sensitive Time Sensitive Targeting ProgramsTargeting Programs

Blue Force Tracking Blue Force Tracking ProgramsPrograms

Payroll ProgramsPayroll Programs

Various

DoD Programs

Key COI Attributes:

COIs are collaborative groups of users who must exchange information in pursuit of their shared interests, missions, or business processes and who have a shared vocabulary for the exchanged information (Net-Centric Data Strategy).

• Formed to solve a mission need

• Actively support information sharing (e.g. information visibility and accessibility)

• Composed of all appropriate stakeholders (i.e. end-users, developers, data owners/producers, architects, project managers)

• Work to define and post their agreed-on vocabulary

There are three primary requirements that are inherent in Service-Oriented Information Sharing

Agility: The ability to quickly and efficiently discover, reposition, and integrate information assets in response to strategic decisions, changing business climate, and unanticipated events

– Example: On-the-fly integration of data sources in response to a terrorist incident or natural disaster

Scalability: The ability to efficiently add new functionality, new information sharing partners, and new information sources to an information sharing environment

– Example: New subject area becomes relevant (or suddenly critical) within an established information sharing environment

Reach: The ability to share information outside one’s own organizational boundaries to the widest degree possible, to support and achieve information sharing mission goals

– Example: Sudden need to exchange information with international entities to support an incident investigation

These requirements are satisfied by SOA principles such as location and processing transparency, open standards, vendor- and platform-neutrality, and more

There are numerous techniques/technologies that can enable information sharing, which we may place on a Service-Oriented Information Sharing “Suitability Spectrum”

– Database-to-Database Transfer: Direct transfer of information from one database to another

– Electronic Data Interchange (EDI): An electronic information exchange standard involving the transfer of positional files between trading partners, often using Value Added Network (VAN) services

– Enterprise Data Warehouse (EDW): An enterprise-level repository of integrated information covering multiple subject areas that is primarily used for query and analysis purposes

– Shared Spaces: A technique in which data providers make information available in common (“shared”) spaces (such as a database), from which data consumers may subsequently retrieve the information

– Enterprise Portals: Web sites that act as electronic “gateways” to information and services within or outside an organization

– Publish/Subscribe: A technique in which data consumers “subscribe” to data (topic-based, source-based, etc.) and are automatically notified upon its availability, as well as upon updates

– Data Services: See earlier

Low

Database-to-Database Transfer

High

EDI Enterprise Data Warehouse (EDW)

Shared Spaces Enterprise Portals

Publish/Subscribe

Data Services

Service-Oriented Information Sharing Suitability Spectrum

Technique/Technology Primary Advantages Primary Disadvantages Support for Primary Capabilities*

Database-to-Database Transfer

Works for repeatable transfers between static (fixed) systems/data partners

Specific to database vendor(s) and version(s); inefficient integration

Agility: Low

Scalability: Low

Reach: Low

EDI Supported by international standards; large installed base, broad product choice

Customization may be cumbersome or infeasible; low semantic enablement of content

Agility: Low

Scalability: Low-Medium

Reach: Medium

Enterprise Data Warehouse (EDW)

Strong support for sharing of historical data; supports cleansing of operational data; unlimited queries within boundaries of subject areas; can be Web service-enabled

Adding new subject area and transferring its data via ETL is time-consuming and cumbersome; EDWs may not always be up-to-date

Agility: Medium

Scalability: Medium

Reach: Medium

Shared Spaces Does not require direct connection between data providers and data consumers; new data assets can be made available in an efficient manner

May be limit to the number of data partners supported; format of data assets may need to be known; transactions are not atomic

Agility: Medium-High

Scalability: Medium

Reach: Medium

Each technique/technology has advantages and disadvantages by which its suitability can be determined

*assume equal weight for each capability

Technique/Technology Primary Advantages Primary Disadvantages Support for Primary Capabilities

Enterprise Portals User interface adds visual dimension to data; can perform analytics on aggregated data; easy to add content

Screen real estate limits amount of information that can be shared; data consumer limited to content provided by portal

Agility: Medium-High

Scalability: Medium-High

Reach: Medium

Publish/Subscribe Does not require direct connection between data providers and data consumers; notifications can be topic-based, event-based, or source-based

Notifications may require further filtering; subscriptions made in error may cause incorrect processing

Agility: High


Reach: Medium-High

Data Services May be easily combined with other data services to produce aggregated data; may be “plugged in” to enterprise portals or publish/subscribe environments; high semantic enablement of content

May be realistic limits to scalability in terms of data assets accessed (i.e. effect on performance); customization for specific data partners may be difficult

Agility: High


Reach: High

Each technique/technology has advantages and disadvantages by which its suitability can be determined (cont’d)

We will now focus on Data Services, and their role in Service-Oriented Information Sharing as specified in the FEA Data Reference Model (DRM) 2.0





Questions

Table of Contents

The DRM enables information sharing and reuse via the standard description and discovery of common data and the promotion of uniform data management practices

The DRM provides a common mechanism to enable data interoperability, harmonization and standardization across the federal government

The DRM also provides guidance to enterprise architects and data architects for helping to increase agency agility in drawing out the value of information as a strategic mission asset

The DRM enables us to begin to address key questions such as the following:

– How do agencies discover what data are available for sharing and re-use?

– How do agencies make such data visible and accessible?

– How do agencies ensure that procedures for security and appropriate use of the data shared are considered and followed?

– How do agencies reduce unnecessary redundancies in the collection and storage of data?

– How do agencies drive down IT system costs by effectively managing data?

– How does the Federal government create rapid information sharing in responding to a time sensitive event or crisis? (most important key question!)

The FEA Data Reference Model (DRM) is a framework to enable information sharing and reuse across the federal government

DRM History and Timelines The original DRM Overview was released in October 2004

– It is known as “The Data Reference Model Volume I, Version 1.0”

Work began on the DRM 2.0 initiative in February 2005

– Initiative sponsored by the Office of Management and Budget (OMB) and the Federal Chief Information Officer (CIO) Council

– DRM Working Group was comprised of 30 agencies (124 representatives)

– Public site: http://colab.cim3.net/cgi-bin/wiki.pl?DataReferenceModel

DRM 2.0 was finalized by OMB in December 2005

– See http://www.whitehouse.gov/omb/egov/documents/DRM_2_0_Final.pdf

The DRM Management Strategy in the process of being updated

– Release anticipated later this year

DRM 2.0 was presented to Congress in December 2005 as a supporting mechanism for the requirements of Section 207d of the E-Gov Act

– See OMB Memorandum M-06-02 “Improving Public Access to and Dissemination of Government Information and Using the Federal Enterprise Architecture Data Reference Model”

http://colab.cim3.net/cgi-bin/wiki.pl?DataReferenceModel

http://www.whitehouse.gov/omb/egov/documents/DRM_2_0_Final.pdf

The DRM is presented as an abstract framework from which “concrete architectures” may be derived

Concrete ArchitectureConcrete

Architecture

Concrete ArchitectureConcrete

Architecture

Reference Model (DRM)

Reference Model (DRM)

Enhanced interoperability

The DRM’s abstract nature will enable agencies to use multiple approaches, methodologies and technologies while remaining consistent with the foundational principles of the DRM

By associating elements of concrete architectures with the DRM abstract model, those elements may therefore be associated with each other, which can help promote interoperability between cross-agency architectures/implementations

Thus, the abstract nature of the DRM as a reference model provides tremendous implementation flexibility

DRM 2.0’s three “standardization areas” represent the various aspects of data that the DRM addresses

Data Sharing

Query Points and Exchange Packages

Query Points and Exchange Packages

Data Description

Data and Data AssetsData and Data Assets

Data Context

TaxonomiesTaxonomies

They are shown below:

The arrangement of the standardization areas in the above figure indicates how:

– Data Sharing is supported by the capabilities provided by the Data Description and Data Context standardization areas, and

– Data Description and Data Context capabilities are mutually supportive

Source: DRM 2.0

The Data Description standardization area provides a means to uniformly capture the semantic and syntactic structure of data

Data Description enables mission-critical capabilities such as:

– Data Discovery:

The capability to quickly and accurately identify and find data that supports mission requirements

– Data Reuse:

The capability to increase utilization of data in new and synergistic ways in order to innovatively and creatively support missions

– Data Sharing/Exchange:

The identification of data for sharing and exchange within and between agencies and COIs, including international, state, local and tribal governments, as appropriate

– Data Entity Harmonization:

An enhanced capability to compare data artifacts across government through a common, well-defined model that supports the harmonization of those artifacts and the creation of “common entities”

The Data Context standardization area facilitates discovery of data through an approach to the categorization of data according to taxonomies

Its purpose is to enable identification and discovery of data, data artifacts, and data assets, and to provide linkages to the other FEA reference models which are themselves taxonomies

Data context: Any information that provides additional meaning to data to relate it to the purposes for which it was created and used

DRM 1.0 specified the FEA Business Reference Model (BRM) as the foundation for Data Context

DRM 2.0 enables specification of context according to any taxonomy

– Can specify context according to FEA reference models (e.g. BRM), authoritative taxonomies, agency-specific taxonomies, etc.

Simple illustration of Data Context: File system folder

The directory in which the “Agenda.doc” file resides provides its

context

The directory in which the “Agenda.doc” file resides provides its

context

The Data Sharing standardization area supports the access and exchange of data

Where:

– Access consists of ad-hoc requests

Example: Query of a Data Asset

– Exchange consists of fixed, re-occurring transactions between parties

Example: Regular exchange of environmental testing data among federal, state, local, and tribal entities

The Data Sharing standardization area is supported by the Data Description and Data Context standardization areas in the following ways:

– Data Description: Robust description of data provides enhanced meaning, which further enables determination of whether that data is pertinent for sharing

– Data Context: Categorization of data facilitates its discovery, which is a prerequisite for sharing

The scope of the DRM is broad, as it may be applied within a single agency, within a Community of Interest (COI), or cross-COI

COI #1 COI #2

COI #3

Agency

Agency

Agency

Agency

Agency

Agency

Agency

DRM 2.0DRM 2.0

COI-specific DRM 2.0

implementation

Agency-specific DRM

2.0 implementatio

n





Questions

Table of Contents

DRM 2.0 facilitates Service-Oriented Information Sharing primarily through its Data Sharing standardization area

Source of base graphic: DRM 2.0

Data Sharing section ofDRM 2.0 Abstract Model

Data supplier

Data consumer

Data exchange requirements (Description,

policies, contracts)

Data exchange topics (subject

areas)

Message payload

requirements

Query “broker”

The Data Sharing standardization area is supported by the Description and Data Context standardization areas

Data Schema

Data Description section of DRM 2.0 Abstract Model: Data Context section of DRM 2.0 Abstract Model:

Service data model

(Description, semantics)

Service categorization

(Discovery)

Documents shared

(Message payload)

Data source(s) queried

FEA reference model

associations

Source of base graphics: DRM 2.0

DRM 2.0 describes various data services that can provide information sharing capabilities

Data Service Description

Extract, Transform, Load (ETL) Services

Perform ETL processing – i.e. reading structured data objects from a data source (extract), changing the format of the data objects to match the structure required by a target database (transform), and updating the target database with the transferred data objects (load).

Publication Services Assemble a document from its component pieces, putting into a desired format and disseminating it to target databases.

Entity/Relationship Extraction Services

Identify and extract specified facts from documents. Typically, the entities identified during an entity/relationship extraction process may be incorporated into the source document as metadata, inserted into a separated document (such as a metadata record used to support discovery), or incorporated into a structured database.

Document Translation Services

Transform documents from their original format to a format required to support a target application.

Context Awareness Services

Allow the users of a collection to rapidly identify the context of the data assets managed by the COI. Context information may be captured in a formalized data architecture, a metadata registry or a separate database.

Structural Awareness Services

Allow data architects and database administrators to rapidly identify the structure of data within a data asset, making the Data Description as defined within the DRM available for use.

DRM 2.0 describes various data services that can provide information sharing capabilities (cont’d)

Data Service Description

Transactional Services Enable transactional create/update/delete operations to be performed on an underlying data store while maintaining business and referential integrity rules, as part of a workflow or business process.

Data Query Services Enable users, services or applications to directly query a repository within a collection.

Content Search and Discovery Services

Enable free text search or search of metadata contained within the documents in a repository. The searchable metadata should include the Data Context as defined within the DRM abstract model.

Retrieval Services Enable applications to request return of a specific document from a repository based upon a unique identifier, such as a URL.

Subscription Services Enable services or users to nominate themselves to automatically receive new documents added to a repository in accordance with a predetermined policy or profile.

Notification Services Automatically alert services or users of changes of the content of a repository in accordance with a predetermined policy or profile.

Further definition of these data services is left to reference architectures and future DRM phases




How DRM 2.0 Can Facilitate Service-Oriented Information Sharing

Questions

Table of Contents

Contact Information

Joseph M. Chiusano

Booz Allen HamiltonWashington, DC(202) [email protected]