ooi cyberinfrastructure common operating...

OOI CyberInfrastructureCommon Operating Infrastructure

• CI

Overview

• Context of COI• Usage scenario• Work plan• Early Results• Logical Architecture• Deployment Architecture• Prototype

Scenario• A scientist is trying to setup up a

facility out of resources (instruments, computing capabilities, storage) spread out over a variety of authority domains

• Challenges–– Resource discovery Resource discovery

(instruments, storage, (instruments, storage, computation)computation)

–– Resource access (seamlessly Resource access (seamlessly across infrastructure)across infrastructure)

–– Resource lifecycle Resource lifecycle (adding/removing an (adding/removing an instrument, ...)instrument, ...)

–– Authentication, authorization, Authentication, authorization, and other policies, and other policies,

–– GovernanceGovernance–– Capability presentationCapability presentation

COI

Instruments Storage Computing

Various Authority Domains User

Legend

ProvidedCapabilities

• COI–– integration platform, integration platform, orchestratororchestrator, communication conduit, communication conduit–– handles crosshandles cross--cutting issues including identity / policy / governance / cutting issues including identity / policy / governance /

resource management => focus of Y1 activitiesresource management => focus of Y1 activities

Development Plan

Date Deliverable StateSep 30 Requirements Document New

Oct 31

Domain Model (AD) NewStandards (SSEP) NewRequirements Document RevisedCOI Prototype with data streams and basic instrument control for MBARI instruments New

Nov 30 Requirements Document RevisedDomain Model Revised

Dec 31 Report on identified Interaction Patterns NewLanguage for defining policies (AD) New

Jan 31

Policy authoring tool (Prototype) NewInteraction Patterns UpdatedLanguage (AD) UpdatedRequirements document (RD) RevisedArchitecture Document (AD) Revised

Feb 27

Software Mapping (SSEP) NewPolicy authoring tool (Prototype) UpdatedRequirements document (RD) RevisedArchitecture Document (AD) Revised

• Calit2 – Model and design the COI Architecture

• NCSA – Policy/Identity federation

• NCSU – Policy/Governance• MBARI – Instruments &

Infrastructure

• Tasks1.1. Requirements ElicitationRequirements Elicitation2.2. Standards AnalysisStandards Analysis3.3. Domain ModelingDomain Modeling4.4. Architecture and DesignArchitecture and Design5.5. Integration and PrototypingIntegration and Prototyping6.6. Documentation and PrototypingDocumentation and Prototyping

Development Plan

• Calit2 – Model and design the COI Architecture

• NCSA – Policy/Identity federation

• NCSU – Policy/Governance• MBARI – Instruments &

Infrastructure

• Tasks1. Requirements Elicitation2. Standards Analysis3. Domain Modeling4. Architecture and Design5. Integration and Prototyping6. Documentation and Prototyping

Release 1• Federated Facility Services• Enterprise Service Bus & Container

• Presentation Framework• Governance Framework• Service Framework• Resource Framework• Distributed State Management• Communication Infrastructure

• Identity Management Services• Resource Lifecycle Services• Resource Catalog and Repository

Services

Release 2• Federated Facility Services• Enterprise Service Bus & Container• Resource Activation Services• Resource Collaboration Services

Technologies

• ESB infrastructure–– MULEMULE–– Hibernate, Spring, Groovy, JMXHibernate, Spring, Groovy, JMX

• Messaging–– ActiveMQActiveMQ/AMQP/AMQP

• Policy/Governance–– GridShibGridShib, , GridGroupperGridGroupper, , myVocsmyVocs

• Leverage existing –– BIRN/BIRN/TelescienceTelescience ATOMICATOMIC–– SDSC Storage Resource Broker (SRB)SDSC Storage Resource Broker (SRB)

Early Results• Results informed by the following models:

–– IEEE1451 IEEE1451 –– Instrument models, relationship with Instrument models, relationship with SensorMLSensorML, , TransducerMLTransducerML

–– ebBPebBP -- Business transaction modelsBusiness transaction models–– Liberty Alliance Liberty Alliance –– Identity managementIdentity management–– SAML SAML –– Identity managementIdentity management–– ……

• Domain models created:–– DoDAF OVDoDAF OV--2, OV2, OV--77–– Focus currently on policy & governance aspectsFocus currently on policy & governance aspects

• Interaction patterns considered:–– Liberty Alliance identity federationLiberty Alliance identity federation

• Early prototype development

CyberPoP Software Platform

Testing prototype:• Hardware: Dual

Xeon• Main OS: Linux• VM: Vmware Server• VM Guest OS: Linux

Security - Authentication and Authorization

ESB BackboneEnd-to-End Data Transformation

Web Portal BPEL Web

ServicesJ2EE/EJB/

Servlet SAP IBM AS400

JBI (JSR-208)

File/FTP/SFTP

JMS, MQ Series,

ActiveMQTCP,

MCAST, SSL

Caching (Distrib.)

Frameworks(Spring)

GRID,JavaSpace

E-CommEmail, IM

Software configuration:• ESB: MULE• Framework: Spring• Messaging: ActiveMQ/AMQP• History: MySQL + JDBC• Feeds: RSS• Visualization: HTML + JavaScript +

Google Maps API

Thank you

From Requirements & State of the Art technologies to CI

From Requirements & State of the Art technologies to CI

Resource

Subject AttributeAttribute

Value Assertion

**

Property

Attribute Authority Attribute

Authority Policy

* Policy

Consent Policy

protects

Responsible Organization

Identity Service

Attribute Authority Manager

*Mapping

configures

configures

Principaldefines

*Rule Context ref

User

Client Application

accessesClient Attributes Requirements

Declaration

Deployment Manager

edits

Developer

creates

ref

reads

ref

Privacy Auditor

checks

gets attributes from

Liberty Alliance

OOI CI Model (simplified)

InstrumentInterface

Serv

ice/

Dat

a C

onne

ctor

Cyb

er-

Infr

astr

uctu

re

CO

IM

ess

agin

g,

Da

ta D

ist,

Po

licy

Dat

a/St

orag

eIn

terfa

ce

Computation/GridInterface

Con

trol

Mod

elin

g

Inst

rum

ent

Dat

a

CE

IPr

oces

sing

InstrumentInterface

Serv

ice/

Dat

a C

onne

ctor

Cyb

er-

Infr

astr

uctu

re

CO

IM

ess

agin

g,

Da

ta D

ist,

Po

licy

Dat

a/St

orag

eIn

terfa

ce


Con

trol

Mod

elin

g

Inst

rum

ent

Dat

a

CE

IPr

oces

sing

InstrumentInterface

Serv

ice/

Dat

a C

onne

ctor

Cyb

er-

Infr

astr

uctu

re

CO

IM

ess

agin

g,

Da

ta D

ist,

Po

licy

Dat

a/St

orag

eIn

terfa

ce


Con

trol

Mod

elin

g

Inst

rum

ent

Dat

a

CE

IPr

oces

sing

InstrumentInterface

Serv

ice/

Dat

a C

onne

ctor

Cyb

er-

Infr

astr

uctu

re

CO

IM

ess

agin

g,

Da

ta D

ist,

Po

licy

Dat

a/St

orag

eIn

terfa

ce


Con

trol

Mod

elin

g

Inst

rum

ent

Dat

a

CE

IPr

oces

sing

OOI CI Model

Inst

rum

ent

Inte

rface

Service/Data Connector

Cyber-Infrastructure

COIMessaging, Data Dist, Policy

Data/StorageInterface

Com

puta

tion/

Grid

Inte

rfaceControlModeling

Instrument Data

CEIProcessing

CO

I Interaction & M

ediation

Process Definition

Observation R

equest

Process Definition

Data R

epresentation

CO

I Interaction & Mediation

CO

I Interaction & M

ediation

CO


CO

I Interaction & M

ediation

CO


Service Agreement

ProposalProcessing Status,

Data Product

Service Agreement

Proposal,Process D

efinition

Data Product,Process

Ontology

Registration, C

omm

unication

COI Detailed needlines and relationships

COI Services Network (internal details)

GPS Clock O10Policy Validator O9

Policy Enforcement O3Identity Management O1

Authentication O2 State Management O4 Logging O6

Router/Interceptor O7

Messenger/Communicator O8

ID Inform

ation O

N1, O

N2

Authentication O

N3, O

N4

Policy, E

vent O

N5, O

N6

State, E

ventO

N7, O

N8

Event ON

11

Time

ON

16, ON

17

Policy

ON

14, ON

15

Instrument SN I0Data SN D0 Modeling SN M0Control SN C0Processing SN P0

Science User/Operator Interface U0

MessageON12, ON13

Service/Data Connector O13

Message ON22, ON23

ON32, ON33

ON24, ON25

ON26, ON27ON28, ON29

ON30, ON31

ON34,ON35 Coastal-Global Scale Node A2 (*)

Regional Scale Node A1 (*)

Research Laboratory A4 (*)

Classroom Facility A5 (*)

Governance O5

Service Registry O12

CEI SN E0

ON36, ON37

Service ID

O

N20, O

N21

Conversation

ON

9, ON

10

Registration, C

omm

unication

CO

I Interaction & M

ediation

AN4, AN9

AN3, AN8

AN2, AN7

Inst

rum

ent

Inte

rface


Cyber-Infrastructure



Com

puta

tion/

Grid

Inte

rfaceControlModeling

Instrument Data

CEIProcessing

Deployment Strategy


CI Core



ControlModeling

Instrument Data

CEIProcessing

Compute Cloud

Data Storage Cloud

Coastal/Global Scale Node

SDC

Regional Scale Node

SDC

Other Observatories/Laboratories

SDC

Science Portals and Applications

OOI Domain Model Overview• Focus is on

Process and Control Networks

• Modeling, Data and Instrument Networks are summarized

Source figure: Exhibit 5 of Ov7

<<resource>>Instrument

<<policy enforcer>>Instrument Proxy

1

1

<<resource>>Process Instancecommunicate via

communicate

*

<<resource>><<policy enforcer>>

Communication Infrastructure

*


Execution Engine

executes

<<resource>>Process Definition

reads

1

*

*

*

<<resource>>Computation

Node

runs*

*

Definition Languageis specified in

understands

1

1

specifies

CEI Services Network

<<policy enforcer>>Computation

Scheduler

setup

Observation Plan

<<policy enforcer>>Resource Planner

*1

Observation Request

receives

creates

*

1

Science Ontology

expressed in

<<policy enforcer>>Instrument Plannersetup

Instrument Services Network

Service AgreementProposal

Resource Setup Protocol

plays consumer

exchanges

plays provider plays provider

Control Services Network

Modeling Services Network

Interaction Role

*

*

plays

constraints

<<resource>>Interaction

Specification

*

*

Communication Channel

*

2..*

communicates over

*

Message Sequence

constraints

1

<<resource>>Message

*Data Message

Command Message

Engineering Data Message

Science Data Message

Data Product Message

Raw Data Message

COI-CoreData Services Network

Interaction

*

1..*

<<policy enforcer>>Dispatcher

<<policy enforcer>>Process Controller

Process Plan

Process Status

<<resource>>Communication

Facility

*

produces

consumes

consumes

produces

consumes

provides

*

Process Services Network

<<policy enforcer>>Data Planner

plays provider

Policy Model

• Informed by Shibboleth/ GridShib, Java Security Model, Liberty Alliance Models, …Source figure: Exhibit 10 of Ov7

Policy

Authentication AuthorizationAuthentication Controller

Interaction Role

Shibboleth Authentication

Controller

Principal Authorization Role

*

Authorization Controller

Shibboleth Authorization

Controller

plays controller plays

* *

Permission

Credential *

*

*

*

Tokenissues

<<resource>>

*

Capabilityaccess

Scope

1

1..*

*

Authentication Domain

1..*

playsprincipal

authenticated by

authenticateplays

1..*

<<policy enforcer>>

check

Governance ModelSource figure: Exhibit 9 of Ov7

Process Model

• COI will provide support to CEI to implement the Process Model

Source figure: Exhibit 11 of Ov7

<<resource>>Process Instance

<<resource>>Communication

Facilitycommunicate via

**


Communication Infrastructure

*


Execution Engine

<<resource>>Process Definition

reads

1

*

*

*

<<resource>>Computation

Node

runs

**

Definition Language

is specified in

understands

1

1specifies

<<policy enforcer>>Computation

Scheduler

delegates

Resource Setup Protocol

plays provider

Interaction Role

*

1..*

plays

<<resource>>Interaction

Specification*

Interaction

Setup

Engineering Data Message

produces

<<resource>>Message

Data Product Message

produces

consumestransmits

Communication Setup Strategy

creates

setup

Model Process Repository

Instrument Process

Repository

Process Definition

Repositoryqueries

queries

queries

Repository

Kepler BRTT Antelope / Unix Java VM ISI Pegasus Matlab

<<RIS>>Policy

*

1..*

emerges

Virtualization Engine:

Xen, Vmware, …

*

<<policy enforcer>>Dispatcher

<<policy enforcer>>Process Controller

Process Plan

Process Status

consumes

consumes

produces

consumes

*

executes

uses

1..*

Logical Model for PrototypeSe

rvic

e/D

ata

Con

nect

or

CTD

/Fl

orom

eter

Con

nect

or

Deployment Model for Prototype

ooi cyberinfrastructure common operating...

Documents