ocean observatories initiative common execution infrastructure (cei) overview michael meisinger...
TRANSCRIPT
![Page 1: Ocean Observatories Initiative Common Execution Infrastructure (CEI) Overview Michael Meisinger September 29, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032605/56649e795503460f94b79afb/html5/thumbnails/1.jpg)
Ocean Observatories Initiative
Common Execution Infrastructure (CEI)
Overview
Michael Meisinger
September 29, 2009
![Page 2: Ocean Observatories Initiative Common Execution Infrastructure (CEI) Overview Michael Meisinger September 29, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032605/56649e795503460f94b79afb/html5/thumbnails/2.jpg)
OOI CI Kick-Off Meeting, Sept 9-11, 20092
Outline
• Subsystem Architecture Overview• Scope of Release 1• Selected Components
– Elastic Computing– Global Network Strategy
![Page 3: Ocean Observatories Initiative Common Execution Infrastructure (CEI) Overview Michael Meisinger September 29, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032605/56649e795503460f94b79afb/html5/thumbnails/3.jpg)
3OOI CI Kick-Off Meeting
Sept 9-11, 2009
Common Execution Infrastructure
• Provides the execution and deployment environment for CI services and user processes
Common Operating Infrastructure
Data Management
(Science)
Sensing & Acquisition
Data Management (Information Distribution)
Analysis & Synthesis
Identity Management
State Management
GovernanceFramework
Resource Management
Planning & Prosecution
Exchange
Service Framework
Presentation Framework
Common Execution
Infrastructure
![Page 4: Ocean Observatories Initiative Common Execution Infrastructure (CEI) Overview Michael Meisinger September 29, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032605/56649e795503460f94b79afb/html5/thumbnails/4.jpg)
Deployment Scenario
4OOI CI Kick-Off Meeting
Sept 9-11, 2009
![Page 5: Ocean Observatories Initiative Common Execution Infrastructure (CEI) Overview Michael Meisinger September 29, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032605/56649e795503460f94b79afb/html5/thumbnails/5.jpg)
Scope of Release 1• Virtual Process Execution Management
– Definition of target execution environment independent processes
– Support of multiple cloud execution environments– Generation of target specific executables
(images)• Elastic Computing
– Elastic scheduling, provisioning, execution– Monitoring and management
• Resource Management– Compute Nodes and Target Environments– Process Repository
OOI CI Kick-Off Meeting, Sept 9-11, 20095
![Page 6: Ocean Observatories Initiative Common Execution Infrastructure (CEI) Overview Michael Meisinger September 29, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032605/56649e795503460f94b79afb/html5/thumbnails/6.jpg)
Elastic Provisioning Example
6OOI CI Kick-Off Meeting
Sept 9-11, 2009
• Imperative to separate mechanism from (smart, specialized) policy
![Page 7: Ocean Observatories Initiative Common Execution Infrastructure (CEI) Overview Michael Meisinger September 29, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032605/56649e795503460f94b79afb/html5/thumbnails/7.jpg)
Cloud Provisioning Architecture
7OOI CI Kick-Off Meeting
Sept 9-11, 2009
![Page 8: Ocean Observatories Initiative Common Execution Infrastructure (CEI) Overview Michael Meisinger September 29, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032605/56649e795503460f94b79afb/html5/thumbnails/8.jpg)
CEI Functional Components
Common Execution Infrastructure
Process Execution & Resource Provisioning
Service Agreement Proposal
Service Agreement Proposal
Status
Fault Analysis
Processing Plan
Deployable UnitDeployable
Type
Execution Environment
Adapter
Fault Monitor
ProvisionerComputation
SchedulerComputation
Controller
OperationalUnit
Execution Engine
Deployable Unit,Control
Status,Fault
Process Definition,Control
Status,Fault
Deployable Type
Deployable Type Repository
Data Management (Information
Distribution) SN
Exchange
Process Definition Repository
Process Definition
Processing Service Requestor
Processing Plan
![Page 9: Ocean Observatories Initiative Common Execution Infrastructure (CEI) Overview Michael Meisinger September 29, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032605/56649e795503460f94b79afb/html5/thumbnails/9.jpg)
Operational Unit
Operational Unit
Process Execution & Resource Provisioning
Fault MonitorProvisioner
Execution Resource
ContextualizerExecution
Monitor & Control
Deployable Unit
Operational Unit Status
Operational Unit Status
Control Commands
Service Agreement Proposal
Registration
Resource & Context
Description
ExecutionContext
Service Agreement Proposal
Computation Controller
![Page 10: Ocean Observatories Initiative Common Execution Infrastructure (CEI) Overview Michael Meisinger September 29, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032605/56649e795503460f94b79afb/html5/thumbnails/10.jpg)
Execution Engine
Process Execution & Resource Provisioning
Execution Engine
Processing Plan
Fault MonitorComputation
SchedulerComputation
Controller
Execution Engine Dispatcher
Execution Engine Processor
Execution EngineFault Monitor
Service AgreementProposal,
Process Definition
Process Definition
Processing Status
Processing Status
Fault Analysis
Service Agreement Proposal
![Page 11: Ocean Observatories Initiative Common Execution Infrastructure (CEI) Overview Michael Meisinger September 29, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032605/56649e795503460f94b79afb/html5/thumbnails/11.jpg)
Example: Data Exchange
![Page 12: Ocean Observatories Initiative Common Execution Infrastructure (CEI) Overview Michael Meisinger September 29, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032605/56649e795503460f94b79afb/html5/thumbnails/12.jpg)
Deployment Concepts
12OOI CI Kick-Off Meeting
Sept 9-11, 2009
Operational Architecture
DeploymentArchitecture
ImplementationArchitecture
![Page 13: Ocean Observatories Initiative Common Execution Infrastructure (CEI) Overview Michael Meisinger September 29, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032605/56649e795503460f94b79afb/html5/thumbnails/13.jpg)
Execution Engines
13OOI CI Kick-Off Meeting
Sept 9-11, 2009
![Page 14: Ocean Observatories Initiative Common Execution Infrastructure (CEI) Overview Michael Meisinger September 29, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032605/56649e795503460f94b79afb/html5/thumbnails/14.jpg)
Deployment - Execution
14OOI CI Kick-Off Meeting
Sept 9-11, 2009
![Page 15: Ocean Observatories Initiative Common Execution Infrastructure (CEI) Overview Michael Meisinger September 29, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032605/56649e795503460f94b79afb/html5/thumbnails/15.jpg)
Cloud Provisioning Basics
15OOI CI Kick-Off Meeting
Sept 9-11, 2009
![Page 16: Ocean Observatories Initiative Common Execution Infrastructure (CEI) Overview Michael Meisinger September 29, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032605/56649e795503460f94b79afb/html5/thumbnails/16.jpg)
Cloud Instance Startup
16OOI CI Kick-Off Meeting
Sept 9-11, 2009
![Page 17: Ocean Observatories Initiative Common Execution Infrastructure (CEI) Overview Michael Meisinger September 29, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032605/56649e795503460f94b79afb/html5/thumbnails/17.jpg)
Observe–Decide–Act Pattern
17OOI CI Kick-Off Meeting
Sept 9-11, 2009
![Page 18: Ocean Observatories Initiative Common Execution Infrastructure (CEI) Overview Michael Meisinger September 29, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032605/56649e795503460f94b79afb/html5/thumbnails/18.jpg)
Network Logical Deployment Model
18OOI CI Kick-Off Meeting
Sept 9-11, 2009
![Page 19: Ocean Observatories Initiative Common Execution Infrastructure (CEI) Overview Michael Meisinger September 29, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032605/56649e795503460f94b79afb/html5/thumbnails/19.jpg)
Component Deployment Model
19OOI CI Kick-Off Meeting
Sept 9-11, 2009
![Page 20: Ocean Observatories Initiative Common Execution Infrastructure (CEI) Overview Michael Meisinger September 29, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032605/56649e795503460f94b79afb/html5/thumbnails/20.jpg)
Hardware Deployment Model
20OOI CI Kick-Off Meeting
Sept 9-11, 2009
![Page 21: Ocean Observatories Initiative Common Execution Infrastructure (CEI) Overview Michael Meisinger September 29, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032605/56649e795503460f94b79afb/html5/thumbnails/21.jpg)
Network Architecture
![Page 22: Ocean Observatories Initiative Common Execution Infrastructure (CEI) Overview Michael Meisinger September 29, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032605/56649e795503460f94b79afb/html5/thumbnails/22.jpg)
22OOI CI Kick-Off Meeting
Sept 9-11, 2009
Network Deployment
![Page 23: Ocean Observatories Initiative Common Execution Infrastructure (CEI) Overview Michael Meisinger September 29, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032605/56649e795503460f94b79afb/html5/thumbnails/23.jpg)
Academic Global Gigabyte Network
![Page 24: Ocean Observatories Initiative Common Execution Infrastructure (CEI) Overview Michael Meisinger September 29, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032605/56649e795503460f94b79afb/html5/thumbnails/24.jpg)
Academic National Gigabit Network
![Page 25: Ocean Observatories Initiative Common Execution Infrastructure (CEI) Overview Michael Meisinger September 29, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032605/56649e795503460f94b79afb/html5/thumbnails/25.jpg)
OOI Operational LocationsMicrosoft
AmazonWest
AmazonEast
Woods Hole
Seattle
Washington DC
San Diego
Portland
Corvallis
Management Point
Acquisition Point
Distribution Point
Execution Point
![Page 26: Ocean Observatories Initiative Common Execution Infrastructure (CEI) Overview Michael Meisinger September 29, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032605/56649e795503460f94b79afb/html5/thumbnails/26.jpg)
Technology Mapping
• Cloud Execution Sites– AWS/EC2, Science Clouds (Nimbus)– GoGrid, FlexiScale (optional)
• National Grid Resource Interface– Condor for high throughput jobs– Teragrid Gateways
Functional Component Technology TRLGrid Scheduler Condor 9Cloud Computing Environment Amazon EC2 9Cloud Computing Environment Nimbus 8VM Packaging & Distribution Elastic Server (CohesiveFT) 8Virtualization Layer Software Individual virtualization vendors, e.g., Amazon AMI)9
![Page 27: Ocean Observatories Initiative Common Execution Infrastructure (CEI) Overview Michael Meisinger September 29, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032605/56649e795503460f94b79afb/html5/thumbnails/27.jpg)
OOI CI Kick-Off Meeting, Sept 9-11, 200927
Thanks !
![Page 28: Ocean Observatories Initiative Common Execution Infrastructure (CEI) Overview Michael Meisinger September 29, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032605/56649e795503460f94b79afb/html5/thumbnails/28.jpg)
Cloud Computing Strategy• Requirements
– a secure, scalable, fault-tolerant network and computing infrastructure that spans the OOI marine instrument networks to the national (academic & commercial) computing/data facilities
– a participation model that allows individual research teams to integrate the computing environments at their intuitions into this infrastructure
• Strategy– provide an effective development and operational process that allow the
OOI members and its operational staff to transparently use and effectively manage the OOI specific computing resources as a coherent cloud resource
• Definition– style of computing where ICT capabilities are provided “as a Service”, – allowing users to access technology-enabled services without
knowledge of, expertise with, or control over the technology infrastructure that supports them
![Page 29: Ocean Observatories Initiative Common Execution Infrastructure (CEI) Overview Michael Meisinger September 29, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032605/56649e795503460f94b79afb/html5/thumbnails/29.jpg)
Cloud Strategy Benefits• Attributes
– Effective coupling of resource acquisition to user demand– Streamlining the preparation, deployment and management of
observatory applications– Removing the management and ownership of physical
computing resources from the direct concern of the Observatory• Results
– Eliminate centralized capital assets associated with networking storage, and computing as well as associated staffing
– Continual transparent tracking of the technology advancements– Ability to respond to changes in usage demand in minutes and
changes in market costs within weeks
![Page 30: Ocean Observatories Initiative Common Execution Infrastructure (CEI) Overview Michael Meisinger September 29, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032605/56649e795503460f94b79afb/html5/thumbnails/30.jpg)
Architectural Concerns• Operational
– Enterprise of Applications, Services and the deployed Modules to support the Enterprise
• Deployment (concern of CEI)
– Secure, Scalable, Self-healing Execution Platform
• Implementation (concern of CEI)
– Configuration framework to compose and condition specific classes deployable execution units (Capability Containers)
• Functional– Decomposition of the system based on functional and
construction concerns
![Page 31: Ocean Observatories Initiative Common Execution Infrastructure (CEI) Overview Michael Meisinger September 29, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032605/56649e795503460f94b79afb/html5/thumbnails/31.jpg)
CEI Components• 40 min: Elastic scaling and provisioning
– Load graph: Start story. From there to CPE drawings or DX drawing. System scales according to load.
– Domain models development, deployment, operation architecture OV7 overview, execution engine OV7
– CPE use case• 20 min Network: Global site selection (Matthew)
– Deployment architecture– Detailed network diagrams (Brian)– National backbone infrastructure– GLIF (Tom) for network environment we’re deploying into.– Story: National DIF framework using Solace technology, link to
COI/Messaging. Higher density computing per enclosure, diversity of deployment. Possibility for inner core messaging architecture (very efficient on the backbone); commodity equipment
OOI CI Kick-Off Meeting, Sept 9-11, 200931