http:// dorii deployment of remote instrumentation infrastructure norbert meyer poznań...
TRANSCRIPT
http://www.dorii.eu/
DORII Deployment of Remote
Instrumentation Infrastructure
Norbert MeyerPoznań Supercomputing and Netwoking Center
DORII Workshop, OGF28Munich, March 17, 2010
DORII Workshop, OGF28, 2010
Outline
DORII project introduction
Applications in DORII
DORII architecture
DORII middleware
Correlated activities
DORII Workshop, OGF28, 2010
DORII – Deployment of Remote Instrumentation Infrastructure
Started – Feb. 1st, 2008 + 30 months
12 partners: scientific community IT partners
Service providers: services + infrastructure SME
7 FP project, started at the Infrastructure Unit Grant agreement for: Combination of Collaborative projects &
Coordination and support action
Call1: Deployment of e-Infrastructures for scientific communities
DORII in a nutshell
DORII Workshop, OGF28, 2010
Project partners
z
z
DORII Workshop, OGF28, 2010
Scientific community
the ICT technology is still not present at the appropriate level
but is demanded by the users to empower the daily work processes
There is strong interest of scientific groups of users that are eager to profit from Remote Instrumentation
e-Infrastructure
This is the reference for DORII, with experimental equipment and instrumentation, which is not integrated or integrated only partially with the European infrastructure
We collected requirements from various owners of equipments, selected scientific disciplines (only areas where hardware instrumentations are required for experimental work)
The current capital investment, collected since a few years and coming from many international and national projects, where we were co-operating together
7+ years experience RINGrid, GRIDCC, Int.EU.Grid and gEclipse
Motivation
DORII Workshop, OGF28, 2010
Remote Instrumentation
What is it?
Remote instrumentation is the method of providing data acquisition and control of scientific instruments from remote locations
Why we need it?
Impact on the science society: to give the possibility to do experimental science using facilities that are extremely rare or unique. This also opens lot of oportunities for scientists from developing countries for sharing very expensive laboratory instruments
Impact on the education: to give students access to experiments remotly even on other continent. Remote instrumentation can allow on-line courses on specific instruments
Impact on the instrument usage: more efficient use of specialized equipment that might otherwise be idle.
DORII Workshop, OGF28, 2010
Buildings:security, safety
Roads and bridges
health care Automotive
Source: Fuji-Keisei USA, Inc.
Market logistics
Internet of things
DORII Workshop, OGF28, 2010
Instrument examples
DORII Workshop, OGF28, 2010
DORII - objectives
Integration of instrumentation and selected applications with e-Infrastructure and maintenance on production level
Adaptation of e-Infrastructure across selected areas of science and engineering
Step forward in accessing scientific instruments combine the experimental science community and its research
facilities with the support given by e-Infrastructure
Deployment and operation of persistent, production quality, distributed instrumentation integrated with e-Infrastructure
to provide added values of e-Infrastructure in the integrated environment of scientific and engineering instrumentation, networking, visualisation and computational infrastructures
Generalize and deploy a framework environment that can be used for fast prototyping
to use expertise and demands collected from various groups/owners of scientific instrumentation
DORII Workshop, OGF28, 2010
DORII structure
DORII Workshop, OGF28, 2010
DORII – scientific communities Earthquake community
Network-centric seismic simulations
Earthquake early warning system
Environment community with selected applications
Mediterranean Ocean Observing Network
Oceanographic and coastal observation and modelling using imaging
Experimental science community
On-line data analysis - synchrotron and free electron lasers
Reference installation mentioned by ESFRI
DORII Workshop, OGF28, 2010
Earthquake Community
DORII Workshop, OGF28, 2010
250 - 500 m
200 m
10- 20 cm/s
Glider: Data sampling and ground transmission
DORII Workshop, OGF28, 2010
Experimental Science: SRF
DORII Workshop, OGF28, 2010
Applications Requirements to Infrastructure
Main requirements Remote access to instruments (control, read data)
Access to computational resources
Access to storage resources
Integration within e-Infrastructure with common accessing methods
More specific requirements Interactivity
Parallel Execution (MPI)
Security
DORII Workshop, OGF28, 2010
DORII Infrastructure
DORII Workshop, OGF28, 2010
Earthquake Community
Actuators
Seismic sensors
Environmental Community
FLOATS, GLIDERS
Cameras
Experimental Science Community
SAXS: Small Angle X-ray Scattering
XRD: X-Ray Diffraction
SYRMEP: SYnchrotron Radiation for MEdical Physics
Demos: LEGO Robot, Robocam
Instrument Resources in DORII
DORII Workshop, OGF28, 2010
DORII Joint Research Activities
The Remote Instrumentation Infrastructure. Layered viewThe Remote Instrumentation Infrastructure. Layered view
The JRA aims at setting up a middleware framework, extended and enhanced towards support of Remote Instrumentation Applications, enabling full-service management, maintenance and exploitation of heterogeneous instrumentation and acquisition devices on the Grid.
The JRA aims at setting up a middleware framework, extended and enhanced towards support of Remote Instrumentation Applications, enabling full-service management, maintenance and exploitation of heterogeneous instrumentation and acquisition devices on the Grid.
DORII Workshop, OGF28, 2010
Parallel application
support (Open MPI)
Visualization and Interactivity (Int.EU.Grid)
Workflow Management
(VLab)
Middleware Services
Middleware Services
Core middleware stack (EGEE)
Core middleware stack (EGEE)
Computation Grid
Storage Grid
Application Development on Grid (g-Eclipse)
Application Development on Grid (g-Eclipse)
Management of Grid resources
(GRIDCC)
Management of Grid resources
(GRIDCC)
The DORII middleware architectureThe DORII middleware architecture
Remote Instrumentation on Grid (RinGRID, GRIDCC)
DORII Middleware FrameworkDORII Middleware Framework
Management & Development
tools
Management & Development
tools
DORII Joint Research Activities
DORII Workshop, OGF28, 2010
DORII Architecture
Introduced new components on top of existing software stacks
Major features:
Hardware virtualisation(makes the approach general, allows fast prototyping)
Workflow composition( automate the process of monitoring and controling
Interaction (quasi real-time control of the instruments)
DORII Workshop, OGF28, 2010
Remote Control of an Instrument
INSTRUMENT
ELEMENTMiddleware
INSTRUMENT
MANAGER
GRID storage
VCR Server
USER logs into the VCR
from a browser(username, password)
Auth withuser’s
proxy cert
DORII Workshop, OGF28, 2010
Workflow Management System
CLIENT SERVERWorkflow Editor
WEB
SERVICES
Workflow Manager
User friendly Swing GUI
Creating and editing workflow
One-click submission on Grid
VCR integration (launched through Java Web Start)
Java and Web Services
Manages workflow's jobs
Common Lib used for Grid interaction
Three layers design (manager, data base, Common Lib)
DORII Workshop, OGF28, 2010
Workflow Editor – Details
Create workflow
VO-driven access to applications
predefined default applications scenarios
SE browsing in workflow's data blocks
Run it on the Grid
one-click workflow submission
Whole workflow's logic is managed by WfMS
failed jobs are re-executed
Monitor execution progress
presentation of state of each workflow's job
DORII Workshop, OGF28, 2010
OGF RISGE-RG Open Grid Forum Remote Instrumentation Services in
Grid Environment Research Group
DORII Workshop, OGF28, 2010
OGF RISGE-RG main goal
The main purpose of this research group is to bring together various existing approaches in defining remote access interfaces to sophisticated laboratory equipment, as well as to come up with use cases that can dictate the requirements for integrating scientific instruments with the Grid
Special interest in the advances of Grid technologies in areas such as interactivity, visualization and Quality of Service
DORII Workshop, OGF28, 2010
BoF of ARI-WGAccess to Remote Instrumentation in Grid environment
Working Group
It is the first WG from RISGE-RG that will focus its efforts on defining services for accessing remote instruments
Goal: To standardize approaches in defining remote access interfaces to instruments
including sophisticated laboratory equipment, sensors as well as large devices
To define services for steering and monitoring of instrument resources, security and authorization (policy), accessing combined or multiple devices, calibration phase
To define services for operations on files and streams
DORII Workshop, OGF28, 2010
Current state and DORII details
The major scientific results and achievement on application and technology levels were described the journal title Computational Methods in Science and Technology (www.man.poznan.pl/cmst) under the special issue “Instrumentation for e-Science” - Volume 15 (1) 2009
Ask about a free copy:
DORII Workshop, OGF28, 2010
Thank you for the attention !
More Information
http://www.dorii.eu
DORII Summer School 2010
INGRID 2010 – Instrumenting the Grid
DORII Summer School 201010 - 11 May 2010Poznan, Poland
Topics
DORII ArchitectureApplication use case - Mediterranean Ocean Observing NetworkBuilding the workflowVirtual Control Room - how to define use casesHow to deploy visualisationBasic installation tutorial