www.csiro.au
OGC and Grid in AU
Dr Robert Woodcock
Executive Manager, e-Science
Stream Leader, Exploration and Mining
Application Access and Workflows
Applications• Web portals
• Desktop applications
Pipelines: data / tool interoperability
• semantics: community agreed standards
• structure: GML, GeosciML for data interchange
• syntax: XML language
• service: WxS, mediator
Traditional Mechanical Modelling Workflow
• “Powerful” desktop computes several models at a time
• Limitations are in the order of ~2 models per week
• Models (mesh + data files) are individually and laboriously constructed
• The manual process is error prone
• Results are manually visualised one at a time
• Screenshots are manually taken and made into “movies”
• Very little, if any, standardised data archiving is done. This results in potential confusion or loss of the originating conditions of the experiments, making it unrepeatable in the long termSlide courtesy of Robert Cheung and Warren Potma
New Refined Workflow
• Parameterised template or wizard driven model geometry/mesh creation
• Boundary condition & model properties parameter sweep utilities
• automatically creates a “family” of model, data files based on varying a set of parameters
• Inversion algorithms
• determine input parameters of future iterations automatically based on the user ranking of previous results
Automated generation of visualisations
Automated movie generation
Automated archiving
3D Time varying volume visualisation
Parameterised Geometry Creation
Multi-site data storage via Storage Resource Broker
Slide courtesy of Robert Cheung and Warren Potma
Client
The Grid Application… Service Interactions
Resource Registry
Data Management Service
HPC Repository
LoginJob
MonitorRun
SimulationEdit Problem Description
Local Repository
Archive Search
Geology S.A
Geology W.AGeochem
N.S.W
Geochem W.A
Information
Authentication Job Management Service
Escript Service
FastfloRT Service
User
Workflow...
Computation
Community Infrastructure
Physical Resource Physical Resource
Semantic Web Semantic Grid
Classical Grid
Increasing scale of data and computation
Sca
le o
f In
tero
per
abil
ity Structured
data
Flat files, unstructured data, images
(Geochem Test Bed)
(NCRIS 5.13)
(APAC Grid Projects)
Classical Web
(SEE Grid)
Minimum accessible through NCRIS 5.13, 5.16: Bandwidth > 1 Gbit/sec, HPC > 800 Gflops, 300-400 TB online storage High
Low
Incr
easi
ng
Str
uct
ure
(Current Web)
Research organisations pressing forward…
AuScope:Earth Model for the Australian Continent
AuScope Infrastructure System
A toolkit for Geoscience Research and Geoscience Applications
minerals, energy, groundwater discovery and management; hazard prediction; environmental monitoring and management
Multiple workflows from observational data to user application
Progressive refinement and enrichment by a community of practice
QLD NSW
APAC Data &Knowledge Gateways
Govt Data &Knowledge Gateways
Industry Data & Knowledge
Gateways
Data and Knowledge Services
AuScope Compute Services
TAS VIC
NT WA
SAGA
QPSF AC3
Environment
MineralsIndustry
Insurance
Expert Scientist
Non-Expert Scientist
UniversityStudent
General Public
A variety of interfaces to suit user capabilities School
Students
TPAC VPAC
IVEC
SoftwareHardware
Finite ElementSolvers Reactive
TransportCode
Fluid FlowModelling
MechanicalModelling
VisualisationTools 3D
ModellingTools
PetroleumIndustry
AuScope
SAPAC APAC
AP
AC
AC
cES
ST
era
wu
lf
APAC
TP
AC
Computing ServicesV
PA
CC
SIR
OA
C3
IVE
C
ChemicalReactions
Coupled Processes
Rock Mechanics
Fluiddynamics
DeformationSurface
ProcessesFluidFlow
SocietalNeed
NumericalBasis
Modelling Codes
ModellingServices
Base Scientific Concepts
Finite Volumes
Inundation
Natural HazardsNatural Hazards
Resources & Mining
Resources & Mining
SustainableEnergy
SustainableEnergy GeodynamicsGeodynamics
TsunamiStorm Surge
Finite Element
Solver(Fastflo)
Reactive Transport
Predictive ore deposition
Mine Waste Disposal
Finite Element Solver (Finley)
Scripting environment for Finite element simulations
(E-Script)
Mine design
Earthquake
AuScope Modelling Framework
MantleConvection
FiniteElements
Chemistry(Log K)
Geodynamics Modelling code
Underworld
Finite Element
Solver(StGFEM)
ESysCrustal
Coupled Mechanics & Reactive Transport
Particle in Cell
Slabsubduction
MantleConvection
Code(CitcomS)
FiniteDifference
Visual-isation
(gLucifer)
SurfaceModelingPackage
(SPModel)
Chemistry(Gibbs)
Particle
Vulnerability Modelling
Interpolation Mesh
The benefits
Benefits for CSIRO and the nation
• Creation of new fields of science enabled by the combination of datasets from many research disciplines.
• Greater science agility by enabling the creation of larger more diverse science teams across Australia and globally
• Greater access to science knowledge by the public
• Greater reuse of science data assets
• Creation of a data processing and distribution models to support CSIRO’s management of Australia’s national science facilities like ATNF and AAHL
eRepository
Gateway
to
Science
Key Challenges
Data acquisition, management and storage
Large data sets (potentially 20 petabytes!)
Structured, spatial scientific databases
Access to
facilities – data stores, HPC, sensors, networks
processing services – simulation, modelling, inversion, and data mining tools
Information services – discovery; structured, spatially enabled, scientific databases
applications – web portals, desktop applications
Integration and Collaboration (separation of concerns)
Community of practice
What do we need?
Unification
OGC and Grid: issues with service chaining and authentication
Adhoc virtual organisations
supporting secure networks over public networks
High performance data transfer
appropriate protocols on appropriate networks
Stateful and asynchronous versions of OGC specs
especially for large data transfers or computational tasks
Toolkits
workflows, User Interfaces…deployment support
Management controls and Scheduling
www.csiro.au
Thank You
Name Dr Robert Woodcock
Title Executive Manager, e-Science
Phone +61 8 6436 8780
Email [email protected]
Web www.csiro.au
www.seegrid.csiro.au
Contact CSIRO
Phone 1300 363 400
+61 3 9545 2176
Email [email protected]
Web www.csiro.au