session9part2 servers detailed
TRANSCRIPT
07/07/2009
Mit
glie
d d
er
Helm
holt
z-G
em
ein
sch
aft
UNICORE Server Components -Detailed View
Bastian [email protected]
07/07/2009 Slide 2
Job Submission: Software Layers
Target System Interface (TSI)
Gateway
Web
Ser
vice
s(W
SR
F)
Target systems
non
WS
Service Container
https
Execution Management (XNJS)
AtomicServices
Security
“web service firewall”,message authentication
and forwarding
Service: coherent chunk of functionality exposed through a web-service
interface
(batch) execution systems, file systems,
databases, ...
AdditionalServices
Client
https
07/07/2009 Slide 3
Deployment Scenario: Workflow Services
ClientGateway
ServiceContainer
GlobalRegistry
Gateway
Service Container
UAS LocalRegistry
lookup
lookup,create TSS,submit job,transfer file
XUUDB XUUDBlookup user
Gateway
Service Container
UAS LocalRegistry
Service Container
UAS LocalRegistry
07/07/2009 Slide 4
WSRF
Web Services Resource Framework
WS Resource
■ Stateful web service
■ Represented by an XML document
■ Resource properties
■ Standard methods: getter, setter, queries
■ Lifetime
Service Group
■ List of WS addresses
■ Used for Registry
WS-BaseFaults
07/07/2009 Slide 5
Configuration
Service Container
■ Web Services to be deployed
■ Address of the shared Registry
■ XUUDB address, “Grid Component ID“
■ Gateway address
Gateway
■ Connection list
Registry
■ Lifetime for entries
Client
■ Registry Address
Everybody:Security settings
(Keystore, certificate, ...)
07/07/2009 Slide 6
UNICORE Atomic Services (UAS)
Target System Interface (TSI)
Gateway
UN
ICO
RE
Site
Service ContainerTarget SystemFactory (TSF)
Security
Target SystemService (TSS)
Target SystemService (TSS)
Job MgmntService (JMS)
Storage MgmntService (SMS)
Job MgmntService (JMS)
Storage MgmntService (SMS)
Storage MgmntService (SMS)
File TransferService (FTS)
Key:reference
file transfer
XUUDB
map grid users
to local users
lookup
07/07/2009 Slide 7
UAS: Target System Factory Service
3. return TSSaddress
Client
TargetSystemService
2. create
4. use TSS
1. createTSSTargetSystemFactory
07/07/2009 Slide 8
UAS: Target System Service
Abstract web service interface to target system
■ List of applications
■ Links to jobs and storages (e.g. user home)
Security
■ User authentication through XUUDB
■ Authorization: Users' target system instances and jobs are protected by configurable XACML policy
■ Secure job submission through message signing
Extensibility
■ Virtualization
■ Exclusive resource reservation
07/07/2009 Slide 9
UAS: Job Management Service
Abstract web service interface to submitted jobs
■ Jobs can be accessed and controlled from anywhere
Job status (queued, running, finished, failed, ...)
Link to storage that represents the working directory (uspace)
■ Used to securely access output files
Detailed execution log, exit code of the application
Applications are abstracted: path of executable invisible
Provide a copy of the job description
■ Can be used for resubmission
Have a lifetime (like all WS-Resources)
■ Used for automatic clean-up
07/07/2009 Slide 10
UAS: Job Management and Storage Services
USpace
Job
1.1 create
1.1.1 return jobaddress
4. stage-out data4. export data
Client
3. start
LocalFilespace Remote
StorageSpaces
2. import data 2. stage-in data
1. submitTargetSystemService
07/07/2009 Slide 11
UAS: Storage and File Transfer Services
3. return FTS address
Client 4. write/read data, monitor
1. importFile() /exportFile()
StorageManagementService
FileTransferService
2. create
07/07/2009 Slide 12
UAS: File Transfer Protocols
Pluggable mechanisms
■ Both for client-server and server-server transfers
Default mechanism: Simple OGSA ByteIO
■ Sends data as SOAP messages through the full stack
■ Needs no additional ports
■ No installation effort (pure Java)
■ Performance of ~400kB/sec
Plain http: ~ 3MB/sec
GridFTP: Speed depends on line & number of parallel TCP ports
■ Drawbacks: Lots of open ports, installation effort
UDT: ~ 100MB/sec on 1Gbit/sec line, C++ Implementation
07/07/2009 Slide 13
Deployment Scenario: Workflow Services
Client
ServiceContainer
GlobalRegistry
Service Container
WorkflowEngine
LocationMapper
Service ContainerService Orchestrator
Service Container
UASService
Container
UASService
Container
UAS
lookup
publish
submit workflow
submit jobs
submit jobs,check job status
callback
Service Container
Tracer
storemessages
traceworkflow
ServiceContainer
InformationService
query
collectdata
07/07/2009 Slide 14
Workflow Engine
3. return workflowaddress
ClientWorkflowInstance
2. create
4. monitor execution
1. submit workflow
07/07/2009 Slide 15
Configurable Security Handlers
Security handler chainUser U
login, group, & role of U?
Request R1User: U
Service: S
U
XUUDB
readXACMLPolicy File
U = SSL partner?Did U sign R1?
Is U allowed to use S?
Service S
SSL
07/07/2009 Slide 16
Trust Delegation
Security handler chainUser U
Request R1User: U
Trusts: WService: S1
U
XUUDB
Workflow Engine W(offers S1)
read
XACMLPolicy File
Request R2Consignor: WService: S2
Request R1User: U
Trusts: W
W = SSL partner?Did W sign R2?
Does U trust W?=> SAML
Is U allowed to use S2?
Service S2
SSLSSL
U
W
07/07/2009 Slide 17
UNICORE as a Web Service Hosting Environment
Security
Platform independence
Lightweight and performing: Jetty, XFire
High level programming APIs => Minimal effort
Hot deployment of web services
Transparent persistence layer using relational databases
07/07/2009 Slide 18
Ongoing Development (Incomplete List!)
European Projects
■ Smart LM: License management
■ Phosphorus: Meta-scheduling, network reservation
■ Etics: Tool for distributed builds on different platforms
German Projects
■ D-Mon: Monitoring in the D-Grid
■ BIS-Grid: Business workflows using BPEL
■ WisNetGrid: Data Management
Other Activities at the JSC
■ Information service (GLUE 2.0)
■ Purely Java based UDT implementation
■ Improved MPI support
