genesis ii open source, ogsa implementation genesis ii: mapping grids into the local file system:...
TRANSCRIPT
![Page 1: Genesis II Open Source, OGSA Implementation Genesis II: Mapping Grids into the Local File System: Access, RNS, and ByteIO Andrew Grimshaw Genesis II Team](https://reader034.vdocuments.mx/reader034/viewer/2022051400/5514d8a1550346935c8b5220/html5/thumbnails/1.jpg)
Genesis II“Open Source, OGSA Implementation”
Genesis II:Mapping Grids into the Local File System:
Access, RNS, and ByteIO
Andrew Grimshaw
Genesis II Team
University of Virginia
“Open Source, OGSA Implementation”
![Page 2: Genesis II Open Source, OGSA Implementation Genesis II: Mapping Grids into the Local File System: Access, RNS, and ByteIO Andrew Grimshaw Genesis II Team](https://reader034.vdocuments.mx/reader034/viewer/2022051400/5514d8a1550346935c8b5220/html5/thumbnails/2.jpg)
Genesis II“Open Source, OGSA Implementation”
Outline
• Background
• Genesis II - The specs in action
![Page 3: Genesis II Open Source, OGSA Implementation Genesis II: Mapping Grids into the Local File System: Access, RNS, and ByteIO Andrew Grimshaw Genesis II Team](https://reader034.vdocuments.mx/reader034/viewer/2022051400/5514d8a1550346935c8b5220/html5/thumbnails/3.jpg)
Genesis II“Open Source, OGSA Implementation”
Classic three layer view
API’s & interfaces, e.g. SAGA, NFS, CIFS
Standard portypes
Resources Layer
Grid Services Layer
Access Layer
![Page 4: Genesis II Open Source, OGSA Implementation Genesis II: Mapping Grids into the Local File System: Access, RNS, and ByteIO Andrew Grimshaw Genesis II Team](https://reader034.vdocuments.mx/reader034/viewer/2022051400/5514d8a1550346935c8b5220/html5/thumbnails/4.jpg)
Genesis II“Open Source, OGSA Implementation”
Classic 3-layer name scheme
…File replica 2File replica 2
WS-name EPRWS-name EPRFile replica 1File replica 1
File replica mFile replica m
RNS file name 1RNS file name 1
RNS file name nRNS file name n
…
Human names Abstract name:EPI, rebinding
WS-Names are WS-Addresses with optionalEPI and resolver EPR
This is essentially a table
![Page 5: Genesis II Open Source, OGSA Implementation Genesis II: Mapping Grids into the Local File System: Access, RNS, and ByteIO Andrew Grimshaw Genesis II Team](https://reader034.vdocuments.mx/reader034/viewer/2022051400/5514d8a1550346935c8b5220/html5/thumbnails/5.jpg)
Genesis II“Open Source, OGSA Implementation”
Four specs
• RNS – directory service that maps strings to EPRs <<EPR> XSD-any>– Can build directed graphs, including trees– Leaves can be most anything, web pages, ByteIO
endpoints, DMI endpoints, BES resources• WS-Naming – A profile on WS-Addressing that
supports identity and rebinding of addresses – migration, failure, and replication transparency
• ByteIO – think POSIX file/steam, read, write, stat• BES – Accepts JSDL documents and executes
them
![Page 6: Genesis II Open Source, OGSA Implementation Genesis II: Mapping Grids into the Local File System: Access, RNS, and ByteIO Andrew Grimshaw Genesis II Team](https://reader034.vdocuments.mx/reader034/viewer/2022051400/5514d8a1550346935c8b5220/html5/thumbnails/6.jpg)
Genesis II“Open Source, OGSA Implementation”
Outline
• Background
• Genesis II – The specs in action
![Page 7: Genesis II Open Source, OGSA Implementation Genesis II: Mapping Grids into the Local File System: Access, RNS, and ByteIO Andrew Grimshaw Genesis II Team](https://reader034.vdocuments.mx/reader034/viewer/2022051400/5514d8a1550346935c8b5220/html5/thumbnails/7.jpg)
Genesis II“Open Source, OGSA Implementation”
User’s First
• A large percentage of a grid’s target audience is unable or unwilling to learn new interaction abstractions
• Instead of asking the user to adapt to the grid, we should adapt the grid to the user
![Page 8: Genesis II Open Source, OGSA Implementation Genesis II: Mapping Grids into the Local File System: Access, RNS, and ByteIO Andrew Grimshaw Genesis II Team](https://reader034.vdocuments.mx/reader034/viewer/2022051400/5514d8a1550346935c8b5220/html5/thumbnails/8.jpg)
Genesis II“Open Source, OGSA Implementation”
User Abstractions
• One of the most ubiquitous user interaction abstractions is the file system– Drag-and-drop– Double Click– /proc filesystem– Named pipes
• RNS, ByteIO, and WS-Naming provide the foundation for building these abstractions
![Page 9: Genesis II Open Source, OGSA Implementation Genesis II: Mapping Grids into the Local File System: Access, RNS, and ByteIO Andrew Grimshaw Genesis II Team](https://reader034.vdocuments.mx/reader034/viewer/2022051400/5514d8a1550346935c8b5220/html5/thumbnails/9.jpg)
Genesis II“Open Source, OGSA Implementation”
(Most) everything is a file or directory
• Files and directories can be accessed without knowing anything about Grids, Web Services, or anything thanks to OGRSH/IFS– OGRSH/IFS map the Grid into the file system
• BES resources, queues are directories– “ls” to list the jobs, “cat” a job to see its state– “cp” a JSDL file into the directory -> start the job up– A shell script can start jobs by copying
• Genesis II containers are directories– “ls” to see the services and porttypes
• IDP are files/directories• IS’s are directories
– “cp” a query file to IS, creates result• RDBMS’s will be directories• The user can access all of these services without dealing with Web
Services!!
![Page 10: Genesis II Open Source, OGSA Implementation Genesis II: Mapping Grids into the Local File System: Access, RNS, and ByteIO Andrew Grimshaw Genesis II Team](https://reader034.vdocuments.mx/reader034/viewer/2022051400/5514d8a1550346935c8b5220/html5/thumbnails/10.jpg)
Genesis II“Open Source, OGSA Implementation”
Genesis II - Data
• FTPd for Windows
• OGRSH for Linux (shim)
• IFS for Windows
• ExportDir
• Replicated ExportDir
![Page 11: Genesis II Open Source, OGSA Implementation Genesis II: Mapping Grids into the Local File System: Access, RNS, and ByteIO Andrew Grimshaw Genesis II Team](https://reader034.vdocuments.mx/reader034/viewer/2022051400/5514d8a1550346935c8b5220/html5/thumbnails/11.jpg)
Genesis II“Open Source, OGSA Implementation”
Localhost Fdpd
• Fully secure– My X.509– SSL– No bits in
the clear
![Page 12: Genesis II Open Source, OGSA Implementation Genesis II: Mapping Grids into the Local File System: Access, RNS, and ByteIO Andrew Grimshaw Genesis II Team](https://reader034.vdocuments.mx/reader034/viewer/2022051400/5514d8a1550346935c8b5220/html5/thumbnails/12.jpg)
Genesis II“Open Source, OGSA Implementation”
Windows IFS
![Page 13: Genesis II Open Source, OGSA Implementation Genesis II: Mapping Grids into the Local File System: Access, RNS, and ByteIO Andrew Grimshaw Genesis II Team](https://reader034.vdocuments.mx/reader034/viewer/2022051400/5514d8a1550346935c8b5220/html5/thumbnails/13.jpg)
Genesis II“Open Source, OGSA Implementation”
OGRSH (Linux)
![Page 14: Genesis II Open Source, OGSA Implementation Genesis II: Mapping Grids into the Local File System: Access, RNS, and ByteIO Andrew Grimshaw Genesis II Team](https://reader034.vdocuments.mx/reader034/viewer/2022051400/5514d8a1550346935c8b5220/html5/thumbnails/14.jpg)
Genesis II“Open Source, OGSA Implementation”
Using RNS to name non-file-system components
• BES resources are also RNS directories
• We can schedule a job on a resource simply by “dropping” it into the directory
![Page 15: Genesis II Open Source, OGSA Implementation Genesis II: Mapping Grids into the Local File System: Access, RNS, and ByteIO Andrew Grimshaw Genesis II Team](https://reader034.vdocuments.mx/reader034/viewer/2022051400/5514d8a1550346935c8b5220/html5/thumbnails/15.jpg)
Genesis II“Open Source, OGSA Implementation”
Export Directory
• Map a Unix or Windows file system into the Genesis II RNS name space so that others can securely Create/Read/Write/Destroy
![Page 16: Genesis II Open Source, OGSA Implementation Genesis II: Mapping Grids into the Local File System: Access, RNS, and ByteIO Andrew Grimshaw Genesis II Team](https://reader034.vdocuments.mx/reader034/viewer/2022051400/5514d8a1550346935c8b5220/html5/thumbnails/16.jpg)
Genesis II“Open Source, OGSA Implementation”
Data publisher
Mapping data into the GridData clients Data clients
LinuxWindowsWindows
• Links directories and files from source location to data grid directory and user-specified name
• Presents unified view of the data across platforms, locations, domains, etc.
• Data publisher controls authorization policy.
Data publisherData publisher
![Page 17: Genesis II Open Source, OGSA Implementation Genesis II: Mapping Grids into the Local File System: Access, RNS, and ByteIO Andrew Grimshaw Genesis II Team](https://reader034.vdocuments.mx/reader034/viewer/2022051400/5514d8a1550346935c8b5220/html5/thumbnails/17.jpg)
Genesis II“Open Source, OGSA Implementation”
Export Directory Tool
![Page 18: Genesis II Open Source, OGSA Implementation Genesis II: Mapping Grids into the Local File System: Access, RNS, and ByteIO Andrew Grimshaw Genesis II Team](https://reader034.vdocuments.mx/reader034/viewer/2022051400/5514d8a1550346935c8b5220/html5/thumbnails/18.jpg)
Genesis II“Open Source, OGSA Implementation”
Create an Export DirSelect local path
![Page 19: Genesis II Open Source, OGSA Implementation Genesis II: Mapping Grids into the Local File System: Access, RNS, and ByteIO Andrew Grimshaw Genesis II Team](https://reader034.vdocuments.mx/reader034/viewer/2022051400/5514d8a1550346935c8b5220/html5/thumbnails/19.jpg)
Genesis II“Open Source, OGSA Implementation”
Create an Export DirSelect target RNS path
![Page 20: Genesis II Open Source, OGSA Implementation Genesis II: Mapping Grids into the Local File System: Access, RNS, and ByteIO Andrew Grimshaw Genesis II Team](https://reader034.vdocuments.mx/reader034/viewer/2022051400/5514d8a1550346935c8b5220/html5/thumbnails/20.jpg)
Genesis II“Open Source, OGSA Implementation”
Moral of the story
• RNS allows us to place arbitrary resources into a traditional directed graph/tree structure
• OGRSH/IFS map RNS namespaces into the local file system
• Users can interact with the grid without knowing anything about grids