challenge in building national scale grid infrastructure somsak sriprayoonsakul thai national grid...
TRANSCRIPT
Challenge in Building Challenge in Building National Scale Grid National Scale Grid InfrastructureInfrastructure
Somsak SriprayoonsakulSomsak SriprayoonsakulThai National Grid CenterThai National Grid CenterSoftware Industry Promotion Agency (SIPA)Software Industry Promotion Agency (SIPA)Ministry of Information and CommunicationMinistry of Information and Communication
Thai National Grid ProjectThai National Grid Project• A 4 years project
under Ministry of Information and Communication Technology
• 5.5 Million US$ program
• Start from 13 universities and 1 government organizations
• Extending rapidly
MinisterMICT
National Grid Committee
DirectorSIPA Board
SIPA
AdminStaffs
Member Institutes
DirectorThai National Grid Center
Researchers
BoardThai National Grid Center Associate Director
Thai National Grid Center
Partner InstitutesPartner Institutes
Bangkok
Chaingmai University (CMU)
Chulalongkorn University (CU)Kasetsart University (KU)King Mongkut’s Institute of Technology North Bangkok (KMITNB)King Mongkut’s University of Technology Thonburi (KMUTT)King Mongkut’s Institute of Technology Ladkrabang (KMITL)Mahidol Unversity (MU)Silpakorn University (SU)Meteorological Department of Thailand (TMD)
Walailak University (WU)
Prince of Songkla University (PSU)
Khonkhan University (KKU)
Suranaree University of Technology (SUT)
Nakhon Ratchasima
Khonkhan
Chaingmai
Nakhonsrithammarat
Songkla
Potential Application Potential Application DomainDomain• Life-science Grid
• Cheminformatics Grid
• Geoinformatics Grid
• Digital Archive and E-learning
• Computational Fluid Dynamics Grid
• Simulation and Modeling Grid
• Digital Media and Animation
2006 2007 2008 2009
Pilot Grid R&E Grid Enterprise Grid
•Pilot Grid and Cluster infrastructure•Interoperable Middleware•Basic computational Grid•Basic Data Grid•POC applications
•Stable Grid and Cluster infrastructure•Interoperable Middleware and basic services•Basic computational Grid•Basic Data Grid•Grid Portal•Grid applications
•R&E •Non production enterprise app
•Broad academic adoption•Limited commercial adoption
•Production Grid and Cluster infrastructure•Rich set of services
•Computational Grid•Enterprise analysis Grid•Data and knowledge Grid•Collaborative Grid
•Broad commercial adoption
Capability
Time
ปั�ตตานี�
ยะลานีราธิ�วาส
แม่�ฮ่�องสอนี
เชี�ยงราย
ล�าปัาง
พะเยา
นี�านี
แพร�
อ�ตรดิ�ตถ์�
ส�โขทั ย
ล�าพ!นี
ตาก
ก�าแพงเพชีร เพชีรบู!รณ์�
เลย
ชี ยภู!ม่�พ�จิ�ตร
หนีองบู วล�าภู!
อ�ดิรธิานี�
หนีองคาย
สกลนีครนีครพนีม่
กาฬส�นีธิ��
ม่หาสารคาม่
ม่�กดิาหาร
ร*อยเอ+ดิ ยโสธิรอ�านีาจิเจิร�ญ
บู�ร�ร ม่ย� ส�ร�นีทัร� ศร�สะเกษ
นีครสวรรค�
อ�ทั ยธิานี�
กาญจินีบู�ร�ส�พรรณ์บู�ร�
ลพบู�ร�
สระบู�ร�
ปัราจิ�นีบู�ร�สระแก*ว
ฉะเชี�งเทัรา
จิ นีทับู�ร�ระยอง
ราชีบู�ร�
เพชีรบู�ร�
ตราดิ
พระนีครศร�
ส�งห�บู�ร �
ปัทั�ม่ธิานี�นีนีทับู�ร �
สม่�ทัรสงคราม่สม่�ทัรสาคร
สม่�ทัรปัราการ
ปัระจิวบูค�ร�ข นีธิ�
ชี�ม่พร
ระนีอง
ส�ราษฏร�ธิานี�
พ งงา
ภู!เก+ต
กระบู�1
ตร ง
สต!ล
พ ทัล�ง
นีครนีายก
เชี�ยงใหม่�
นีครราชีส�ม่าอ�บูลราชีธิานี�
สงขลา
นีครศร�ธิรรม่ราชี
Inter-University Network (UniNet) Inter-University Network (UniNet) Commission on Higher Education (Center)
24 Public Universities
39 Rajabhat Universities29 IT Campus
8 Private Universities13 Mahachulalongkorn Rajawitayalai9 Other agencies9 Rajamangala Institue of Technology (44 Campus) Ministry of Education (for Under Higher Education)
155 Mbps- STM134 Mbps - E38 Mbps - 4*E12 Mbps - E164 k – 2Mbps
CMUCMUSUTSUT
MUAMUA
SU_NMJSU_NMJ
BUBU
PSUPSU
DomesticDomestic1Gbps1Gbps
Internet - 622 MbpsInternet - 622 Mbps
(to Internet 2 155 Mbps)(to Internet 2 155 Mbps)
2.5 Gbps- STM161 Gbps
Update on 24 Jan 2006Update on 24 Jan 2006
THAISARNTHAISARN
1Gbps1Gbps
MOEnet (K-12)MOEnet (K-12)
1Gbps 1Gbps
(coming soon)(coming soon)
Grid InfrastructureGrid Infrastructure
ThaiRen(Uninet/ThaiSarn)
KMITNB
KMUTT
KMITL
TMD
WU
KU
PSU
CMU
SUT
KKU SU
MU
CU
AIT
Teraflop Clusterat TNGC
2.5Gb
NECTEC
ImplementationImplementation
C C C C
H
C C C C
H C C C C
H
C C C C
H
Network
GH Grid Scheduler
Super ClusterGrid Model
Super ClusterGrid Model
Users see grid as a large single cluster
Thai National Grid Thai National Grid Standard Software StackStandard Software Stack
Applications
Core: Gamess , AutodockOption: Blast
Portal: GridSphereOption: GridPort
Management
Core: SCMSOption: MOGAS
Scheduler
Core : SQMS , SGEOption: PBS, Condor, CSF
Programming
Ninf-GMPI -G2
Grid Middleware
Globus Toolkit 4.0
Fabric
Core: Linux / Rocks on x86_64/32 Option: Windows
Network
Building the GridBuilding the Grid
Grid MiddlewareBioinformatics
Digital Media HEP Medical
Technical ContactTechnical Contact
• Each ThaiGrid member institute need to assign at least one person as technical contact point for all ThaiGrid activity
• Technical contact tasks– Software setup (both cluster and Grid)– Contact point for people inside institute
• Our communication medium are mailing-list, webboard, wiki, skype, and telephone– http://rswg.thaigrid.or.th are the central point for all
technical contacts
ThaiGrid deployment stepThaiGrid deployment step
1. Site firewall compliance test
2. Site deployment• Install NPACI Rocks 4.1• Install ThaiGrid Roll
3. Site setup• Grid CA setup• Setup of Grid services• Register site with Grid Control Center node
4. Site Certify
Site firewall compliance Site firewall compliance testtest
• ThaiGrid using Globus 4.0 software that comes with NAPCI Rocks
• Focus on using Pre-WS Grid component – 2119 – Globus Gatekeeper– 2811 – Grid FTP– 2135 – Globus MDS– 80 – For web access– 22 – For SSH access– 10001 – 20000 – reserved for applications
• Firewall is a nightmare for Grid deployment– A Firewall testing tools is needed to identify firewall
problem
Firewall testing toolFirewall testing tool
• A firewall tester program “TGCheckport” is developed.
[bank@tgcc TGcheckport-1.3]$ ./TGcheckport.sh maeka.ku.ac.thStart testing maeka.ku.ac.thTue May 16 16:48:31 ICT 2006determining globus-gatekeeper service [2119] ... Passdetermining gsiftp service [2811] ... Passdetermining globus-mds service [2135] ... Passdetermining scmsweb service [80] ... Passdetermining globus-mpi-required-spare-port service [10001-20000] ... PassCongratulation, you pass all the test. Your system is ThaiGrid FirewallCompliance.Finish testingTue May 16 16:48:32 ICT 2006[bank@tgcc TGcheckport-1.3]$
Site DeploymentSite Deployment
• Site deployment process– OS installation using – Grid core middleware (Globus) installation– Grid additional middleware installation
• MPICH-G2, ninf-G, GSI-ssh
ChallengeChallenge
• All these must be done on every cluster nodes to ensure compatibility for all components
• Manual installation is time consuming, costly, and error prone
• Rocks simplify all of these process due to the roll concept– OS and Middleware installation are highly automatic
ChallengeChallenge
• Many Grid middleware components are needed– SGE/NINF-G/GLOBUS/MPICH/SCE– Each has already been packed into Rocks Roll– Nevertheless, to completely install all software require many
rolls
• Solution: every software are bundled to a new ThaiGrid Roll– A Jumbo Roll bundle all standard Grid software together
ThaiGrid Roll
HPC Grid
SGE
Java
Ninf-G
SCE
Site Setup for ThaiGridSite Setup for ThaiGrid
• Setup host certificates for Grid• Metadata Service Setup• Enable some services
– Jobmanager-sge– Grid-FTP– GSI-SSH
• Joining with Grid Control Center– Currently using SCMSWeb for central web-base monitoring– Technical contact of each site will send an e-mail with URL of
cluster monitoring nodes– Grid meta-scheduler on GCC will distribute jobs to each
institutes cluster• Setup Grid user
Site SetupSite Setup
• Traditional Grid setup are based on manual setup– Complicate– Error prone
• ROCKS 4.1 provide mechanism for automatic grid installation and configuration– Rocks has already configured almost every services for basic
Grid system
• Anyway, some additional manual setup is needed• We currently use a well-documented admin guide to
provide a step-by-step to join ThaiGrid
Thai Grid Control CenterThai Grid Control Center
• Grid Control Center Software– Monitoring
• Automatic testing• Service Monitoring• Accounting• System monitoring• Netwwork Bandwidth
– Grid-level Job scheduling
Thai Grid Control Center Thai Grid Control Center (con’t)(con’t)• Automatic Grid services status monitoring• Command line for immediate status checking
Site CertifySite Certify
• Site certification program– Level 1: Network firewall
compliance– Level 2: Middleware
compliance (MPI/G2, Ninf-g)
– Level 3: Job scheduling compliance
– Level 4: Data Grid compliance
• Standard– Define clear software
stack
ThaiGrid Firewall StatusThaiGrid Firewall Status
Overall : 15Pass : 12Partial : 1Fail : 2
ThaiGrid Middleware ThaiGrid Middleware StatusStatus
Overall : 15Pass : 4Partial : 1Fail : 10
Grid ApplicationsGrid Applications
• ThaiGrid Drug Discovery Infrastructure• Grid based data assimilation using RS data• Implicit Narrowband Level Set Method for Fire
Spread Model• The Parallel Simulation Model for Thin Film Dep
osition Using the DSMC Method• Medical Grid
• Access Grid
ThaiGrid Drug Discovery ThaiGrid Drug Discovery InfrastructureInfrastructure
• Partners:– Dr. Jak Saengma, KU– Somsak Sriprayoonsakul, TNGC– IBM Thailand
• Challenges– From over 10000 active compounds available
from Thai medicinal plants database, find a smallest set of compounds that has a potential to be used as a drug
– Very compute intensive. Several month of computing time
• Solution: Use grid to increase computing power to 10-100 times
– Distributed screening process to hundred of hosts on the Grid
• Benefit– Speeding up time to results– Allow scientists to spend more time analyzing
results
Grid based data assimilation Grid based data assimilation using RS datausing RS data
• Dr. Kiyoshi Honda, AIT• Dr. Putchong Uthayopas, THNC• Challenges
– Using GA based data assimilation for agricultural monitoring
– Very long computation time from months to years
– Large amount of RS data needed to be moved around and process
• Solution: – Using Grid to harvest more
computing power– Hide data assimilation process
behind the service using MPI, GridRPC (ninf-g)
• Benefit– Speeding up the calculation
User
Portal
Cluster A
Cluster B Cluster C
Host DSWAP-SERVICE
SWAP-SERVICE
SWAP-SERVICESWAP-SERVICE
Grid
Implicit Narrowband Level Implicit Narrowband Level Set Method for Fire Spread Set Method for Fire Spread Model Model
Dr. Pallop HuabsomboonMahidol University
• Challenges– Simulation of large scale firespred
for forest fire– Very time consuming process
• Solution– Distribute the computation on the
Grid
• Benefit– Expect 10x faster simulation speed– Ability to model much larger
problem within a reasonable time
• Part of TNGP/ Hewlett Packard Join Research Program
The Parallel Simulation Model for Thin FiThe Parallel Simulation Model for Thin Film Deposition Using the DSMC Methodlm Deposition Using the DSMC Method
Dr. Rardchawadee SilapuntDr. Tiranee AchalakulKMUTT, Thailand• Thin film deposition process is
used for many manufacturing industries including very large scale integrated circuits, optics, magnetic, hard and corrosion resistant coatings, micro-mechanics.
• Grid system will be used to accelerate the computation and
simulation process • Part of TNGP/ Hewlett
Packard Join Research Program
simulation domain S
ub
-D
om
ain
Processor
Processor
Processor
MP
IM
PI
Su
b-
Do
ma
inS
ub
-D
om
ain
Medical GridMedical GridSuwachai Sieanoon, Somkiat
Chatchuenyot, Dr. Panjai Tantasnawong
Computer Science Department, Silpakorn University
• To build and develop a distributed information management system using DataGrid technology.
• Apply the distributed information management system to test on exchanging and managing the Patient Records System, which is record complexity and, different format and large size of data.
Grid Services(Internet)Option
Hospital_E
Grid Services
MetadataHospital_A
MetadataHospital_B
MetadataHospital_C
Data Agent
Hospital_F
Hospital_D
GRID Services
Data Agent
Service Registr
y
Meta Data(HL7)
XML
IMAGE
Access Grid Access Grid
• First APEC Virtual Symposium on Pandemic Preparedness using Access Grid, January 20, 2006
• Presentation from Medical Organization in many economies– China, Taiwan, USA,
Vietnam, Thailand, Singapore, Philippine
• Access Grid are used as a key communication infrastructure
Future WorkFuture Work
• Grid Infrastructure– Include “Grid ZeroConf” roll to auto configure the
cluster to join ThaiGrid • Install additional CA certificates• Generate host certificate request• Modify yum repository
• Grid Application– Drug Design application will be deployed fully on
ThaiGrid clusters before July– GIS-base Grid-RPC application due on August– More application coming
End End
Additional SlidesAdditional Slides
Technical ContactTechnical Contact
• Each ThaiGrid member institute need to assign at least one person as technical contact point for all ThaiGrid activity
• Technical contact tasks– Software setup (both cluster and Grid)– Contact point for people inside institute
• Our communication medium are mailing-list, webboard, wiki, skype, and telephone– http://rswg.thaigrid.or.th are the central point for all
technical contacts
Grid: What and WhyGrid: What and Why• Grid is a technology that incorporates distributed
computing systems into a massive computing and storage pool with efficient resource sharing.
• Grid can strengthen national competitiveness in ICT industries as well as promote sciences and engineering research.
• Grid benefits several application domains:– E-commerce, E-government, E-learning– Digital Media and Animation– Geoinformatics, Bioinformatics– Financial Analysis– Etc.
Solving network problemSolving network problem
• General problem– Firewall Compliance– Dealing with smart intrusion detection system– Running Grid on Shared REN/Service network– Network underutilized, unbalance bandwidth
• Develop a network measurement infrastructure – Collaboration PSU/TNGC/KU/Uninet
• Collaboration with network provider (Uninet)
Network Bandwidth OUT (Mbps)
0
5
10
15
20SPIRIT(CU)
GRIDCMU(CMU)
ENQUEUE(KMITNB)
INCA(KMUTT)
SRITHAN(KKU)RDCGRID1(NECTEC)
RDCGRID2(NECTEC)
PSU-GRID(PSU)
SUTGRID(SUT) Network Bandwidth IN (Mbps)
0
20
40
60
80SPIRIT(CU)
GRIDCMU(CMU)
ENQUEUE(KMITNB)
INCA(KMUTT)
SRITHAN(KKU)RDCGRID1(NECTEC)
RDCGRID2(NECTEC)
PSU-GRID(PSU)
SUTGRID(SUT)
ThaiGrid Central ThaiGrid Central MonitoringMonitoring• ThaiGrid currently use OpenSCE to monitor the whole
ThaiGrid• OpenSCE is installed during Rocks installation using
SCE Roll• Each cluster provides monitored information through
HTTP protocol to central monitoring site
Grid Collector
RRD Database Grid
Configure file
Grid Summary Database
Grid Summary
xml file
Network
Presentation Layer
Collector Layer
System API Layer
Web
Summary XML Display
HistoryConfigurati
on
System Informatio
n
Grid Viewer
SCMS
Cluster Collector
Error Notification Email
XML Display
XML Display
Network
scms-getinfo
Summary xml file
SCE config file
Uptime Analysis
Network
Globus
Scheduler
Probe Jobs
driver