jaxa grid update september 2005 jaxa/eorc kengo aizawa keio univhiromichi fukui
DESCRIPTION
JAXA GRID Update September 2005 JAXA/EORC Kengo Aizawa KEIO UNIVHiromichi Fukui. Background. JAXA and KEIO Univ start to try establishing “Digital Asia” system In Digital Asia, it required to - semi-real time data processing, analyzing - PowerPoint PPT PresentationTRANSCRIPT
JAXA GRID Update
September 2005
JAXA/EORC Kengo Aizawa
KEIO UNIV Hiromichi Fukui
BackgroundBackground
JAXA and KEIO Univ start to try establishing “Digital Asia” system
In Digital Asia, it required to - semi-real time data processing, analyzing
JAXA and Keio Univ had computing GRID experiments as a data processing part.
PurposePurpose
Accumulate knowledge, know-how to process remote sensing data by GRID Computing.
To study technology elements to realize near realtime system on developing Digital Asia.
Image processing
Geometric rectify processGeometric rectify process
Composit processComposit process
Raidometric rectify process Raidometric rectify process
NDVI process NDVI process
Computational Computational GRIDGRID
Computational Computational GRIDGRID
Trial AreaTrial Area
Test Environment
Computing Server
Computing Server
・ GLI250m processing program LTSK10_250m （ Composite） LTSK1_250m （ Radiometric）
・ GRID middleware CyberGRIP
・ GLI250m processing program LTSK10_250m （ Composite） LTSK1_250m （ Radiometric）
・ GRID middleware CyberGRIP
GigabitEthernet
SoftwareSoftware
System System
HardwareHardware
Management Server
Management Server
category server CPUmemo
ryOS
ManagementServer
PRIMERGY TX200
Xeon 3.06GHz
2.0GBLinu RedHat ES V3
ComputingServer
PRIMERGY BX600
Xeon 3.06GHz
512MB
Linu RedHat ES V3
User
User
Computing node ＰＣComputing node ＰＣ
Management server ＰＣ
Management server ＰＣ
Processing
program sceine
シーン 1シーン 2
シーン 11シーン 12
シーン 1シーン 2
シーン 11シーン 12
シーン 1シーン 2
シーン 11シーン 12
シーン 1シーン 2
シーン 11シーン 12
シーン 1シーン 2
シーン 11シーン 12
SystemwalkerCyberGRIP Master
SystemwalkerCyberGRIP Master
SystemwalkerCyberGRIP Agent
SystemwalkerCyberGRIP Agent
SystemwalkerCyberGRIP Agent
SystemwalkerCyberGRIP Agent
SystemwalkerCyberGRIP Agent
SystemwalkerCyberGRIP Agent
SystemwalkerCyberGRIP Agent
SystemwalkerCyberGRIP Agent
Processed image
・ＯＳ [Windows] Windows SV 2003SE/EE Windows 2000 SV/AS/Pro Windows XP Professional [Linux] Red Hat Enterprise Linux AS(v. 3)/ES(v. 3) Red Hat Linux 9・ Disk ５ MB・ memory ５０ MB
・ＯＳ [Windows] Windows SV 2003SE/EE Windows 2000 SV/AS/Pro Windows XP Professional [Linux] Red Hat Enterprise Linux AS(v. 3)/ES(v. 3) Red Hat Linux 9・ Disk ５ MB・ memory ５０ MB
・ＯＳ Red Hat Enterprise Linux AS(v. 3)/ES(v. 3)・ Disk ２０ MB・ memory ３０ MB+ User X 13MB)
・ＯＳ Red Hat Enterprise Linux AS(v. 3)/ES(v. 3)・ Disk ２０ MB・ memory ３０ MB+ User X 13MB)
Required spec of nodeRequired spec of node
Required spec of serverRequired spec of server
Computing GRID Overview
imageprocessing
imageprocessing
Image processing
Image processing
Image processing
Image processing
imageprocessing
imageprocessing
５２ Hours -> １７ hours （ 66% ） reduction
※ ( コンポジット処理 + 大気補正処理 ) × 5 ジョブの場合
Estimation of scalability
51.94
17.20
0
10
20
30
40
50
60
Non Grid Grid env
Pro
cess
ing
ours
5 machines
200 ４ /11
Cyber GRIP
• Fujitsu is making efforts to standardize to OGSA,JDSL …
Next step For semi-realtime data processing
Satellite
Processing flow
Alignment among the process
Automatic flow control
data
Data processingData processing
（センサ A 用）（センサ A 用）
Data processingData processing
（センサ X 用）（センサ X 用）
datapreservation
datapreservation
・・・・・・
Data GRIDData GRID
ComputingGRID
ComputingGRID
Exec multiple job pallarelly
データ
データ
データ
データ
Realize by JGES functionRealize by JGES functionRealize by OJC functionRealize by OJC function
Receiving StationReceiving StationRecording systemRecording system
データ
Data GRIDData GRID
datapreservation
datapreservation
データ
Next step management of flow control
Geo rectifyGeo rectifycompositcomposit Radio rectifyRadio rectify
NDVI NDVI
Parameter file
Parameter
file
Parameter
file
Processing flow control Processing flow control by OJCby OJC
・ dynamic feedback by the results and resubmit jobs
Processing flow control Processing flow control by OJCby OJC
・ dynamic feedback by the results and resubmit jobs
データ
Parameter
File
Automatic parameter generation
Automatic parameter generation
データ データ データ
Data acquisition or submit job after complete previous job
Data acquisition or submit job after complete previous job
データ
低クォリティ結果時に
処理パラメータや入力データ選択へ
のフィードバック／
再実行
低クォリティ結果時に
処理パラメータや入力データ選択へ
のフィードバック／
再実行
制御ファイル
Thank you