a case study on multi-component multi-cluster interaction...
TRANSCRIPT
![Page 1: A Case Study on Multi-Component Multi-Cluster Interaction ...hpc.pnl.gov/conf/wolfhpc/2013/talks/atanasov.pdfWOLFHPC 2013 A. Atanasov, H.-J. Bungartz, K. Unterweger,T. Weinzierl, R](https://reader035.vdocuments.mx/reader035/viewer/2022071405/60f9f6fd26782737977b3e12/html5/thumbnails/1.jpg)
Technische Universität München
A Case Study on Multi-ComponentMulti-Cluster Interaction with an AMR Solver
WOLFHPC 2013
A. Atanasov, H.-J. Bungartz, K. Unterweger,T. Weinzierl, R. Wittmann
November, 18th 2013
A. Atanasov: A Case Study on Multi-Component Multi-Cluster Interaction with an AMR Solver
WOLFHPC 2013, November, 18th 2013 1
![Page 2: A Case Study on Multi-Component Multi-Cluster Interaction ...hpc.pnl.gov/conf/wolfhpc/2013/talks/atanasov.pdfWOLFHPC 2013 A. Atanasov, H.-J. Bungartz, K. Unterweger,T. Weinzierl, R](https://reader035.vdocuments.mx/reader035/viewer/2022071405/60f9f6fd26782737977b3e12/html5/thumbnails/2.jpg)
Technische Universität München
Outline
Motivation
Communication
Software Architecture
Results
Conclusion & Outlook
A. Atanasov: A Case Study on Multi-Component Multi-Cluster Interaction with an AMR Solver
WOLFHPC 2013, November, 18th 2013 2
![Page 3: A Case Study on Multi-Component Multi-Cluster Interaction ...hpc.pnl.gov/conf/wolfhpc/2013/talks/atanasov.pdfWOLFHPC 2013 A. Atanasov, H.-J. Bungartz, K. Unterweger,T. Weinzierl, R](https://reader035.vdocuments.mx/reader035/viewer/2022071405/60f9f6fd26782737977b3e12/html5/thumbnails/3.jpg)
Technische Universität München
Motivation
IO IO IO
IO IO IOIO IO IO
IO IO IO
NM
M̂
• Multi-cluster interactions between non-monolithic applications
• Examples : Multi-Physics, simulation-postprocessing coupling, . . .
• M request sources, N data sources, M̂ destination nodes
A. Atanasov: A Case Study on Multi-Component Multi-Cluster Interaction with an AMR Solver
WOLFHPC 2013, November, 18th 2013 3
![Page 4: A Case Study on Multi-Component Multi-Cluster Interaction ...hpc.pnl.gov/conf/wolfhpc/2013/talks/atanasov.pdfWOLFHPC 2013 A. Atanasov, H.-J. Bungartz, K. Unterweger,T. Weinzierl, R](https://reader035.vdocuments.mx/reader035/viewer/2022071405/60f9f6fd26782737977b3e12/html5/thumbnails/4.jpg)
Technische Universität München
Use Case
View
Model
Controller
• Simulation Model: Parallel AMR solver for hyperbolic PDEs [4]
• Visualisation: Query-based data retrieval [3]
• Controller : User located in VR environment
A. Atanasov: A Case Study on Multi-Component Multi-Cluster Interaction with an AMR Solver
WOLFHPC 2013, November, 18th 2013 4
![Page 5: A Case Study on Multi-Component Multi-Cluster Interaction ...hpc.pnl.gov/conf/wolfhpc/2013/talks/atanasov.pdfWOLFHPC 2013 A. Atanasov, H.-J. Bungartz, K. Unterweger,T. Weinzierl, R](https://reader035.vdocuments.mx/reader035/viewer/2022071405/60f9f6fd26782737977b3e12/html5/thumbnails/5.jpg)
Technische Universität München
Outline
Motivation
Communication
Software Architecture
Results
Conclusion & Outlook
A. Atanasov: A Case Study on Multi-Component Multi-Cluster Interaction with an AMR Solver
WOLFHPC 2013, November, 18th 2013 5
![Page 6: A Case Study on Multi-Component Multi-Cluster Interaction ...hpc.pnl.gov/conf/wolfhpc/2013/talks/atanasov.pdfWOLFHPC 2013 A. Atanasov, H.-J. Bungartz, K. Unterweger,T. Weinzierl, R](https://reader035.vdocuments.mx/reader035/viewer/2022071405/60f9f6fd26782737977b3e12/html5/thumbnails/6.jpg)
Technische Universität München
M
SPoC
IO IO IO
IO IO IOIO IO IO
IO IO IO
N
M̂Visualisation
Simulation
Controller
A. Atanasov: A Case Study on Multi-Component Multi-Cluster Interaction with an AMR Solver
WOLFHPC 2013, November, 18th 2013 6
![Page 7: A Case Study on Multi-Component Multi-Cluster Interaction ...hpc.pnl.gov/conf/wolfhpc/2013/talks/atanasov.pdfWOLFHPC 2013 A. Atanasov, H.-J. Bungartz, K. Unterweger,T. Weinzierl, R](https://reader035.vdocuments.mx/reader035/viewer/2022071405/60f9f6fd26782737977b3e12/html5/thumbnails/7.jpg)
Technische Universität München
M
SPoC
IO IO IO
IO IO IOIO IO IO
IO IO IO
N
M̂Visualisation
Simulation
0
0
Controller
A. Atanasov: A Case Study on Multi-Component Multi-Cluster Interaction with an AMR Solver
WOLFHPC 2013, November, 18th 2013 7
![Page 8: A Case Study on Multi-Component Multi-Cluster Interaction ...hpc.pnl.gov/conf/wolfhpc/2013/talks/atanasov.pdfWOLFHPC 2013 A. Atanasov, H.-J. Bungartz, K. Unterweger,T. Weinzierl, R](https://reader035.vdocuments.mx/reader035/viewer/2022071405/60f9f6fd26782737977b3e12/html5/thumbnails/8.jpg)
Technische Universität München
M
SPoC
IO IO IO
IO IO IOIO IO IO
IO IO IO
N
M̂Visualisation
Simulation
0
0
1 1
Controller
A. Atanasov: A Case Study on Multi-Component Multi-Cluster Interaction with an AMR Solver
WOLFHPC 2013, November, 18th 2013 8
![Page 9: A Case Study on Multi-Component Multi-Cluster Interaction ...hpc.pnl.gov/conf/wolfhpc/2013/talks/atanasov.pdfWOLFHPC 2013 A. Atanasov, H.-J. Bungartz, K. Unterweger,T. Weinzierl, R](https://reader035.vdocuments.mx/reader035/viewer/2022071405/60f9f6fd26782737977b3e12/html5/thumbnails/9.jpg)
Technische Universität München
M
SPoC
IO IO IO
IO IO IOIO IO IO
IO IO IO
N
M̂Visualisation
Simulation
0
0
1 1
2
Controller
A. Atanasov: A Case Study on Multi-Component Multi-Cluster Interaction with an AMR Solver
WOLFHPC 2013, November, 18th 2013 9
![Page 10: A Case Study on Multi-Component Multi-Cluster Interaction ...hpc.pnl.gov/conf/wolfhpc/2013/talks/atanasov.pdfWOLFHPC 2013 A. Atanasov, H.-J. Bungartz, K. Unterweger,T. Weinzierl, R](https://reader035.vdocuments.mx/reader035/viewer/2022071405/60f9f6fd26782737977b3e12/html5/thumbnails/10.jpg)
Technische Universität München
M
SPoC
IO IO IO
IO IO IOIO IO IO
IO IO IO
N
M̂Visualisation
Simulation
0
0
1 1
2
3
Controller
A. Atanasov: A Case Study on Multi-Component Multi-Cluster Interaction with an AMR Solver
WOLFHPC 2013, November, 18th 2013 10
![Page 11: A Case Study on Multi-Component Multi-Cluster Interaction ...hpc.pnl.gov/conf/wolfhpc/2013/talks/atanasov.pdfWOLFHPC 2013 A. Atanasov, H.-J. Bungartz, K. Unterweger,T. Weinzierl, R](https://reader035.vdocuments.mx/reader035/viewer/2022071405/60f9f6fd26782737977b3e12/html5/thumbnails/11.jpg)
Technische Universität München
Routing Information (0 − 3)
A. Atanasov: A Case Study on Multi-Component Multi-Cluster Interaction with an AMR Solver
WOLFHPC 2013, November, 18th 2013 11
![Page 12: A Case Study on Multi-Component Multi-Cluster Interaction ...hpc.pnl.gov/conf/wolfhpc/2013/talks/atanasov.pdfWOLFHPC 2013 A. Atanasov, H.-J. Bungartz, K. Unterweger,T. Weinzierl, R](https://reader035.vdocuments.mx/reader035/viewer/2022071405/60f9f6fd26782737977b3e12/html5/thumbnails/12.jpg)
Technische Universität München
Merges : Greedy Merge Version(4 + 5)
M
SPoC
IO IO IO
IO IO IO
IO IO IO
IO
25%
10
IO
35%
60%
IO
10%10%
20%
N
M̂
Visualisation
Simulation
0
0
1 1
2
3
4
Controller
40%
A. Atanasov: A Case Study on Multi-Component Multi-Cluster Interaction with an AMR Solver
WOLFHPC 2013, November, 18th 2013 12
![Page 13: A Case Study on Multi-Component Multi-Cluster Interaction ...hpc.pnl.gov/conf/wolfhpc/2013/talks/atanasov.pdfWOLFHPC 2013 A. Atanasov, H.-J. Bungartz, K. Unterweger,T. Weinzierl, R](https://reader035.vdocuments.mx/reader035/viewer/2022071405/60f9f6fd26782737977b3e12/html5/thumbnails/13.jpg)
Technische Universität München
Merges : Greedy Merge Version(4 + 5)
M
SPoC
IO IO IO
IO IO IO
IO IO IO
IO
25%
10
IO
35%
60%
IO
10%10%
20%
N
M̂
Visualisation
Simulation
0
0
1 1
2
3
4
Controller
40%
A. Atanasov: A Case Study on Multi-Component Multi-Cluster Interaction with an AMR Solver
WOLFHPC 2013, November, 18th 2013 13
![Page 14: A Case Study on Multi-Component Multi-Cluster Interaction ...hpc.pnl.gov/conf/wolfhpc/2013/talks/atanasov.pdfWOLFHPC 2013 A. Atanasov, H.-J. Bungartz, K. Unterweger,T. Weinzierl, R](https://reader035.vdocuments.mx/reader035/viewer/2022071405/60f9f6fd26782737977b3e12/html5/thumbnails/14.jpg)
Technische Universität München
Merges : Greedy Merge Version(4 + 5)
M
SPoC
IO IO IO
IO IO IO
IO IO IO
IO
25%
10
IO
35%
60%
IO
10%10%
20%
N
M̂
Visualisation
Simulation
0
0
1 1
2
3
4
Controller
40%
A. Atanasov: A Case Study on Multi-Component Multi-Cluster Interaction with an AMR Solver
WOLFHPC 2013, November, 18th 2013 14
![Page 15: A Case Study on Multi-Component Multi-Cluster Interaction ...hpc.pnl.gov/conf/wolfhpc/2013/talks/atanasov.pdfWOLFHPC 2013 A. Atanasov, H.-J. Bungartz, K. Unterweger,T. Weinzierl, R](https://reader035.vdocuments.mx/reader035/viewer/2022071405/60f9f6fd26782737977b3e12/html5/thumbnails/15.jpg)
Technische Universität München
Merges : Greedy Merge Version(4 + 5)
M
SPoC
IO IO IO
IO IO IO
IO IO IO
IO
25%
10
IO
35%
60%
IO
10%10%
20%
N
M̂
Visualisation
Simulation
0
0
1 1
2
3
4
Controller
40%
• Global identification of the nodes holding most data• Merge data on identified nodes• Forward merged data to destinations
A. Atanasov: A Case Study on Multi-Component Multi-Cluster Interaction with an AMR Solver
WOLFHPC 2013, November, 18th 2013 15
![Page 16: A Case Study on Multi-Component Multi-Cluster Interaction ...hpc.pnl.gov/conf/wolfhpc/2013/talks/atanasov.pdfWOLFHPC 2013 A. Atanasov, H.-J. Bungartz, K. Unterweger,T. Weinzierl, R](https://reader035.vdocuments.mx/reader035/viewer/2022071405/60f9f6fd26782737977b3e12/html5/thumbnails/16.jpg)
Technische Universität München
M
SPoC
IO IO IO
IO IO IOIO IO IO
IO IO IO
N
M̂Visualisation Simulation
0
0
1 1
2
3
4
Controller
4
4
A. Atanasov: A Case Study on Multi-Component Multi-Cluster Interaction with an AMR Solver
WOLFHPC 2013, November, 18th 2013 16
![Page 17: A Case Study on Multi-Component Multi-Cluster Interaction ...hpc.pnl.gov/conf/wolfhpc/2013/talks/atanasov.pdfWOLFHPC 2013 A. Atanasov, H.-J. Bungartz, K. Unterweger,T. Weinzierl, R](https://reader035.vdocuments.mx/reader035/viewer/2022071405/60f9f6fd26782737977b3e12/html5/thumbnails/17.jpg)
Technische Universität München
Hierarchical Merge Version(4)
Level 0
Level 1
Level 2
Level 3
Query
Level 0
v0 v1 v2 v3 v4 v5
cutoff level
cutoff level
• Use a logical tree to merge data• Merge data up to given cutoff ℓ
• Destinations merge the remaining data fragments
A. Atanasov: A Case Study on Multi-Component Multi-Cluster Interaction with an AMR Solver
WOLFHPC 2013, November, 18th 2013 17
![Page 18: A Case Study on Multi-Component Multi-Cluster Interaction ...hpc.pnl.gov/conf/wolfhpc/2013/talks/atanasov.pdfWOLFHPC 2013 A. Atanasov, H.-J. Bungartz, K. Unterweger,T. Weinzierl, R](https://reader035.vdocuments.mx/reader035/viewer/2022071405/60f9f6fd26782737977b3e12/html5/thumbnails/18.jpg)
Technische Universität München
Tree Merge, cont.
Properties
• Requires a logical master-worker hierarchy
• Fully asynchronous
• Additional memory requirements dependent on ℓ
• ℓ can be chosen dynamically
• Merge distributed between N and M̂
A. Atanasov: A Case Study on Multi-Component Multi-Cluster Interaction with an AMR Solver
WOLFHPC 2013, November, 18th 2013 18
![Page 19: A Case Study on Multi-Component Multi-Cluster Interaction ...hpc.pnl.gov/conf/wolfhpc/2013/talks/atanasov.pdfWOLFHPC 2013 A. Atanasov, H.-J. Bungartz, K. Unterweger,T. Weinzierl, R](https://reader035.vdocuments.mx/reader035/viewer/2022071405/60f9f6fd26782737977b3e12/html5/thumbnails/19.jpg)
Technische Universität München
Outline
Motivation
Communication
Software Architecture
Results
Conclusion & Outlook
A. Atanasov: A Case Study on Multi-Component Multi-Cluster Interaction with an AMR Solver
WOLFHPC 2013, November, 18th 2013 19
![Page 20: A Case Study on Multi-Component Multi-Cluster Interaction ...hpc.pnl.gov/conf/wolfhpc/2013/talks/atanasov.pdfWOLFHPC 2013 A. Atanasov, H.-J. Bungartz, K. Unterweger,T. Weinzierl, R](https://reader035.vdocuments.mx/reader035/viewer/2022071405/60f9f6fd26782737977b3e12/html5/thumbnails/20.jpg)
Technische Universität München
Software Architecture
Scientific IDL
Proxygenerator
Component1 Component2
ASCoDT Framework
ASCODT Builder
Static Repository
DynamicRepository
CCAPorts
Static RepositoryAPI AbstractconfigurationAPI
Communicationspecificpartof theCCAPorts
Frameworkcalls CCAcodegenerator calls
Independentcomponentcalls
• A lightweighted distributed framework based on CCA [1]• Communication through BSD sockets or MPI Ports• IDE for component development [2]• Generate glue code from SIDL annotations
A. Atanasov: A Case Study on Multi-Component Multi-Cluster Interaction with an AMR Solver
WOLFHPC 2013, November, 18th 2013 20
![Page 21: A Case Study on Multi-Component Multi-Cluster Interaction ...hpc.pnl.gov/conf/wolfhpc/2013/talks/atanasov.pdfWOLFHPC 2013 A. Atanasov, H.-J. Bungartz, K. Unterweger,T. Weinzierl, R](https://reader035.vdocuments.mx/reader035/viewer/2022071405/60f9f6fd26782737977b3e12/html5/thumbnails/21.jpg)
Technische Universität München
Outline
Motivation
Communication
Software Architecture
Results
Conclusion & Outlook
A. Atanasov: A Case Study on Multi-Component Multi-Cluster Interaction with an AMR Solver
WOLFHPC 2013, November, 18th 2013 21
![Page 22: A Case Study on Multi-Component Multi-Cluster Interaction ...hpc.pnl.gov/conf/wolfhpc/2013/talks/atanasov.pdfWOLFHPC 2013 A. Atanasov, H.-J. Bungartz, K. Unterweger,T. Weinzierl, R](https://reader035.vdocuments.mx/reader035/viewer/2022071405/60f9f6fd26782737977b3e12/html5/thumbnails/22.jpg)
Technische Universität München
Results
1 2 4 8 16 32 640
1020304050607080
Level 3 full query
Avg. BandwidthMin.BandwidthMax. Bandwidth
[Mprocesses][Mprocesses]^
[Gbp
s]
1 2 4 8 16 32 6405
101520253035404550
Level 31/2 full query
Avg. BandwidthMin.BandwidthMax. Bandwidth
[Mprocesses][Mprocesses]^
[Gbp
s]
1 2 4 8 16 32 640
20
40
60
80
100
120
full query4096x4096
L3 TreeMerge TCPL3 TreeMergeMPIGreedy Merge TCP
[Mprocesses]
[Gbps]
^1 2 4 8 16 32 64
01020304050607080
1/2 full query4096x4096
L3 TreeMerge TCPL3 TreeMergeMPIGreedy Merge TCP
[Mprocesses]
[Gbps]
^
• Fully refined computational domain with 98 (6561 × 6561) cells• Two queries : full domain query, half domain query (4096 × 4096)
A. Atanasov: A Case Study on Multi-Component Multi-Cluster Interaction with an AMR Solver
WOLFHPC 2013, November, 18th 2013 22
![Page 23: A Case Study on Multi-Component Multi-Cluster Interaction ...hpc.pnl.gov/conf/wolfhpc/2013/talks/atanasov.pdfWOLFHPC 2013 A. Atanasov, H.-J. Bungartz, K. Unterweger,T. Weinzierl, R](https://reader035.vdocuments.mx/reader035/viewer/2022071405/60f9f6fd26782737977b3e12/html5/thumbnails/23.jpg)
Technische Universität München
Outline
Motivation
Communication
Software Architecture
Results
Conclusion & Outlook
A. Atanasov: A Case Study on Multi-Component Multi-Cluster Interaction with an AMR Solver
WOLFHPC 2013, November, 18th 2013 23
![Page 24: A Case Study on Multi-Component Multi-Cluster Interaction ...hpc.pnl.gov/conf/wolfhpc/2013/talks/atanasov.pdfWOLFHPC 2013 A. Atanasov, H.-J. Bungartz, K. Unterweger,T. Weinzierl, R](https://reader035.vdocuments.mx/reader035/viewer/2022071405/60f9f6fd26782737977b3e12/html5/thumbnails/24.jpg)
Technische Universität München
Conclusion & Outlook
Enabling technologies
• On-demand data retrieval
• Destination-aware data routing
• Asynchronous partial merges
• Hide technical details through CCA
Outlook
• Autotuning approaches
• User-defined communication protocols
• Multi-physics applications
A. Atanasov: A Case Study on Multi-Component Multi-Cluster Interaction with an AMR Solver
WOLFHPC 2013, November, 18th 2013 24
![Page 25: A Case Study on Multi-Component Multi-Cluster Interaction ...hpc.pnl.gov/conf/wolfhpc/2013/talks/atanasov.pdfWOLFHPC 2013 A. Atanasov, H.-J. Bungartz, K. Unterweger,T. Weinzierl, R](https://reader035.vdocuments.mx/reader035/viewer/2022071405/60f9f6fd26782737977b3e12/html5/thumbnails/25.jpg)
Technische Universität München
References I
R. Armstrong, D. Gannon, A. Geist, K. Keahey, S. Kohn,L. McInnes, S. Parker, and B. Smolinski.Toward a common component architecture for high-performancescientific computing.In High Performance Distributed Computing, 1999. Proceedings. TheEighth International Symposium on, pages 115–124. IEEE, 1999.
A. Atanasov, H.-J. Bungartz, and T. Weinzierl.A toolkit for the code development in advanced computing.Technical Report TUM-I1330, 2013.
A. Atanasov: A Case Study on Multi-Component Multi-Cluster Interaction with an AMR Solver
WOLFHPC 2013, November, 18th 2013 25
![Page 26: A Case Study on Multi-Component Multi-Cluster Interaction ...hpc.pnl.gov/conf/wolfhpc/2013/talks/atanasov.pdfWOLFHPC 2013 A. Atanasov, H.-J. Bungartz, K. Unterweger,T. Weinzierl, R](https://reader035.vdocuments.mx/reader035/viewer/2022071405/60f9f6fd26782737977b3e12/html5/thumbnails/26.jpg)
Technische Universität München
References II
A. Atanasov, M. Srinivasan, and T. Weinzierl.Query-driven parallel exploration of large datasets.In Large Data Analysis and Visualization (LDAV), 2012 IEEESymposium on, pages 23 –30, Oct. 2012.doi 10.1109/LDAV.2012.6378972.
K. Unterweger, T. Weinzierl, D. I. Ketcheson, and A. Ahmadia.Peanoclaw—a functionally-decomposed approach to adaptive meshrefinement with local time stepping for hyperbolic conservation lawsolvers.Technische Universität München, Technical Reports, 2013.
A. Atanasov: A Case Study on Multi-Component Multi-Cluster Interaction with an AMR Solver
WOLFHPC 2013, November, 18th 2013 26