towards asynchronous and mpi-interoperable active … · towards asynchronous and mpi-interoperable...
TRANSCRIPT
![Page 1: TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE … · TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE MESSAGES ... James Dinan, David Goodell, Pavan Balaji, Rajeev Thakur,](https://reader033.vdocuments.mx/reader033/viewer/2022051308/5afadab37f8b9a32348e4a30/html5/thumbnails/1.jpg)
TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE MESSAGES Xin Zhao, Darius Buntinas, Judicael Zounmevo, James Dinan, David Goodell, Pavan Balaji, Rajeev Thakur, Ahmad Afsahi, William Gropp
University of Illinois at Urbana-Champaign, Argonne National Laboratory, Queen’s University
13th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing
![Page 2: TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE … · TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE MESSAGES ... James Dinan, David Goodell, Pavan Balaji, Rajeev Thakur,](https://reader033.vdocuments.mx/reader033/viewer/2022051308/5afadab37f8b9a32348e4a30/html5/thumbnails/2.jpg)
Motivation • Many new important large-scale applications • Bioinformatics, social network analysis
2
![Page 3: TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE … · TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE MESSAGES ... James Dinan, David Goodell, Pavan Balaji, Rajeev Thakur,](https://reader033.vdocuments.mx/reader033/viewer/2022051308/5afadab37f8b9a32348e4a30/html5/thumbnails/3.jpg)
Motivation • Many new important large-scale applications • Bioinformatics, social network analysis
• Different from “traditional” applications • Many small messages sent to random
targets • Communication pattern is irregular • Computation is data-driven
3
![Page 4: TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE … · TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE MESSAGES ... James Dinan, David Goodell, Pavan Balaji, Rajeev Thakur,](https://reader033.vdocuments.mx/reader033/viewer/2022051308/5afadab37f8b9a32348e4a30/html5/thumbnails/4.jpg)
Motivation • Many new important large-scale applications • Bioinformatics, social network analysis
• Different from “traditional” applications • Many small messages sent to random
nodes • Communication pattern is irregular • Computation is data-driven
• Approaches for “traditional” applications are not well-suited
4
![Page 5: TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE … · TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE MESSAGES ... James Dinan, David Goodell, Pavan Balaji, Rajeev Thakur,](https://reader033.vdocuments.mx/reader033/viewer/2022051308/5afadab37f8b9a32348e4a30/html5/thumbnails/5.jpg)
while any rank’s queue is not empty: for i in ranks: out_queue[i] ⟵ empty for vertex v in my queue: if color(v) is white: color(v) ⟵ black for vertex w in neighbors(v): append w to out_queue[owner(w)]
Breath-First Search
for i in ranks: start receiving in_queue[i] from rank i for j in ranks: start sending out_queue[j] to rank j synchronize and finish communica:ons
rank 0 rank 1 rank 2
out
in
5
![Page 6: TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE … · TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE MESSAGES ... James Dinan, David Goodell, Pavan Balaji, Rajeev Thakur,](https://reader033.vdocuments.mx/reader033/viewer/2022051308/5afadab37f8b9a32348e4a30/html5/thumbnails/6.jpg)
Active Messages (AM)
6
rank 0 rank 1
Messages handler
Reply handler
• Proposed by von Eicken et al for Split-C in 1992 • Sender explicitly sends message • Upon message’s arrival, message handler is triggered
• Receiver is not explicitly involved • A suitable paradigm for data-driven applications • Data is sent immediately • Communication is asynchronous
![Page 7: TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE … · TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE MESSAGES ... James Dinan, David Goodell, Pavan Balaji, Rajeev Thakur,](https://reader033.vdocuments.mx/reader033/viewer/2022051308/5afadab37f8b9a32348e4a30/html5/thumbnails/7.jpg)
Breadth-First Search in Active Messages while queue is not empty: new_queue ⟵ empty begin AM epoch for vertex v in queue: for vertex w in neighbors(v) send AM to owner(w) end AM epoch queue ⟵ new_queue
AM_handler (vertex v) if color(v) is white: color(v) ⟵ black insert v to new_queue
rank 0 rank 1 rank 2
7
![Page 8: TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE … · TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE MESSAGES ... James Dinan, David Goodell, Pavan Balaji, Rajeev Thakur,](https://reader033.vdocuments.mx/reader033/viewer/2022051308/5afadab37f8b9a32348e4a30/html5/thumbnails/8.jpg)
Motivation • Most applications are MPI-based, but we don’t want to rewrite the entire application • An incremental approach is needed to modify the
application
8
![Page 9: TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE … · TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE MESSAGES ... James Dinan, David Goodell, Pavan Balaji, Rajeev Thakur,](https://reader033.vdocuments.mx/reader033/viewer/2022051308/5afadab37f8b9a32348e4a30/html5/thumbnails/9.jpg)
Motivation • Most applications are MPI-based, but we don’t want to rewrite the entire application • An incremental approach is needed to modify the
application • This paper tackles the challenge of supporting Active Messages within MPI framework, so that both traditional MPI communication and Active Messages can be simultaneously utilized by applications
9
![Page 10: TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE … · TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE MESSAGES ... James Dinan, David Goodell, Pavan Balaji, Rajeev Thakur,](https://reader033.vdocuments.mx/reader033/viewer/2022051308/5afadab37f8b9a32348e4a30/html5/thumbnails/10.jpg)
API DESIGN
![Page 11: TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE … · TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE MESSAGES ... James Dinan, David Goodell, Pavan Balaji, Rajeev Thakur,](https://reader033.vdocuments.mx/reader033/viewer/2022051308/5afadab37f8b9a32348e4a30/html5/thumbnails/11.jpg)
Challenges • Active Messages should work correctly with other MPI messages
• Memory consistency semantics • Consistency between two different active messages
11
![Page 12: TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE … · TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE MESSAGES ... James Dinan, David Goodell, Pavan Balaji, Rajeev Thakur,](https://reader033.vdocuments.mx/reader033/viewer/2022051308/5afadab37f8b9a32348e4a30/html5/thumbnails/12.jpg)
Challenges • Active Messages should work correctly with other MPI messages
• Memory consistency semantics • Consistency between two different active messages
• Leverage MPI One-sided (RMA) Interface, which already address those challenges
12
![Page 13: TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE … · TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE MESSAGES ... James Dinan, David Goodell, Pavan Balaji, Rajeev Thakur,](https://reader033.vdocuments.mx/reader033/viewer/2022051308/5afadab37f8b9a32348e4a30/html5/thumbnails/13.jpg)
MPI RMA Interface
13
• “One-sided” communication • Origin process specifies all communication parameters • Target process exposes memory (“window”) accessed
by other processes
• Three basic operations: Put / Get / Accumulate • Accumulate: simple updates on target process
MPI_Put MPI_Get MPI_Accumulate
![Page 14: TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE … · TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE MESSAGES ... James Dinan, David Goodell, Pavan Balaji, Rajeev Thakur,](https://reader033.vdocuments.mx/reader033/viewer/2022051308/5afadab37f8b9a32348e4a30/html5/thumbnails/14.jpg)
API Design • Extend MPI_Accumulate to support user function
• User-defined function • MPI_User_function (void *invec, void *inoutvec, int *len,
MPI_Datatype *dtype)
14
![Page 15: TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE … · TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE MESSAGES ... James Dinan, David Goodell, Pavan Balaji, Rajeev Thakur,](https://reader033.vdocuments.mx/reader033/viewer/2022051308/5afadab37f8b9a32348e4a30/html5/thumbnails/15.jpg)
API Design • Extend MPI_Accumulate to support user function
• User-defined function • MPI_User_function (void *invec, void *inoutvec, int *len,
MPI_Datatype *dtype)
• Operation creation • MPI_Op_create (MPI_User_function *user_fn, int commute,
MPI_Op *user_op)
15
![Page 16: TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE … · TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE MESSAGES ... James Dinan, David Goodell, Pavan Balaji, Rajeev Thakur,](https://reader033.vdocuments.mx/reader033/viewer/2022051308/5afadab37f8b9a32348e4a30/html5/thumbnails/16.jpg)
API Design • Extend MPI_Accumulate to support user function
• User-defined function • MPI_User_function (void *invec, void *inoutvec, int *len,
MPI_Datatype *dtype)
• Operation creation • MPI_Op_create (MPI_User_function *user_fn, int commute,
MPI_Op *user_op)
• Operation registration • MPIX_Op_register (MPI_Op user_op, int id, MPI_Win win) • Collective call on the window
16
![Page 17: TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE … · TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE MESSAGES ... James Dinan, David Goodell, Pavan Balaji, Rajeev Thakur,](https://reader033.vdocuments.mx/reader033/viewer/2022051308/5afadab37f8b9a32348e4a30/html5/thumbnails/17.jpg)
rank 1 rank 0
Example void func0 (void *in, void *inout, int *len, MPI_Datatype *dtype);
MPI_Win_create (…, &win); MPI_Op_create (func1, ..., &user_op); MPIX_Op_register (user_op, 0, win); /* func1 is triggered at rank1 */ MPIX_Op_deregister (user_op, win); MPI_Win_free (&win);
MPI_Win_create (…, &win); MPI_Op_create (func0, ..., &user_op); MPI_Win_lock (…, 1, …, win); MPI_Accumulate (…, 1, …, user_op, win); MPI_Win_unlock (1, win); MPIX_Op_deregister (user_op, win); MPI_Win_free (&win);
void func1 (void *in, void *inout, int *len, MPI_Datatype *dtype);
MPIX_Op_register (user_op, 0, win);
func0 and func1 have the same functionality
17
MPIX_Op_register (user_op, 0, win);
![Page 18: TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE … · TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE MESSAGES ... James Dinan, David Goodell, Pavan Balaji, Rajeev Thakur,](https://reader033.vdocuments.mx/reader033/viewer/2022051308/5afadab37f8b9a32348e4a30/html5/thumbnails/18.jpg)
ASYNCHRONOUS PROCESSING
![Page 19: TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE … · TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE MESSAGES ... James Dinan, David Goodell, Pavan Balaji, Rajeev Thakur,](https://reader033.vdocuments.mx/reader033/viewer/2022051308/5afadab37f8b9a32348e4a30/html5/thumbnails/19.jpg)
Asynchronous execution models • “NON-ASYNC”
• No asynchronous processing, messages are processed by single thread
• Disadvantage: messages cannot be processed upon arrival
19
![Page 20: TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE … · TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE MESSAGES ... James Dinan, David Goodell, Pavan Balaji, Rajeev Thakur,](https://reader033.vdocuments.mx/reader033/viewer/2022051308/5afadab37f8b9a32348e4a30/html5/thumbnails/20.jpg)
Asynchronous execution models • “NON-ASYNC”
• No asynchronous processing, messages are processed by single thread
• Disadvantage: messages cannot be processed upon arrival
• “THREAD-ASYNC” • Asynchronous processing is provided by thread on top of
MPI library • Disadvantage: “active polling” for intra-node messages
20
![Page 21: TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE … · TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE MESSAGES ... James Dinan, David Goodell, Pavan Balaji, Rajeev Thakur,](https://reader033.vdocuments.mx/reader033/viewer/2022051308/5afadab37f8b9a32348e4a30/html5/thumbnails/21.jpg)
Asynchronous execution models • “NON-ASYNC”
• No asynchronous processing, messages are processed by single thread
• Disadvantage: messages cannot be processed upon arrival
• “THREAD-ASYNC” • Asynchronous processing is provided by thread on top of
MPI library • Disadvantage: “active polling” for intra-node messages
• “INTEGRATED-ASYNC” • Asynchronous processing is supported internally from
MPI library 21
![Page 22: TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE … · TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE MESSAGES ... James Dinan, David Goodell, Pavan Balaji, Rajeev Thakur,](https://reader033.vdocuments.mx/reader033/viewer/2022051308/5afadab37f8b9a32348e4a30/html5/thumbnails/22.jpg)
Inter-node Communication • Thread-based approach
• Spawn thread in TCP network module
• Block polling for AM • Separate sockets for MPI
messages and AM
spawn AM thread
22
MPI AM
![Page 23: TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE … · TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE MESSAGES ... James Dinan, David Goodell, Pavan Balaji, Rajeev Thakur,](https://reader033.vdocuments.mx/reader033/viewer/2022051308/5afadab37f8b9a32348e4a30/html5/thumbnails/23.jpg)
Intra-node Communication • “Origin computation”
• During window creation, processes on the same node allocate a shared-memory region
• Origin process directly fetches data from target process, completes computation locally and writes data back to target process
23
![Page 24: TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE … · TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE MESSAGES ... James Dinan, David Goodell, Pavan Balaji, Rajeev Thakur,](https://reader033.vdocuments.mx/reader033/viewer/2022051308/5afadab37f8b9a32348e4a30/html5/thumbnails/24.jpg)
Network
Integrated Idea
rank 0 (target)
rank 1 (origin)
rank 2 (origin)
Shared-memory
NODE 0 NODE 1
24
![Page 25: TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE … · TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE MESSAGES ... James Dinan, David Goodell, Pavan Balaji, Rajeev Thakur,](https://reader033.vdocuments.mx/reader033/viewer/2022051308/5afadab37f8b9a32348e4a30/html5/thumbnails/25.jpg)
Network
Integrated Idea Shared-memory
NODE 0 NODE 1
25
rank 0 (target)
rank 1 (origin)
rank 2 (origin)
![Page 26: TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE … · TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE MESSAGES ... James Dinan, David Goodell, Pavan Balaji, Rajeev Thakur,](https://reader033.vdocuments.mx/reader033/viewer/2022051308/5afadab37f8b9a32348e4a30/html5/thumbnails/26.jpg)
Network
Integrated Idea Shared-memory
NODE 0 NODE 1
26
rank 0 (target)
rank 1 (origin)
rank 2 (origin)
![Page 27: TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE … · TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE MESSAGES ... James Dinan, David Goodell, Pavan Balaji, Rajeev Thakur,](https://reader033.vdocuments.mx/reader033/viewer/2022051308/5afadab37f8b9a32348e4a30/html5/thumbnails/27.jpg)
Network
Integrated Idea Shared-memory
NODE 0 NODE 1
27
rank 0 (target)
rank 1 (origin)
rank 2 (origin)
![Page 28: TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE … · TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE MESSAGES ... James Dinan, David Goodell, Pavan Balaji, Rajeev Thakur,](https://reader033.vdocuments.mx/reader033/viewer/2022051308/5afadab37f8b9a32348e4a30/html5/thumbnails/28.jpg)
Network
Integrated Idea Shared-memory
NODE 0 NODE 1
28
rank 0 (target)
rank 1 (origin)
rank 2 (origin)
![Page 29: TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE … · TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE MESSAGES ... James Dinan, David Goodell, Pavan Balaji, Rajeev Thakur,](https://reader033.vdocuments.mx/reader033/viewer/2022051308/5afadab37f8b9a32348e4a30/html5/thumbnails/29.jpg)
Network
Integrated Idea Shared-memory
NODE 0 NODE 1
29
rank 0 (target)
rank 1 (origin)
rank 2 (origin)
![Page 30: TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE … · TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE MESSAGES ... James Dinan, David Goodell, Pavan Balaji, Rajeev Thakur,](https://reader033.vdocuments.mx/reader033/viewer/2022051308/5afadab37f8b9a32348e4a30/html5/thumbnails/30.jpg)
PERFORMANCE
![Page 31: TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE … · TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE MESSAGES ... James Dinan, David Goodell, Pavan Balaji, Rajeev Thakur,](https://reader033.vdocuments.mx/reader033/viewer/2022051308/5afadab37f8b9a32348e4a30/html5/thumbnails/31.jpg)
Stencil Benchmark
31
Number of processes: 2x2 to 20x20, 8 processes per node (“Fusion” Cluster at ANL: 320 nodes, 8 cores per node, QDR Infiniband)
![Page 32: TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE … · TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE MESSAGES ... James Dinan, David Goodell, Pavan Balaji, Rajeev Thakur,](https://reader033.vdocuments.mx/reader033/viewer/2022051308/5afadab37f8b9a32348e4a30/html5/thumbnails/32.jpg)
Graph500 Benchmark
32 (TEPS: Traversed Edges Per Second, higher is better)
• Breadth-First Search (BFS) • Large number of small messages to random targets • Optimization: origin process accumulates local data and sends only one message to each target process
![Page 33: TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE … · TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE MESSAGES ... James Dinan, David Goodell, Pavan Balaji, Rajeev Thakur,](https://reader033.vdocuments.mx/reader033/viewer/2022051308/5afadab37f8b9a32348e4a30/html5/thumbnails/33.jpg)
Conclusion • Why to support Active Messages in MPI • Leverage MPI RMA Interface • API Design • Asynchronous processing
• Intra-node messages • Inter-node messages
• Stencil benchmark and Graph500 benchmark
33
![Page 34: TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE … · TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE MESSAGES ... James Dinan, David Goodell, Pavan Balaji, Rajeev Thakur,](https://reader033.vdocuments.mx/reader033/viewer/2022051308/5afadab37f8b9a32348e4a30/html5/thumbnails/34.jpg)
QUESTIONS
![Page 35: TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE … · TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE MESSAGES ... James Dinan, David Goodell, Pavan Balaji, Rajeev Thakur,](https://reader033.vdocuments.mx/reader033/viewer/2022051308/5afadab37f8b9a32348e4a30/html5/thumbnails/35.jpg)
BACKUP SLIDES
![Page 36: TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE … · TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE MESSAGES ... James Dinan, David Goodell, Pavan Balaji, Rajeev Thakur,](https://reader033.vdocuments.mx/reader033/viewer/2022051308/5afadab37f8b9a32348e4a30/html5/thumbnails/36.jpg)
Message Passing Models • Two-sided communication
• Explicit sends and receives • One-sided communication
• Explicit sends, implicit receives • Simple operations on remote process
• Active Messages • Explicit sends, implicit receives • User-defined operations on remote process
36
![Page 37: TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE … · TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE MESSAGES ... James Dinan, David Goodell, Pavan Balaji, Rajeev Thakur,](https://reader033.vdocuments.mx/reader033/viewer/2022051308/5afadab37f8b9a32348e4a30/html5/thumbnails/37.jpg)
Active Message Models • Libraries and runtime systems
• Low level: GASNet, IBM’s LAPI, DCMF and PAMI, AMMPI
• High level: Charm++, X10 • Middle: AM++
AM++ AMMPI
![Page 38: TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE … · TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE MESSAGES ... James Dinan, David Goodell, Pavan Balaji, Rajeev Thakur,](https://reader033.vdocuments.mx/reader033/viewer/2022051308/5afadab37f8b9a32348e4a30/html5/thumbnails/38.jpg)
MPI RMA Interface • Two synchronization modes
Unlock
origin target
ACC(X) epoch
Lock PUT(Y)
epoch
Complete
origin target
ACC(X)
Start PUT(Y)
Post
Wait
Passive Target Mode (Lock-Unlock)
Active Target Mode (PSCW, Fence)
![Page 39: TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE … · TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE MESSAGES ... James Dinan, David Goodell, Pavan Balaji, Rajeev Thakur,](https://reader033.vdocuments.mx/reader033/viewer/2022051308/5afadab37f8b9a32348e4a30/html5/thumbnails/39.jpg)
Inter-node Communication
39
• Separate sockets for Active Messages
origin_sc
origin_sc rma_sc
rma_sc main thread
AM thread
![Page 40: TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE … · TOWARDS ASYNCHRONOUS AND MPI-INTEROPERABLE ACTIVE MESSAGES ... James Dinan, David Goodell, Pavan Balaji, Rajeev Thakur,](https://reader033.vdocuments.mx/reader033/viewer/2022051308/5afadab37f8b9a32348e4a30/html5/thumbnails/40.jpg)
Inter-node Communication • Separate sockets for Active Messages
origin_sc
origin_sc rma_send_sc
rma_send_sc
main thread AM thread
rma_recv_sc
rma_recv_sc
rma_resp_sc
rma_resp_sc