software overhead in messaging layers pitch patarasuk
TRANSCRIPT
![Page 1: Software Overhead in Messaging Layers Pitch Patarasuk](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0241a28abf838cd4e4b/html5/thumbnails/1.jpg)
Software Overhead in Messaging Layers
Pitch Patarasuk
![Page 2: Software Overhead in Messaging Layers Pitch Patarasuk](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0241a28abf838cd4e4b/html5/thumbnails/2.jpg)
Messaging Layer
• Bridge between the hardware functionality and the user communication requirement
• Network hardware feature– Arbitrary delivery order– Finite buffering– Limited fault handling
• User communication requirement– In-order delivery– End-to-end flow control– Reliable transmission
![Page 3: Software Overhead in Messaging Layers Pitch Patarasuk](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0241a28abf838cd4e4b/html5/thumbnails/3.jpg)
Messaging Layer (cont.)
![Page 4: Software Overhead in Messaging Layers Pitch Patarasuk](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0241a28abf838cd4e4b/html5/thumbnails/4.jpg)
Software overhead
• Main communication cost = hardware cost + software cost
• Software cost dominates hardware cost
• 50-70% of the software cost are the direct gap between the network features and user requirements
![Page 5: Software Overhead in Messaging Layers Pitch Patarasuk](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0241a28abf838cd4e4b/html5/thumbnails/5.jpg)
Software overhead in messaging layers
• Analyze the costs of communication functionality and network features
• What overhead might be reduced if the underlining network provide higher level of service
![Page 6: Software Overhead in Messaging Layers Pitch Patarasuk](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0241a28abf838cd4e4b/html5/thumbnails/6.jpg)
CM-5 active message layer (CMAM)• CM-5
– Parallel machine up to a few thousand nodes connected with an incomplete fat tree topology
• Active message
– Communication mechanism intended to expose full hardware flexibility and performance of modern networks
– Each message contain an address of a user-level handler which is executed on message arrival with the message body as an argument
– The handler extract data from the network and integrate it into the ongoing computation
• CMAM vs Send/Recv– User direct access to network, no OS involve, the data go directly to the
user space computation
![Page 7: Software Overhead in Messaging Layers Pitch Patarasuk](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0241a28abf838cd4e4b/html5/thumbnails/7.jpg)
Software overhead cost analysis
• Consider implementation of 3 protocols– Single-packet delivery– Finite sequence, multi-packet delivery– Indefinite sequence, multi-packet delivery
• Use instruction counts for measurement
![Page 8: Software Overhead in Messaging Layers Pitch Patarasuk](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0241a28abf838cd4e4b/html5/thumbnails/8.jpg)
Single-packet delivery
Description Source Destination
Call/Return 3 10
NI Setup - -
Write to NI 2 -
Read from NI - 3
Check NI status 7 12
Control Flow 3 2
20 27
![Page 9: Software Overhead in Messaging Layers Pitch Patarasuk](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0241a28abf838cd4e4b/html5/thumbnails/9.jpg)
Finite sequence, multi-packet delivery
![Page 10: Software Overhead in Messaging Layers Pitch Patarasuk](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0241a28abf838cd4e4b/html5/thumbnails/10.jpg)
Indefinite sequence, multi-packet delivery
![Page 11: Software Overhead in Messaging Layers Pitch Patarasuk](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0241a28abf838cd4e4b/html5/thumbnails/11.jpg)
Message size = 16 words
![Page 12: Software Overhead in Messaging Layers Pitch Patarasuk](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0241a28abf838cd4e4b/html5/thumbnails/12.jpg)
Message size = 1024 words
![Page 13: Software Overhead in Messaging Layers Pitch Patarasuk](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0241a28abf838cd4e4b/html5/thumbnails/13.jpg)
Messaging layer with high-level network feature
• Given that CMAM is considered to be very efficient, there are 2 choices to reduce software overhead– Lower user requirement– Raise level of service provided by the network
• Compressionless Routing (CR)– Order-preserving transmission– Deadlock freedom independent of packet acceptance
guarantees– Fault-tolerant transmission at packet level
![Page 14: Software Overhead in Messaging Layers Pitch Patarasuk](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0241a28abf838cd4e4b/html5/thumbnails/14.jpg)
Single-packet delivery
• Has the same cost as the previous CMAM case
• However, it is guaranteed to be fault-free, no deadlock or buffer overflow
![Page 15: Software Overhead in Messaging Layers Pitch Patarasuk](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0241a28abf838cd4e4b/html5/thumbnails/15.jpg)
Finite sequence, multi-packet delivery
![Page 16: Software Overhead in Messaging Layers Pitch Patarasuk](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0241a28abf838cd4e4b/html5/thumbnails/16.jpg)
Indefinite sequence, multi-packet delivery
![Page 17: Software Overhead in Messaging Layers Pitch Patarasuk](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0241a28abf838cd4e4b/html5/thumbnails/17.jpg)
Message size = 16 words
![Page 18: Software Overhead in Messaging Layers Pitch Patarasuk](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0241a28abf838cd4e4b/html5/thumbnails/18.jpg)
Message size = 1024 words
![Page 19: Software Overhead in Messaging Layers Pitch Patarasuk](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0241a28abf838cd4e4b/html5/thumbnails/19.jpg)
Discussion
• Larger packet sizes– Reduce overhead, especially for indefinite-sequence
protocol
• Improved network interfaces and DMA hardware– Network interface:
• only make basic cost faster, but not protocol cost in messaging layer
• Make it more important for messaging layer to be effective
– DMA:• only reduce cost in moving large amounts of data
![Page 20: Software Overhead in Messaging Layers Pitch Patarasuk](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0241a28abf838cd4e4b/html5/thumbnails/20.jpg)
Discussion (Cont.)
• Implication of network design– Improving routing performance may increase software
cost, e.g. out-of-order delivery
• Reducing communication features– Fault-tolerance, in-order-delivery
– Put burden on parallel software programmer
• Use instruction counts for measurement instead of latency– Latency is hard to measure in a portable fashion
![Page 21: Software Overhead in Messaging Layers Pitch Patarasuk](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0241a28abf838cd4e4b/html5/thumbnails/21.jpg)
Conclusion
• Software overhead is much larger than routing time and 50-70% are messaging layer overhead
• Software overhead in messaging layer is the cost of the gap between the network feature and the user requirement
• This cost can be reduced to zero if the underlining network provides functionality that the user requires.
![Page 22: Software Overhead in Messaging Layers Pitch Patarasuk](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697c0241a28abf838cd4e4b/html5/thumbnails/22.jpg)
Questions?