compi: enhancing mpi based applications performance and scalability using run-time compression. rosa...
TRANSCRIPT
CoMPI: Enhancing MPI based applications performance and
scalability using run-time compression.
Rosa Filgueira, David E.Singh, Alejandro Calderón and Jesús Carretero
University Carlos III of Madrid.
Summary
• Problem description• Main objectives• CoMPI• Study of compression algorithms. • Evaluation of CoMPI • Results• Conclusions
Summary
• Problem description• Main objectives• CoMPI• Study of compression algorithms. • Evaluation of CoMPI • Results• Conclusions
Main objectives (2/2)
CoMPI: Optimization of MPI communications by using
compression.
• Compression in all MPI primitives.• Fit any MPI application.• Transparent to user.• Run-time compression.
• Studding of compression algorithms.• Selecting the best algorithm based on
message characteristics.
Summary
• Problem description• Main objectives• CoMPI
– How we have integrated compression into MPI– Set of compression algorithms proposed
• Study of compression algorithms. • Evaluation of CoMPI • Results• Conclusions
Compression of MPI Messages (2/2)
• Header in the exchanged message to inform:– Compression used or not, algorithm and length.
• All compression algorithms are included in a single Compression Library:– CoMPI can be easily updated .– New compression algorithms can be included .
Summary
• Problem description• Main objectives• CoMPI• Study of compression algorithms.
– Conclusion of compression study. • Evaluation of CoMPI • Results• Conclusions
Study of compression algorithms (1/7)
• To select the most appropriated algorithm for each datatype based on:– Buffer size.– Redundancy level.
• To Increase the transmission speed by using compression depends on:– Number of bits sent.– Time required to compress.– Time required to decompress.
€
Speedup =Time _ Sent _Orig.
Time _ Sent _Compr.+ time_compress.+ time_ decompr.
Study of compression algorithms (2/7)
• For each algorithm, datatype, buffer size and redundancy level we will study theComplexity and Compression ratio.
Study of compression algorithms (7/7)
• Double precision WITH pattern: Data sequence 50001.0, 50003.0 , 50005.0 …
Summary
• Problem description• Main objectives• CoMPI• Study of compression algorithms. • Evaluation of CoMPI • Results• Conclusions
Summary
• Problem description• Main objectives• CoMPI• Study of compression algorithms. • Evaluation of CoMPI • Results
– Real Applications– Benchmarks
• Conclusions
Results (4/5)
• IS :– Integer data. – Improves to x1.2 with LZO.– Rice obtains good results with 32 processes.
Results (5/5)
• LU:– Double precision.– No better performance. Only with 64 processes
by using FPC we obtain a speedup of x1.1
Summary
• Problem description• Main objectives• CoMPI• Study of compression algorithms. • Evaluation of CoMPI • Results• Conclusions
– Principal Conclusion .– On going.
Principal conclusions (1/2)
• New Compression library integrated into MPI using MPICH distribution CoMPI.
• CoMPI includes five different compression algorithms and compress all MPI primitives.
• Main characteristics: – Transparent for the users.– Fit any application without any change in it.
• We have evaluated CoMPI using:– Synthetic traces.– Real applications.
Principal conclusion (2/2)
• The results of evaluations demonstrated that in most of the cases, the compression:– Reduce the overall execution time.– Enhance the scalability.
• When compression is not appropriated:– Little performance degradation.