development environments for hpc: the view from...
TRANSCRIPT
Development Environments for HPC: The View from NCSA
Jay Alameda National Center for Supercomputing Applications, University of Illinois at Urbana-Champaign
DEHPC ‘15
San Francisco, CA 18 October 2015
Acknowledgements ª Portions of this material are supported by or based upon work supported by the
Defense Advanced Research Projects Agency (DARPA) under its Agreement No. HR0011-07-9-0002, the United States Department of Energy under Contract No. DE-FG02-06ER25752, the Blue Waters sustained petascale computing project, which is supported by the National Science Foundation under award number OCI 07-25070, and the SI2-SSI Productive and Accessible Development Workbench for HPC Applications, which is supported by the National Science Foundation under award number OCI 1047956
ª The SI2-SSI team is lead by Jay Alameda (NCSA), Greg Watson (IBM), Steven Brandt (LSU), and Allen Malony (U Oregon). Team members and senior personnel include Beth Tibbitts (IBM), Ralph Johnson (U Illinois), Chris Navarro (NCSA), Sameer Shende (U Oregon), Wyatt Spear (U Oregon), Brian Jewett (U Illinois), Galen Arnold (NCSA), and Rui Liu (NCSA)
ª Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation
1
Outline
ª Overview of Eclipse and the Parallel Tools Platform (PTP)
ª Motivation for Workbench for High Performance Computing (WHPC)
ª Advantages of Eclipse for HPC Software Developers
ª Sustainability, Investment, Collaboration: Building Community
ª Challenges with DEHPC
ª Opportunities (for discussion)
2
Eclipse
ª Integrated development environment (IDE) ª Edit code, compile, run, debug without leaving Eclipse ª Graphical user interface
3
Eclipse
4
Eclipse
ª Integrated development environment (IDE) ª Edit code, compile, run, debug without leaving Eclipse ª Graphical user interface
ª Plug-ins add new functionality to Eclipse
ª Support languages: C/C++, Fortran, Python, … ª Support version control systems: Git, Subversion, … ª Support issue tracking systems: Bugzilla, Jira, … ª Support HPC development (?!)
5
Parallel Tools Platform (PTP)
ª PTP: a set of plug-ins that extend Eclipse to support HPC development ª Write code on your laptop
ª Build it on a remote HPC system using its compilers
ª Run it on the remote HPC system (qsub)
ª Debug it on the remote system (MPI debugger)
ª Tune it for performance on the remote system
ª …all inside Eclipse! 6
7
Eclipse Parallel Tools Platform (PTP) Coding & Analysis
Code completion
8
Launching & Monitoring
Eclipse Parallel Tools Platform (PTP) Coding & Analysis
9
Launching & Monitoring
Eclipse Parallel Tools Platform (PTP)
Debugging
Coding & Analysis
10
Launching & Monitoring
Eclipse Parallel Tools Platform (PTP)
Debugging
Coding & Analysis
Performance Tuning
11
Launching & Monitoring
Eclipse Parallel Tools Platform (PTP)
Debugging
Coding & Analysis
Performance Tuning
Motivation for Workbench for High Performance Computing (WHPC)
ª Context ª NSF 1047956: SI2-SSI: A Productive and Accessible
Development Workbench for HPC Applications Using the Eclipse Parallel Tools Platform
ª Stable, portable platform for tool development ª Focus on tool functionality, manage rapid evolution of HPC
platforms ª Encourage consistent tool look and feel ª Support for HPC application development practices
ª Why Parallel Tools Platform? ª High potential to meet needs of a WHPC. ª Target next generation of HPC developers growing up with
IDEs (Eclipse, Visual Studio, ...) ª Need to cultivate community of users!
12
Improvements
ª Work within Eclipse release cycle ª Major (API-breaking) improvements with coordinated
June release ª Last major release Eclipse 4.5 “Mars” released
June 2015 ª Minor enhancements and bug-fixes with two
coordinated service releases in September and February ª Eclipse 4.5 SR1 Released September, 2015.
ª Foci of improvements ª Improve usability ª Improve productivity
13
Consider 2 possible types of users of Eclipse Parallel Tools … ª Science code users/modelers
ª Need to build science code ª May need to modify science code (and rebuild)
ª Software specialists enabling modeling projects ª Lots of software engineering concerns
ª Next set of slides address some of those concerns.
14
Science code users/modelers
ª Some of the challenges ª Complex codes (eg WRF) ª Codes + HPC architectures can be daunting ª Adding user code not always easy
ª WRF: from http://wrf-model.org/PRESENTATIONS/2000_04_18_Klemp/sld007.htm
15
Navigating Computational Science Codes
16
Code navigation
Syntax-aware editing (navigate to program units and declarations)
Code Outline
Eclipse aiding in a typical code workflow…
ª May want to add a model output variable ª Eclipse PTP makes it easy to navigate source,
make changes ª Drive remote builds on HPC resources
ª Make, autotools…
ª Run – can generate a run configuration for particular system, batch environment
17
Software Specialists enabling modeling projects
ª Need a wider array of software engineering tools ª Source repository ª Issue tracking ª Documentation ª Performance tuning…
ª Eclipse Parallel Tools can help with many of these concerns
18
Source Code Control: “Team” Features
ª Eclipse supports integration with multiple version control systems (VCS) ª CVS, SVN, Git, and others ª Collectively known as “Team” services
ª Many features are common across VCS ª Compare/merge ª History ª Check-in/check-out
ª Some differences ª Version numbers ª Branching
19
Issue Tracking
ª Mylyn Bridge ª Tracks tasks, links to source and bug repositories
20
Connections to Jira, bugzilla, …
Eclipse Documentation
ª Eclipse Help System – built in and standalone (http://help.eclipse.org)
21
Performance Tuning: PTP TAU plug-ins http://www.cs.uoregon.edu/research/tau
ª TAU (Tuning and Analysis Utilities) ª First implementation of External Tools Framework (ETFw) ª Eclipse plug-ins wrap TAU functions, make them available
from Eclipse ª Full GUI support for the TAU command line interface ª Performance analysis integrated with development
environment
22
Sustainability, Investment, Collaboration: Building Community
ª Stakeholders/investors ª Funding agencies à innovative capability ª HPC vendors
ª contribute to open source foundational software ª basis for proprietary or open-source add-ons
ª Supercomputing centers ª user support and training ª xml documents for batch system and local policy
integration
23
Sustainability, Investment, Collaboration: Building Community II ª Stakeholders/investors continued
ª Supercomputing integrating organizations (eg, XSEDE): ª user support ª single sign-on support (?) ª training ª education support
ª Tool providers ª xml documents for integration of command line
tools ª extra value-add from tool-specific plugins?
ª Anyone else?
24
Challenges for DEHPC
ª Budgetary pressure on supercomputing centers and integrating organizations
ª Challenges successfully competing proposals in open calls (eg, NSF SI2). ª Innovation, impact are paramount here
ª User community is large, disperse (eg lots of downloads of PTP, hard to connect to user community)
25
Opportunities (for discussion)
ª Can we build a community to support a DEHPC? ª PIs proposing to funding agencies for innovative
additions? ª Vendor contributions to DEHPC? ª Supercomputing centers and integrating
organizations support? ª Eg, User support, training, configuration (xml)
documents
26
Eclipse Parallel Tools Platform
www.eclipse.org/ptp
27