Issue 60, summer 2007The newsletter of ePCC: europe’s premier high-performance computing and technology transfer centre.

multi-skilled, multi-talented: education at ePCC

In this issue...

msc graduate Zhaoyang Dong plays the pipa at the reception for graduates.

2 HeCTor signing

3 HPC-europa Access Grid surgeries

4 FHPCA: maxwell supercomputer and new training course

6 OGsA-DAI 3.0 on its way

7 FilmGrid: Grid technology for the film industry

8 Cancer research project msc in High Performance Computing

9 PhD students at ePCC

10 msc in High Performance Computing update

11 msc graduations

12 2006 msc graduates: what happened next?

14 eDIKT 2 on tour ePCC at the edinburgh International science Festival

15 event reports

16 ePCC High end Computing training

2 3

Training and education

Britain’s new national supercomputing service, HECToR, will be based on a large Cray system located at the University of Edinburgh’s Advanced Computing Facility (ACF). EPCC will direct and operate the service.

Contracts for the £113-million project, which will last for six years, were signed in February. HECToR (High End Computing Terascale Resources) will be funded by the UK Research Councils, and will start work in October this year. Computational science and engineering support will be provided by NAG Ltd.

The service will be delivered in 3 phases. First phase hardware will include a total of 11,328 processors with more than 33 TB of main memory, giving a peak performance of 59 Tflops - equivalent to about 10,000 desktop PCs.

HECToR will be available for scientists at UK universities. Areas of work which it will support include:• Ocean currents, weather forecasting and climate change• Materials science• Chemistry, Biochemistry and drug development• Computational physics• Modelling disasters and emergency response

• New sources of energy, such as wave, wind and fusion• Aircraft design.

As the problems which scientists address have become more complex, the importance of high-performance computing as a research technique has increased. Prof Arthur Trew, director of EPCC said: ‘Traditionally progress in science has been made through theory and experiment, but an increasing range of problems now require to be simulated computationally. HECToR is critical for UK scientists to compete internationally.’

EPCC has been providing national supercomputing services since the early nineties. The Cray T3D/T3E service, which ran from 1994 to 2002, delivered 309 Gflops at its peak; the first phase of HECToR will be about 190 times as powerful. EPCC also directs the current HPCx national service. As Arthur Trew said: ‘We are delighted that EPCC has again been chosen to manage this facility. The choice of Edinburgh demonstrates the University’s leadership in the field.’

HECToR: www.hector.ac.uk EPSRC: www.epsrc.ac.uk/hpc

EPCC News

editorial Catherine Inglis

HeCTor signingJohn Fisher

Signing the contracts: Allan Digance, Assistant

Director of Finance, University of Edinburgh; Tim O’Shea, Principal of University of Edinburgh; Alan Simpson, a director

of HPCX UoE Ltd

2 3

Training and education

The next issue of EPCC News will be available in electronic and print versions. Subscribers now have two options: the electronic version only; or the electronic and print versions. You can also choose to be removed completely from our mailing list.

To change your subscription, please send your request, along with your name and email address to: subscriptions@epcc.ed.ac.uk

AccessGrid is a central technology within HPC-Europa, being the focus of one of its sub-projects (Collaboration Support Infrastructure and Tools in AccessGrid, developing tools to aid remote collaborative research), as well as the means by which regular project meetings and selection panel meetings are held. These AccessGrid meetings bring together participants in several different European countries, without them having to leave their workplace.

HPC-Europa runs a regular series of ‘surgeries’ via AccessGrid, at which an expert from one of the partner centres gives a presentation on an HPC-related topic. Visitors are invited to submit questions in advance, so that the expert can cover any areas of particular interest to the visitors.

Users can attend the surgeries from any of the six European centres running the visitor programme. The sharing of expertise between centres in this way not only benefits the visitors, who are offered more learning opportunities as a result, but also the staff, who have also found the sessions fruitful.

The AccessGrid surgeries have covered many topics of general interest to HPC users, such as MPI and OpenMP programming, performance optimisation and parallel I/O, as well as more specialised topics, such as Paraver (a performance analysis tool developed by Barcelona Supercomputing Centre, a partner in HPC-Europa), program optimisation for single processor performance, and data archives in distributed environments.

The most recent surgery, AccessGrid Tools and Technology, was presented jointly by EPCC and HLRS (Stuttgart), and was on the subject of the AccessGrid itself. The experts explained the basics of the technology and demonstrated the AccessGrid tools developed within HPC-Europa. They also provided some details on the requirements for setting up an AccessGrid node.

For more information about the HPC-Europa visitor programme or the work on Collaboration Support Infrastructure and Tools in AccessGrid, see: www.hpc-europa.org/

The next closing dates for applications to the HPC-Europa visitor programme are 15th May and 31st August.

AccessGrid is a sophisticated means of internet video-conferencing which allows users to share applications such as PowerPoint presentations, virtual whiteboards and web-browsing sessions.

sharing HPC knowledge across europe Adam Carter

Change to ePCC News delivery

HPC-europa Access Grid surgeries

4 5

Training and education

This spring saw the switch-on of the FPGA High-Performance Computing Alliance’s (FHPCA’s) massively-parallel, reconfigurable supercomputer Maxwell. The FHPCA partners – Algotronix, Alpha Data, EPCC, the Institute for System Level Integration, Nallatech, Scottish Enterprise and Xilinx – have successfully delivered this large-scale demonstration of the potential of high-performance reconfigurable computing (HPRC). Maxwell – a traditional name here at Edinburgh for cutting edge research computers – lives!

The FHPCA Supercomputer project began early in 2005 with the goal of linking together 64 FPGA (field-programmable gate array) devices into a parallel computer. Our aim was to exploit the tremendous potential of FPGAs for high-performance computing in a way that is as efficient and portable as possible.

HPRC and FPGAs have been the buzz at Supercomputing conferences for a few years now, but why? What’s the big deal here? Well, high-performance computing (HPC, no ‘R’) is renowned for its insatiable demand for compute cycles. Certain grand challenges in science and engineering will use all the computing power they can get their hands on and still come back for more. The trouble is, with the very real physical barriers now appearing in chip manufacture, where is this power going to come from tomorrow? Multi-core architectures, say some chip vendors; reconfigurable devices say others.

FPGAs can be reconfigured at runtime with a whole algorithm encoded as an electronic circuit – your algorithm, perhaps. If you’re lucky your whole HPC application can be cast into a highly efficient single circuit and zapped onto the FPGA, using exactly the instructions it needs and no more. This tremendous efficiency at low (electrical) power is what attracts the cycle-hungry users of traditional HPC. For the right kind of application kernel FPGAs can offer speedups of two to three orders of magnitude over current best microprocessors.

There are several approaches to HPRC on the market today; vendors like SGI, Cray and SRC have been selling FPGA-based hardware acceleration capabilities for a while. Many current solutions take a ‘co-processor’ approach or an ‘acceleration library’ approach, deploying standard functional units in FPGA logic to accelerate applications running still in large part on attendant microprocessors.

The FHPCA approach to building Maxwell has been to investigate the suitability of FPGAs as main processors not coprocessors. To this end Maxwell comprises 64 Xilinx Virtex-4 FPGAs each with four RocketIO connections, allowing up to 3.125 Gb/s bandwidth point to point in four directions. This whole is hosted in an IBM Bladecentre cluster of 32 diskless Xeon-based blades. Our goal, if you like, is to use the FPGAs as main compute nodes and the Xeons as coprocessors.

It lives!The FPGA supercomputer maxwell turns on Rob Baxter

It’s been a real challenge but we’re there.

4 5

Training and education

New training course: FPGA supercomputingRob Baxter

As reported opposite, EPCC has begun the final commissioning process for the FPGA HPC Alliance (FHPCA)’s supercomputer, Maxwell. Based on a direct-connect massively-parallel model of FPGA computing, Maxwell will be a tricky beast to get to grips with. Consequently EPCC is delighted to have received additional support from FHPCA sponsors Scottish Enterprise to develop a full training course for potential users of the machine.

The training course currently under development with FHPCA partners Nallatech will cover the spectrum of issues in high-performance reconfigurable computing – everything from the low-level nitty-gritty of FPGAs to the effective exploitation of

arrays of these devices for HPC applications, with a particular emphasis on the hardware-neutral approach adopted by the FHPCA in its Parallel Toolkit.

The material will be presented as a series of lectures with accompanying practical sessions, grouped into days. We aim to keep each day’s material as self-contained as possible so that potential users can dip in, based on their existing experience of FPGAs or HPC. All materials will be provided for download from both EPCC’s webpages and the FHPC Alliance’s website, free of charge for non-commercial use.

On the software side of things, rather than take the approach of building a library of generic FPGA cores for linear algebra and the like – a BLAS-for-FPGA approach – we took the view that getting the most performance out of the FPGAs really means optimising memory bandwidth.

Most scientific and technical HPC codes tend to be limited not by the speed of the microprocessors they run on but by the speed of the memory subsystems – keeping the codes fed with data is the key. HPRC faces the same problems, only more so. FPGAs can, in principle, run codes hundreds of times faster than their microprocessor rivals – but they use the same DDR SRAM and DRAM memory systems. What’s more, programs that run on an FPGA machine still start off by running on the host microprocessors (and their main memory) and then transfer control – and all necessary data – to the FPGAs to perform the compute-intensive parts. Communication between host microprocessor and FPGA can be over a fast network fabric, or it can be over PCI – either way it’s a lot slower than communication between the microprocessor and its main memory, especially when sophisticated cache systems are factored in.

The danger of the BLAS-for-FPGA approach is that too much time is spent transferring data between microprocessor and FPGA over less-than-ideal channels (in the worst case, PCI

compared to L1 cache) – thus killing any performance gains stone dead. The approach we have adopted to programming Maxwell involves converting as much of the key application code as possible to run on the FPGAs, ideally ensuring there is one big data transfer at the start but that the rest of the computational load is worth it – once the data are FPGA-side the FPGAs can crunch through them, exchanging data with each other in full parallel fashion as required, until the problem is solved. No swapping large arrays back and forth between FPGA and microprocessor.

To illustrate this approach the FHPCA has ported three real application demonstrators to Maxwell – real commercial codes from the oil and gas and medical imaging sectors and a typical simulation code in financial services. In each demo significant effort has been spent in getting as much of the application as possible to run on the FPGA, leaving the microprocessors to do little more than start jobs running and wrap them up at the end. Early performance results are very encouraging, with all demos showing at least a factor of 6 performance improvement per node over 3 GHz Xeon systems. Once full scalability tests across the 64-node system are complete later this year we expect to have a very interesting story to tell about large-scale reconfigurable computing for HPC!

6 7

Training and education

OGSA-DAI 3.0’s features will include:OGSA-DAI 3.0 will now allow users to:OGSA-DAI: www.ogsadai.org.uk. OMII-UK: www.omii.ac.uk.

Coming soon! OGsA-DAI 3.0Mike Jackson

Some of the OGSA team at EPCC.

6 7

Training and education

In our summer report of the BEinGRID kick-off we deferred discussion of FilmGrid, the Business Experiment led by EPCC. It is an 18-month, EC-funded collaboration aiming to enrich the film post-production process with bespoke, Grid-based solutions that offer secure, reliable data movement and ensure efficient, repeatable workflows. With EPCC the data-management expert, the project benefits from the involvement of two UK-based, motion-picture specialists: The Hat Factory [1] and Joe Dunton and Co. [2].

The Hat Factory was set up as a completely new operation for Digital Film Grading and Finishing. Being recently established, it has state-of-the-art equipment for film grading and outputting. The Hat Factory has been engaged to provide services to the veteran film director, Dino de Laurentiis, for his next trilogy of films, which have a total production budget of over one half of a billion dollars (The Decameron: Angels & Virgins, The Last Legion, and Hannibal Rising). Their facility contains the ‘Kubrick Theatre’, Soho’s largest screen post-production projection facility for film and digital material.

Joe Dunton and Co. is a well-established company specialising in the rental of cameras, lenses, grip and video assist equipment to the UK, US and international motion picture industries. The company is extremely well-known in the motion picture industry. For example, it provided cameras, cranes and grip equipment to the fifth movie in the Harry Potter franchise, The Order of the Phoenix.

The two companies work closely together. Managing the transfer and workflow of digital data in the motion picture industry is of crucial importance as the industry moves towards full digital cinematography. By utilising Grid and work scheduling technologies the companies believe they can significantly improve the creative process.

The project is well suited to BEinGRID’s over-arching goal to showcase the benefits of integrating established Grid technologies into small- and medium-sized enterprises. In a motion-picture production, the data and resources are distributed amongst various locations and collaborating partners. Presently, management and monitoring of these resources are done in an ad hoc and inefficient manner (eg by making telephone calls to numerous partners and locations). FilmGrid will create a Grid solution that could be used from production to production. We will endeavour to minimise data movement, while maximising data visibility and accessibility with our bespoke browser for distributed assets.

We have just finished our general design, including GUI prototypes demonstrating the new features. The project runs until February 2008, when a usable proof-of-concept implementation is expected.

[1] http://thehatfactory-dmc.com/ [2] http://www.joedunton.co.uk/

21st Century FilmGridKostas Kavoussanakis

Training and education

EPCC offers a one-year taught MSc in HPC. This well-established programme provides an excellent grounding in HPC technologies and their practical application.

EPCC has an international reputation as a leading centre of expertise in HPC and Grid computing. EPCC supports HPCx, which is the UK’s flagship academic supercomputer service and one of the largest machines in Europe. EPCC also hosts an IBM Blue Gene system, the architecture used to build the most powerful supercomputer in the world today.

This MSc will appeal if you have a keen interest in programming and would like to learn about HPC and parallel programming. The course has a strong practical focus and you will have access to leading-edge HPC platforms and technologies. Applications are encouraged from people who are competent programmers in Java, C++, C or Fortran, and are graduates in science, engineering, computer science and mathematics or are currently working in a relevant field. The entrance requirement is a good honours degree or equivalent work experience. No prior HPC knowledge is assumed.

For more information and application details see: www.epcc.ed.ac.uk/msc Or email msc@epcc.ed.ac.uk

msc in High Performance Computing

The MSc can also be undertaken on a part-time basis by PhD students at other UK universities, through the EPSRC-sponsored High End Computing Centre at EPCC (www.epcc.ed.ac.uk/hec). Applications for this programme should be made by PhD supervisors.

Some of this year’s HEC students are pictured above (Geraldine McKenna, Alastair Smith, Emilia Tang, Martin Chorley).

We reported last year that EPCC has teamed up with the Colon Cancer Genetics Group (CCGG) at the University of Edinburgh to research the relationship between genetic markers and colorectal cancer. The ultimate goal of this long-term programme is to be able to identify individuals at risk of the

disease by examining their genetic make-up. At the same time, understanding the multiple genes involved will allow us to understand the biology and natural history of bowel cancer.

This project is an introduction to the ambitious goals of the programme. Last summer, we gained access to a unique and extensive set of 565,000 genetic markers with real data from 1000 cancer cases and 1000 matched controls. During the first phase we ported to our Blue Gene/L system, optimised and parallelised a Fortran serial code that investigates the effects of each genetic marker individually. This analysis was projected to require four months to run on a dedicated standard PC. After optimisations and parallelisation, it completed in 6.5 hours on 128 Blue Gene/L processors. The results are encouraging and the same computation will be run against a new set of data.

Research has moved on to the next stage, where we will investigate the interactions between pairs of different genetic markers. The problem size involved is extremely challenging. The full problem requires sorting 1.6 billion floating point numbers, occupying 1.2 terabytes. To address the challenges, we devised a complex, 2-dimensional decomposition of the data so as to fit the data on the Blue Gene/L memory and are working on a parallel heap-sort algorithm. Even with these optimisations, we will sort only a subset of the 1.6 billion probabilities we will calculate. The computation may exceed three months of run-time on as many as 1,024 Blue Gene/L nodes.

The project ended in February 2007, but follow-on projects are planned to continue the collaboration in the following years. Bowel cancer is one of the most common causes of cancer death in developed countries and it deserves our attention.

The primary research work, including patient recruitment and genetic analysis, is funded by Cancer Research UK, the Medical Research Council (MRC), the Scottish Executive and CORE. EPCC effort is additionally funded by the eDikt2 project. The CCGG is a University of Edinburgh research group based at the MRC Human Genetics Unit at the Western General Hospital in Edinburgh.

Bowel cancer research updateKostas Kavoussanakis

The MSc in HPC is supported by EPSRC, which offers a number of funded studentships for UK/EU students. SAAS funding is also available.

8

Training and education

Over the past three years, EPCC has established a successful PhD programme, taking on students who are either working wholly in EPCC, or on collaborative projects with other research groups, both inside the University and further afield.

Russell Sommerford was EPCC’s first PhD student, and is now in his third year. Russell is supervised by Mark Bull and is working on automated optimisation of programs. In particular, he is looking at the effects on program performance of the layout of data structures, and has developed optimisation methods based on a type of genetic algorithm for searching the potentially very large space of solutions for candidates which offer good performance. This work is particularly applicable to molecular dynamics simulations, where the data structures representing particles play a key role in the overall performance of the codes.

Owain Kenway’s PhD is part of a collaborative project with Prof. Peter Coveney at University College London, looking at molecular dynamics simulations of complex systems. This project is exploiting a set of world-leading HPC resources: in addition to access to the UK’s National HPC Resources and to local university systems, Owain has access to several supercomputers on the US TeraGrid and within Europe, ensuring this project has access to some of the biggest and fastest supercomputers in the world. The research harnesses these systems interactively through Grid technology, allowing the combined power of these diverse resources to be exploited through orchestrated workflows in an unprecedented manner to permit turn-around of exceptionally large simulations within hours. The position is associated with the EPSRC-funded

Reality Grid e-Science project (www.realitygrid.org), a project dedicated to high-performance scientific grid computing research.

Owain’s co-supervisor at EPCC is Lorna Smith, who is also a co-supervisor for Steven Shave. Steven’s research is in the area of Virtual Screening, a universally accepted approach in the pharmaceutical industry for identifying novel molecules as drug leads. Stephen is based in the Institute of Structural and Molecular Biology (ISMB), with Professor Malcolm Walkinshaw and Dr Paul Taylor. Much of his work is focused on developing the LIDAEUS code for massively parallel virtual screening. This is being used to predict the binding modes involved in the docking of small ligands to proteins.

Our most recent student is Tom Edwards, who is working on a project with UKAEA Fusion in Culham. He is co-supervised by Peter Knight at Culham and Joachim Hein in EPCC, and is supported by an EPSRC CASE award from Culham. This project aims to produce an efficient code for the simulation of the turbulent plasma flow inside a nuclear fusion reactor. By utilising a fluid model, the application is capable of simulating the entire fusion reactor. With the next generation fusion reactor ITER scheduled to become operational in 2016, there is at present a large interest in all aspects of fusion research, and this is a very timely project. Tom’s contributions are focused on the computational aspects of the project. This includes the investigation of the serial efficiency of the code, the development and implementation of the parallelisation strategy, and the use of efficient algorithms.

PhD students at ePCCMark Bull

Turbulent plasma flow inside a nuclear fusion reactor.

9

10 11

Most of our courses are run in the traditional way with weekly sessions over a whole semester. However, some are offered in intensive blocks, generally across three days. This style of delivery means that these courses are suitable for external attendees as well as for MSc students.

The MSc also includes a series of guest lectures with external speakers from academia and industry. If you are interested in giving a talk please get in touch!

The msc in High Performance Computing: what’s new?Judy Hardy

The msc in High Performance Computing (HPC) is now in its sixth year and is a well-established activity within ePCC. The courses that make up the core of the programme have remained essentially the same since the msc was first established. more recently, as an ePsrC-sponsored HeC Training Centre we have

been able to offer several new courses covering more specialised and advanced topics: ‘HPC Techniques in Computational Chemistry’ and ‘HPC Techniques in Computational engineering’, both taught by colleagues from CCLrC Daresbury Laboratory, and ‘Performance scaling on modern HPC Architectures’.

The MSc courses offered by EPCC this year are...

• Fundamental Concepts of HPC

• Practical Software Development

• Shared Memory Programming

• Message-Passing Programming

• Parallel Decomposition

• Performance Scaling on Modern HPC Architectures

• Object Oriented Programming for HPC

• Hardware, Compilers and Performance Programming

• Applied Numerical Algorithms

• Scientific Visualisation

• Advanced Topics in HPC and e-science

• HPC Techniques in Computational Chemistry

• HPC Techniques in Computational engineering

HPC Techniques in Computational Chemistry The course provides an overview of modern quantum and classical methods in computational chemistry, including an assessment of the accuracy and cost of the associated techniques. An overview of parallelisation strategies, focusing on the key implementation details of both quantum and simulation codes, is provided. The practical sessions provide students with the opportunity to perform both quantum and classical simulations on HPCx.

HPC Techniques in Computational Engineering This course includes an overview of CFD methods, including an assessment of incompressible and compressible techniques together with an overview of parallelisation strategies, focusing on both structured and unstructured grids. The key implementation details of a number of practical codes are provided. In the practical sessions students have the opportunity to perform CFD simulations on HPCx.

Performance Scaling on Modern HPC ArchitecturesThis course covers the detailed architecture of current HPC systems and discusses the challenges faced in obtaining good performance on large numbers of processors. The use of performance analysis tools is covered, and practical techniques for performance optimisation are introduced. Practical sessions give students experience both of using the tools and of applying optimisation techniques.

For more information on the MSc see: www.epcc.ed.ac.uk/msc Email msc@epcc.ed.ac.uk

Training and education

10 11

Training and education

On Tuesday 28th November 2006, EPCC hosted a pre-graduation sherry reception for the 2005/2006 graduates of our MSc in High Performance Computing.

One of our graduates, Zhaoyang Dong, gave a recital on the pipa (Chinese lute). This instrument gets it name from two Chinese words which describe movements used in playing: the word ‘Pi’ means ‘to play forward’ and ‘Pa’ means ‘to play backward’. The pipa is a traditional Chinese instrument with a long history, having been played for over two thousand years in China.

Zhaoyang played ‘Shi Mian Mai Fu’, which translates into English as ‘Ambush on all sides’. This piece describes the fierce fighting between Chu and Han in 202 BC when Liu Bang defeated Xiang Yu by creating ambushes on all sides. The second piece of music played was Auld Lang Syne, which Zhaoyang had modified to suit the pipa.

As well as her academic achievement in successfully undertaking the MSc in High Performance Computing, Zhaoyang is a very accomplished musician. We were honoured to have her play for us.

msc graduation receptionMargaret Jackson

EPCC recently awarded the first ever class medal from its MSc in High-Performance Computing to Derek Murray.

This medal was awarded in recognition of Derek’s outstanding all-round achievement in the MSc course, including the work done on his project on the use of OGSA-DAI to retrieve Binary Large Objects (BLOBs) from a relational database.

You will find Derek’s dissertation, along with all the other dissertations from last year, at: www.epcc.ed.ac.uk/msc/dissertations/2005-2006

Presentation of the first msc course medal Margaret Jackson

Derek Murray being awarded the class medal by EPCC Director,

Prof. Arthur Trew.

12 13

Training and education

I was looking for an MSc course, but, having interest in programming for science, this was the main criterion. I heard about the MSc in HPC from my friend, who did the course the year before. Not only had he told me about the high teaching standards and very interesting course content, but also about

a friendly atmosphere. Having graduated, I can honestly confirm that everything he said was true.

I have always enjoyed programming, but, as my background was in physics, I hadn’t received any proper programming

training. The ‘Practical Software Development’ course filled this gap and taught me how to employ best industrial practices in order to develop high-quality software. On top of this, I became familiar with and had a chance to use some of the cutting-edge technologies. Most importantly, however, I gained knowledge of HPC and became skilled at the art of parallel programming. I wholeheartedly recommend the ‘Hardware, Compilers and Performance Optimisation’ and ‘Message Passing Programming’ courses.

I enjoyed my time at EPCC so much that I decided to apply for the post of Applications Consultant. I was successful in this and now I am a member of the QCDgrid project team. I work in the area of my interest, as the project is concerned with developing a piece of software to be used by physicists throughout the UK.

Radek Ostrowski

I first noticed this MSc course by its online poster, which showed me a well-established, ideal course with distinct subjects, substantial content, and experienced directors. This course opened to me a wide picture of HPC which I believed would become the biggest highlight of computer development in the future. My interest in parallel computing and computer science background finally pushed me to apply for the course. Now I rejoice at the right choice I made at that time.

The year was absolutely not an easy time for any MSc student, especially for a foreigner who had to complete 12 courses plus a 4-month project in a second language he was not so familiar with. But I have never seen myself make such huge progress. As the old saying goes, no pain, no gain. I learnt sufficient fundamental HPC knowledge and technologies through the lectures and improved my practical abilities and skills in the final project. What’s most significant is I have managed to establish

self-confidence in this entirely new environment.

The skills acquired from the MSc course helped me find a job soon after graduation. I’m currently working for the Information Service of Edinburgh University on the ‘Edinburgh Computing and Data Facility (ECDF)’ project which aims to build and maintain a powerful cluster (256 dual-core CPU in phase 1 and a 256 quad-core CPU in phase 2) plus a massive fibre channel storage device (155TB SAN). Sounds exciting? I really enjoy my current job and life in Edinburgh and it was all started off by my application for the MSc.

Yuan Wan

Here, some of last year’s graduates tell us why they chose to do the msc in High Performance Computing at ePCC, what they enjoyed about the course, and where it led them afterwards.

msc 2006 graduates

12 13

Training and education

I worked as a computational fluid dynamics (CFD) engineer in the UK nuclear industry for two years before enrolling in the HPC MSc at EPCC. I had been using parallel commercial CFD code on a day-to-day basis as part of my everyday job. However, as I had been trained mainly in physics, applied maths and engineering, I did not have a clear understanding of how parallel codes work ‘under the bonnet’. This proved to be problematic when attempting to parallelise our in-house code. I found it

difficult to train myself in parallel programming and was delighted to be offered a place on the perfectly-suited MSc.

One of the key factors in my decision to undertake the course was its practical emphasis. The modules are generally taught in half-day blocks which consist of excellently-presented succinct lectures, usually followed

by practical sessions. This allows you to apply the theory you have just been taught on the EPCC machines. The core courses such as message-passing programming, shared-memory programming and performance programming are extremely effective at getting across the fundamental concepts of HPC. I was amazed how quickly I could put the techniques to good use tackling real-world problems. The coursework assignments were extremely well thought out, satisfying and even fun!

My classmates’ diverse backgrounds provided a rich source of knowledge to complement the teaching. I did not have a formal computing background, and was worried that I could get left behind very quickly. However, my more tech-savvy classmates were very willing to answer my ‘dumb’ questions, which I hope I managed to repay by answering quantum mechanics queries during the Computational Chemistry elective! The MSc’s fascinating subject matter was motivation enough for me to work hard, achieve a good grade and develop a real interest in HPC. After the course, I started working for EPCC as an Applications Consultant.

Upon completing a BEng in Electronics Engineering, I realised that I could understand code and computers better than I could circuits and electrons. This led me to venture into employment in the computer science industry, trying on different hats while seeking my niche (and dreaming of fame and fortune). After four years of ‘soul searching’, I learnt two things: where my interest lay, and that interest alone was not enough.

To give my career a boost in the right direction, I started ‘googling’ for postgraduate and professional courses. The search did not take long. As I stumbled upon the MSc in HPC course information, I was amazed at how well the course fitted the bill. The

entry requirements were flexible enough for me to qualify, and the subjects covered were exciting and up-to-date.

Just over a year later, I completed the MSc and was immediately offered a job in the CCLRC Rutherford Appleton Laboratory. Thanks to the strong practical focus of the MSc course, I had the skills to contribute effectively in my employment right from the beginning.

While fame and fortune are still a little out of reach, I am now on track to build my dream career, all thanks to EPCC!

HPC plays a more and more important role in research and industry nowadays and has a very wide foreground in the future. I was interested in HPC and decided to take this MSc course after my undergraduate study of Computer Science in China.

The one-year Master’s course provided by EPCC was really well-established and I enjoyed it a lot. The course covers a wide range of HPC areas and has a strong practical focus, which enabled me to acquire not only the leading-edge HPC knowledge and technologies, but also the ability to solve real parallel problems using

suitable techniques. One of the most exciting parts of the course was that I had the chance to access some world-leading HPC platforms and technologies, such as HPCx, the UK national HPC supercomputer, and BlueSky, a leading-edge IBM BlueGene system operated by EPCC, which were all very precious experiences.

The MSc course in HPC also led me to a better career. After graduation, I joined EPCC as an applications consultant. The job gives me more opportunities to apply what I learned from the course to real research and projects, which is really great.

Xu Guo

Lee-Shawn Chin

Howard Price

14 15

Training and education

For the last few years, EPCC has been delighted to be part of the Edinburgh International Science Festival. Our work at the edge of high-performance computing has provided Festival goers with a good number of ‘a-ha!’ moments – and shown that computers are for much more than instant messaging and playing games.

This year we changed tack slightly and took a look at the Internet. The Internet has gone from a scientific tool – almost a scientific curiosity – to an entity which pervades our everyday lives in a remarkably short time.

However, over 40% of users have suffered at the digital hands of ‘malware’, and 8% report being the victims of online fraud,

despite 9 in 10 reportedly taking security precautions online. The Internet’s current state is almost breathtaking in scope – and not a little scary. Like any other stratum of human life the Internet has its fun side, its seedy side and its downright criminal side. The big difference, of course, is that you don’t need to travel to the wrong part of town – it’s all a couple of mouse clicks away.

EPCC’s talk took a holistic look at the Internet, its history, its technology, trying to analyse how it has got to the state it’s in – and what can be done about it.

ePCC at the edinburgh International science Festival 2007Rob Baxter

eDIKT2 is a four year project that is coming to the end of its first very successful year. EPCC has used this project to promote the use of research computing around the University of Edinburgh, with fantastic results.

eDIKT2 aims to produce new knowledge by using computational science to extract information from vast datasets, and generate models through simulation. Researchers across the University are already doing this but many do not know about the range of resources and expertise available within EPCC. A talk to publicise these aspects of EPCC has been given in several departments of the University. It aims to show researchers what computer simulation can do for them. The main message is that high-performance computing allows the simulation of the very small (quarks and other sub-atomic particles) and the very big (the whole Earth to the entire Universe). A whistle-stop tour of past and current EPCC projects certainly shows this range of application sizes. EPCC has been at the forefront of HPC research for the last fifteen years and is well placed to help researchers exploit this productive opportunity in research.

So far, the talk has been given in three Schools: GeoSciences, Biology and Engineering, with more to follow. The talks were well attended and produced both stimulating debate

and an interest in pursuing research computing from many researchers. We have already encouraged researchers to apply for time on HPCx and there are potential collaborative projects as a result of the presentation. In addition, a series of shorter presentations are planned in Chemistry, in collaboration with the School’s chemistry simulation expert, to introduce experimental chemists to the prospect of simulation. It is hoped that it will encourage more academic collaborations between EPCC and researchers across the University. The talk is available on-line from the eDIKT2 website: www.edikt.org/edikt2/PromotingResearchCollaborationsActivity

eDIKT2 also runs quarterly technical workshops to allow eDIKT2 participants to meet and discuss issues occurring in their work. These meetings are also open to other interested parties around the University. In previous meetings there have been a wide range of talks regarding the upcoming ECDF, how to image the brain, performing chemical simulations and Field Programmable Gate Arrays (FPGAs). The next workshop will be held in Glasgow and will be available over Access Grid, pushing the outreach of eDIKT2 beyond Edinburgh. This and future workshops will be publicised on the eDIKT2 website so look out for them! www.edikt.org

ePCC on tour with eDIKT2Jon Hill

Rob giving his talk at the Science Festival.

14 15

Training and education

The third DEISA Symposium will be held at the Bavarian Academy of Sciences in Munich on Monday and Tuesday, the 21st and 22nd of May, 2007.

The theme will be Towards PetaScale Computing in Europe and the event will feature key scientists from both within DEISA and far beyond. Further, there will be a poster session featuring results of scientific simulations run by users of the DEISA Extreme Computing Initiative.

The event will include lunch on both days and a cocktail reception will be held on the Monday evening.

For a detailed agenda, travel information, on-line registration and how to apply for the third DEISA Extreme Computing Initiative, see: www.deisa.org.

DeIsA symposium: ‘Towards Petascale Computing’21-22 may, 2007, munich, GermanyGavin J Pringle

SC2006 was held this year in Tampa, Florida. Flying from a windy and wet Glasgow airport in November, I have to say that the warm sunshine was a welcome sight!

As the premier conference on high performance computing, networking and storage, SC2006 attracts many scientists and researchers interested in learning about new and important developments in High Performance Computing. The tutorial programme is always well attended and covers a broad range of new topics in HPC. This year, EPCC was selected to present a tutorial on the performance of applications on the Blue Gene architecture.

Since obtaining the first IBM eServer Blue Gene system in Europe in December 2004, EPCC has generated a great deal of expertise in porting and optimising scientific applications on this architecture. This tutorial offered us the opportunity to pass on some of this knowledge. Four staff presented the tutorial: Mark Bull, Alan Gray, Joachim Hein and myself.

Blue Gene was designed as a special purpose system, offering unprecedented computing performance coupled with very low power consumption and cost for a limited set of applications. Adoption has however become more widespread and Blue Gene is now being used in a relatively diverse range of scientific disciplines. It is therefore highly topical to explore the range and type of application that can make effective use of these systems, and the relevant architectural features influencing performance.

The tutorial focused on the performance of a series of applications and considered the techniques required to achieve optimal performance and scaling on the Blue Gene architecture. We discussed the factors and bottlenecks that influence performance and covered techniques for compiler, memory and communication optimisation. Attendees had the opportunity to gain hands-on experience through a series of practical sessions on our Blue Gene system.

sC2006 Tutorial: Application Performance on the Blue Gene Architecture Lorna Smith

EPCC is a European centre of expertise in developing high performance, novel computing solutions, managing advanced systems and providing HPC training. Our clients and partners include local and global industry, government institutions and academia.

EPCC’s combination of advanced computing resources and expertise is unique and unmatched by any European university.

www.epcc.ed.ac.uk info@epcc.ed.ac.uk

Phd supervisors: hec studentships provide funding for incorporating a year’s training in high end computing into a 4-year phd studentship.

The HEC Training Centre provides postgraduate training at masters level. The students who come to the Centre will be on a four-year studentship programme leading eventually to a PhD; during this, they take a number of courses at the Training Centre followed by an HEC research project. This all leads to an additional qualification, the MSc in High Performance Computing from the University of Edinburgh. The training programme is supported by the UK’s Engineering and Physical Sciences Research Council (EPSRC). EPSRC and PPARC both have HEC studentships available, which take advantage of access to EPCC’s HEC Training Centre.

To combine the PhD and MSc programmes successfully, HEC students are formally enrolled as part-time students at the University of Edinburgh. The will usually visit us for two blocks of seven or eight weeks for the taught part of the course, spread out over their first three years of study. This will allow them to meet and share experiences with their fellow MSc students without having to spend too long away from the institution where they are doing the PhD research.

The training that we provide covers a wide range of useful and exciting areas in HEC. This includes how to write high-quality computer programs, how to make programs go fast on the world’s largest supercomputers, how to use Grid technologies to harness the internet effectively, visualisation of scientific results and the mathematical tools and techniques that underpin computational science and engineering. Graduates of the MSc in HPC will have acquired skills that make them employable in a wide range of future careers ranging from postdoctoral academic research beyond their PhD, to jobs in commercial Information Technology.

HEC students undertake the same MSc in High Performance Computing (HPC) as our full time students. This well-established MSc programme provides an excellent grounding in HPC technologies and their practical application.

Applications

Applications are made by PIs and PhD supervisors and not by students. We require that our students are competent programmers in C, C++, Fortran or Java and are graduates in science, engineering, computer science or mathematics, or are currently working in a relevant field. The entrance requirement for the MSc in HPC is a good honours degree or equivalent work experience; no prior HPC knowledge is assumed. PIs should bear this in mind when selecting a student for this course of study.

Taught courses

Students take courses from: Fundamental Concepts of HPC Practical Software Development Shared Memory Programming Message Passing Programming Parallel Decomposition Performance scaling on Modern HPC Architectures Object Oriented Programming for HPC Hardware, Compilers and Performance Programming Applied Numerical Algorithms Scientific Visualisation Advanced Topics in HPC and e-Science HPC Techniques in Computational Chemistry HPC Techniques in Computational Engineering plus additional optional courses from other Edinburgh University MSc programmes including Informatics and e-Science. All students also undertake a 16-week research dissertation.

EPCC is an institute within the School of Physics at the University of Edinburgh. It has an international reputation as a world-leading centre of expertise in HPC and Grid computing. The MSc is supported by EPSRC.

More information at www.epcc.ed.ac.uk/hec or contact hec@epcc.ed.ac.uk

EPCC High End Computing Training Centre

hec@epcc.ed.ac.ukwww.epcc.ed.ac.uk/msc/

www.epcc.ed.ac.uk/hec/PhD supervisors: HEC studentships provide funding for incorporating a year’s training in High End Computing into a 4-year PhD studentship.

The HEC Training Centre provides postgraduate training at masters level. Students who come to the Centre will be on a four-year studentship programme leading eventually to a PhD; during this, they take a number of courses at the Training Centre followed by an HEC research project. This all leads to an additional qualification, the MSc in High Performance Computing from the University of Edinburgh. The training programme is supported by the UK’s Engineering and Physical Sciences Research Council (EPSRC). EPSRC and STFC both have HEC studentships available, which take advantage of access to EPCC’s HEC Training Centre.

To combine the PhD and MSc programmes successfully, HEC students are formally enrolled as part-time students at the University of Edinburgh. They will usually visit us for two blocks of seven or eight weeks for the taught part of the course, spread out over their first three years of study. This will allow them to meet and share experiences with their fellow MSc students without having to spend too long away from the institution where they are doing the PhD research. The training that we provide covers a wide range of useful and exciting areas in HEC. This includes how to write high-quality computer programs, how to make programs go fast on the world’s largest supercomputers, how to use Grid technologies to harness the internet effectively, visualisation of scientific results and the mathematical tools and techniques that underpin computational science and engineering. Graduates of the MSc in HPC will

have acquired skills that make them employable in a wide range of future careers ranging from postdoctoral academic research beyond their PhD, to jobs in commercial Information Technology.

HEC students undertake the same MSc in High Performance Computing (HPC) as our full time students. This well-established MSc programme provides an excellent grounding in HPC technologies and their practical application.

ApplicationsApplications are made by PIs and PhD supervisors and not by students. We require that our students are competent programmers in C, C++, Fortran or Java and are graduates in science, engineering, computer science or mathematics, or are currently working in a relevant field. The entrance requirement for the MSc in HPC is a good honours degree or equivalent work experience; no prior HPC knowledge is assumed. PIs should bear this in mind when selecting a student for this course of study.

EPCC is an institute within the School of Physics at the University of Edinburgh. It has an international reputation as a worldleading centre of expertise in HPC and Grid computing. The MSc is supported by EPSRC.

More informationwww.epcc.ed.ac.uk/hec or contact hec@epcc.ed.ac.uk

EPCC High End Computing Training Centre: hec@epcc.ed.ac.uk www.epcc.ed.ac.uk/msc/ www.epcc.ed.ac.uk/hec/

See page 10 for a list of MSc courses.