research computing @ uq · pdf filehristev systems administrator [email protected] william...
TRANSCRIPT
1
Research Computing @ UQResearch Computing @ UQ
1 August 20131 August 2013 Issue 1Issue 1
The Research Computing Centre supports collaboration to facilitate discoveriesin science and engineering, humanities, and social sciences, through advancedcomputation, data analysis and other digital research tools.
Message from the Director
I am delighted tohave joined theUniversity ofQueensland asDirector of theResearch ComputingCentre (RCC), andlook forward tobuilding on itsachievements overthe past few years.
RCC plays a crucialrole in UQ’s researchinfrastructure by
working with researchers to better leverage newcomputing technologies. Many of these systems areat the leading edge but are not always easy to use.On the other hand, they promise significantbenefits for UQ researchers. Our job is to workclosely with you to realise these benefits.
Over the past 4 months I have been moving aroundthe University to meet people and discuss how wecan work more closely. In response to the feedback,I have been reformulating the RCC structure andengagement model, and have been testing this on
some key stakeholders. Over coming months wewill be releasing the details and seek further input.Overall, the model develops and fosters a uniquecombination of deep technical IT knowledge withdomain expertise, supporting collaboration andsupport with our clients. We look forward toworking with you.
David AbramsonDirector
A new homeRCC has a new home in Axon Bld, Level 5.
We are co-located with the Queensland Cyber-InfrastructureFoundation(QCIF) andthe ResearchData StorageInfrastructureproject(RDSI).Together theseorganisationsoffer a unique portfolio of expertise to UQ,Brisbane and Queensland.
We look forward to seeing you in Axon 5 in thenear future.
2
cloud computingIs it hot air?
Just what is it?
Cloud computing is a hot topic - everyone is talkingabout it and it seems to be a panacea for all ills.Let’s take a quick look at the key ideas and outline alittle about local plans at UQ.
No single technology in Cloud computing is new.High performance compute clusters with multiplecomputing cores underpin modern computinginfrastructure. High speed networks have beendeployed for many years, particularly in the researchenvironment. So called “thin clients”, mobiledevices and enormous data archives and databaseshave also been around for some time. Virtualmachine technology was invented in the 1960s andeven pay-as-you-go computing has been attemptedover the years.
However, Clouds leverage all of these things, andthe combination makes them particularly attractivefor both research and business. Putting it simply, aCloud is a place where you can put your data,perform your computations and interact with others.You may not even be aware where the Cloud islocated - it is available 24/7 from anywhere that is
networked. It means that organisations can shiftCapital Expenditure (so called CAPEX) toOperational Expenditure (OPEX), and this makes iteasier to budget for infrastructure needs over thelong term. Some Clouds support “pay-as-you-go”meaning you can scale your infrastructure needs asdemand changes.
Unfortunately, Cloud computing is also the latesthype, and has been promised as a solution to everyailment known to mankind. This makes it difficultfor researchers to decide what’s in it for them, andwhat the appropriate choices are. At RCC, we havebuilt a team of experts in Cloud computing, and areworking with our partners (QCIF, NeCTAR andRDSI) to provide simple and effective solutions inresearch.
Cloud computing really embodies a few ideas:
On demand and scalable accessApplication servicesLocation transparencyVirtual machines
On demand and scalable access means that usersshould be able to scale resources dynamicallywithout the need to purchase hardware. If you need1 compute core today, but 1000 tomorrow, it shouldbe as simple as filling in a web form and requestingthese. Amazon calls this “elastic computing” andpromise unbounded resources, although clearly it isultimately limited by the size of their cloud. But,this can be very large.
Application services means that the cloud providermight expose high level services that meets somebusiness (or research) need. This currently workswell for tools like e-mail and calendar, but the
3
number of applications of value to researchers isgrowing all the time; some of these will be verydomain specific.
Location transparency means just that; you don’tknow where your services are being sourced.Whilst this is liberating and simple for many users,there are clearly applications where physicallocation is important. For example, whilst you maynot care where your e-mail is handled, you mightwant data to be held in a particular data centre.Thus, most cloud providers allow you to indicate alocale that restricts the actual domain. (A lot hasbeen said about legal issues such as the US PatriotAct and censorship laws. This is a very tricky areathat suggests the legal profession will ultimatelyhave to deal with virtual locations rather than lawsthat govern physical domains. But that won’thappen overnight!).
Virtual machines are a technical trick that simplifythe deployment of new services, and are part ofproviding rapid scale up and scale down. Putting itsimply it means you can build a software stacklocally and deploy it on a many physical machinesas required.
Commercial clouds also include pay-as-you-goeconomic models. This means that you pay only forthe infrastructure you use - whether that’s machinetime or data storage space. Companies such asAmazon are happy to take your credit card andprovide services according to a charging schedule.This is incredibly useful for business applications,but not commonly used in research circles.
What’s happening at UQ?
UQ researchers have access to data storage via theResearch Data Storage Infrastructure (RDSI)project to compute services via the NationaleResearch Collaboration Tools and Resources(NeCTAR) project. Researchers can get virtualmachines for their own use.
RDSI and NeCTAR are Australian Governmentprojects conducted as part of the Super Scienceinitiative and financed by the Education InvestmentFund. The Queensland node is called QCloud, and
its main data centre is currently located at TheUniversity of Queensland’s St Lucia campus andoperated by QCIF and the RCC.
QCloud is an integral part of new, national researchinfrastructure, leveraging data collections stored inthe local and national RDSI nodes, and integratingwith access to Queensland-based HPC facilitiesand specialised cloud data services. It will providelarge scale storage for UQ researchers andcomputational capacity suited to Cloud computing.
Virtual Machines/Research Cloud
NeCTAR offers eResearch infrastructure in fourkey areas:
Research CloudVirtual LaboratorieseResearch ToolsA secure and robust hosting service – the NationalServers Program
The research cloud is a computing resource for allAustralian researchers. If offers virtual machineswhere researchers can develop and deployapplications and collaborate in a uniformenvironment with controlled sharing of data..
Research Data Collections
QCloud makes available the RDSI Research DataServices (ReDS) data collections. Researchers arewelcome to apply to QCloud for RDSI ReDSstorage. The aim of ReDS is to fund infrastructureto store research data collections of national meritand collections of interest for future research.Collections that do not qualify for ReDS storagewill be considered for an allocation of locallyapproved data storage for their research datacollection, including collections still indevelopment.
Want to know more?
See the QCIF Cloud information page.
4
Need High Performance Computing?
The Research Computing Centre has access to a number of high performance computers - both at UQ andnationally. We would be happy to discuss your HPC needs and help secure access on the most appropriateresource.
Specifically, UQ has access to Barrine (at UQ) and the National Computational Infrastructure (NCI) atANU.
Barrine is a High-Performance Computingcluster located on The University of Queensland's(UQ) St. Lucia campus. It consists of 384compute nodes with over 3000 cpu coresconnected via an Infiniband network. Data storagefacilities include a 92TB parallel networkfilesystem (Panasas) and archived storagefeaturing a Hierarchical Storage Management(HSM) system and remote file mirroring. Jobsubmission and execution is handled by PBSProfessional, a commercial-grade job schedulingsoftware.
The cluster has been funded by a variety of groups and represents a major increase in capability for UQ andits partners. Its usage is closely monitored to ensure that the entitlements of all stakeholders are satisfied.The maintenance and management of Barrine including system upgrades and configuration is handled byUQ's Research Computing Centre (RCC).
Barrine is used by researchers from all over Australia, and also hosts Bioinformatics Resource Australia-EMBL web services & EBI FTP Mirror. Whilstbioinformatics represents a substantial amount of the workdone on Barrine, there are a number of other groupsincluding physics and mechanical engineering using thecluster.
NCI operates two large-scale peak systems, Raijin (aFujitsu Primergy cluster, commissioned in 2013, and whichentered production in June 2013), and Vayu (a Sun/OracleConstellation cluster, commissioned in 2009, and due to bedecommissioned later in 2013), and a small specialisedsystem, Fujin (a Fujitsu PrimeHPC FX10 system,commissioned in March 2013).
A key characteristic of both of the production systems, Vayu and Raijin, is their balance. System balance isthat important characteristic that provides hardware and applications scalability by requiring that crucialsystem parameters such as performance of memory, the interconnect fabric, and the filesystem all scale withnode and aggregate processor performance. Such balance, which is displayed in the hardware solutionarchitected by the HPC systems team at NCI, provides the facility with strong and scalable performance, anddelivers a quality user experience.
5
RCC Seminars
RCC launches the Semester II e-Research seminarsOn 2 August, RCC launched an exciting range of seminars in Axon 5. These feature high profile keynotespeakers on an exciting range of topics from e-Research infrastructure to advanced applications. We inviteyou to come along and see what’s happening in some of the most important e-research centres. The seminarsare pitched at undergraduates, graduates and academic staff. See the seminar page for details.
Time: 9:00 AM Fridays Place: Axon Level 5 seminar room RSVP: is essential as space islimited.
02/08/2013 Prof Michael Norman, SDSC Application Experiences with Gordon - A Flash-based HPC System
09/08/2013 Dr Sameer Tilak, CalIT2, UCSD Storage and Analysis of Big Data from SensorNetworks: Challenges and Opportunities
16/08/2013 Dr Wayne Pfeiffer, SDSC Bioinformatics Meets Big Data
23/08/2013
Prof Geoffrey Fox, U. Indiana Data Science and the Cloud
30/08/2013
Dr Shonali Krishnaswamy, I2R, A-star, Singapore
Big Data Analytics: When Research Meets Reality
06/09/2013
Prof Jack Dongarra, U. Tennessee Emerging Heterogeneous Technologies for HighPerformance Computing
13/09/2013
Dr Nancy Wilkins-Diehr, SDSC Science in the Digital Age: The Science GatewaysInstitute and NSF Software Investments
20/09/2013
Dr Arun Konagurthu, Monash U.
27/09/2013
Dr William Gropp, NCSA
11/10/2013
Dr Mark Miller, SDSC Embedding CIPRES Science Gateway Capabilitiesin Phylogenetics Software Environments
18/10/2013
Dr Ilkay Altintas, SDSC On Workflow-Driven Science Using ScientificWorkflows and Provenance
25/10/2013 Prof Peter Fox, RPI How Environmental Informatics is Preparing Usfor the Big Data Era
6
Name Position e-Mail
DavidAbramson
Director [email protected]
BrendonCarter
QCloud systemsprogrammer
JustinClark
eResearch analyst [email protected]
SharonCook
Executive Officer [email protected]
Dr DavidGreen
Deputy Director [email protected]
Ron Horst Project Manager - GVL [email protected]
RyurickHristev
Systems Administrator [email protected]
WilliamHsu
Senior IT Officer -BRAEMBL
MinhHuynh
QCloud SystemsProgrammer
AdrianLee
QCloud SystemsProgrammer
Dr IgorMakunin
Research Fellow - GVL [email protected]
MichaelMallon
Technical Assistant [email protected]
FranMoore
Administration Officer [email protected]
MartinNicholls
Specialist SystemsProgrammer
Dr EdanScriven
QCloud SystemsProgrammer
Contacts
6