maintainability, sustainability, testing and deployment of qe
TRANSCRIPT
«WhatIcannotcompute,Idonotunderstand.»(adaptedfromRichardP.Feynman)
Maintainability,Sustainability,TestingandDeploymentofQE
FilippoSpiga1,2 <[email protected] >
1 HeadofResearchSoftwareEngineering,Univ.ofCambridge2 QuantumESPRESSOFoundation
Elevatorpitch:ResearchSoftwareEngineering
"Software isacornerstoneofscience.Withoutsoftware, twenty-firstcenturysciencewouldbeimpossible.Withoutbettersoftware,sciencecannotprogress.”-- SCIENCECODEMANIFESTO
BETTERSOFTWAREBETTERRESEARCH
Thishappens…lotoftimes!
Becauseeverybody…
The(invalid)argument
Whybotheraboutallofthis,wealwaysdidwithoutthoseanditwasgood
enough…
WRONG
3principles
• Versioning(SVNà SVN+GIT)• ContinuousIntegration(BuildBot)• Testing(test-code)
…appliedtoQuantumESPRESSO.
Demoincluded,referencesincluded.
Versioning(classic)
SVN
Versioning(ideal)
GIT
Versioning(sustainable)
SVN(legacy)
GIT(mirror)
NEWRELEASE
ContinuousIntegration(C-I)
Automate,Automate,Automate…
BuildBot
WritteninPython,verysimple!
BuildBot
Buildbot overview
17
http://buildbot.net/buildbot/docs/current/manual/introduction.html Upon branch update, build is created, and test suite is run
Source: buildbot team
Whatbuildbot doesforme…• Uponbranchupdateoratspecific
timeinterval,abuild iscreatedandtestsuiteisrun
• Developer isalerted(viaUIoremail)whenatestfails,cansubmitfix,andre-launchtest(evenmanually)
• Uploadtestresultsorcompiledapplicationstoanexternalserver
DeployyourBuildBot (slave)
mkdir -p $HOME/my_buildbot_slavecd $HOME/my_buildbot_slave
virtualenv --no-site-packages buildbot_sandboxsource buildbot_sandbox/bin/activate
buildslave create-slave slave \<my-public-IP>:9989 my_slave_1 <password>
buildbot start my_slave_1
DEMO
test-code
• Projectinitiated byJamesSpencer(ICL)fortesting lookingatregressionerrorsinscientific software.
• Itrunsasetofcalculations, andcomparestheoutputdatatothatgeneratedbyapreviouscalculation (whichisregardedtobe"correct").
• Written inpython,designed tobelightweight andhighlyportable.• Itcanrunasetoftestsandcheckthecalculated dataiswithinathedesired
toleranceofresultscontained inpreviousoutput(dataextractionfeatures)• Theprogramstobetested canberuninserialandinparallelandtests canberun
ineither locallyorsubmitted toacomputecluster.
test-code- capabilities
• compare =comparesetoftestoutputsfromaprevioustestcode runagainstthebenchmarkoutputs.
• diff =diffsetoftestoutputsfromaprevioustestcode runagainstthebenchmarkoutputs.• make-benchmarks =createanewsetofbenchmarksandupdatetheuserconfig filewiththe
newbenchmarkid.Alsorunsthe‘run’action.• recheck =comparesetoftestoutputsfromaprevioustestcode runagainstbenchmark
outputsandrerunanyfailedtests.• run =runasetoftestsandcompareagainstthebenchmarkoutputs.• tidy =removefilesfromprevioustestcode runsfromthetestdirectories.
test-code- configuration
Twoconfigurationfiles:• jobconfig defines theteststorun• userconfig definesaprogramtobetested
DEMO
ProductionRoadmap
Phase1-- April2015,v5.3.1:• QE-FORGEContinuous Integrationgoesliveasservice• All“slaves”welcome (butwevaluediversity)• Development branchescovered ifpushedontheQE-FORGE
(Possible)Phase2-- September2015,v5.x.x:• Regular (howregular?)nightlybinaries• Morepackageswithinsuite(PH,TDDFPT,…)• Morecomplex