maintainability, sustainability, testing and deployment of qe

«WhatIcannotcompute,Idonotunderstand.»(adaptedfromRichardP.Feynman)

Maintainability,Sustainability,TestingandDeploymentofQE

FilippoSpiga1,2 <filippo.spiga@quantum-espresso.org >

1 HeadofResearchSoftwareEngineering,Univ.ofCambridge2 QuantumESPRESSOFoundation

Elevatorpitch:ResearchSoftwareEngineering

"Software isacornerstoneofscience.Withoutsoftware, twenty-firstcenturysciencewouldbeimpossible.Withoutbettersoftware,sciencecannotprogress.”-- SCIENCECODEMANIFESTO

BETTERSOFTWAREBETTERRESEARCH

Thishappens…lotoftimes!

Becauseeverybody…

The(invalid)argument

Whybotheraboutallofthis,wealwaysdidwithoutthoseanditwasgood

enough…

WRONG

3principles

• Versioning(SVNà SVN+GIT)• ContinuousIntegration(BuildBot)• Testing(test-code)

…appliedtoQuantumESPRESSO.

Demoincluded,referencesincluded.

Versioning(classic)

SVN

Versioning(ideal)

GIT

Versioning(sustainable)

SVN(legacy)

GIT(mirror)

NEWRELEASE

ContinuousIntegration(C-I)

Automate,Automate,Automate…

BuildBot

WritteninPython,verysimple!

BuildBot

Buildbot overview

17

http://buildbot.net/buildbot/docs/current/manual/introduction.html Upon branch update, build is created, and test suite is run

Source: buildbot team

Whatbuildbot doesforme…• Uponbranchupdateoratspecific

timeinterval,abuild iscreatedandtestsuiteisrun

• Developer isalerted(viaUIoremail)whenatestfails,cansubmitfix,andre-launchtest(evenmanually)

• Uploadtestresultsorcompiledapplicationstoanexternalserver

DeployyourBuildBot (slave)

mkdir -p $HOME/my_buildbot_slavecd $HOME/my_buildbot_slave

virtualenv --no-site-packages buildbot_sandboxsource buildbot_sandbox/bin/activate

buildslave create-slave slave \<my-public-IP>:9989 my_slave_1 <password>

buildbot start my_slave_1

DEMO

test-code

• Projectinitiated byJamesSpencer(ICL)fortesting lookingatregressionerrorsinscientific software.

• Itrunsasetofcalculations, andcomparestheoutputdatatothatgeneratedbyapreviouscalculation (whichisregardedtobe"correct").

• Written inpython,designed tobelightweight andhighlyportable.• Itcanrunasetoftestsandcheckthecalculated dataiswithinathedesired

toleranceofresultscontained inpreviousoutput(dataextractionfeatures)• Theprogramstobetested canberuninserialandinparallelandtests canberun

ineither locallyorsubmitted toacomputecluster.

test-code- capabilities

• compare =comparesetoftestoutputsfromaprevioustestcode runagainstthebenchmarkoutputs.

• diff =diffsetoftestoutputsfromaprevioustestcode runagainstthebenchmarkoutputs.• make-benchmarks =createanewsetofbenchmarksandupdatetheuserconfig filewiththe

newbenchmarkid.Alsorunsthe‘run’action.• recheck =comparesetoftestoutputsfromaprevioustestcode runagainstbenchmark

outputsandrerunanyfailedtests.• run =runasetoftestsandcompareagainstthebenchmarkoutputs.• tidy =removefilesfromprevioustestcode runsfromthetestdirectories.

test-code- configuration

Twoconfigurationfiles:• jobconfig defines theteststorun• userconfig definesaprogramtobetested

DEMO

ProductionRoadmap

Phase1-- April2015,v5.3.1:• QE-FORGEContinuous Integrationgoesliveasservice• All“slaves”welcome (butwevaluediversity)• Development branchescovered ifpushedontheQE-FORGE

(Possible)Phase2-- September2015,v5.x.x:• Regular (howregular?)nightlybinaries• Morepackageswithinsuite(PH,TDDFPT,…)• Morecomplex