tool-supported collaborative requirements prioritisation

Tool-supportedCollaborativeRequirementsPrioritisationP.Busetta,F.M.Kifetew,D.Muñante,A.Perini,A.Siena,A.Susi

6July2017COMPSAC2017

MotivationsandKeyPoints ataglance:• Decidingwhatismoreandwhatislessimportantiscriticalwhendevelopingsoftware,inparticulartodeterminethecontentsofadevelopmentcycle,aswellasinanyactivitywithlimitedtimeandresources• However,differentstakeholders(developers,users,operations,marketing…)mayhavedifferentperspectivesandsometimesconflictingopinionsontherelativepriorityofrequirementstobeaddressed

• Supportingdecision-makingtechniques/toolsaboundbutpracticaluseisrelativelyrare• … especiallyinsmallanddistributedorganizations,becauseofperceivedcostsandeffortsw.r.t.benefits

• Theprocessweproposeanditsaccompanyingtooltackletheseissuesby:• Minimizingthecostofset-upandexecutionofadecision-makingsession• Allowingittobeeasily repeated,withthesameordifferentparticipants/criteria/items• Distributingitintimeandspace(i.e.,notrequiringeverybodytobeatthesametimeinthesameplace)

• Insertinggamification elementstomakedecision-makingorganizationallyflexible,engagingfortheparticipants,andpotentiallyitselfsourceoffurtherorganizationalknowledge

• Experimentedinafewcasestudiessofar,theveryfirstbeingillustratedhere

06July2017 Tool-SupportedRequirementPrioritisation 2

Outline

• Context:Involvingtheend-usersdirectlyintheloopandfosteringcollaborationwithinthedevelopmentteamandwithstakeholders• Introducinggamificationintheprocess• Anexperiment:configurationandconclusions• CurrentstateandFuturework


ProjectContext:End-userfeedbackandmonitoringdataasdriversofsoftwareevolutionandadaptation


Usersarefirst-classactorswithin theloop–particularlyrelevantwiththeincreasingadoptionofCI/CDpipelines

Prioritizationisacriticalsteptoinformreleaseplanning.Criteriamaybemultiple;thus,thisactivitycanbeseenasaformofmulti-objectiveoptimizations

ResearchperformedaspartoftheH2020SUPERSEDEproject

(www.supersede.eu)

Usersfeedbackanddatafrommonitorsbecomerequirements tobeaddressedbythenextsoftwarerelease

Prioritizinginacollaborativeenvironment• Collaborationisrequiredtocapturedifferentperspectivesonaproduct,aswellasdifferentcompetencesandexperiencewithinateam• Theprocess(anditsaccompanyingtool)shouldallowtheinvolvementofdecision-makersthat donotcoveranytraditional,organization-determinedrole• Whichmayincludemorethanthesole“productowner”(inSCRUMterminology)• Thisisevenmoreimportantifend-users“sitatthetable”,notjustbyprovidingfeedbackbut,indeed,participatingtothedecision-makingprocess

• Ontheotherside,howtokeepparticipantsengaged,willingtoparticipate?• Especiallynon-developersorothersthatdonothaveaspecificdutyintheorganization


Gamifyingtheprocess(1/2)”Gamification”simplymeans“introducingelementstypicalofgames”forvariouspurposes,typicallytomakeaprocesssmootherandmoreengaging.Wetookafewtoaddresstherequirementsidentifiedbefore:• Multipleroleswithdifferentallowedactions.Theyareunrelated toanyorganizationalstructure,justlocaltoaspecific(game)session• Identifiedsofararesupervisor,opinionprovider,negotiator,thelatterintroducingawholeworldof(potentiallyrich)interactionsamongplayersthatgoesbeyondthedeterministicresultsofatechniqueoralgorithm

• Anon-linearworkflow,wheremovingfromonephasetoanotherisdeterminedeitherby(game)logicorbyaspecificrolewhile,withineachphase,allrolesinvolvedcanplaytheiractionsinanarbitraryorderandpossiblymultipletimes• Identifiedsofarare(1)set-up,(2)opinionprovisionandnegotiation,(3)closure.Set-upandclosurearecurrentlyforthesupervisoronly(butcouldbeautomatede.g.withreasoningaboutthestateofthegame),whilethemiddlephaseisforallotherplayers


Gamifyingtheprocess(2/2)• Playfulness (howmuchoneis“entertained”bythesystem),tobeintroducedinvariousways.Weexperimentedwith:• Competition andReward bymeansofpoints-ification (i.e.collectionofpointsaccordingtotheactionsperformedandtheirmodalityofexecution,inturnleadingtotheintroductionofleaderboards,badgesandsoon)

• Timepressure isboththeresultsofpoints-ification (thefasteroneplays,themorepointsonecollects)andbythebarepsychologicaltrickofshowingaclockticking

• Lightweight-nessofgameplay(i.e.thesimplicityandspeedatwhichplayerscanperformtheiractions),notonlytoimproveplayfulnessbuttosupportrepeatability

• Easyrepeatability,enablingmultiplerunswiththesameordifferentstakeholders(participants),thesameordifferentsetsofrequirements,thesameordifferentsetsofprioritizationcriteria,andsoon• Further,theintroductionofsomeformofKPIs(e.g.,pointscouldbecomputedaccordingtovariouscriteriaandkeptseparated)maybeexploitedtoletcompetenceandengagementemergeovertime


GRP(GamifiedcollaborativeRequirementsPrioritisation):Experimentalset-up• TheexperimentedprocessisbasedonthewellknownAHP (AnalyticHierarchyProcess)

• Pairwisecomparisons:decision-makerexpresseswhichoneoftwoitems(requirementsinourcase)ispreferredaccordingtoaspecificcriterion(e.g.uservalue,developmentcomplexity,…)

• 9-valuescaleusedtoexpresspreferences,rangingfrom“AisgreatlypreferredtoB”to“indifferent”to“BisgreatlypreferredtoA”.

• Numberofcomparisons=½items2 *criteria• But,astopping criteriacanbeimposed,andsupervisormaydecidetostopanyway.Computationscanusepartial

data

• Allopinionproviderscanexpresstheirpreferencesonallrequirements.Negotiation isautomaticallyrequestedbythesystemwhentwoormoreplayershavestronglydifferentopinions(contrastingpreferencesexceedathreshold)• Resolutioncurrentlyimplementedasnegotiatorexpressingfinalopinionorignoringandlettingtheunderlyingcomputationsuseaverages

• ThetoolprovidesaWebinterface.Playerscanintervenefromanylocationattheirleisurebutwithinatimewindowdefinedbythesupervisor• Gamificationthusincludedmanyplayfulnessaspects,butnopoints-ification (competition&rewards)fortheexperimentreportedhere


Webinterfaces

• Gameset-upandconclusionwithcomputations(supervisor)• Pairwisepreferences(opinionproviders)• Conflictresolution(negotiator)


Pointsandbadges(NOTE:laterversion)

Preferenceexpression

Playerhistory

Theexperiment:dataandcharacteristics• DevelopmentteamofR&Ddepartment• DeltaInformatica’s owngroupworkingonauthoringtoolsforvirtualreality-basedtraining(PRESTOtoolsuite)

• SME.Littleornouserinputonmostnovelaspects• 16requirements,groupedin4clusters• 2criteriaforevaluation(effortanduservalue)• 8participants:engineersofvarioussenioritylevelsandtheircoordinator• Allasopinionproviders• Thecoordinatorhadalsotheroleofnegotiator

• 24hoursfor“playing”4games(onepercluster)• Questionnairesandinterviewsafterconclusion

06July 2017 Tool-SupportedRequirementPrioritisation 10

Thesubjectofprioritization:Averycontroversialuserinterface

PRESTOScripting’scontrolleristhekeytoolforatrainertostartanddirectatrainingsession

inavirtualreality

(seediscussioninP.Busetta,AddressingTeamAwarenessByMeansOfARequirementPrioritizationTool,PRIORE’17)

Resultsfromtheexperiment• GRPperceivedaseffective forreachingacceptabledecisionsinanefficientmanner• Participantsagreedtoahighdegreewiththefinaloutcomeoftheprocessforallclustersofrequirements• Thisinspiteofahighlevelofdisagreementinpairwisepreferences• ThecomparisonsofGRPoutcomesbeforeandafterthenegotiator’sinterventionshowedthatthenegotiatorplaysanimportantrole inmanagingconflictingpreferences,affectingresults

• Reportedadvantagesincludetheabilitytotrackdecisions• Suggestedpotentialimprovementsincludemechanismsfor“livecommunication”duringgameplay• Themeasuredtimerequiredtoexpresspreferencesisquiteshort


Currentstate

• Improvedprototypeandextendedscopeofprocess• Points-ifcation fullysupported• PrioritizationtechniquesalternativetoAHPhavebeenintroduced,mostimportantlynon-pairwisecomparisons(usingGAstocomputeagreements,measuringindividualchoice’sdistancefromgroup’sconsensus,…)

• Revisedtheentireprocesstogivemoreimportancetothesupervisorandtointroduceplanningtools,e.g.toselectthemostsuitablealgorithm/technique,todecidehowmanygamestoplayandwithwhichparticipants,andsoon

• Triedoutinotherexperimentsandvalidatedagainsttheproject’susecases• Resultsavailableasproject’sreports• Overall,approachalwayswellreceived


Futuredirections

• Integrationofprioritizationprocesswithsomeexistingapproach/methodology• LookinginparticularatSCRUMandatitsSprints’“backlog”selectionand“planning”meeting

• Integrationofprioritizationtoolasplug-inofmarkettools• Currently,developingplug-insforAtlassian’s JIRA,extendingitsAgiletools



Thanksforyourattention!

tool-supported collaborative requirements prioritisation

Science