WhyDataCanBeBothInvaluableandDetrimentaltoChangeInitiativeIdeas MICHAELS.MCCALLA,LeanAgileIntelligenceInc.

I have learned the hard way that if an organization lacks an Agile mindset, and does not foster an environment of safety, trust, andexperimentation,collectingteammetricsandinsightscanactuallyleadtomoreharmthangood.However,oncetheprerequisiteofsafetyisestablished,datanolongerbecomesyourenemyasachangeagent,butyourally,andapowerfultoolinyourAgilecoachingtoolbox.

1. INTRODUCTION

Anothermovement sweeping through the corporateworldwith the same steamasAgile is that ofBusinessIntelligence&BigData.Moreandmoreorganizationsareturningtodataanalyticstoidentifypatternstohelpthemimprove.Therefore,itshouldcomeasnosurprisethatorganizationsundergoingAgileTransformationsseekdatatomeasureperformanceoftheinvestmentandidentifyareasthatcanbeimproved. However, thesadtruthisiftheorganizationisnotmatureenoughtoutilizethisdataintherightway,itcandomoreharmthangood.

2. BACKGROUND

HavinggrownupasBA, Ihavealwaysgravitated towardsdatabecause I enjoy identifyingpatterns to tell astory.IamalsoapassionateAgilist,playingtheroleofchangeagentinAgileTransformations.Therefore,mynaturalinclinationhasalwaysbeentocouplethetwotogethertodrivechange.

I liketothinkofcollectingteamdataas justanotherfeedbackloopinanAgileenvironment.Afterall, thepeople doing the work know best! Therefore, why not collect data points from teams to identify areas forcontinuousimprovementthroughouttheorganization?Ihavehadsomesuccessinthisarea,butunfortunatelythisapproachhasnotalwaysleadtothedesiredoutcome.

Likeeverythingelsethatcouldbeusedforgood,datacanalsobeleveragedforevil.OrganizationsthathavenotacquiredtheAgilemindsetandembracedtheAgilevaluesandprincipleswillbemorelikelytousethedatato drive a more command-and-control leadership style. This approach does not foster an environment ofsafety, trust,andtransparencywhich leads tomorerisk that thedatawillnotreflectreality.However, if theorganizationdoesembracethevaluesandprinciples,andpeopletrusttheycanbehonestandtransparent,thedataismorelikelytobeaccurate.Leadershipcanthentrustthedatatoactupon.

2.1 TheFailureIconsidermyselfvery luckytohavespent the first tenyearsofmycareer inasmallsoftwarecompanythatembracedAgilevaluesandprinciplesevenbeforeAgilewasmainstream.TheAgilemindsetwasinstilledinmefromthebeginning.ThemindsetwaseasytoadoptsincetheAgileprinciplesandvaluesaligndirectlywithmypersonal beliefs. Agile has directly impactedmy career, outlooks, and personal growth. Therefore, when itcametimeforachange,itfeltrighttopursueanAgilecoachingcareerinordertohelpotherssharethesameexperience.

Myfirstengagementwaswitharathertraditionalmonolithicbank.TheorganizationwantedtobeAgileinname only, but I was too early in my journey to recognize it. I become part of the Agile Coach Center of

Author'semail:Michael.s.mccalla@gmail.comCopyright2017isheldbytheauthor.

WhyDataCanBeBothInvaluableandDetrimentaltoChangeInitiatives:Page-2

Excellence,acommunityresponsibleforassisting55teams,atallordergiventherewereonlythreeofus!Iwaseagertoshowmyknowledgeandskills,soIjumpedonanopportunitytodosomestatisticalanalysisonteamsdue tomy affinity for data. The assignmentwas a perfectmatch formy skills. All teambacklogs resided inindividualexcelsheets!Needless tosay, theorganization lackedaholisticviewofeach initiativegivenmanyteamswerecontributingtoit.

Isawitasopportunityto immediatelyaddvalue. Icreatedaprogramtoconsolidateallexcelsheets intoone.Aftera fewexcelmacros,pivot tables,andcharts, leadershiphadtheviewthat theyhavebeenseeking.Thetaskquicklymorphedawayfromaone-timeexercise,andbecameafull-timejob!Thespreadsheetbecameknownas the “AgileDashboard,” adocument inwhich Iwasnotonly asked to constantly enhance, but alsoadministrate.

Atfirst,Iacceptedtherole.Isawitasanopportunitytocoachleadershiponhowtoappropriatelyusethedata to drive positive change. I encouraged them to leverage the data to identify behavioral patterns,organizational constraints, coaching needs, and continuous improvement opportunities. Unfortunately,leadershipwasnotreceptivetothecoaching.Infact,theydidtheexactopposite!Mydatawasusedtopunishteams,identifypoorperformers,andholdAgilePMsaccountableforsomethingtheycouldnotcontrol.

IfoundmyselfinaroleinwhichIdidnotsignup.IlefttheofficeeverydaywithasenseofbetrayaltotheAgilecommunityandmyfriendsatthebank.ThiswasnottheAgileenvironmentthatIcametolove;onethatfostered safety, trust, and autonomy.Here Iwas, a so-called coach, fueling anti-patterns such as abuse andreprimand.When I left the engagement, I vowed that I would never putmyself in that position again, andfrankly,wantedtostayclearofdataaltogether.

2.2 WhatILearnedfromtheFailureTheexperiencemademeabettercoachandperson.Complexitytheorytellsusthatorganizationsarecomplexadaptive systems because they are a dynamic network of interactions that collectively self-organize whenchange is introduced.Asayoungcoach, Iwasnaïvetothis factanddidn’trealizethemagnitudeofdifficultyinvolved in organizational change. The word “transformation” is very misleading when it comes to Agileadoptioninitiativesbecauseitimpliesthereisanendstate.Instead,thesetypesofinitiativesshouldbeseenasanevolution,orajourney,characterizedbyrelentlessinspectionandadaptiontoimprovethecurrentstate.

Bob Hartman coined the term “doing” Agile vs. “being” Agile. In the earlier stages of an Agile adoptioninitiative, organizations that are transitioning froma traditionalmethodology are simply “doing”Agile. ThisadoptionstageischaracterizedbypeoplestartingtolearnAgilepracticesandthe“mechanics”behindthem.Innoway,shape,orformdoesthistranslatetoanorganization“being”Agile.

Atthisstage,theAgilemindsethasnotbeenacquiredbytheorganizationanditsleaders.Thisshouldbeofno surprise! As part of the change, leaders and individual contributors, who were very successful in thetraditionalorganization,arebeingaskedtochangetheirbehaviorsandthinking.Thisisachangethatdoesnothappen overnight. Unfortunately, the sad truth is this change in mindset will never occur for manyorganizations.Agileframeworkstypicallyexposeanorganization’sdysfunction,butwhetherornotleadershipwants to resolve it is a whole different story. Frommy experience, organizations often tend to change theframeworktoaccommodatethedysfunction,ratherthanappropriatelyresolvingthedysfunctiontobecomeanAgileorganization.

Organizationswillstillgainsomebenefitswhen“doing”Agile,suchasincreasedvisibility,productivity,andthe opportunity to change direction. However, the magnitude of these benefits does not compare to thosereaped by the organizations that do acquire the Agile mindset and “become” Agile. Ahmed Sidky definescompaniesthat“become”Agileasthosethatembodythevaluesandprinciples,andworkinsmallincrementstodelivervaluecontinuouslyinordertoreceivefeedback.Thankstothedigitalrevolution,powerhasshiftedfromthesupplier to thecustomer.Customersnowhavemanydifferentoptionsrightat their fingertips.Theorganizations that deliver value early andoften and incorporate customer feedback into their productswillhaveabetterchanceofachievingbetterbusinessoutcomes.

WhyDataCanBeBothInvaluableandDetrimentaltoChangeInitiatives:Page-3

Figure1.AgileAdoptionBenefitTimeline

Organizationsearlyintheirjourneyoronesthatdecidetonever“be”Agilewillstillhaveleaderswhouseacommand-and-controlapproach.Command-and-control leadersusedataandmetrics todrivebehaviorsandmeasurepeople’sperformance.Peopleworkinginacommandandcontrolenvironmenttypicallydonottrustthosearound them,and thereforedonot feel safe.When trustandsafety isabsent,peoplewillbe fearfuloftransparency and honesty. As a result, the data is typically fabricated, gamed, and not reflective of reality.Sadly, this leads to strategic decisions being made on false data which compounds the dysfunction, andultimatelypreventstheorganizationfromreachingitsgoalofhappyemployeesandbetterbusinessoutcomes.

For example, I once asked a team why they were committing to more story points then their averagevelocity. It turns out that theywere fearful of reprimandbecause if theyused their velocity to forecast, theprojectwouldappearofftrackandtheirmanagerwouldlookbad.Thisisaperfectexampleofwhythelackoftrustandsafetywithinanenvironmentleadstodetrimentalresultsandisalosingpropositionforeveryone.

Iftheteamwouldhavefeltsafetobehonestabouttheircapacityandforecast,itwouldhaveprovidedtheopportunityfortheProductOwnertomanagestakeholderexpectations,anddiscussreprioritization.However,sincetheteamdidnot,thecustomerwasnotawareoftheriskuntilthelastminute,leavingnotimetomanageexpectationsanddiscusstrade-offs.Ultimately,thegoalofsatisfyingthecustomerwasnotachievedduetolackofsafetyandtrustwithintheenvironment.

As coaches and change agents, we need to proceed with caution when utilizing data early in an Agileadoptioninitiative,asitcouldleadtomoreharmthangood.

WhyDataCanBeBothInvaluableandDetrimentaltoChangeInitiatives:Page-4

Figure2.DataEffectivenessforPositiveChange–Detrimental

2.3 TheSuccessAsluckwouldhaveit,afewengagementsdowntheroad,IstumbledacrossanorganizationthathadamatureAgilemindsetandthegoal to“be”Agile.Therewasneveradoubt thatAgilewasagoodfit.Theculturewasbuilt on alignment, autonomy, trust, and safety. They wanted to take the next step forward in their AgilejourneysotheyhiredafewAgilecoachestohelpthemgetthere.

AsanAgileCoachingCenterofExcellencetaskedwithimprovingtheflowofworkwithineachvaluestreamacrosstheorganization,wesawdataasourally.Wecreatedadirect feedbackchannel fromthosedoingtheworktothosemakingdecisionsbycollectingmultipledatapointsfromteamstoidentifypatternsimpedingtheorganizationfromachievingitsdesiredoutcomes.

Theorganizationhadanopen-mindedleadershipteamwhoembracedservantleadershipandwaswillingtobecoached.Theywerereceptivetoviewingthedataasanadditionalfeedbackloop,helpingthemidentifytoidentify behavioral patterns, organizational constraints, coaching needs, and continuous improvementopportunities.Thedatawasseenmoreasameanstoanend;helpingpeopleandtheorganizationachieveitsgoals.

As a result of trust and safety beingpresent, teammembers reported accurate data. Itwas reliable andactionable. It was used to make strategic decisions that benefited everyone in the organization, ultimatelyleadingtobetteroutcomes.Forexample,velocitywasusedtoforecastthecompletiondateoftheproject.Ifthedate was unacceptable due to business needs, it gave Product Owners and stakeholders ample time to re-prioritize. Once the AgileMindset is acquired and people feel safe, data becomes a change agent’s ally, andinvaluabletoolstodrivepositivechange.

WhyDataCanBeBothInvaluableandDetrimentaltoChangeInitiatives:Page-5

Figure3.DataEffectivenessforPositiveChange–Invaluable

3. DATAGATHERINGTECHNIQUESTODRIVEPOSITIVECHANGEINASAFEENVIRONMENT

3.1 AgileTeamSelf-AssessmentsThewordassessmenthasbecomeadirtywordintheAgilecommunityandIcanunderstandwhy.Themarkethasbeenlitteredwithanabundanceofassessments,bringingtheircredibilityandvalueintoquestion.Ontopof that, organizations are abusing them, utilizing them as team audits. We thought long and hard aboutwhetherornottoadoptassessments,butconcludedthatgiventhesafeenvironment,theycouldbeusedasatool for learning Agile practices, identifying team coaching needs, and recognizing organization systemicconstraints.

Werecognizedassessmentsarenotonesizefitsall!Rather,theyshouldbetailoredtoboththeteamandorganization’schallengesandobjectives.AllAgilecoachescollaboratedonacatalogofover100triedandtrueAgileandLeanPracticesdirectlycompiledfromthespecificframeworkliterature.Eachpracticewasmappedto one or many general business outcomes such as time to market, employee satisfaction, customersatisfaction,innovation,reliability,responsiveness,andpredictability.Thecatalogwasusedtoidentify“when”toexperimentwithapracticeand“how”toproperlyapplyandmeasuretheeffectivenessof it.Eachpracticeconsisted of a set of criteria for each of the five agility stages; blocked, developing, emerging, adapting,optimizing.

Theteamwasempoweredtoassessthemselves.Afterall,thevalueisintheconversation!Weworkedwithteams on facilitation and consensus voting techniques, and helped conduct assessments in which teammembersfeltcomfortabletoexpresstheiropinions.Therewasnoscoringsystem,justasetofcriteriaforeachagilitystageenablingteamstoassesstheirfluencyoneachpracticeandsetgoalsforimprovement.

Finally,wewereabletocreatedashboardsthatconsolidatedcross-assessmentandcross-teamresultsinaconsumableandactionableformat.Aggregatedresultswerecreatedfromteamswithinaspecificbusinessunitoraprogram.Thisapproachprovidedaholisticviewofthegroup’sperformanceonpracticesthatcontributedto each general business outcome. Each unit or program had slightly different objectives andwould assessthemselvesonthepracticesthatleadtotheirdesiredbusinessoutcome.

WhyDataCanBeBothInvaluableandDetrimentaltoChangeInitiatives:Page-6

Figure4.Self-AssessmentDashboard

For example, we worked closely with the business unit responsible for supporting the organization’s

payments application. Given that this was a mission critical application, the team’s focus was mainly onreliabilityandquality.After twoassessments, itbecameobvious the team felt they lackedknowledge inkeytechnical engineering practices that lead to high quality and reliability. This was a scenario in which ourcustomizedself-assessment frameworkenabledusto identifyacoachingneedfora teamwhichsupportedamission-criticalapplicationvital to theorganization’ssuccess.The teamwasassigneda technical coachwhofocused on the adoption of Agile engineering practices such as test-driven development, continuousintegration,andtestautomation.

Theoverallagilitystageofeachteamandgroupwasalsotracked.Theoverallagilitystageoftheteamwasdeterminedbycalculating theaverageofallquestionsansweredby them inagivenassessment.ThemotivebehindthisdatapointwastoensurethattheoverallAgilefluencyoftheteamwascontinuingtoimprove.Inone scenario, we identified a team that did not improve their overall agility stage after six quarters. Afterengagingtheteam,wefoundtheylackedtheskillsetstosupportthetechnicalstackoftheproduct,andwereforced to hire contractors to fill this void. The organization did not retain a contractor for more than sixmonths,sotheteamreceivednewteammembersonthesamecadence.Itwasobviousthatthechurnofteammembersprevented the team from forming teamnorms, and forced them toessentially startoverevery sixmonths.Afterfurtherdiscussion,itturnedoutthegroup’smanagementdidnotquiteunderstandtheproduct’stechnology,andthelevelofeffortandknowledgeneededtobestsupportcustomers.Theconclusionwasthatthebenefitofhiringafull-timeemployeeinthisrolefarexceededthecost.

3.2 Cross-TeamImpedimentRoll-UpsAnotherdatacollectingapproachweutilizedwascollatingteamimpedimentsthattheycouldnotcontrol.Thegoalwastoidentifycommonimpedimentsandattempttoquantifytheimpactinordertodriveprioritizationofcontinuous improvement opportunities. We created an internal tool that enabled scrummasters to recordeventsandblockersinrealtimeoverthecourseofthesprint.Itwasdubbedthe“SprintDiary.”Attheendofeach sprint we would collect the data and identify common impediments encountered by the teams. Theoutputofthisexercisewasitemsforourcontinuousimprovementbacklog.

In order to prioritize the continuous improvement backlog, we implemented a lightweight impactcategorizationsystemtohelpteamsclassifyblockersandquantifytheimpacts.Aftercollaboratingwithmanyteam members and leaders, we concluded that a majority of team blockers fell within one of six generalcategories;staffing,unplannedwork,dependencies,environments,administration,andbusinesscollaboration.Theimpactofeachblockertoteamperformancewasratedonascaleofonetothree,withthreerepresentingthehighestimpact.Weencouragedteamstorankeachblockerrelativetooneanother,butalsoprovidedsomeguidelinesbasedonthepercentageofproductivitylostduetotheblocker.Weacknowledgedthattherewasno

WhyDataCanBeBothInvaluableandDetrimentaltoChangeInitiatives:Page-7

hardandfastsciencetocalculatetheimpact.Instead,weempoweredtheteamstodeterminetheimpactinamannertheydeemedappropriatefortheirgivencontext.

Attheendofeachsprint,datawouldbecollatedacrossallteamsinordertoprovideaholisticpictureof“what”wasimpactingteamperformance.Thecoachesinconjunctionwithleadershipwouldanalyzethedatainsearchforcommonpatternsandtrendsofblockersinhibitingteamperformance.Forexample,thistechniqueenabledthecoachingpracticetoidentifythatthesharedQAenvironmentaveragedowntimewasnearlyadayand a half per sprint, impacting over 13 teams. When we presented this case to executive leadership andquantified the delays, there was an investment made in upgrading the QA environment and increasing itsstability.

Figure5.CrossTeamImpedimentImpactTrends

3.3 HappinessIndexEmployeeengagementandhappinesswasakeypillarof theorganization.Therefore,wewouldperiodicallytake the pulse of individuals on the teams to see how theywere feeling and how it changed given specificevents. The data gathering technique become known as the “Happiness Index.” Thanks to some of the IOSdevelopers,wecreatedanappthatrandomlypushedoutnotificationstoteammembersoverthecourseofthesprint.Thenotificationwouldasktherecipient“howareyoufeeling?”Theresponseoptionsweremad(1),sad(2), indifferent (3), and happy (4). Teammembers could respond autonomously, or not respond at all. Allresponseswereaggregatedsothatteamscouldseetheirhappinesstrendsovertime.Thismetricbecametheonemostmonitoredbyleadershipbecausetheysawhappyemployeesasaleadingindicatortosuccess.

Figure6.TeamSprintHappinessIndexTrends

Oncewehaveenoughdatapoints,weoverlaidthehappinessindexwiththesprintdiary.Combiningthese

two data sets was a game changer for the organization. The view enabled everyone to see how team andorganizationeventswerealteringpeople’sfeelingsandemployeesatisfaction.

02468101214161820

Sprint1 Sprint2 Sprint3 Sprint4 Sprint5

AggregateIm

pacts

Q1SprintRollupImpactTrends

Staffing

UnplannedWork

Dependencies

Environments

Admin

BizCollaboraEon

01020304050

MON TUE WED THU FRI MON TUE WED THU FRI

VipersSprint5AggregatedHappinessIndex

WhyDataCanBeBothInvaluableandDetrimentaltoChangeInitiatives:Page-8

Figure7.TeamSprintHappinessIndex/SprintDiary

4. LESSONSLEARNEDFROMMYEXPERIENCE

• CollectingdatainanAgileEnvironmentisanadditionalfeedbackloopyoucanusetomakedecisionsandidentifyorganizationconstraints.

• Trustandsafetyistypicallyabsentinacommandandcontrolworkenvironment.Thiswillleadpeopleto be fearful of transparency and honesty. As a result, the data collected from teams is typicallyfabricated, and therefore strategic decisions are made on false data, which compounds theorganization’s dysfunction, andultimately prevents the organization from reaching its goals. In thiscase,changeagentsshouldbeverycautionarywhenintroducingnewteamdatagatheringtechniques.

• Once an Organization acquires the Agilemindset, and embraces the values and principles of Agile,peoplewillfeelsafetobetransparentandhonest.Inthesetypesofenvironments,datacanbecountedon to make strategic decisions. In this case, change agents should encourage new data gatheringtechniques.

• Aggregating data points to tell a meaningful story helps Agile coaches and leadership to identifybehavioral patterns, organizational constraints, coaching needs, and continuous improvementopportunities.

• AgileTeamself-assessmentsandhappinessareleadingindicatorsofwhetherornotorganizationsaretherightpathofachievingbetterbusinessoutcomes.

5. ACKNOWLEDGEMENTS

Iliketotaketheopportunitytothankmycolleagues,TroyLightfootandJamesGifford.TroyandJamesactivelycollaborated with me to produce these unique data gathering techniques. Together, we have createdwww.leanagileintelligence.com so other organizations can utilize the techniques to achieve better businessoutcomes.

I also like to thank Sue Burk, my mentor in this process. This is the first time I have published anexperiencereportsoIleanedheavilyonSueforguidanceandfeedback.Hervastamountofexperienceinthisareawasevident.Ihavelearnedsomuchfromherduringthisshorttime.

6. REFERENCESGupta,Amit”InsightsFromComplexityTheory:UnderstandingOrganizationsBetter”:http://tejas.iimb.ac.in/articles/12.phpSidky,Ahmed:https://www.stickyminds.com/presentation/agile-vs-agility-doing-vs-beingSahota,Michael:http://agilitrix.com/2016/04/doing-agile-vs-being-agile/Hartman,Bob:https://www.slideshare.net/lazygolfer/doing-agile-isnt-the-same-as-being-agile