risk register - fermilab
TRANSCRIPT
HL-LHC CMS Detector Upgrades Project CMS-doc-13480 Version 3.0
Lucas Taylor, 06-Mar-2019 1
Risk Register For DOE Scope
ThisdocumentisasnapshotoftheWeb-basedriskregisterthatisusedtohelpidentify,assess,andmanage risks. The register includes threats (with negative impacts); opportunities (with positiveimpacts);andproject-wideuncertainties(witheithernegativeorpositiveimpacts,suchasexchangerateuncertainties).Theserisksaredescribedherein.
The general procedures and techniques used to identify and assess risks, perform quantitativeanalysis,developmitigationandresponseplans,andcontrolandreportonrisksaredescribedhere:
[1]“FermilabRiskManagementProcedureforProjects”,PPP-doc-65.
http://ppp-docdb.fnal.gov/cgi-bin/ShowDocument?docid=65.
Themodestamountofproject-specifictailoringofthisprocessisdescribedhere:
[2]HL-LHCCMSUpgrades“RiskManagementPlan”,CMS-doc-13749.https://cms-docdb.cern.ch/cgi-bin/DocDB/ShowDocument?docid=13749
TheMonteCarlomethodologies thatweuse to:analyze theconsequencesof risks;aggregatecostand schedule impacts, including cost estimate uncertainties; and establish the cost and schedulecontingencyneedsoftheprojectaredocumentedelsewhere:
[3]HL-LHCCMSUpgrades“RiskandContingencyAnalysis(DOEScope)”,CMS-doc-13481.https://cms-docdb.cern.ch/cgi-bin/DocDB/ShowDocument?docid=13481
TheriskshereinarelistedinorderofRiskID,whichhasthesyntaxRx-402-n-iii.• Rx=RT(forthreats),RO(foropportunities),andRU(foruncertainties);• 402denotestheP6IDoftheproject(theriskregisteriscommonacrossFermilab);• ndenotestheL2WBSID:
o 0=ProjectManagement (NSFscope–notshownhere);o 1=ProjectManagement (DOEscope);o 2=OuterTracker (DOEscope);o 3=BarrelCalorimeter (NSFscope–notshownhere);o 4=EndcapCalorimeter (DOEscope);o 5=EndcapMuons (NSFscope–notshownhere);o 6=TriggerandDAQ (DOEscope);o 7=ForwardPixels (NSFscope–notshownhere);o 8=TimingLayer (DOEscope);ando 9=TriggerandDAQ (NSFscope–notshownhere).
iii=001,002,etc.isanumericalriskidentifier.
HL-LHC CMS Detector Upgrades Project CMS-doc-13480 Version 3.0
Lucas Taylor, 06-Mar-2019 2
Document Change Log
Version Date Changedby ChangesV-0.1 2018-01-18 LucasTaylor FirsttemplateversioncreatedbasedonNSFPDRversion.V-1.0 2018-03-15 LucasTaylor VersioncreatedforDOECD-1Director’sReview.V-2.0 2018-05-08 LucasTaylor VersioncreatedforDOECD-1Review,includingnewsummarytableofrisks.
V-2.1 2018-05-15 LucasTaylor Updatecontributedlaborrisk:newTimingLayerdataandcostedlaborrate.
V-3.0 2019-03-06 LucasTaylor VersioncreatedforDOECD-1(v2)Director’sReview,March2019.
Admin note: how to update this document:
(1) ConnecttotheWindowsTerminalServer(youneedWindowsExplorerfortheSharePointexport).(2) Openthewebbasedriskregister1.(3) Openview=ADMIN--Exceldumpforproject=XXX.(4) Checkthefiltersareappropriate(e.g.project,fundingsource,riskstatus,etc.)(5) Exportregister toExcel.TheExcel filehashiddencharactersand“dataconnections” thatcorrupt the
restoftheprocess.SoopentheExcelfile,selectallthetextanddo“ClearFormats”.Thencopy-pastethecellsofinterestBYVALUEintoafreshExceldocumentandsaveit.
(6) InthefreshExceldocument,searchfordouble-quotesintheExcelfile.ReplacethemwithsinglequotesinthewebRiskRegisterbecausetheycorruptthemailmerge.ThenredotheexporttoExcel.
(7) Adjust the Excel formatting (e.g. set date format, set percentage format and precision, set P*Impactprecisions).Wherepossiblefixtherootcauseintheriskregisterandredotheexport.
(8) DeletespuriouscolumnsintherightoftheExceltable(theextraonesthatSharePointexportsforyouwithoutyourasking).Theysometimeshangthemailmerge.
(9) Update theMSWordmail merge template (e.g.2019-03-06---CMS-HL-LHC---Risk-Register---mail-merge-template.docx,storedinCMS-doc-13480)withthenewExceldata.
(10) CheckWordformattingandtidyupifnecessary(preferablyfixtherootcauseintheriskregisterandredotheexport).Forsomereasonrandomtextfieldscangetcorrupted(eventhoughtheylookfineinSharePointandExcel).Ifyoucannotfixtherootcause,copypastetheuncorruptedtextbyhand.
(11) InWordriskregister:usespecialmenuofReplacemenutochangeallSectionbreakstoPagebreaks.(12) Addheaderandfooterwithpagenumbers,ifdesired.(13) Pastethefinalresultsfromthemailmergeintothisdocument(e.g.2019-03-06---CMS-HL-LHC---Risk-
Register---cover-page.docx, stored in CMS-doc-13480), check it formats OK, and generate a PDF.AlternativelymaketwoPDFsandmergethemwithe.g.AdobeAcrobatPro.
(14) UploadeverythingtoDocDB:CMS-doc-13480.
1https://fermipoint.fnal.gov/organization/ocoo/ippm/SitePages/ProjectRiskManagement.aspx
HL-LHC CMS Detector Upgrades Project CMS-doc-13480 Version 3.0
Lucas Taylor, 06-Mar-2019 3
Summary table of risks The table below shows the risks that are included in the MC model, including their estimatedprobabilities, and cost and schedule impacts. A risk identifier (RI-ID)with a prefix RT denotes athreat, RO denotes an opportunity, and RU denotes a risk uncertainty. The cost and scheduleimpactshaveeitherone,twoor(usually)threevaluesdependingonwhetheritisa1-pointestimate(singlevalue),a2-pointestimate(i.e.aflatrangep.d.f.),ora3-pointestimate(i.e.atrianglep.d.f.).
HL-LHC CMS Detector Upgrades Project CMS-doc-13480 Version 3.0
Lucas Taylor, 06-Mar-2019 4
HL-LHC CMS Detector Upgrades Project CMS-doc-13480 Version 3.0
Lucas Taylor, 06-Mar-2019 5
TheremainderofthisdocumentlistsallrisksorderedfirstbyWBS,thenbyriskID.
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 1 Updated: 2019-03-06
RO-402-2-03-DOT-Moduleassemblycanbeautomated
RiskRank: 3(High)Scores:Probability:5(VH);Cost:2(M)Schedule:1(L)) RiskStatus: OpenSummary: Ifautomationinmoduleassemblycomestofruition,thenlaborcostsandscheduledurationsbothdecreaseRiskType: Opportunity Owner: LeonardGSpiegelWBS: 402.2OT-OuterTracker RiskArea: ExternalRisk/FacilitiesProbability(P): 66% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =1-point-singlevalue
Minimum =k$Mostlikely =-1000k$Maximum =k$Mean =-1000k$P*<Impact> =-660k$
ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =-2monthsMaximum =monthsMean =-2monthsP*<Impact> =-1.3months
BasisofEstimate: Areductionof$1.6Minfullyburdenedlaborcoststhroughouttheproductionperiodisdeterminedassumingthat2techniciansperassemblysitearereplacedwith2students(uncostedgraduatestudentsatFNALandmoderatelycostedundergraduatesatBrown=$30/hr).Thecostoftwogantryrobots($150keach)andengineering+programmingdevelopmentcosts($300k)leadstoa$1.0Mopportunity.Theimpactonthescheduleisexpectedtobe2monthsofsavingsbasedonthecurrentestimateofthelaborrequiredtocarryourthestepsbeingconsideredforautomation.
CauseorTrigger: ImpactedActivities: AllPSand2SModuleassemblyactivitiesinaggregate.Implementedasariskhookbetweenstartandcompletionofproductionof2Smodules.
Startdate: 1/Oct/2019 Enddate: 1/Oct/2022RiskMitigations: IntheR&Dphasewewillpro-activelyexploreautomatingcertainstepsinthemoduleassemblyprocesswiththeaimofrealisingtheassociatedcost
andschedulesavings.RiskResponses: Accepttherisk,producemodulesmoreefficientlyMoredetails:
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 2 Updated: 2019-03-06
RO-402-2-08-DOT-CanusecheaperCarbonFoamforrings
RiskRank: 2(Medium)Scores:Probability:4(H);Cost:1(L)Schedule:0(N)) RiskStatus: OpenSummary: IfcheaperCarbonFoamshowsacceptableperformancefortherings,thepotentialdecreaseinthecostofthecarbonfoammaybenefittheproject
budgetRiskType: Opportunity Owner: StefanGruenendahlWBS: 402.2OT-OuterTracker RiskArea: ExternalRisk/MarketProbability(P): 50% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =1-point-singlevalue
Minimum =k$Mostlikely =-60k$Maximum =k$Mean =-60k$P*<Impact> =-30k$
ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =0monthsMaximum =monthsMean =0monthsP*<Impact> =0.0months
BasisofEstimate: Estimatethatthecostcouldbehalved(reducedfrom$120kto$60k).CauseorTrigger: Ringprototypesbuiltwithmechanicalgrade(cheaper)carbonfoammeet
QCcriteria(precision,stiffness)
ImpactedActivities: Carbonfoamprocurements
Startdate: 1/Jan/2020 Enddate: 31/Dec/2024RiskMitigations: Weareevaluatingringprototypesbuiltwithmechanicalgrade(cheaper)carbonfoam.Resultsobtainedsofar(firstpairofprototypes)look
promising.RiskResponses: AcceptopportunityMoredetails:
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 3 Updated: 2019-03-06
RO-402-8-01-DETL-UseAltiROC
RiskRank: 2(Medium)Scores:Probability:2(L);Cost:2(M)Schedule:3(H)) RiskStatus: OpenSummary: IftheAltiROCmeetsourspecification,wewillbeabletouseitandwewillnothavetopayforthecostoftheproductionmaskset.
RiskType: Opportunity Owner: TiehuiLiuWBS: 402.8.4ETL-EndcapTimingLayer RiskArea: ManagementRisk/PlanningProbability(P): 10% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =1-point-singlevalue
Minimum =k$Mostlikely =-760k$Maximum =k$Mean =-760k$P*<Impact> =-76k$
ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =-8monthsMaximum =monthsMean =-8monthsP*<Impact> =-0.8months
BasisofEstimate: Theprobabilityisinitiallysetto10%sincetheinitialtestresultsoftheAlitROCv0doesnotmeetourspecification.WewillevaluatetheperformanceoftheAltiROCasnewversionsbecomeavailableandwewillremaininclosecontactwiththeAltiROCdevelopers.Thecostsavingscorrespoondstothecostoftheproductionmaskset.
CauseorTrigger: ImpactedActivities: Startdate: 1/Jan/2022 Enddate: 1/Jan/2023RiskMitigations: RiskResponses: Moredetails:
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 4 Updated: 2019-03-06
RT-402-1-05-DPM-Significantfundingdelayduringprojectexecution(DOE)
RiskRank: 3(High)Scores:Probability:3(M);Cost:3(H)Schedule:3(H)) RiskStatus: OpenSummary: ItisassumedthatDOEfundswillnotnecessarilyarrivepromptlyatthestartofeachfiscalyearforvariousadministrativeorotherreasons.In
addition,theProjectmaytemporarilyreceivesomewhatlessthantheexpectedleveloffundingifthereisaContinuingResolutioninplace.TheProjectexpectstobeabletocontinueworkingthroughsuch(non-catastrophic)scenarioswiththehelpofcarry-overfunds,suchasunspentcontingency.Thereis,however,amoresignificantfundingdelayrisk:ifthedecisiontoapprovetheProject’sinitialuseofMIEfunds(i.e.CD-3A)coincideswithaContinuingResolution(CR)–thatprohibitsany'newstarts'–thenallMIE-supportedworkwouldbeheldupforthedurationoftheCR.Thisdelaysmanyactivitiesandhasancostimpactduetoescalationandstandingarmycosts.
RiskType: Threat Owner: VivianO'DellWBS: 402.1PM-ProjectManagement(DOE) RiskArea: ManagementRisk/FundingorResourcesProbability(P): 21% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular
Minimum =0k$Mostlikely =0k$Maximum =3339k$Mean =1113k$P*<Impact> =234k$
ScheduleImpact: PDF =3-point-triangularMinimum =0monthsMostlikely =0monthsMaximum =9monthsMean =3monthsP*<Impact> =0.6months
BasisofEstimate: CongressionalResearchServicedataonCRdurations(https://fas.org/sgp/crs/misc/R42647.pdf)indicatethatinthelasttwodecadestherehavebeenmorethanfiveCRsperyearlastinganaverageof138daysperyear(4.5monthsor38%ofthewholeyear)and,inthreecasesforanentirefiscalyear.WeassumethatwewillseekCD-3Aapprovalatleast3monthsbeforethefundsareactuallyneeded.ThereforethefirstthreemonthsofaCRwouldnothaveanimpact.Wefindthaton21%ofthedaysfromFY1998–FY2016aCRwoulddelayCD-3Aapprovalbybetween0monthsand9months(i.e.21%ofthedaysanalyzedhaveaCRofdurationbetween3monthsand12months).Wemodelthisriskasatrianglep.d.f.withP=21%andmin/likely/max=0/0/9months.Thefullprojectburnrateis$371kpermonth(CMS-doc-13481)comprising$249kforstandingarmycostsand$122kforescalation.Thisestimatedstandingarmycostincludesthepossibilitythatsome,butnotall,personnelcanchargetheirtimetootheroff-projectactivitiesduringthedelay.9monthsat$371k/monthis$3,339k.
CauseorTrigger: Changesinfundingagencypriorities,orinCongressionalappropriationsduetopoliticalorotherreasons.
ImpactedActivities:
Startdate: 1/Jan/2018 Enddate: RiskMitigations: Itisassumedthatmodestfundingdelaysareinevitable.Theprojectplanwillmitigatethepotentialimpactsbyavoidinglargeprocurementsearly
ineachfiscalyearandusingcarry-overfunds(e.g.unspentcontingency).TheProjectwillaimtoseekapprovaltospendMIEfunds(i.e.'newstart'approval)inthesecondhalfofthefiscalyearifthisispossiblegiventhevariouspre-requisites(e.g.R&Dresults,vendorinformation,etc.).ThiswillreducethelikelihoodofcoincidingwithaCR.
RiskResponses: Ifsignificantdelaysduetolimitedavailabilityoffundsstarttolookinevitable,theprojectwillpro-activelyre-prioritiseworktominimisetheconsequencesforcriticalpathactivities.Theprojectwillworkcloselywiththeagenciestooptimizetheflowofevenlimitedfundstoensuretheprojectdoesnotgrindtoahalt.
Moredetails: CMS-doc-13481
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 5 Updated: 2019-03-06
RT-402-1-09-DPM-ES&HincidentatUS-CMSinstitute(DOE)
RiskRank: 1(Low)Scores:Probability:1(VL);Cost:2(M)Schedule:1(L)) RiskStatus: OpenSummary: Ifacontractororlaboratory/universityemployeeisinjuredorifmaterialisseverelydamagedinansafetyincident,orifanunplannedenvironmentalreleaseoccurs
duringtheconstructionorinstallationphases,theresultinginvestigationandimplementationofcorrectiveactionscouldshutdowntheprojectandcausescheduledelays,dependingontheseverityoftheincidentanditsimpactoncriticalpathwork.
RiskType: Threat Owner: GregoryRSnowWBS: 402.1PM-ProjectManagement(DOE) RiskArea: TechnicalRisk/ES&HProbability(P): 5% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular
Minimum =4k$Mostlikely =52k$Maximum =471k$Mean =176k$P*<Impact> =9k$
ScheduleImpact: PDF =3-point-triangularMinimum =0.25monthsMostlikely =0.5monthsMaximum =2monthsMean =0.92monthsP*<Impact> =0.0months
BasisofEstimate: ThisriskassessmentisbasedonexperiencewithCMSPhase1Upgrades(only1minorES&Hincidentin5years)andexpertinputfromAmberKenney,DeputyHeadofFermilabESH&Q.Theapparentlyhighvalueoftheprobability(forasafety-relatedrisk)reflectstheinvolvementormorethan30institutesforaperiodofabout5years.Ifasafetyincidentoccurs,theinstitute,hostLab,DOE,NSForotherstakeholdersmayrequireaworkstoppageuntiltheyaresatisfiedtheissuesareresolvedandworkcanresume.Historicalexperienceindicatesthisisusuallyafewdaysorweeksbutinexceptionalcasesitcouldbemonths,particularlyifthereisafatality.Thecostimpactmayincludereworkorrepairs(beyondwhatiscoveredbyinsurance)andburnrates(asdescribedinCMS-doc-13481).Thescheduleimpactincludesdelaysduetoworkstoporderswhiletheincidentisanalysedandcorrectiveactionistaken.Tocapturearangeofpossibleconsequencesweconsiderthreescenarios:Minimpact:OneL3areaisimpactedfor1weekwithoutanydamage.TheaverageL3burnrate($18k/month)resultsinacostimpactofabout$4k(for1week).Likelyimpact:OneL2areaisimpactedfor2weeks.Thereis$20kofcorrectiveactioncost.TheaverageL2burnrate($68k/month)resultsinacostimpactof$32k(for2weeks).Totalcostimpact=$52k.Maximpact:Aseriousincidentresultsin$100kofcorrectiveactioncost.Aworkstoporderhaltsrelatedactivitiesproject-widefor2monthsasfailingsareaddressed.Someactivitiescontinue(e.g.design,firmware,software,etc.).Weassumethemonthlyburnrateinthisscenariois50%ofthetotalprojectburnrateof$371k/month.Thetotalcostimpact=$100k+2months*50%*$371k=$471k.
CauseorTrigger: DespiterigorousES&Hprogramatcollaboratinginstitutes,anaccidentcouldhappenduetounsafepractices,beingdistractedwhileworking,beinglaxaboutattentiontoenvironmentalregulatoryrequirements,orpoorenforcementofcontractES&Hprovisions.
ImpactedActivities:
Startdate: 1/Jan/2018 Enddate: RiskMitigations: ComprehensiveuseofISMprogramacrossentireprojectincludingcompleteHAprocess;goodESHcontractualprovisionsaswellasenforcement;workingupfront
duringconstructioncontractstogainbuy-inofconstructioncontractormanagementtoESHissueprevention.Oversightrolesandresponsibilitiesforcontractorandprojectteamtoincludeensuringadequatetrainingandattentiontositespecificrisks,andcompliancewiththeirESHprogram.ThehostLabsandagenciesprovidereview/oversightonregularbasis.
RiskResponses: Moredetails: CMS-doc-13481
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 6 Updated: 2019-03-06
RT-402-1-12-DPM-Majorimportorexportissue(DOE)
RiskRank: 2(Medium)Scores:Probability:2(L);Cost:1(L)Schedule:2(M)) RiskStatus: OpenSummary: Thereisamajorunforeseenimport/exportissue.Forexample,ruleschangeinsuchawayastodelayorincreasethecostofimportingorexporting
certainitems(e.g.radhardcomponents).Ifinsufficientinteragencyandinternationalregulatorycoordinationhasoccurred,thenitemsmovedbetweencountriesmaybedelayedorsubjecttoadditionalcosts.
RiskType: Threat Owner: VivianO'DellWBS: 402.1PM-ProjectManagement(DOE) RiskArea: ExternalRisk/RegulatoryProbability(P): 20% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular
Minimum =18k$Mostlikely =56k$Maximum =172k$Mean =82k$P*<Impact> =16k$
ScheduleImpact: PDF =3-point-triangularMinimum =1monthsMostlikely =2monthsMaximum =4monthsMean =2.33monthsP*<Impact> =0.5months
BasisofEstimate: Themainimpactisadelaywhileissuesareresolved.Unexpectedcustomsdelaysofseveralmonthshavehappenedinthepast.Allowforalongtail(butunlikely)toevenhighervaluesif,forexample,thereisanissuethatalsorequirestheinvolvementofthehostLabs,fundingagencyortheStateDepartment.Directcostimpactcouldbesay10%ofextradutyongoodsworthupto$1Moruptoabout$100k.Directlaborcostsarealreadycoveredon-project(e.g.management)orbyindirectcosts(e.g.Labprocurementofficers).Theburnratecosts(standingarmyandescalation)aredescribedinCMS-doc-13481.WeassumethedelayimpactsoneL3systemwithatypicalburnrateof$18kpermonth.Minscenario:1monthdelay.Nodirectcostimpact.$18kofburnrate.Likelyscenario:2monthsdelay.$20kofdirectcosts.$36kofburnrate.Maxscenario:4monthsdelay.$100kofdirectcosts.$72kofburnrate.
CauseorTrigger: ImpactedActivities: Startdate: 1/Jan/2018 Enddate: RiskMitigations: Pro-activelyworkwithhostlabandfundingagencyprocurement,shipping,andimport/exportexpertstoidentifypotentialissues.Prepareplans
andworkinadvancetoensurethatallrulesandregulationsarecompliedwith.Monitorshippingactivitiesclosely,talktogovermentagencies,putassumptionsaboutwhopaysdutiesintocontracts.Considerholdinglogisticsreviews.Talktootherprojectstobetterunderstandprobabilities&impactsandmitigationstrategies.
RiskResponses: Workwiththehostlabandfundingagenciesprocurement,shipping,andimport/exportexpertstoaddresstheissues.Moredetails: CMS-doc-13481
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 7 Updated: 2019-03-06
RT-402-1-90-DPM-KeyProjectOfficepersonnelneedtobereplaced(DOE)
RiskRank: 1(Low)Scores:Probability:3(M);Cost:1(L)Schedule:0(N)) RiskStatus: OpenSummary: Keyengineerorseniortechnicianwithspecialknowledgeleavestheprojectandneedstobereplaced.Ifthetransitioncanbemanagedsuchthat
theincomingandoutgoingpersonneloverlapandexchangeknowledge,thenthereismainlyalaborcostimpact.Ifthetransitionismoreabrupt,thenthereisnocostimpactofoverlappingpersonnelbuttherecanbeadelaytotheprojectactivitiesastheincomingpersongetsfullyuptospeed.Thisriskdoesnotincludetheriskoflosingkeymanagersbecausetheprojectensuresthateachmanagerhasawell-traineddeputy.
RiskType: Threat Owner: VivianO'DellWBS: 402.1PM-ProjectManagement(DOE) RiskArea: ManagementRisk/FundingorResourcesProbability(P): 25% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular
Minimum =30k$Mostlikely =60k$Maximum =90k$Mean =60k$P*<Impact> =15k$
ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =0monthsMaximum =monthsMean =0monthsP*<Impact> =0.0months
BasisofEstimate: Thehas2staffwhosefunctionsareparticularlyhardtoreplaceonashorttimescale,inparticulartheAssociatePMforCost,ScheduleandRisk,andtheProjectControlsLead.ExperienceintheCMSPhase1Upgradeprojectsuggeststheprobabilityperkeypersonisroughly25%(probabilitytheyleaveduringtheentireproject).Thelaborcostperperson(acanonicalrate)is$15k/month(fullyburdenedrate=$185k/year).Minimumscenerio:personleavingisreplacedbyanexistingskilledperson.1monthoffulloverlapcosting$15k.Noscheduledelay.Likelyscenario:personleavingisreplacedbyanexistingpersonlessfamiliarwiththespecificwork.2monthsoverlaptotrainnewpersoncosting$30k.Maximumscenario:personleavesunexpectedlywithnotransferofknowledge.Significanttimehireanewpersonplus3monthsoflearningtimetobecomefullyproductiveatacostof3*$15k=$45kextralaborcost.Totalfor2persons:Min/Likely/Maxcostimpact=2*$(15/30/45)k=$(30/60/90)k.Ineachscenario,itisassumedthatothersintheteam(projectmanagersandprojectcontrols)cancovercriticaltaskssuchthatthereisnonetscheduleimpact.Inefficienciesinthistransitionarecoveredbythecostimpactsabove.
CauseorTrigger: ImpactedActivities: IndividualpersonsareindependentlymodeledintheMC(eachwithcorrespondingfractionofthetotalcostimpact).Typicallyhooktherisktothepersonsmainactivityduringtheproductionphase(maximumconsequences).
Startdate: 1/Oct/2018 Enddate: RiskMitigations: KeyprojectofficestaffarematrixedatFermilabandcanbereplaced(withsomeinitiallossofefficiency)byotherLabstaff,contractors,ornew
hires.Intheveryshortterm,theprojectmanagerscanstepintosustainkeyfunctionsoftheprojectoffice.RiskResponses: Aimforoutgoingandincomingpersonneltooverlapifpossibletoensureasmoothtransition.Hirefromskilledmembersoftheexisitngteamif
appropriate.InterimsupportmaybepossibleusingstaffintheFermilabOfficeofProjectSupportServices.Moredetails:
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 8 Updated: 2019-03-06
RT-402-2-01-DOT-Sensorqualityproblemduringproduction
RiskRank: 3(High)Scores:Probability:4(H);Cost:1(L)Schedule:2(M)) RiskStatus: OpenSummary: Ifthesensorvendordeliverssensorsthatdonotmeetspecificationsthenthedegradedperformanceofthetrackerjeopardizesthephysics
performanceoftheupgradeddetector.RiskType: Threat Owner: UlrichHeintzWBS: 402.2OT-OuterTracker RiskArea: ExternalRisk/VendorsProbability(P): 50% TechnicalImpact: 2(M)-significantlysubstandardCostImpact: PDF =3-point-triangular
Minimum =46k$Mostlikely =79k$Maximum =163k$Mean =96k$P*<Impact> =48k$
ScheduleImpact: PDF =3-point-triangularMinimum =2monthsMostlikely =3monthsMaximum =6monthsMean =3.67monthsP*<Impact> =1.8months
BasisofEstimate: ThecontractwillbewrittenforthevendortodeliveraspecifiednumberofgoodsensorsthatsatisfyCMSspecifications.Thuswedonothavetopayforsensorsthatdonotsatisfythespecificationsandthereisnoimpactonsensorcost.TheonlycostimpactisthatwewillhavetorepeattheQCtestingofthereplacementsensors.Minimalimpact:thishappensduringproductionandiscorrectedquicklyafterfeedbackfromsensorQCleadingtoadelayofabout2monthsandnegligibledirectcost.Maximalscheduleimpact:thishappensduringpreproductionandthepreproductioncyclehastoberepeated,leadingtoadelayofabout6monthsandextralaborcostofabout$25k(costforpreproductioncycleofonesensortype).TheL3burnrateduetothedelayofdownstreamactivitiesis$23k/month(CMS-doc-13481).Mincost=$0k+2months*$23kburnrate=$46k.Likelycost=$10k+3month*$23kburnrate=$79k.Maxcost=$25k+6months*$23kburnrate=$163k.TheproblemhastoeitherpersistovermanybatchesornotbenoticedduringQCatthevendor(forexampleadegradationofperformanceoversometime).Problemsthataffectasinglebatchofsensors(egbecauseofsomecontaminationorprocessingmistake)willnotleadtoasignificantdelaybecausereprocessingabatchwillonlyaddaweekortwototheproductionperiod.Basedonpastexperiencewiththevendorweexpectthistohappenatleastonceduringproductionandweassign50%probabilityforeachsensortype.
CauseorTrigger: Sensorsdonotsatisfyspecifications ImpactedActivities: Sensorprocurementactivitiesanddownstreamactivities.Thisappliestoeachtypeofsensor,buttheprobabilityshouldbe5%pertype(PS-s,PS-p,2S)
Startdate: 1/Apr/2020 Enddate: 31/Dec/2024RiskMitigations: Wecarryoutextensiveprototypingworkwiththevendorspriortoplacingthecontractforsensorproductiontomakesurethatvensorsunderstand
ourspecificationsandcanmeetthem.ThevendorwillcarryoutafirstsetofQCmeasurementsbeforethesensorsareshippedtoCERNanddistributedtoQCcenters.Thisensuresthatmostproblemswillbecaughtquicklyanddonotleadtosignificantimpactontheproject.Thecostofthesemeasurementsisfacturedintothesensorcost.
RiskResponses: Ifamodestproblemoccurs,workcloselywithvendortosolveit(e.g.testing).Replacetheflawedsensors.Moredetails: CMS-doc-13481
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 9 Updated: 2019-03-06
RT-402-2-06-DOT-TemporarylossofSensorQCSite
RiskRank: 2(Medium)Scores:Probability:2(L);Cost:1(L)Schedule:2(M)) RiskStatus: OpenSummary: IfaSensorQCfacilitytemporarilybecomesinoperableduetolossordamageofcriticalequipment(e.g.duetoawaterleak)thentheresultantdip
insensorthroughputmayjeopardizetimelycompletionoftheproject.RiskType: Threat Owner: UlrichHeintzWBS: 402.2OT-OuterTracker RiskArea: TechnicalRisk/ES&HProbability(P): 20% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular
Minimum =22k$Mostlikely =48k$Maximum =86k$Mean =52k$P*<Impact> =10k$
ScheduleImpact: PDF =3-point-triangularMinimum =1monthsMostlikely =2monthsMaximum =4monthsMean =2.33monthsP*<Impact> =0.5months
BasisofEstimate: Probability=20%isapproximatelyestimatedfrom2sites*10%persite.ThisisbasedonexperiencefromoriginalCMStracker,originalpixel,andPhase1pixelwhereoneincidentoccurredinO(10)sites.Ifonecenterhasamajorequipmentfailurethesecondcentercanpickuptheadditionalloadwithinthe100%cushion.Min/likely/maxdelay=1/2/4monthsdelayfortheinefficiencyinthelogisticstotransfermaterialsandpeoplebackandforth.Min/likely/maxrepairestimateis10/25/40k$.Thisassumesinsurancewillcoverloss/damageofmajorequipment.TheL3burnrateduetothedelayofdownstreamactivitiesis$23k/month(CMS-doc-13481).Mincost=$10k+1month*$23kburnrate=$33k.Likelycost=$25k+2months*$23kburnrate=$71k.Maxcost=$40k+4months*$23kburnrate=$132k.
CauseorTrigger: ImpactedActivities: ThisisimplementedastwoindependentriskeventsforthetwoQCsites(BrownandRochester).Ateachsite,3tasksareimpactedinacorrelatedway,representingtheQCworkonthe3sensortypes.TheimpactismodeledinthemiddleoftheQCwork(Lot5).
Startdate: 1/Apr/2020 Enddate: 31/Dec/2024RiskMitigations: Havingtwositesisalreadyahedgeagainstthecompletestoppageofsensortesting,andshouldonesitebecometemporarilyinoperable,sensorsw
ouldberedirectedtotheothersitetemporarilytomitigatetheimpact.RiskResponses: Sensorscanbedivertedtotheunaffectedsitetoutilizeitsfullthroughput,andadditionalresourcesaddedtoincreasemoduleproduction
throughputatbothsites(oncetheaffectedoneisre-established)toregaintimeintheschedule.Moredetails: CMS-doc-13481
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 10 Updated: 2019-03-06
RT-402-2-09-DOT-MaPSAyieldislowerthanexpected
RiskRank: 2(Medium)Scores:Probability:2(L);Cost:2(M)Schedule:0(N)) RiskStatus: OpenSummary: IfMaPSAyieldislowerthanexpected,theadditionalwastagealsosacrificestheassociatedsensorsandMPAchips,whichwouldneedtobereplaced
attheproject'scost.RiskType: Threat Owner: RonLiptonWBS: 402.2OT-OuterTracker RiskArea: TechnicalRisk/QualityProbability(P): 15% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =2-point-flatrange
Minimum =370k$Mostlikely =k$Maximum =640k$Mean =505k$P*<Impact> =76k$
ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =0monthsMaximum =monthsMean =0monthsP*<Impact> =0.0months
BasisofEstimate: Foreachlossof10%inyield,wewouldneed10%moresensors,estimatedat245k,and10%moreMPAchips,estimatedat125k.Therangecoverswastagebetween10and20%.ImplementedbetweenTestingbatch3andVendorproducingbatch5.
CauseorTrigger: Amyriadnumberofproblemsatthebumpbondingstagemightreducetheyield,orhandlingduringtheassembly.
ImpactedActivities: IncreasedwastageduringtheMaPSAassemblywouldrequireadditionalcomponents,namelyPSsensorsandMPAchips
Startdate: 2/Sep/2021 Enddate: 20/Sep/2023RiskMitigations: RiskResponses: Moredetails:
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 11 Updated: 2019-03-06
RT-402-2-10-DOT-VendorcannotperformMaPSAqualificationtests
RiskRank: 2(Medium)Scores:Probability:3(M);Cost:2(M)Schedule:0(N)) RiskStatus: OpenSummary: MaPSAqualificationisdoneatthevendorsite.Thecurrentcostestimatemayincreaseconsiderablyifthevendorsdonothavetheproper
infrastructuretoqualifytheparts.RiskType: Threat Owner: RonLiptonWBS: 402.2OT-OuterTracker RiskArea: TechnicalRisk/ComplexityProbability(P): 33% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular
Minimum =200k$Mostlikely =400k$Maximum =600k$Mean =400k$P*<Impact> =132k$
ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =0monthsMaximum =monthsMean =0monthsP*<Impact> =0.0months
BasisofEstimate: QualificationofMaPSAsmayrequiresophisticatedprobingequipment,whichcancostupbetween200-600kforprocurement,installation,andcomissioningoftherequisiteequipment,potentiallyatseveralvendors.
CauseorTrigger: ImpactedActivities: MaPSAprocurementcostswouldincrease.Implementedasacostincreaseafterround2ofMaPSAprototyping.
Startdate: 1/Jan/2019 Enddate: 14/Jun/2023RiskMitigations: RiskResponses: Workwithvendortoimprovetheirinfrastructureormovetestingtodifferentsite(othervendororcollaborator)Moredetails:
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 12 Updated: 2019-03-06
RT-402-2-11-DOT-MaPSAbumpbondingcostincreases
RiskRank: 2(Medium)Scores:Probability:2(L);Cost:3(H)Schedule:0(N)) RiskStatus: OpenSummary: CurrentlywehaveseveralMaPSAestimates,withaverybroadrangebetweenhighandlow,indicatingtheindustrydoesnotgiveaclearindication
oftheactualcost.Thisriskistocoverthepossibilitythatthishighcostitemexceedsthenominalestimateuncertainty,M5atthemoment.RiskType: Threat Owner: RonLiptonWBS: 402.2OT-OuterTracker RiskArea: ExternalRisk/VendorsProbability(P): 20% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular
Minimum =500k$Mostlikely =1000k$Maximum =1500k$Mean =1000k$P*<Impact> =200k$
ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =0monthsMaximum =monthsMean =0monthsP*<Impact> =0.0months
BasisofEstimate: CurrentlywehaveseveralMaPSAestimates,withaverybroadrangebetweenhighandlow,indicatingtheindustrydoesnotgiveaclearindicationoftheactualcost.Thisriskistocoverthepossibilitythatthishighcostitemexceedsthenominalestimateuncertainty,M5atthemoment.
CauseorTrigger: ImpactedActivities: ThecostsofMaPSAbumpbondingwouldincrease,increasingthecostsofPSmodulefabrication.
Startdate: 1/Jan/2019 Enddate: 14/Jun/2023RiskMitigations: Prototypeswillbeusedtovalidatelowbidders,forwhichthereisnotyetconfidenceofdeliveringwithrequisitequality.Thereismoreconfidence
forhighcostbidders,whichwillalsobevalidatedintheprototypingphase,buteventherethequotesarestillpreliminary.RiskResponses: Moredetails:
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 13 Updated: 2019-03-06
RT-402-2-14-DOT-Systemtesthardwarehasinsufficientcapacity
RiskRank: 2(Medium)Scores:Probability:2(L);Cost:2(M)Schedule:2(M)) RiskStatus: OpenSummary: Ifunforeseenproblemsoccurduringassemblyandtesting,thenthebaselinetestingsystemsmaynotbesufficienttomaintaintherequired
throughput.Thiswouldnecessitatetheprocurementandcommissioningofadditionaltestsystemsandadditionallabourfortesting.
RiskType: Threat Owner: AnadiCanepaWBS: 402.2OT-OuterTracker RiskArea: TechnicalRisk/ReliabilityorPerformanceProbability(P): 10% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular
Minimum =71k$Mostlikely =169k$Maximum =292k$Mean =177k$P*<Impact> =18k$
ScheduleImpact: PDF =3-point-triangularMinimum =2monthsMostlikely =3monthsMaximum =4monthsMean =3monthsP*<Impact> =0.3months
BasisofEstimate: Themaximumcostimpactcorrespondstoascenarioinwhichthecapacityofalltheproductiontestsystems(totalcostabout$200k)needstobedoubled,i.e.acostimpactof$200k.Theminimumandlikelycostsreflecttheneedtoduplicatepartsofthetestsystems.Themin/likely/maxscheduleimpactof2/3/4monthsisestimatedassumingthattheproblembecomesapparentduringproduction.TheL3burnrateduetothedelayofdownstreamactivitiesis$23k/month(CMS-doc-13481).Minimpact=$25k+2months*$23k/month=$71k.Likelyimpact=$100k+3months*$23k/month=$169k.Maximpact=$200k+4months*$23k=$292k.
CauseorTrigger: ImpactedActivities: Moremoduletestingequipmentwouldberequired,possiblymorecoldboxes,singlemoduletesting,orhybridtestingequipment.ImplementedasacostimpactonFNALPSmoduleproduction,A,EastCoastPSModuleproduction(B),FNAL2SModuleProduction(C),andEastCoast2Smoduleproduction(D).Sitesshouldbedelayedthesameamount,buttheprobabilityshouldbesplitevenlybetweenPS(A,C)and2S(B,D)
Startdate: 1/Jan/2022 Enddate: 31/Dec/2024RiskMitigations: Thetestinghardwareisanexternaldeliverable.WewillmonitortheprogressoftheUSCMSmoduleproductionandensurethatnewtesting
equipmentispurchasedwhennecessaryanddeliveredwhentheproductionrateisincreasedtomeettheschedule.Labourisincreasedaccordinlytosupportthehigherproductionrate.
RiskResponses: Moredetails: CMS-doc-13481
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 14 Updated: 2019-03-06
RT-402-2-23-DOT-Vendorisunabletoproducesensorstospecifications
RiskRank: 2(Medium)Scores:Probability:1(VL);Cost:3(H)Schedule:3(H)) RiskStatus: OpenSummary: IfvendorisunabletoproducesensorsthatmeetCMSSpecificationthentheadditonalcostanddelayofidentifyinganewvendorjeopardizesthetimelyandon-
budgetcompletionoftheprojectRiskType: Threat Owner: UlrichHeintzWBS: 402.2OT-OuterTracker RiskArea: ExternalRisk/VendorsProbability(P): 5% TechnicalImpact: 3(H)-extremelysubstandardorKPPinjeopardyCostImpact: PDF =3-point-triangular
Minimum =210k$Mostlikely =315k$Maximum =2720k$Mean =1082k$P*<Impact> =54k$
ScheduleImpact: PDF =3-point-triangularMinimum =6monthsMostlikely =9monthsMaximum =12monthsMean =9monthsP*<Impact> =0.5months
BasisofEstimate: Iftheselectedvendorisunabletoproducesensorstospecificationsanewvendorhastobedeveloped.Ataminimumthiswillrequireanotherpreproductionrun(6monthdelay).Atamaximumonetotwoprototyperunsmayalsoberequired(12monthsdelay).TheburnratefortheentireOuterTrackeris$70k/month(CMS-doc-13481).WeassumehalfoftheOTscopeisimpactedbyadelayfromthisrisk,therebyincurringaburnrateof$35k/month.Minimpact=nodirectcostincrease.Burnrate=6*$35k=$210k.Likelyimpact:costincreaseiscoveredbythe30%sensorestimateuncertainty.Burnrate=9*$35k=$315k.Maximpact:theworstcasescenariobasedoninformalcostinformationreceivedduringthemarketsurveyisanincreaseinthecostofthesensorsby2/3=66%.30%arecoveredbythecostuncertainty.Theadditionalcostof36%ofthe$6.5Msensorpurchaseis$2.3M.Burnrate=12*$35k=$420k.Total=$2,720k.Wehaveidentifiedavendor(HPK)whohasalreadyproducedsensorsofalltypesthatsatisfyourspecifications.TogetherwiththehistoricallyreliableperformanceofHPKitisveryunlikelythatthisthreatwilloccur.WearenotawarethatHPKhaseverfailedtoproducesensorstospecificationsafterapurchasewasnegotiated.Hencetheprobabilityisconsideredtobelow.
CauseorTrigger: Sensorsdeliveredbyvendoraresubstandardandvendorisunabletofixtheproblem.
ImpactedActivities: SensorproductionandQC.Costriskisimplementedasasinglerisk.Scheduleriskisimplementedasthreeseperaterisks(probabilitydependsonsensortype).Therearethreeriskhooksforthethreesensortypes,butbecause2SandPS-saresimilar,wouldsplittheprobability:1%forPS-s(hookA),2%for2S(hookC),2%forPS-p(hookB).Note:PRAdoesnotsupportfractionsofpercent.
Startdate: 1/Apr/2020 Enddate: 3/Dec/2024RiskMitigations: CERNiscarryingoutamarketsurveytoidentifypossiblevendors.CompaniesareselectedbasedontheircapabilitytoproducesensorsthatsatisfyCMS
specificationsandtoproduceallthesensorsneededbyCMSandATLASwithinatwo-yearperiod.CompanieshavetobequalifiedbyproducingprototypesensorstoCMSspecifications.Thisminimizestheprobabilitythattheselectedcompanycannotdelivertheorder.
RiskResponses: Anewvendorhastobeidentifiedandproductionrestarted.Moredetails: CMS-doc-13481
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 15 Updated: 2019-03-06
RT-402-2-24-DOT-Problemwithmodulemechanicalpartsvendor
RiskRank: 2(Medium)Scores:Probability:2(L);Cost:2(M)Schedule:2(M)) RiskStatus: OpenSummary: Majorproblemswiththevendorofmechanical(bridges,spacers,etc)partsformodulesRiskType: Threat Owner: LeonardGSpiegelWBS: 402.2OT-OuterTracker RiskArea: Probability(P): 20% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular
Minimum =0k$Mostlikely =0k$Maximum =324k$Mean =108k$P*<Impact> =22k$
ScheduleImpact: PDF =3-point-triangularMinimum =0monthsMostlikely =0monthsMaximum =6monthsMean =2monthsP*<Impact> =0.4months
BasisofEstimate: Minimumimpact:noextracostnordelayMaximumimpact:significantdelayscausingastandingarmycostof$324k(54k/monthmoduleassemblylaborburnratefor6months).Theburnrateisassumedtobe1/3timesthetotalmoduleassemblylaborcostof5Mduringtheproductiondividedbytheproductionintervalof31months.The1/3assumesthatacertainfractionofthelaborforceiseithermatrixedouttootherareasorcanadvancesomeoftheotheractivitieswhilewaitingforparts.
CauseorTrigger: ImpactedActivities: Implementedasadelaybetweenbatch3and4ofAL-CFspacerproductionfor2S(A)andPS(B),probabilityshouldbesplitbetweenthetwo.
Startdate: 1/Apr/2020 Enddate: 31/Dec/2024RiskMitigations: Vendorqualificationwillprovidesomeexperiencewithreliability,andthecontractwillincludescheduleexpectations.RiskResponses: IncreasedresourcesforlaborandinfrastructuretoparallelizedownstreamactivitiesinModuleAssemblyandPlank/LayerAssemblytorecoup
delaysMoredetails: CMS-doc-13481
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 16 Updated: 2019-03-06
RT-402-2-25-DOT-Moduleassemblyyieldislow
RiskRank: 2(Medium)Scores:Probability:2(L);Cost:1(L)Schedule:2(M)) RiskStatus: OpenSummary: IftheyieldfromModuleAssemblyislowerthanexpected,thentheadditionalresourcesneededtocompensatejeopardizeprojectscheduleand
budgetRiskType: Threat Owner: LeonardGSpiegelWBS: 402.2OT-OuterTracker RiskArea: Probability(P): 10% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular
Minimum =0k$Mostlikely =40k$Maximum =240k$Mean =93k$P*<Impact> =9k$
ScheduleImpact: PDF =3-point-triangularMinimum =0monthsMostlikely =0monthsMaximum =6monthsMean =2monthsP*<Impact> =0.2months
BasisofEstimate: Minimum:nocostimpact,nodelay.Likely:nodelay.40k$isthecostofadditionallabourandadditionalcomponents.Maximum:240k$isthecostofadditionallabourandadditionalcomponents.6monthdelay:timetodiagnosetheproblemandimprovethemoduleassemblyprocedure.
CauseorTrigger: ImpactedActivities: ImplementedasaRiskHookbetweencompletionof2S/PSModulesBatch3(assumingthatiswhenyouassesstheyield)andMechanicalAssemblyofBatch5(Batch4isinprogress).ShareprobabilityequallybetweenPS(hookA)and2S(hookB).
Startdate: 1/Apr/2020 Enddate: 31/Dec/2024RiskMitigations: ExtensiveprototypingshouldmitigatetheriskofoverestimatedyieldsinModuleassembly.RiskResponses: AdditionalresourcesbothtocompensatefortheloweryieldandfurtherparallelizetheassemblyproceduretoregainschedulewouldbeneededMoredetails: CMS-doc-13481
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 17 Updated: 2019-03-06
RT-402-2-33-DOT-Morepreproductionmodulesneeded
RiskRank: 2(Medium)Scores:Probability:3(M);Cost:2(M)Schedule:2(M)) RiskStatus: OpenSummary: Ifmorepre-productionmodulesareneededtoqualifytheproductioncomponentsandassembly,thentheremaybecostincreasesanddelays.RiskType: Threat Owner: MeenakshiNarainWBS: 402.2OT-OuterTracker RiskArea: Probability(P): 25% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular
Minimum =0k$Mostlikely =0k$Maximum =330k$Mean =110k$P*<Impact> =28k$
ScheduleImpact: PDF =3-point-triangularMinimum =0monthsMostlikely =0monthsMaximum =6monthsMean =2monthsP*<Impact> =0.5months
BasisofEstimate: Additionallabour:22k/monthandadditionalmechanicalcomponents10k/month.Total=6months*($22k+$10k)=$192k.TheL3burnrateduetothedelayofdownstreamactivitiesis$23k/month(CMS-doc-13481).Totalmaximpact=$192k+6*$23k=$330k.
CauseorTrigger: ImpactedActivities: Modulepreproductionactivitieswouldstretchanadditional3months.Implementedasahooksbetween2S(hookB)orPS(hookA)PreproductionModuleAssemblyactivitiescompleteand2S/PSReadyforproduction.Shareprobabilityequally.
Startdate: 1/Apr/2020 Enddate: 31/Dec/2020RiskMitigations: Preproductionactivitiesareadequatelyestimatedassumingnomajorproblemsarise,afterextensiveprototypingRiskResponses: Moredetails: CMS-doc-13481
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 18 Updated: 2019-03-06
RT-402-2-35-DOT-Temporarylossofmoduleassemblyfacility
RiskRank: 2(Medium)Scores:Probability:4(H);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: IfaModuleAssemblyfacilitytemporarilybecomesinoperableduetolossofcriticalequipmentorActofGod,thentheresultantdipinmodule
throughputmayjeopardizetimelycompletionoftheprojectRiskType: Threat Owner: MeenakshiNarainWBS: 402.2OT-OuterTracker RiskArea: Probability(P): 50% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =1-point-singlevalue
Minimum =k$Mostlikely =50k$Maximum =k$Mean =50k$P*<Impact> =25k$
ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =1monthsMaximum =monthsMean =1monthsP*<Impact> =0.5months
BasisofEstimate: Itisassumedthatin1monththeequipmentwouldbereplacedandassemblycouldthenproceedattheaffectedsite.Riskimpactisestimatedfrom1/2ofthemodulelaborburnrate($84k/month)plusanestimateof$8ktomakeemergencyrepairs.
CauseorTrigger: ImpactedActivities: Allassemblyactivitiesattheaffectedsite.ImjplementedasahookbetweenPS/2Smechanicalassemblyofsubsequentbatches(notethoseshouldbecorrelated-ifyoucannotbuildPS,youalsocannotbuild2S)atFermilab(B,D)andBrown(A,C)
Startdate: 1/Apr/2020 Enddate: 31/Dec/2024RiskMitigations: Havingtwositesisalreadyahedgeagainstthecompletestoppageofmoduleproduction,andshouldonesitebecometemporarilyinoperable,
componentscouldberedirectedtotheothersitetemporarilytomitigatetheimpactRiskResponses: Modulecomponentscanbedivertedtotheunaffectedsitetoutilizeitsfullthroughput,andadditionalresourcesaddedtoincreasemodule
productionthroughputatbothsites(oncetheaffectedoneisre-established)toregaintimeinthescheduleMoredetails: CMS-doc-13481
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 19 Updated: 2019-03-06
RT-402-2-43-DOT-Problemwithcarbonfibervendor
RiskRank: 2(Medium)Scores:Probability:3(M);Cost:1(L)Schedule:2(M)) RiskStatus: OpenSummary: IfthereisaproblemwiththeCarbonFibervendorthentherecouldbedelaysorcostincreases.Examplesofproblemsinclude:vendorgoingoutof
business,vendordeliveringsubstandardCF,orthevendorcannotdeliveraccordingtotheagreedschedule.
RiskType: Threat Owner: StefanGruenendahlWBS: 402.2OT-OuterTracker RiskArea: ExternalRisk/VendorsProbability(P): 25% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular
Minimum =23k$Mostlikely =79k$Maximum =158k$Mean =87k$P*<Impact> =22k$
ScheduleImpact: PDF =3-point-triangularMinimum =1monthsMostlikely =3monthsMaximum =6monthsMean =3.33monthsP*<Impact> =0.8months
BasisofEstimate: Minimumimpact:theproblemisrelativelymodestandissolvedbythevendorwithin1monthswithoutanydirectcostimpacttotheproject.Likelyimpacts:theproblemismoresignificantandrequiresadditionalprototypeswiththevendorresultinginadelayof3monthsandacostof$10kforCFandmachining.Maximumimpact:ifweneedtoswitchvendorwewouldneedtoqualifyascondvendorresultingina6monthdelay.Thecostincreasecouldbe$20kforprototypingandsetupcostswiththenewvendor.TheL3burnrateduetothedelayofdownstreamactivitiesis$23k/month(CMS-doc-13481).Mincost=$0k+1month*$23kburnrate=$23k.Likelycost=$10k+3month*$23kburnrate=$79k.Maxcost=$20k+6months*$23kburnrate=$158k.
CauseorTrigger: ImpactedActivities: Carbonfiberbasedstructuresinbothmodulemechanicsandflatbarrelmechanics.Implementedasadelayinprocurementforproductionplanks(hookA)andspacers(hookB).
Startdate: 1/Jan/2019 Enddate: 31/Dec/2024RiskMitigations: Prototypingexperienceinthenexttwoyearsshouldresultinareliablecostestimateofthenecessarycarbonfiberforbothmoduleandflatbarrel
structures.RiskResponses: Moredetails: CMS-doc-13481
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 20 Updated: 2019-03-06
RT-402-2-46-DOT-Problemwithcarbonfoamvendor
RiskRank: 3(High)Scores:Probability:3(M);Cost:2(M)Schedule:3(H)) RiskStatus: OpenSummary: IfthereisaproblemwiththeCarbonFoamvendorthentherecouldbedelaysorcostincreases.Examplesofproblemsinclude:vendorgoingoutof
business,vendordeliveringsubstandardCarbonFoam,orthevendorcannotdeliveraccordingtotheagreedschedule.RiskType: Threat Owner: StefanGruenendahlWBS: 402.2OT-OuterTracker RiskArea: ExternalRisk/VendorsProbability(P): 25% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular
Minimum =23k$Mostlikely =158k$Maximum =396k$Mean =192k$P*<Impact> =48k$
ScheduleImpact: PDF =3-point-triangularMinimum =1monthsMostlikely =6monthsMaximum =12monthsMean =6.33monthsP*<Impact> =1.6months
BasisofEstimate: TheL3burnrateduetothedelayofdownstreamactivitiesis$23k/month(CMS-doc-13481).Minimumimpact:theproblemisrelativelymodestandissolvedbythevendorwithin1monthswithoutanydirectcostimpacttotheproject.Costimpact=1month*$23k/month(burnrate).Likelyimpacts:theproblemismoresignificantandrequiresadditionalprototypeswiththevendorresultinginadelayof6monthsandacostof$20kforCarbonFoamandtechnicalwork.Totallikelyimpact=$20k+6*$23k=$158k.Maximumimpact:ifweneedtoswitchvendorwewouldneedtoqualifyasecondvendorresultingina12monthdelay.Thecostincreasecouldbe$120kforincreasedvendorcosts,prototypingandsetupcostswiththenewvendor.Totallikelyimpact=$120k+12*$23k=$396k.
CauseorTrigger: ImpactedActivities: FabricationofthecarbonfoamstructuresinMechanics-implementedasadelayinthecarbonfoamprocurementforproductionplanks.
Startdate: 1/Jan/2019 Enddate: 31/Dec/2024RiskMitigations: RiskResponses: JoinwiththerestoftheLHCcommunityinseekingoutanewvendorofCarbonFoamMoredetails: CMS-doc-13481
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 21 Updated: 2019-03-06
RT-402-2-54-DOT-Mechanicsmaterialsdegradedbyradiation
RiskRank: 2(Medium)Scores:Probability:2(L);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: Ifthemechanicalmaterialsaresusceptibletointegritydegradationduetoradiationexposure,theresultingmaterialmodificationsanddesign
changesjeopardizetimelycompletionoftheflatbarrel.RiskType: Threat Owner: StefanGruenendahlWBS: 402.2OT-OuterTracker RiskArea: TechnicalRisk/QualityProbability(P): 10% TechnicalImpact: 2(M)-significantlysubstandardCostImpact: PDF =3-point-triangular
Minimum =48k$Mostlikely =96k$Maximum =144k$Mean =96k$P*<Impact> =10k$
ScheduleImpact: PDF =3-point-triangularMinimum =1monthsMostlikely =2monthsMaximum =3monthsMean =2monthsP*<Impact> =0.2months
BasisofEstimate: Totalcostofthematerialsinvolvedisabout$300k(MandS)and$220kinlabortomaketheproductionplanks(x80)andrings(x6).Itisassumedthatanyproblemsarefoundearlyinthemechanicsproductionsuchthatthelikelycostimpactisabout10oftheMandSandlabor:min/likely/max=25/50/75k$andadelayof1/2/3months.TheL3burnrateduetothedelayofdownstreamactivitiesis$23k/month(CMS-doc-13481).Mincost=$25k+1months*$23kburnrate=$48k.Likelycost=$50k+2month*$23kburnrate=$96k.Maxcost=$75k+3months*$23kburnrate=$144k.
CauseorTrigger: ImpactedActivities: Plankandringfabricationwouldbedelayed.Startdate: 1/Jan/2019 Enddate: 1/Apr/2021RiskMitigations: AllmaterialsandassemblieswillberadiationtestedintheprototypingphaseRiskResponses: Moredetails: CMS-doc-13481
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 22 Updated: 2019-03-06
RT-402-2-57-DOT-Majorfailureoflayerassemblyinfrastructure
RiskRank: 1(Low)Scores:Probability:1(VL);Cost:1(L)Schedule:2(M)) RiskStatus: OpenSummary: Ifthereisalongertermnon-availabilityofFNALCO2system,survey/alignmentequipment,autoclave/oven,etc,thenthesubsequentdelayuntila
alternativeisinoperationjeopardizestimelycompletionoftheflatbarrelRiskType: Threat Owner: StefanGruenendahlWBS: 402.2OT-OuterTracker RiskArea: ExternalRisk/FacilitiesProbability(P): 5% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular
Minimum =56k$Mostlikely =112k$Maximum =178k$Mean =115k$P*<Impact> =6k$
ScheduleImpact: PDF =3-point-triangularMinimum =2monthsMostlikely =4monthsMaximum =6monthsMean =4monthsP*<Impact> =0.2months
BasisofEstimate: Estimateweneedmin/likely/max=2/4/6monthstocompleterepairsorprocurereplacementatacostofmin/likely/max=10/20/30k$.TheL3burnrateduetothedelayofdownstreamactivitiesis$23k/month(CMS-doc-13481).Mincost=$10k+2months*$23kburnrate=$56k.Likelycost=$20k+4month*$23kburnrate=$112k.Maxcost=$40k+6months*$23kburnrate=$178k.
CauseorTrigger: majorcomponentfailure(e.g.pump,accumulator) ImpactedActivities: Implementedasthreeriskhooks,oneeachforInner(hookA),Middle(hookB),andOuter(hookc)Layerassembly,delaybetweenstartmilestoneandactualassembly.Suggestsplittingtheprobabilityequally.
Startdate: 13/Jun/2017 Enddate: 31/Dec/2025RiskMitigations: monitorsystemperformance&performregularmaintenance
RiskResponses: repairsystem;buyreplacement(mobile)systemMoredetails: CMS-doc-13481
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 23 Updated: 2019-03-06
RT-402-2-58-DOT-DamagetoFlatBarrelPlanks
RiskRank: 2(Medium)Scores:Probability:1(VL);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: Ifanaccidentdamagesaflatbarrelcomponent(e.g.aplankoranendring)thenrepairworkwouldcauseadelayandcostincrease.Thedamage
couldbepurelyphysical(e.g.thecomponentsarecrushed),chemical(e.g.watercontamination),orotherhazard.RiskType: Threat Owner: StefanGruenendahlWBS: 402.2OT-OuterTracker RiskArea: ManagementRisk/LogisticsProbability(P): 5% TechnicalImpact: 3(H)-extremelysubstandardorKPPin
jeopardyCostImpact: PDF =3-point-triangular
Minimum =30k$Mostlikely =91k$Maximum =141k$Mean =87k$P*<Impact> =4k$
ScheduleImpact: PDF =3-point-triangularMinimum =1monthsMostlikely =1monthsMaximum =2monthsMean =1.33monthsP*<Impact> =0.1months
BasisofEstimate: MinimumImpact:thephysicaldamageisnotpermanentandonlyreworkisneededtoeffectarepair.Theestimateddelayis1monthatacostof2FTE-monthsofengineeringorabout$30k.Minimpacttotal=$30k.Likelyimpact:OneentireLayer2plank(plusassociatedmodules)needstobereplacedincludingabout11modulesmaximum(11*$5k=$55k).Tothisisaddedthemechanicscostandlabor(about10hoursperplank)foratotalofabout$6k.Thedelayis1month(assumingthatmodulesareavailable).Likelyimpacttotal=$30k+$55k+$6k=$91k.Maximumimpact:thedamageiscatastrophicforaLayer3plank,includingabout15modules(15*$5k=$75k).Tothisisaddedthemechanicscostandlabor(about10hoursperplank)foratotalofabout$6k.Thedelayis2month(assumingthatmodulesareavailable).Likelyimpacttotal=$60k+$75k+$6k=$141k.Theriskoccurslateinthescheduleandhencedoesnothaveasignificantburnratefromescalation.Thestandingarmycosts(theyarenotidle!)areincludedinthelaborcostsabove.
CauseorTrigger: Damagetoaplankduringplankassemblyorduringlayerassemblyortransport.
ImpactedActivities: ImplementedasadelaybetweenQCtestingofInner,Middle,andOuterLayer(whicheverislast)andmilestoneofcompletionofFlatBarrel
Startdate: 1/Jan/2021 Enddate: 31/Dec/2024RiskMitigations: Plankhandlingproceduresaredesignedtoreducedamageduetohandling.Transportenclosuresaredesginedtominimizerisktodetector.RiskResponses: Thedamagedplankwouldneedtobereplaced,increasingbothtimeandlabordependingontheextentofthedamageMoredetails: CMS-doc-13481
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 24 Updated: 2019-03-06
RT-402-2-59-DOT-DamagetoFlatBarrelLayer
RiskRank: 2(Medium)Scores:Probability:1(VL);Cost:3(H)Schedule:3(H)) RiskStatus: OpenSummary: Ifanaccidentdestroysamajorflatbarrelcomponent(i.e.awholelayer)thenrepairworkwouldcauseadelayandcostincrease.Thedamagecould
bepurelyphysical(e.g.thecomponentsarecrushed),chemical(e.g.watercontamination),orotherhazard.RiskType: Threat Owner: StefanGruenendahlWBS: 402.2OT-OuterTracker RiskArea: Probability(P): 1% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular
Minimum =930k$Mostlikely =1880k$Maximum =3150k$Mean =1987k$P*<Impact> =20k$
ScheduleImpact: PDF =3-point-triangularMinimum =6monthsMostlikely =9monthsMaximum =12monthsMean =9monthsP*<Impact> =0.1months
BasisofEstimate: Costtorebuildanentirelayerusingplanksat$6klaborperplankandmodulesat$5k/module,foreitherLayer1(minimalrisk),Layer2(likelyrisk)orLayer3(maximumrisk).ThecosttoreplacethemodulesforLayer3(assumingthetotalnumberofsparesneededbyCMSremainsthesame)=540*$5k=$2.7M(includingM&Sandmoduleassemblylabor).Thecostforthemechanicsofalayerisabout$150k(1/3ofthetotal).Laborcostestimate(plankconstruction+modulemounting)rangesfrom200k$(layer1)to400k$(layer3)or$300konaverage.Maximumdirectcostimpact=$3.15M$.Layers1and2scaledaccordingly.
CauseorTrigger: catastrophicdamagetoamajorityofcomponentsofaFlatBarrellayer.
ImpactedActivities: ImplementedasadelaybetweenQCtestingofInner,Middle,andOuterLayer(whicheverislast)andmilestoneofcompletionofFlatBarrel
Startdate: 1/Jan/2021 Enddate: 31/Dec/2024RiskMitigations: DesignandenforcementofFlatBarrelhandlingandtransportprocedures,anddesignofFlatBarrellayertranportenclosures.
RiskResponses: rebuildplanksusingsparemodules,andreassemblelayer.
Moredetails:
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 25 Updated: 2019-03-06
RT-402-2-60-DOT-Problemswithwirebonding
RiskRank: 2(Medium)Scores:Probability:5(VH);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: TemporarylossofabondingmachineatFNAL,Princeton,orRutgers.Whiletherepaircostsarecoveredtherewouldbeadditionalexpensesinthe
hold-upofmoduleassemblyuntilthemachineisrepairedortheworkloadcouldberebalance.RiskType: Threat Owner: LeonardGSpiegelWBS: 402.2OT-OuterTracker RiskArea: TechnicalRisk/ReliabilityorPerformanceProbability(P): 80% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =2-point-flatrange
Minimum =13.5k$Mostlikely =k$Maximum =27k$Mean =20k$P*<Impact> =16k$
ScheduleImpact: PDF =2-point-flatrangeMinimum =1monthsMostlikely =monthsMaximum =2monthsMean =1.5monthsP*<Impact> =1.2months
BasisofEstimate: Probabilityisassumedtobe20%permachine(fourmachines:twoatFNAL,oneatPrinceton,oneatRutgers)soroughly80%probabilityintotal(thenumberthatisusedinriskranking).IntheMCtheprobabilityistreatedas4independenteventseachwithP=20%.BasedonaModuleAssemblymonthlylaborburnrateof$54kandtheassumptionthatasinglewirebonderfailsoverthecourseoftheproductionperiod,thusaffecting1/4ofthemoduleassembly(burnrate=$54k/4=$13.5kpermonth).Thisismodeledasadelayof1-2monthsinwirebondingateachof3sitesindependently,bothforPSand2Smodules(correlated).
CauseorTrigger: Failureofawirebonder,forexamplefromthecrashofabondinghead. ImpactedActivities: WirebondingandallOTmoduleassemblyactivitiesthataredownstreamofbonding.IntheMCtheprobabilityistreatedas4independentriskeventswithP=20%each--oneperbondingmachine.
Startdate: 20/Dec/2021 Enddate: 2/Aug/2024RiskMitigations: Withtimethebondingloadcanberebalancedamongstthe3bondingfacilities.Itmaybepossibletopayforexpeditedservicefromthebonding
machinevendors.RiskResponses: Understandhowquicklyamachinecanbebroughtbackintoservice.Moredetails:
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 26 Updated: 2019-03-06
RT-402-2-90-DOT-KeyOuterTrackerpersonnelneedtobereplaced
RiskRank: 2(Medium)Scores:Probability:3(M);Cost:2(M)Schedule:1(L)) RiskStatus: OpenSummary: Keyengineerorseniortechnicianwithspecialknowledgeleavestheprojectandneedstobereplaced.Ifthetransitioncanbemanagedsuchthattheincomingand
outgoingpersonneloverlapandexchangeknowledge,thenthereismainlyalaborcostimpact.Ifthetransitionismoreabrupt,thenthereisnocostimpactofoverlappingpersonnelbuttherecanbeadelaytotheprojectactivitiesastheincomingpersongetsfullyuptospeed.Thisriskdoesnotincludetheriskoflosingkeymanagersbecausetheprojectensuresthateachmanagerhasawell-traineddeputy.
RiskType: Threat Owner: StevenC.NahnWBS: 402.2OT-OuterTracker RiskArea: ManagementRisk/FundingorResourcesProbability(P): 25% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular
Minimum =75k$Mostlikely =225k$Maximum =570k$Mean =290k$P*<Impact> =73k$
ScheduleImpact: PDF =3-point-triangularMinimum =0monthsMostlikely =0monthsMaximum =3monthsMean =1monthsP*<Impact> =0.3months
BasisofEstimate: hisL2areahas5seniortechnicalstaffwhosefunctionsareparticularlyhardtoreplaceonashorttimescalebyexistingteammembers:4seniorengineersand1seniortechnician.ExperienceintheCMSPhase1Upgradeprojectsuggeststheprobabilityperkeypersonisroughly25%(probabilitytheyleaveduringtheentireproject).Someinputnumberstothecostimpactanalysis:--Laborcostperperson=$15k/month(fullyburdenedseniorenginer=$185k/year).--AverageburnrateperL3areais=$23k/month(CMS-doc-13481).Minimumscenerio:personleavingisreplacedbyanexistingskilledpersonintheteam.1monthoffulloverlapcosting$15koflabor.Noscheduledelay.Likelyscenario:personleavingisreplacedbyapersonlessfamiliarwiththespecificwork.3monthsmanagedoverlapoftheoutgoingandincomingpeoplecosts3*$15k=$45koflabor.Noscheduledelay.Maximumscenario:personleavesunexpectedlywithnotransferofknowledge.Ittakesabout6monthstofindanewpersonandgetthemfullyuptospeed.Thereislossofproductivityduringthisdifficulttransitionperiodresultinginanet3monthdelaytoL3activities.Costoframp-upeffortofthenewperson(learningbutnotfullycontributingtodeliverables)=3FTE-months*$15k=$45k.Theburnratecostis3*$23k=$69k.Totalcostimpact=$114kTotalfor5persons:Min/Likely/Maxcostimpact=5*$(15/45/114)k=$(75/225/570)$.
CauseorTrigger: ImpactedActivities: EachkeypersonsisindependentlymodeledintheMC(eachatP=25%andwithcorrespondingshareofthetotalcostimpact).Typicallyhooktherisktothepersonsmainactivityduringtheproductionphase(maximumconsequences).
Startdate: 1/Jan/2018 Enddate: RiskMitigations: Anumberofengineersandtechnicianshaveoverlappingskills,bothwithinagiveninstituteandacrossUS-CMSinstitutes.Thisprovidessomebackupincaseof
non-availabilityofakeyperson.Pro-activecross-trainingofengineersandtechnicianshelpsensurekeyskillsarenotcompletelylostifakeypersonisnolongeravailable.
RiskResponses: Aimforoutgoingandincomingpersonneltooverlapifpossibletoensureasmoothtransition.Hirefromskilledmembersoftheteamifappropriate.Interimsupportmaybepossibleusingengineersortechnicainsfromelsewhereintheinstituteorproject.
Moredetails: CMS-doc-13481
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 27 Updated: 2019-03-06
RT-402-2-91-DOT-ShortfallinOuterTrackerscientificlabor
RiskRank: 3(High)Scores:Probability:3(M);Cost:3(H)Schedule:0(N)) RiskStatus: OpenSummary: Ifasignificantamountofthe(uncosted)scientificlaborisunavailable,thentheprojectwouldthenneedtofundadditional(costed)personnelto
performthework.Itisassumedthattheriskistriggeredbyaseriouslyunfavorableoverallbaseprogramfundingsituation.RiskType: Threat Owner: StevenC.NahnWBS: 402.2OT-OuterTracker RiskArea: ManagementRisk/FundingorResourcesProbability(P): 30% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular
Minimum =0k$Mostlikely =0k$Maximum =1049k$Mean =350k$P*<Impact> =105k$
ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =0monthsMaximum =monthsMean =0monthsP*<Impact> =0.0months
BasisofEstimate: InthepastUS-CMShasnotexperiencedasignificantlackofscientificlabor(postdocsandgraduatestudents).WhenshortfallsoccuredtheywereusuallyresolvedbycollaboratorsatUS-CMSinstitutesorsometimesfromiCMS.Weassigna30%probabilitythatasignificantshortfalloccursinfuture,duetounfavorablefundingconditionsinthebaseprogram.ThecontributedlaborforthescopeofthisL2areais60.8FTE-yearsspreadoverabout5years.ThisdoesnotincludethemoresecureL2andL3managerswhoaretenuredfacultyandseniorFermilabscientists.Weestimatethelosscouldbeupto20%ofthetotalcontributedlaboror12.2FTE-years(e.g.thisisalossof50%ofallcontributedlaborforatwoyearperiodoralossof1/3ofthecontributedlaborfor3years).Themissinglaborcouldbereplacedbycostedpersonnel:amixtureofmid-rangetechnicians,juniortechnicians,orundergraduatescostingrespectively62$/hr,50$/hrand18$/hrfully-burdened(43$/houronaverage).Allowingforfouryearsofescalationat3.1%perannumyieldsanaveragecostof49$/hror86k$/FTE-year(1768hrsworkedperyear).The(min/likely/max)costimpactistherefore:86k$perFTE-year*(0/0/12.2)FTE-years=$(0/0/1049)k.
CauseorTrigger: Theriskistriggeredbyanunfavorablebaseprogramfundingsituation. ImpactedActivities: Startdate: 1/Oct/2018 Enddate: RiskMitigations: Workwithinstitutesandagenciestoensuretheanticipatedamountofscientificlaborwillbeavailable.Whereshortfallslooklikelytooccur,seek
alternativesamongstotherUS-CMSinstitutesorevenfromiCMSinstitutes.RiskResponses: Seekreplacementscientificlaborinotherinstitutes.Ifthislaborcannotbefoundthencontingencywillneedtobespenttosupplementtheeffort
withcostedlabor(e.g.technicians).Moredetails: CMS-doc-13509(FTEdataatCD1)
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 28 Updated: 2019-03-06
RT-402-4-01-DCE-AdditionalFEASICengineeringrunrequired
RiskRank: 3(High)Scores:Probability:3(M);Cost:2(M)Schedule:3(H)) RiskStatus: OpenSummary: TheFEASICisnotaUSCMSdeliverable.TherequiredFEASICareexpectedtobereceivedfreeofchargefromtheinternationalprojectinexchange
fortheconcentratorASICsprovidedfortheelectromagneticsectionoftheCE(CE-E).However,theUS,alongwiththerestofthecollaboration,wouldshareaportionofthecostofanadditionalengineeringrunfortheFEASICbeyondthecurrentplannedtwoengineeringruns.Ifasignificantflawisobservedafterthesecondengineeringrun,athirdcyclemayberequired.Inthiscase,USASICengineeringresourcesmayberequiredtoconsult,validate,orfinalizesomespecificpartofthedesign.ThelikelyriskisaradiationeffectinthedigitalpartoftheASIC,sincethatpartofthechipwilllikelybecompletedlastandtheirradiationcampaignwillnotbecompleteduntilaftertheengineeringrun.
RiskType: Threat Owner: JeremiahMichaelMansWBS: 402.4CE-CalorimeterEndcap RiskArea: TechnicalRisk/QualityProbability(P): 25% TechnicalImpact: 2(M)-significantlysubstandardCostImpact: PDF =1-point-singlevalue
Minimum =k$Mostlikely =336k$Maximum =k$Mean =336k$P*<Impact> =84k$
ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =8monthsMaximum =monthsMean =8monthsP*<Impact> =2.0months
BasisofEstimate: Cost:AssumefourmonthsofASICengineering($108k)and$60kMandSbasedona27%cost-sharefortheASIC=$168k.The27%fractionisthestandardUSCMScontributiontointernationalCMScommoncosts.Schedule:Aneight-monthdelaywouldbeincurred.TheL3burnrateduetothedelayofdownstreamactivitiesis$21k/month(CMS-doc-13481).Somedownstreamactivitiesmayneedtobeacceleratedifdeemednecessarytocompleteproductiononschedule.Thecostassociatedwithsuchanaccelerationisnotincludedherebutincludedasaseparaterisk.Totalimpact=$168k+8months*$21kburnrate=$336k.
CauseorTrigger: ImpactedActivities: SiliconmodulePCBcompletion,siliconmoduleassembly,scintillatortile-moduleassembly,andcassetteassembly.
Startdate: 1/Jun/2020 Enddate: 1/Mar/2021RiskMitigations:
ManagerswillworkwithengineeringanddevelopmentteamtoprovidefullqualityassurancefortheASICdesignbeforeengineeringrunsoccur.Wehave1FTEofextraengineeringover2yearstoworkwithCERNandFEASICengineerstohelpvalidatetheFEASIC,inadditiontotheworkthatisneededtointerfacethechipintoUSCMSdeliverables.
RiskResponses: Somedownstreamactivitiesmayneedtobeacceleratedifdeemednecessarytocompleteproductiononschedule.Thecostassociatedwithsuchanaccelerationisnotincludedherebutincludedasaseparaterisk.
Moredetails: CMS-doc-13481
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 29 Updated: 2019-03-06
RT-402-4-02-DCE-Infrastructurefailureatmoduleassemblyfacility
RiskRank: 2(Medium)Scores:Probability:3(M);Cost:2(M)Schedule:2(M)) RiskStatus: OpenSummary: Asignificantfailurecouldoccurinamoduleassemblyfacilitywhichrendersitunabletooperateforsomeperiodoftime.Exampleswouldinclude
floodingofthefacilityorseveredamagetothegantrysystemusedforthegluingofmodules.RiskType: Threat Owner: ManfredPauliniWBS: 402.4CE-CalorimeterEndcap RiskArea: ExternalRisk/FacilitiesProbability(P): 30% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =2-point-flatrange
Minimum =100k$Mostlikely =k$Maximum =336k$Mean =218k$P*<Impact> =65k$
ScheduleImpact: PDF =2-point-flatrangeMinimum =1monthsMostlikely =monthsMaximum =4monthsMean =2.5monthsP*<Impact> =0.8months
BasisofEstimate: Probability=30%for3sites.Probabilitypersite=10%,basedonexperiencefromoriginalCMStracker,originalpixel,andPhase1pixelwhereoneincidentoccurredinO(10)sites.Cost:Costforsufficientmodulestorecommissionthesystemandapossiblestanding-armycosttomaintaintheteamwhilerepairsarecarriedout.Assume40modulesforrecommissioningatacost(includinglabor)of$40k.Standingarmycostsareestimatedat$58k/month.Costsforrepairoffloodingareassumedtobecoveredbyinsurance,whilecostsforadamagedgantryareestimatedas25%oftotalreplacementcostor$64k.Totalof$100kto$336kdependingondamageanddelay.Schedule:delayof1to4months.
CauseorTrigger: ImpactedActivities: IntheriskMCthisriskneedstobeduplicatedonceforeachcentre
Startdate: 1/Jan/2022 Enddate: 30/Sep/2023RiskMitigations: Mitigatescheduledelaysandstandingarmycostsbyusinguniformproceduresatallmoduleassemblysites.Theplanisconstructedtoallowsome
excesscapacityateachsite.RiskResponses: Inthecaseofaseriousfailure,somestaffandproductionworkcanbetransferredtotheothersites,possiblyaddingaweekenddaytotheshiftcycle.
Costsfortravelwouldbeincurred,assumedtobeatthelevelof$4k/month/technican.Moredetails:
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 30 Updated: 2019-03-06
RT-402-4-04-DCE-Concentratordoesnotmeetspecifications
RiskRank: 2(Medium)Scores:Probability:2(L);Cost:3(H)Schedule:3(H)) RiskStatus: OpenSummary: IftheconcentratorASICdoesnotmeetspecificationsaftertheproductionmasksareproducedthenanadditionalsetofmaskswillneedtobe
producedandadditionalengineeringeffortisrequiredtoadjustthedesignsothattheASICmeetspecifications.Theriskprobabilitydependsonhowmanyengineering/multi-projectwaferrunsaredoneandtestedbeforetheproductionmasksareproduced,andalsoontheexactmaturityofthedesign(progressmade),andontheschedule,e.g.ifallirradiationtestscanbedonebeforetheproductionmasksarefabricated.Thefullpackagingcostsarerealizedonlyoncesincewewouldfindtheproblembeforepackaging.ThemaskswillincludebothTRGandDAQchips.
RiskType: Threat Owner: JamesFHirschauerWBS: 402.4CE-CalorimeterEndcap RiskArea: TechnicalRisk/RequirementsProbability(P): 10% TechnicalImpact: 2(M)-significantlysubstandardCostImpact: PDF =3-point-triangular
Minimum =907k$Mostlikely =971k$Maximum =1035k$Mean =971k$P*<Impact> =97k$
ScheduleImpact: PDF =3-point-triangularMinimum =6monthsMostlikely =7.5monthsMaximum =9monthsMean =7.5monthsP*<Impact> =0.8months
BasisofEstimate: TheM&Scostis$690kformasksandwaferand$25kforasinglesetuptopackageafewchips.Thelaborcostfortheadditionalengineeringeffortisa3-pttriangularPDF:0.25-0.375-0.5FTE.Themin/likely/maxcostishence781/813/846k$.TheL3burnrateduetothedelayofdownstreamactivitiesis$21k/month(CMS-doc-13481).Mincost=$781k+6month*$21kburnrate=$907k.Likelycost=$813k+7.5months*$21kburnrate=$971k.Maxcost=$846k+9months*$21kburnrate=$1035k.
CauseorTrigger: ConcentratorASICasreturnedfromproductiondoesnotmeetspecificationdespitethetwopreviousmulti-projectwaferruns(MPWs)
ImpactedActivities:
Startdate: 30/Aug/2021 Enddate: 29/Aug/2022RiskMitigations: Thisriskismitigatedbyprototypingandtestingthefulldesignthroughtwomulti-projectwafersubmissions,andbydesignandproductionreviews.
TheASICsfromthesewillbeusedinfullverticalslicetests,aswellasbeamtestsandirradiationtests.RiskResponses: ToreducetheimpactduetothedelayofthisASICwewillneedtoimplementacombinationofthefollowing:acceleratetheproductionandtestingof
motherboards(MB)oncetheASICisavailable,andalsotheassemblyofcassettes;domoreassemblyandtestinginparallelforpiecesthatdonotrequiretheASIC(fortheMB),orthatdonotrequiretheMB,tominimizethetimetocompleteassemblyoftheMBandcassettes,andtominimizethetimeneededtofortests.
Moredetails: CMS-doc-13481
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 31 Updated: 2019-03-06
RT-402-4-09-DCE-ModulePCBbatchfailure
RiskRank: 1(Low)Scores:Probability:1(VL);Cost:1(L)Schedule:2(M)) RiskStatus: OpenSummary: Onebatchworth1000modulePCB'shaslowqualityyieldandisunusable.Thisriskcoversanumberofscenarioswheretheassembledmodule
PCB'sareunusable.ThiscouldincludeproblemsintheperformanceofthepartsorinthemodulePCBsorintheassemblyprocess,orinshippingorhandling.WeassumeallPCBswillberemadeandthepartsontheassembledPCBarelost.
RiskType: Threat Owner: ManfredPauliniWBS: 402.4CE-CalorimeterEndcap RiskArea: ExternalRisk/VendorsProbability(P): 5% TechnicalImpact: 1(L)-somewhatsubstandardCostImpact: PDF =2-point-flatrange
Minimum =144k$Mostlikely =k$Maximum =186k$Mean =165k$P*<Impact> =8k$
ScheduleImpact: PDF =2-point-flatrangeMinimum =2monthsMostlikely =monthsMaximum =4monthsMean =3monthsP*<Impact> =0.2months
BasisofEstimate: Onebatchworth1000modulePCB'shaslowqualityyieldandisunusable.Costimpactis$102kand2-4monthdelay.Costincludestotalcostofpartsandassembly.TheL3burnrateduetothedelayofdownstreamactivitiesis$21k/month(CMS-doc-13481).Mincost=$102k+2months*$21kburnrate=$144k.Maxcost=$102k+4months*$21kburnrate=$186k.
CauseorTrigger: ImpactedActivities: Moduleassemblyatmodulesassemblysites.
Startdate: 1/Oct/2019 Enddate: 2/Oct/2022RiskMitigations: Thedesign,andmodulePCBwillgothroughtwoprototypestagesandverticalslidetests,aswellasbeamtests.Theproductionpartswillgothrough
QC,andthevendorswillbequalifiedthroughtheprototypecycles.Costswillbemitigatedbywritingthepurchasecontractssoweonlypayforgoodparts.
RiskResponses: WewillneedtorequestadditionalfundsfromcontingencytopayforthereplacementassembledPCBsandwemayneedtoacceleratemoduleproduction,and/orcassetteproductionintheseareonthecriticalpath.
Moredetails: CMS-doc-13481
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 32 Updated: 2019-03-06
RT-402-4-10-DCE-Siliconsensorhaslowyield
RiskRank: 1(Low)Scores:Probability:1(VL);Cost:2(M)Schedule:2(M)) RiskStatus: OpenSummary: Siliconsensorsshowlessthananticipatedquality,orareunsableinsomeway.Thisriskismeanttocoveranumberofissuesthatcouldaffectthe
usabilityofthesensor,likehandling,QCprocessissues,radiationeffects,post-assemblyeffectsonleakagecurrents,andsingle-vendor-relatedproblems.Weuseaneffective10%yieldforthepurposeofquantifyingthisrisk.
RiskType: Threat Owner: ManfredPauliniWBS: 402.4CE-CalorimeterEndcap RiskArea: ExternalRisk/VendorsProbability(P): 1% TechnicalImpact: 2(M)-significantlysubstandardCostImpact: PDF =2-point-flatrange
Minimum =542k$Mostlikely =k$Maximum =784k$Mean =663k$P*<Impact> =7k$
ScheduleImpact: PDF =2-point-flatrangeMinimum =2monthsMostlikely =monthsMaximum =4monthsMean =3monthsP*<Impact> =0.0months
BasisofEstimate: Usea10%yieldissuetoquantifythisrisk.Costimpactis500-700k$andadelayof2-4months.TheL3burnrateduetothedelayofdownstreamactivitiesis$21k/month(CMS-doc-13481).Mincost=$500k+2months*$21kburnrate=$542k.Maxcost=$700k+4months*$21kburnrate=$784k.
CauseorTrigger: ImpactedActivities: Implementedbetween'deliveryofproductionsensorsbatch3'and'fabricationofproductionsensorsbatch4'
Startdate: 1/Oct/2019 Enddate: 1/Oct/2021RiskMitigations: Thedesignwillbeprototypestartingwithteststructuresandproceedingtofull-sizedsesnorsoffinaldesign.Testsofthesewithincludeirradiation
studiesandtestbeamstudies,aswellasassemblyintoprototypemodulesthatwillundergotwomajorsystemprototypesthatserveasverticalslicetests.Thecontractforthesensorswillhavefullspecificationstoensurewepayonlyforeventualgoodsensors,andwewillqualifyasecondvendorthathasthecapacitytofabricateatleastsomefractionofthesensors.Orderthesensorsasearlyaspossible.
RiskResponses: Requestadditionalfundingfromcontingencytopurchaseadditionalsensors.Activateasecondvendorifneeded.Acceleratedownstreamproductionifneeded.Uselessthanidealsensorsifneededinregionssotheydon'toverlapandthatwouldaffectlesstheperformanceofthecalorimeter.
Moredetails:
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 33 Updated: 2019-03-06
RT-402-4-13-DCE-HGCROCfrontendchipisdelayed
RiskRank: 2(Medium)Scores:Probability:2(L);Cost:2(M)Schedule:3(H)) RiskStatus: OpenSummary: IftheHGCROCfrontendchipdeliverydateisdelayedthentheassemblyofmoduleboardsandthereforetheassemblyofthemodulesgetsdelayed.
Thismayjeopardizethedeliveryofcassettesontime.ThisriskcanincludedelaysinanyoftheprototypeHGCROCchipsubmissionsbesidesthefinalproductionrun.
RiskType: Threat Owner: JeremiahMichaelMansWBS: 402.4CE-CalorimeterEndcap RiskArea: ExternalRisk/CollaboratorsProbability(P): 20% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular
Minimum =21k$Mostlikely =126k$Maximum =252k$Mean =133k$P*<Impact> =27k$
ScheduleImpact: PDF =3-point-triangularMinimum =1monthsMostlikely =6monthsMaximum =12monthsMean =6.33monthsP*<Impact> =1.3months
BasisofEstimate: Themin/likely/maxdelaysareestimatedtobe1/6/12months.TheL3burnrateduetothedelayofdownstreamactivitiesis$21k/month(CMS-doc-13481).Noadditioncostshavebeenassumeinthisriskentryforapossibleaccelerationofmoduleandcassetteproduction.TheseareincluidedasaseparateriskeventMincost=1month*$21kburnrate=$21k.Likelycost=6months*$21kburnrate=$126k.Maxcost=12months*$21kburnrate=$252k.
CauseorTrigger: ImpactedActivities: Assemblyofmoduleboards.JM:Implementedbetween:(A)'vendorfabricatesHGROC'and'vendordeliversHGROC'forsilicon,and(B)'prepareandplaceorders'and'deliveryofHGROC-SiPM'forscintillator.LT:assumeburnratecostisequallysharedbetweenthesetwoitems.
Startdate: 1/Dec/2019 Enddate: 1/Dec/2020RiskMitigations: DevelopteststandssothatcomponentsthatdependontheHGCROCcanbetestedasmuchaspossibleusingemulatorsandolder/prototype/non-
productionversionsoftheHGCROCchip.IncludesignificantengineeringsothatwecanhelpverifyandvalidatetheprototypeandproductionHGCROCchipfunctionalityandqualityasquicklyaspossible.Useprototypeandpre-productionHGCROCchipsinthe2majorsystemprototypesforverticalslicetestsandinbeamtests,andirradiationstudies.Ensurethemoduleandcassetteproductionfacilitiesaresetuptobeabletoaccelerateproductionifneeded.
RiskResponses: HelpvalidateortesttheHGCROCchipsasquicklyaspossibletoreducethetestingtime.Acceleratedownstreamproduction(modulesandcassette)asneeded.
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 34 Updated: 2019-03-06
RT-402-4-14-DCE-Cassettecoolingplatefabricationfailure
RiskRank: 1(Low)Scores:Probability:2(L);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: Ifcassettecoolingplatefabricationfailsspecificationintermsofflatness,locationoffeatures,orcoolingtubeperformancethentheassemblyofthe
cassettescannotproceedandmayjeopardizethedeliveryofassembledcassettestoCMS.RiskType: Threat Owner: ZoltanGecseWBS: 402.4CE-CalorimeterEndcap RiskArea: ManagementRisk/ExperienceorCapabilityProbability(P): 10% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =2-point-flatrange
Minimum =73k$Mostlikely =k$Maximum =213k$Mean =143k$P*<Impact> =14k$
ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =3monthsMaximum =monthsMean =3monthsP*<Impact> =0.3months
BasisofEstimate: Thecoolingplatemanufacturingprocesscontainsanon-standardoperationofembeddingacoolingtube.Thisisthereasonforanon-negligiblefailureduringproductionevenifthevendorqualificationwassuccessful.The3monthdelaywasestimatedasatimeneededtoshiftproductiontoanalternativevendororin-houseatFNAL.Thecostimpact($10k-$150k)isbasedontheassumptionofafailedbatchofcoolingplates,andusesFNALlaborratesforfabricationasifitisdonein-house.WehavenotyetqualifiedanexternalvendorsoFNALratesareassumedforcostestimes.Theentirecostofabatchisusedbecausethisriskcoversarangeofsourcesoffailuresthatmaynotbecoveredbythevendoror/andinatimelymanner.TheL3burnrateduetothedelayofdownstreamactivitiesis$21k/month(CMS-doc-13481).Mincost=$10k+3months*$21kburnrate=$73k.Maxcost=$150k+3months*$21kburnrate=$213k.Aburnrateisusedtocovertheriskthatwemaynotbeabletoscheduleproductionofcoolingplatesearlyenoughtoavoidbeingonthecriticalpath,orsomeotherdelaypushesthisintothecriticalpath.
CauseorTrigger: ImpactedActivities: CassetteAssembly.Insertedbetweendeliveryofbatch1andfabricationofbatch2
Startdate: 1/Jan/2021 Enddate: 12/Dec/2023RiskMitigations: Thecassetteprototypingprogramalreadyunderwayaddressesissuesofmanufacturabilityofthecoolingplatesandexperiencegainedwillfeed
backtothedesign.Threeroundsofprototypecassettesareplannedpriortothestartofproduction.Qualifyseveralvendorsforcoolingplatefabricationsuchthattheycanpick-upproductionifonevendorfails.RequirearigorousQA/QCprogrambythevendor.Begincoolingplateproductioninadvanceofcassetteassemblytoensureabufferofcomponentssothatinterruptionofthesupplydoesnotaffectcassetteproduction.
RiskResponses: Workwithvendortofixproductionproblemsordistributeproductiontoothervendors,orfabricationin-houseatFNAL.Increasethecassetteassemblyrate,utilizingplannedbackupcapacityoftheassemblysites,toreducethescheduleimpact.Thecostofacceleratedcassetteproductioniscoveredbyaseparaterisk.
Moredetails: CMS-doc-13481
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 35 Updated: 2019-03-06
RT-402-4-15-DCE-Motherboardandinterfaceboardfabricationfailure
RiskRank: 1(Low)Scores:Probability:2(L);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: Iflargerthanexpectedfailurerateisidentifiedduringmotherboardandinterfaceboardproductionthenthecassetteassemblywillslowdownand
mayjeopardizethedeliveryofcassettestoCMSontime.RiskType: Threat Owner: ZoltanGecseWBS: 402.4CE-CalorimeterEndcap RiskArea: TechnicalRisk/QualityProbability(P): 10% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =2-point-flatrange
Minimum =73k$Mostlikely =k$Maximum =193k$Mean =133k$P*<Impact> =13k$
ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =3monthsMaximum =monthsMean =3monthsP*<Impact> =0.3months
BasisofEstimate: Thecostimpact(10-130k$)isbasedontheassumptionthatatmaximum10%ofthemotherboardsforthewholedetectorhavetobeproducedduetoMotherboardfailures.Thebasecostincludesanadditional10%sparemotherboardsontopofthatneededforthedetector.Thesesparespartlycovermoreminorfailuresandyieldofindividualcomponents,ormotherboardvariants,andmaynotcoveramoresignificantfailurecoveredbythisriskwheretheproductionmotherboardsaredoneintwobatches.Weassumeafailurescenariowhere10%ofthemotherboardswouldneedtobeproducedandassembledtoensurethatwehavesufficientsparestocoverlostedmotherboardsofanyofthevariantsbytheendoftheproduction.Thedurationimpact(3months)isbasedontheassumptionthatanothervendormightbeneededtobequalifiedtotakeovertheproduction,orsomespecificvariantshavetobemadeagainbeyondwhatwasorderedforspares.TheL3burnrateduetothedelayofdownstreamactivitiesis$21k/month(CMS-doc-13481).Mincost=$10k+3months*$21kburnrate=$73k.Maxcost=$130k+3months*$21kburnrate=$193k.
CauseorTrigger: ImpactedActivities: Startdate: 1/Jan/2021 Enddate: 12/Dec/2023RiskMitigations: Thoroughlyqualifythedesignthroughthecassetteprototypeprogramand,asappropriate,focusedmockups.
Qualifyseveralvendorsforboardfabricationsuchthattheycanpick-upproductionifonevendorfails.RequirearigorousQA/QCprogrambythevendor.Beginmotherboardandinterfaceboardproductioninadvanceofcassetteassemblytoensureabufferofcomponentssothatinterruptionofthesupplydoesnotaffectcassetteproduction.Designcassetteassemblysitestoallowreworkwithoutinterruptingproductionflow.
RiskResponses: Dependingonthescheduleandwhichmotherboardvariantsneedtobereplacedwemayneedtoorderanewroundofmotherboardsandassembly.Ifotrherdelaysorthisdelaycausesthistasktobeonthecritcialpathwewillacceleratethecassetteproductionandtesting.Thepossibilityofneedingtoacceleratethecassetteproductioniscoveredbyaseparaterisk.
Moredetails: CMS-doc-13481
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 36 Updated: 2019-03-06
RT-402-4-16-DCE-Cassettesdamagedorlostinassembly,testingorshipping
RiskRank: 1(Low)Scores:Probability:1(VL);Cost:2(M)Schedule:1(L)) RiskStatus: OpenSummary: Ifacassettegetsdamagedduringassemblyorabatchof15cassettesgetdamagedduringcoldtestingorabatchof15cassettesgetlostduring
shipping,thenthelostcassettesneedtobefabricatedandassembledagain,whichmayjeopardizethedeliveryofcassettestoCMSontime.RiskType: Threat Owner: ZoltanGecseWBS: 402.4CE-CalorimeterEndcap RiskArea: ManagementRisk/ExperienceorCapabilityProbability(P): 5% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =2-point-flatrange
Minimum =100k$Mostlikely =k$Maximum =1000k$Mean =550k$P*<Impact> =28k$
ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =3monthsMaximum =monthsMean =3monthsP*<Impact> =0.2months
BasisofEstimate: Thecostestimateisbasedonthecostoflosingupto15cassettes.Wewillbeassemblyingandtestinginbatchesof15,andshippingthemtoCERNinbatchesof15.Wecalculatethecostbysimplyrollingupthecostofproducing375cassettes,andtaking(15/375)ofthiscostforthemaximumcostimpact.Thecostofproducingandtestingallcomponents,includingsiliconmodulesandscintillatortile-modules,andthecostofcassetteassembly,testing,andshipping,areincluded.WedonotincludethecostofHGCROCandECONASICasenoughsparesareexpectedtobepurchasedsowedonotneedanadditionproductionrunofthesechips.Thedelaysisbasedonthetimeneededtoreplacethelostcassettes.
CauseorTrigger: ImpactedActivities: LinkedtoShipcassettes331-360
Startdate: 1/Jan/2021 Enddate: 12/Dec/2023RiskMitigations:
Setinplacecarefullydesignedtoolingandsafehandlingprocedures.Donothandlemanycassettesatthesametime;limitnumberofcassettesinashipmentto15andnomorethanoneofeachtypepershipment.Plannedproductionincludes1sparecassetteofeachtype(forthetestbeamwedge).Ensureadequatequantityofsparepartstoallowrapidassemblyofreplacementcassettes.Ensurethatallshipmentsareadequatelyinsured.Contractstoincludeoptionsforlaterdeliveryofadditionalcomponents.
RiskResponses: Responsedependsontheexactlost,iflosinganentirebatchof15cassettesinshipping,ordamagingsomecassettesinabatch(es)duringhandlingintheassembly,testing,orshipping.Intheworstcaseoflosingall15cassettesinabatch,wewillorderadditionalpartsasneededandmaketheadditional15cassettes.The15testbeamwedgecassettesmaybeusedinthedetectorifthisisneededtgoavoidsignificantdelays.Asneededwewillacceleratethecassetteassemblyandtesting.Thecostofacceleratingthecassetteproductionisincludedinaseparateriskentry.
Moredetails:
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 37 Updated: 2019-03-06
RT-402-4-17-DCE-Cassetteassemblysitefailure
RiskRank: 1(Low)Scores:Probability:2(L);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: IfthecleanroomareaofthecassetteassemblysitegetsdamagedoriftheCO2coolingplantfailsthentheassemblyandtestingprocedurewillstop
untiltheproblemsarefixedanditmayjeopardizethedeliveryofcassettestoCMSontime.RiskType: Threat Owner: ZoltanGecseWBS: 402.4CE-CalorimeterEndcap RiskArea: ExternalRisk/FacilitiesProbability(P): 10% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =2-point-flatrange
Minimum =73k$Mostlikely =k$Maximum =163k$Mean =118k$P*<Impact> =12k$
ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =3monthsMaximum =monthsMean =3monthsP*<Impact> =0.3months
BasisofEstimate: Theestimateisbasedontherangeofcostsneededtoreplacethedamagedequipment=10-100k$.The3monthdelayisestimatedbasedonthetimeitmaytaketofixtheproblems.TheL3burnrateduetothedelayofdownstreamactivitiesis$21k/month(CMS-doc-13481).Mincost=$10k+3months*$21kburnrate=$73k.Maxcost=$100k+3months*$21kburnrate=$163k.
CauseorTrigger: ImpactedActivities: JM:Insertedintoassemblybetween150and151
Startdate: 1/Jan/2021 Enddate: 12/Dec/2023RiskMitigations: Tomitigatetheimpactontheschedule,thecapacityoftheassemblyandtestingfacilityisplannedtotwicelargerthanrequiredfornormal
operations.RiskResponses: Moredetails: CMS-doc-13481
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 38 Updated: 2019-03-06
RT-402-4-18-DCE-AdditionalconcentratorASICengineering(MPW)runisrequired
RiskRank: 3(High)Scores:Probability:4(H);Cost:2(M)Schedule:3(H)) RiskStatus: OpenSummary: CurrentplanningincludesthecostoftwoMulti-ProjectWafer(MPW)prototyperuns.Ifasignificantflawisobservedafterthesecondrun,athird
cyclemayberequired.Thisriskaddressestheneedanadditionalsinglerun(eitherDAQorTRG).
RiskType: Threat Owner: JamesFHirschauerWBS: 402.4CE-CalorimeterEndcap RiskArea: TechnicalRisk/QualityProbability(P): 50% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular
Minimum =164k$Mostlikely =241k$Maximum =385k$Mean =263k$P*<Impact> =132k$
ScheduleImpact: PDF =3-point-triangularMinimum =6monthsMostlikely =7.5monthsMaximum =9monthsMean =7.5monthsP*<Impact> =3.8months
BasisofEstimate: CurrentplanningincludesthecostoftwoMulti-ProjectWafer(MPW)prototyperuns.Ifasignificantflawisobservedafterthesecondrun,athirdcyclemayberequired.Thisriskaddressestheneedanadditionalsinglerun(eitherDAQorTRG).TheM&Scostsare$136kfortheMPWrunand$28kforpackaging.Thelaborcostforadditionaldesignworktofixtheobservedflawisa3-pttriangularPDF:0/0.17/0.5FTE.Itisassumedtheflawisfairlyeasytofixandtheexacttimeneededdependsonwhenwhentheissueisdiscovered,ifinthebeginning,middle,orendofthecycle.TheadditionaltestingisassumedtobemodestandcoveredbythelaborincludedforASICtestinginthebaseline.Themin/likely/maxcostishence164/209/295k$.TheL3burnrateduetothedelayofdownstreamactivitiesis$21k/month(CMS-doc-13481),butthereare6monthsoffloatavailablebeforecassetteassemblyisaffected.therefore:Mincost=$164k+0=$164k.Likelycost=$209k+1.5months*$21kburnrate=$241k.Maxcost=$295k+3months*$21kburnrate=$385k.
CauseorTrigger: ImpactedActivities: JM:ImplementedinP6between'ECONp2testparts'and'ECONv1finalizedesign'
Startdate: 31/Aug/2020 Enddate: 30/Aug/2021RiskMitigations: Wewillworkcloselywithengineerstomakesurethattwoplannedengineering(MPW)runsaresufficient.Wewillbeusingsomecodeblocksfrom
thelpGBTthatwouldhaveundergonesometestingbeforethetwoMPWruns,andwewillbeincludingtheprototypeECONASICinamajorsystemprototypeverticalslicetestbeforetheproductionrun.
RiskResponses: RequestadditionalfundingforathirdMPWrunandfordesignchanges.Reducetheneededtestingtime.Accelerateorparallelizetheproductionofthemotherboardsifpossibleandifneededacceleratethecassetteassemblyandtesting.Thecostofacceleratingtheassemblyofcassettesiscoveredbyaseparaterisk.
Moredetails: CMS-doc-13481
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 39 Updated: 2019-03-06
RT-402-4-20-DCE-BoundarybetweenSiandscintillatorsectionsismoved
RiskRank: 2(Medium)Scores:Probability:2(L);Cost:2(M)Schedule:0(N)) RiskStatus: OpenSummary: Ifthecombinedperformanceofthescintillator+SiPMsystemcannotmeettheperformancerequirementsoftheproject,wemaybeforcedtomove
backtheboundarybetweenthesiliconandscintillatorsections.Weassumeheretheimpactisequivalenttomovingthefirstmixedlayerbackbyonelayer.
RiskType: Threat Owner: JeremiahMichaelMansWBS: 402.4CE-CalorimeterEndcap RiskArea: TechnicalRisk/RequirementsProbability(P): 10% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =1-point-singlevalue
Minimum =k$Mostlikely =252k$Maximum =k$Mean =252k$P*<Impact> =25k$
ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =0monthsMaximum =monthsMean =0monthsP*<Impact> =0.0months
BasisofEstimate: Estimatedasthecostoftwoadditionalall-siliconlayersminusthe(small)costofonelessscintillatorlayer.Weassumethatthedecisionismadeearlyenoughthatitwouldnothaveascheduleimpactsincetheassemblycenterswillhavesufficientcapacityforanyforeseenchange.,andwecanmakemodificationstothecoolingplatesintheshadowofothertasks.
CauseorTrigger: ImpactedActivities: Startdate: 29/Jan/2018 Enddate: 1/May/2020RiskMitigations: Performdetailedsimulationstudiestoensurewehaveanaccurateradiationmap.Doirradiationstudiesoftheperformanceofprototypesiliconand
scintillatordetectormodulestomakesuretheymeetrequirements.Dofulldetectorphysicsstudiestoensurefullunderstandingofthephysicsimpactoftheboundarychange.Setupmoduleassemblycenterstohavesufficientcapacityifextrasiliconmoduleshavetobeconstructed.Wedesignthecoolingplateandcoolingtuberouting,tile-modules,andmodule-to-motherboardconnectionsthatareflexibletohandleavariableboundaryfromonelayertothenext.
RiskResponses: Adjustbudget,andschedulestoproducemore/lesssiliconmodules,andcorrespondingless/morescintillatortile-modiules.Modifytheaffectedcoolingplateswiththecorrectmountingholes,andproducethemodifiednumberofeachtypeofmotherboards.
Moredetails:
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 40 Updated: 2019-03-06
RT-402-4-22-DCE-Additionalproductionaccelerationrequired
RiskRank: 2(Medium)Scores:Probability:2(L);Cost:2(M)Schedule:1(L)) RiskStatus: OpenSummary: Ifupstreamprojectdelaysabsorballfloatbeforeseriesproductionbeginsthenadditionalfundswouldberequiredtoaccelerateproductiontoavoid
jeoparizingtheinternationalneed-bydateRiskType: Threat Owner: JeremiahMichaelMansWBS: 402.4CE-CalorimeterEndcap RiskArea: TechnicalRisk/RequirementsProbability(P): 20% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular
Minimum =564k$Mostlikely =564k$Maximum =777k$Mean =635k$P*<Impact> =127k$
ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =1monthsMaximum =monthsMean =1monthsP*<Impact> =0.2months
BasisofEstimate: Minimalcostimpactisbasedonaccelerationofmoduleproductionfrom120modules/weekto160modules/week.Thisrequiresastaffingincreaseof33%duringthisperiodwhichisbalancedbyadecreasedproductionperiod.Duetothelimitationsinhowquicklystaffcanbetrained,however,weassumethatthelaborlevelduringtheramp-upofproductionwouldbe100%insteadoftherampedprocessinthebaselineplan.(costsdeterminedfromP6tobe$564k)Maximalcostimpactincludesthemoduleproductionaccelerationandalsoincludeslabor(1FNALtechnicianfull-time=$113k)andinfrastructure(CO2plantenhancements=$100k)toacceleratethecassettetestingcapacityfrom2/dayto3/day.
CauseorTrigger: Primarycausesareadelayononeofthecriticalinputsforproduction:HGCROC,concentrator,siliconsensor,modulePCB,motherboards
ImpactedActivities:
Startdate: 15/Oct/2021 Enddate: 15/Jan/2023RiskMitigations: Activemanagementandcarefultrackingofmilestonesandacompleteprototype2validationofthesystemtoavoidsurprisesatthestartof
production.RiskResponses: Moredetails:
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 41 Updated: 2019-03-06
RT-402-4-23-DCE-SiMotherboardcomplexityismuchhigherthanexpected
RiskRank: 2(Medium)Scores:Probability:2(L);Cost:2(M)Schedule:0(N)) RiskStatus: OpenSummary: Thesiliconmotherboardisexpectedtoexistinseveralvarients(~5).Dependingondecisionsmadeelsewhereintheproject,thenumberofdesigns
couldincreasebyafactoroftwooreventen.Ifthisweretooccur,theunitcostformotherboardswouldincreaseandthedesigntimewouldalsosignificantlyincrease.
RiskType: Threat Owner: ZoltanGecseWBS: 402.4CE-CalorimeterEndcap RiskArea: TechnicalRisk/InterfacesProbability(P): 20% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular
Minimum =383k$Mostlikely =575k$Maximum =767k$Mean =575k$P*<Impact> =115k$
ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =0monthsMaximum =monthsMean =0monthsP*<Impact> =0.0months
BasisofEstimate: Lowendofcostrangeassumes6monthsofadditionalengineering($120k)andafactorof50%inunitcostincreaseforPCBsandassemblyonly($263k),whilehighendassumes12monthsofadditionalengineering($240k)andafactorof100%increaseinunitcostforPCBsandassembly($527k).Thescheduledelayassumesthattheissuesareidentifiedsufficientlyinadvancethatadditionalengineeringcanbebroughtontotheprojecttoavoiddelays.
CauseorTrigger: Triggeredbydesignchoiceswhichrequiresignificantadditionalmotherboarddesigns,separatefromcostuncertaintyinasingledesign.
ImpactedActivities:
Startdate: 1/Jan/2019 Enddate: 11/Dec/2020RiskMitigations: WorkingwithinternationalcollaboratorstoadjustdesignsofcassetteandmodulePCBtoavoidrequiringadditionaldesignstomeetoveralldesign
constraints.RiskResponses: Moredetails:
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 42 Updated: 2019-03-06
RT-402-4-90-DCE-KeyCalorimeterEndcappersonnelneedtobereplaced
RiskRank: 2(Medium)Scores:Probability:3(M);Cost:2(M)Schedule:1(L)) RiskStatus: OpenSummary: Keyengineerorseniortechnicianwithspecialknowledgeleavestheprojectandneedstobereplaced.Ifthetransitioncanbemanagedsuchthattheincomingand
outgoingpersonneloverlapandexchangeknowledge,thenthereismainlyalaborcostimpact.Ifthetransitionismoreabrupt,thenthereisnocostimpactofoverlappingpersonnelbuttherecanbeadelaytotheprojectactivitiesastheincomingpersongetsfullyuptospeed.Thisriskdoesnotincludetheriskoflosingkeymanagersbecausetheprojectensuresthateachmanagerhasawell-traineddeputy.
RiskType: Threat Owner: JeremiahMichaelMansWBS: 402.4CE-CalorimeterEndcap RiskArea: ManagementRisk/FundingorResourcesProbability(P): 25% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular
Minimum =75k$Mostlikely =225k$Maximum =555k$Mean =285k$P*<Impact> =71k$
ScheduleImpact: PDF =3-point-triangularMinimum =0monthsMostlikely =0monthsMaximum =3monthsMean =1monthsP*<Impact> =0.3months
BasisofEstimate: ThisL2area5seniortechnicalstaffwhosefunctionsareparticularlyhardtoreplaceonashorttimescalebyexistingteammembers.ExperienceintheCMSPhase1Upgradeprojectsuggeststheprobabilityperkeypersonisroughly25%(probabilitytheyleaveduringtheentireproject).Someinputnumberstothecostimpactanalysis:Laborcostperperson=$15k/month(fullyburdenedseniorenginer=$185k/year).AverageburnrateperL3areais=$21k/month(CMS-doc-13481).Minimumscenerio:personleavingisreplacedbyanexistingskilledpersonintheteam.1monthoffulloverlapcosting$15koflabor.Noscheduledelay.Likelyscenario:personleavingisreplacedbyapersonlessfamiliarwiththespecificwork.3monthsmanagedoverlapoftheoutgoingandincomingpeoplecosts3*$15k=$45koflabor.Noscheduledelay.Maximumscenario:personleavesunexpectedlywithnotransferofknowledge.Ittakesabout6monthstofindanewpersonandgetthemfullyuptospeed.Thereislossofproductivityduringthisdifficulttransitionperiodresultinginanet3monthdelaytoL3activities.Costoframp-upeffortofthenewperson(learningbutnotfullycontributingtodeliverables)=3FTE-months*$15k=$45k.Theburnratecostis3*$21k=$66k.Totalcostimpact=$111kTotalfor5persons:Min/Likely/Maxcostimpact=5*$(15/45/111)k=$(75/225/555)$.
CauseorTrigger: ImpactedActivities: EachkeypersonsisindependentlymodeledintheMC(eachatP=25%andwithcorrespondingshareofthetotalcostimpact).Typicallyhooktherisktothepersonsmainactivityduringtheproductionphase(maximumconsequences).Note:currentMCmodeltakescareofthetotalprobabilityandcostimpact.Butweshouldultimatelymodelthe5engineersindividuallyatappropriatepointsoftheschedule.
Startdate: 1/Jan/2018 Enddate: RiskMitigations: Anumberofengineersandtechnicianshaveoverlappingskills,bothwithinagiveninstituteandacrossUS-CMSinstitutes.Thisprovidessomebackupincaseof
non-availabilityofakeyperson.Pro-activecross-trainingofengineersandtechnicianshelpsensurekeyskillsarenotcompletelylostifakeypersonisnolongeravailable.
RiskResponses: Aimforoutgoingandincomingpersonneltooverlapifpossibletoensureasmoothtransition.Hirefromskilledmembersoftheteamifappropriate.Interimsupportmaybepossibleusingengineersortechnicainsfromelsewhereintheinstituteorproject.
Moredetails: CMS-doc-13481
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 43 Updated: 2019-03-06
RT-402-4-91-DCE-ShortfallinCalorimeterEndcapscientificlabor
RiskRank: 2(Medium)Scores:Probability:3(M);Cost:2(M)Schedule:0(N)) RiskStatus: OpenSummary: Ifasignificantamountofthe(uncosted)scientificlaborisunavailable,thentheprojectwouldthenneedtofundadditional(costed)personnelto
performthework.Itisassumedthattheriskistriggeredbyaseriouslyunfavorableoverallbaseprogramfundingsituation.RiskType: Threat Owner: JeremiahMichaelMansWBS: 402.4CE-CalorimeterEndcap RiskArea: ManagementRisk/FundingorResourcesProbability(P): 30% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular
Minimum =0k$Mostlikely =0k$Maximum =982k$Mean =327k$P*<Impact> =98k$
ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =0monthsMaximum =monthsMean =0monthsP*<Impact> =0.0months
BasisofEstimate: InthepastUS-CMShasnotexperiencedasignificantlackofscientificlabor(postdocsandgraduatestudents).WhenshortfallsoccuredtheywereusuallyresolvedbycollaboratorsatUS-CMSinstitutesorsometimesfromiCMS.Weassigna30%probabilitythatasignificantshortfalloccursinfuture,duetounfavorablefundingconditionsinthebaseprogram.ThecontributedlaborforthescopeofthisL2areais57.1FTE-yearsspreadoverabout5years.ThisdoesnotincludethemoresecureL2andL3managerswhoaretenuredfacultyandseniorFermilabscientists.Weestimatethelosscouldbeupto20%ofthetotalcontributedlaboror11.4FTE-years(e.g.thisisalossof50%ofallcontributedlaborforatwoyearperiodoralossof1/3ofthecontributedlaborfor3years).Themissinglaborcouldbereplacedbycostedpersonnel:amixtureofmid-rangetechnicians,juniortechnicians,orundergraduatescostingrespectively62$/hr,50$/hrand18$/hrfully-burdened(43$/houronaverage).Allowingforfouryearsofescalationat3.1%perannumyieldsanaveragecostof49$/hror86k$/FTE-year(1768hrsworkedperyear).The(min/likely/max)costimpactistherefore:86k$perFTE-year*(0/0/11.4)FTE-years=$(0/0/982)k.
CauseorTrigger: Theriskistriggeredbyanunfavorablebaseprogramfundingsituation. ImpactedActivities: Startdate: 1/Oct/2018 Enddate: RiskMitigations: Workwithinstitutesandagenciestoensuretheanticipatedamountofscientificlaborwillbeavailable.Whereshortfallslooklikelytooccur,seek
alternativesamongstotherUS-CMSinstitutesorevenfromiCMSinstitutes.RiskResponses: Seekreplacementscientificlaborinotherinstitutes.Ifthislaborcannotbefoundthencontingencywillneedtobespenttosupplementtheeffort
withcostedlabor(e.g.technicians).Moredetails: CMS-doc-13509(FTEdataatCD1)
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 44 Updated: 2019-03-06
RT-402-6-02-DTD-Boardorpartsvendornon-performance(DOE)
RiskRank: 2(Medium)Scores:Probability:2(L);Cost:2(M)Schedule:2(M)) RiskStatus: OpenSummary: Ifthevendorhasintermittentproblemsduringtheproductioncausingsub-standardbatchestobedeliveredthenpartsfrombatchNdonotmeet
electricalspecscausingadelayoforderonebatchlengthoranothervendorissought.Similarlyifthereissignificantvendordelaytoduetointernaltechnicalorinventoryproblems.TheriskisrepeatedperL3area.
RiskType: Threat Owner: JeffreyWBerryhillWBS: 402.6TD-TriggerandDAQ(DOE) RiskArea: ExternalRisk/VendorsProbability(P): 20% TechnicalImpact: 2(M)-significantlysubstandardCostImpact: PDF =3-point-triangular
Minimum =60k$Mostlikely =180k$Maximum =310k$Mean =183k$P*<Impact> =37k$
ScheduleImpact: PDF =3-point-triangularMinimum =1monthsMostlikely =3monthsMaximum =6monthsMean =3.33monthsP*<Impact> =0.7months
BasisofEstimate: Basedonpreviousdealingswithsimilarvendorsandeffectsfromproductionissues.Possiblecostincreasesduetonewvendorchoiceandadditionalprocurementofboardsandparts:Min/likely/maxcost=50/150/250k$for2/6/10newboardequivalents.Min/likely/maxdelay=1/3/6monthsforanotherproductionbatchorvendorselection.TheaverageL3burnrateduetothedelayofdownstreamactivitiesis$10k/month(CMS-doc-13481).Mincost=$50k+1month*$10kburnrate=$60k.Likelycost=$150k+3months*$10kburnrate=$180k.Maxcost=$250k+6months*$10kburnrate=$310k.
CauseorTrigger: ImpactedActivities: Procurementofproductionelectronics.ProbabilityissharedbetweenCaloTriggerandCorrelatorTrigger.
Startdate: 1/Jan/2021 Enddate: 1/Jan/2024RiskMitigations: Baselineboardproductionproceedsinwelldefinedsteps:(1)apreproductionofafewboardsensuresthatthequalityofthepartsprovidedbythe
vendormeetsthespecifications,(2)apilotproductionoffurtherensuresthatthequalityofthepartsprovidedbythevendormeetsspecifications,beforethefinalproductionisstarted.Thisphasedapproachprovidesreasonableassurancethatanyqualityperformanceissueswiththevendorwillbediscoveredandmitigatedbeforefullproductionbegins.
RiskResponses: Diagnosetheproblem.Ifasimpletechnicalfixisavailable,considerreordering.Ifnot,findanothervendor.Moredetails: CMS-doc-13481
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 45 Updated: 2019-03-06
RT-402-6-03-DTD-I/Operformancedoesnotmeetrequirements(DOE)
RiskRank: 2(Medium)Scores:Probability:2(L);Cost:2(M)Schedule:3(H)) RiskStatus: OpenSummary: IftheI/Orequirementsofatriggersubsystemchange,thusjeopardizingtheabilityofthetriggertoreceive/transmitalltriggerprimitivedata
from/tothesource/sinksystem.RiskType: Threat Owner: JeffreyWBerryhillWBS: 402.6TD-TriggerandDAQ(DOE) RiskArea: TechnicalRisk/RequirementsProbability(P): 20% TechnicalImpact: 2(M)-significantlysubstandardCostImpact: PDF =3-point-triangular
Minimum =280k$Mostlikely =550k$Maximum =820k$Mean =550k$P*<Impact> =110k$
ScheduleImpact: PDF =3-point-triangularMinimum =3monthsMostlikely =5monthsMaximum =7monthsMean =5monthsP*<Impact> =1.0months
BasisofEstimate: 10-30%upscopeofa100-boardproductiontomeetchangedrequirements,withmin/likely/maxcostof250/500/750k$.Scheduledelaysof3/5/7monthsduetotheacquisitionofmorebatches.TheaverageL3burnrateduetothedelayofdownstreamactivitiesis$10k/month(CMS-doc-13481).Mincost=$250k+3month*$10kburnrate=$280k.Likelycost=$500k+5months*$10kburnrate=$550k.Maxcost=$750k+7months*$10kburnrate=$820k.
CauseorTrigger: Theinterfacebetweenthedetectorbackendelectronicsandthetriggersubsystemscouldchangetoaccomodateincreasedbandwidthrequirements.Thiscantaketheformofeithermorelinksorhigherlinkspeeds.Ifmorelinksarerequired,thiscouldcausethenumberofboardsrequiredtoabsorbthenumberoflinkstoincrease.Alternatively,ifthelinkspeedschange,thiscouldnecessitatenewopticaltransceiversontheboards,tomeetthenewlinkspeedrequirements.
ImpactedActivities: Productionprocurement,assembly,andtesting.ProbabilityissharedbetweenCaloTriggerandCorrelatorTrigger.
Startdate: 1/Jan/2021 Enddate: 1/Jan/2024RiskMitigations: Thebaselineincludesextralinkcapacityforalltriggersubsystems,toreasonabllyaccountforpossiblechangesinthenumberofinputlinkstothe
triggersubsystem.Thebaselinealsoincludesopticaltransceiversthatoperateacrossarangeof10-25Gb/slinkspeeds,allowingforreasonablechangesinlinkspeedinterfaces.
RiskResponses: (1)Attempttooptimise(reduce)thewordbitdefinitionofthereceivedortransmittedtriggerprimitivedata,sothatthetotalbandwidthin/outisconsistentwiththeI/Ocapacityoftheboard,whileattemptingtopreservethesciencegoals.(2)Increasethenumberofelectronicboardstomatchthenumberofinputlinks.
Moredetails: CMS-doc-13481
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 46 Updated: 2019-03-06
RT-402-6-04-DTD-Additionalboardredesignisrequired(DOE)
RiskRank: 2(Medium)Scores:Probability:2(L);Cost:2(M)Schedule:2(M)) RiskStatus: OpenSummary: Aprototypeorproductionbatchhasdesignflaws.RiskType: Threat Owner: JeffreyWBerryhillWBS: 402.6TD-TriggerandDAQ(DOE) RiskArea: TechnicalRisk/ReliabilityorPerformanceProbability(P): 10% TechnicalImpact: 2(M)-significantlysubstandardCostImpact: PDF =3-point-triangular
Minimum =60k$Mostlikely =180k$Maximum =310k$Mean =183k$P*<Impact> =18k$
ScheduleImpact: PDF =3-point-triangularMinimum =1monthsMostlikely =3monthsMaximum =6monthsMean =3.33monthsP*<Impact> =0.3months
BasisofEstimate: 2-10boardsat25keacharefoundtorequireatechnicalredesign.Min/likely/maxcost=50/150/250k$for2/6/10newboardprocurements.Min/likely/maxdelay=1/3/6monthsforredesignandprocurement.TheaverageL3burnrateduetothedelayofdownstreamactivitiesis$10k/month(CMS-doc-13481).Mincost=$50k+1month*$10kburnrate=$60k.Likelycost=$150k+3months*$10kburnrate=$180k.Maxcost=$250k+6months*$10kburnrate=$310k.
CauseorTrigger: ImpactedActivities: Designofnextprotoype/batch.ProbabilityissharedbetweenCaloTriggerandCorrelatorTrigger.
Startdate: 1/Jul/2017 Enddate: 1/Jan/2024RiskMitigations: BaselineboardR&Dandproductionproceedsinwelldefinedsteps:TherearetwoprototypedesignsduringtheR&Dphase.Ifthefirstprototype
doesnotmeettherequirements,thesecondprototypewillberedesignedtomeettherequirements,basedontheexperiencelearned.Ifthesecondprototypealsodoesnotmeettherequirements,aredesigncanbeincorporatedintothepreproductioncycles(involvingafewboards),whichprovidesreasonableassurancethattheproductionboardwillmeettherequirements.Ifthepreproductionboarditselfalsodoesnotcompletelymeetallrequirements,thepilot-productioncyclecanbeusedtotweakthedesignsothatallrequirementsaremetbeforegoingintofinalproduction.
RiskResponses: Ifaprototypeorpreproductionboarddoesnotmeettherequirements,redesigntheboardinthenextcyclesothatitdoesmeettherequirements.Moredetails: CMS-doc-13481
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 47 Updated: 2019-03-06
RT-402-6-05-DTD-Additionalfirmwaredevelopmentisrequired(DOE)
RiskRank: 1(Low)Scores:Probability:2(L);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: Firmwaredoesnotmeettechnicalorscientificrequirementsattimeofmilestonereleases.RiskType: Threat Owner: JeffreyWBerryhillWBS: 402.6TD-TriggerandDAQ(DOE) RiskArea: TechnicalRisk/ReliabilityorPerformanceProbability(P): 20% TechnicalImpact: 1(L)-somewhatsubstandardCostImpact: PDF =3-point-triangular
Minimum =10k$Mostlikely =30k$Maximum =60k$Mean =33k$P*<Impact> =7k$
ScheduleImpact: PDF =3-point-triangularMinimum =1monthsMostlikely =2monthsMaximum =3monthsMean =2monthsP*<Impact> =0.4months
BasisofEstimate: Anecessaryfirmwareblockrequires1/3/6monthsofreworkorredesign,withafirmwareengineerat0.5FTE.Nostandingarmycost(thereisnoidleworkforceintheeventofadelay).TheaverageL3burnrateduetothedelayofdownstreamactivitiesis$10k/month(CMS-doc-13481).Mincost=1month*$10kburnrate=$10k.Likelycost=3months*$10kburnrate=$30k.Maxcost=6months*$10kburnrate=$60k.
CauseorTrigger: ImpactedActivities: Prototypeandproductionfirmwarereleases.ProbabilityissharedbetweenCaloTriggerandCorrelatorTrigger.
Startdate: 1/Jul/2017 Enddate: 1/Jan/2024RiskMitigations: Infrastructureandalgorithmfirmwarearedevelopedincyclesandprogresstracked.Ifthetechnicalorsciencerequirementsarenotmetfora
particularmilestone,thisallowsengineerstomodifythefirmwaresothatitmeetsthetechnicalandsciencerequirementsinthenextcycle.Theuseofhigh-levelsynthesistoolsenablescientiststotranslatehigh-levelalgorithmsintofirmware,reducingthepotentialforinconsistenciesbetweenfirmwareandsoftwareemulation,aswellasreducingthepotentialtointroducebugsinthefirmwareimplementation,comparedwiththescientificintentofthealgorithm.
RiskResponses: (1)Ifthetechnicalorsciencerequirementsarenotmetforaparticularmilestone,thenallowtheengineerstomodifythefirmwaresothatitmeetsthetechnicalandsciencerequirementsinthenextcycle.(2)hireadditionalfirmwareengineering,eitherbyallocatingmoretimetoon-projectengineers,orhiringadditionalpersons.
Moredetails:
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 48 Updated: 2019-03-06
RT-402-6-06-DTD-BaselineFPGAdoesnotsatisfyrequirements(DOE)
RiskRank: 2(Medium)Scores:Probability:2(L);Cost:2(M)Schedule:2(M)) RiskStatus: OpenSummary: IfthecomputationrequirementsofcalorimetertriggerandcorrelatorarenotsatisfiedbythebaselineFPGA,thenthesciencerequirementsare
jeopardized.RiskType: Threat Owner: JeffreyWBerryhillWBS: 402.6TD-TriggerandDAQ(DOE) RiskArea: TechnicalRisk/ReliabilityorPerformanceProbability(P): 20% TechnicalImpact: 2(M)-significantlysubstandardCostImpact: PDF =3-point-triangular
Minimum =136k$Mostlikely =282k$Maximum =564k$Mean =327k$P*<Impact> =65k$
ScheduleImpact: PDF =3-point-triangularMinimum =1monthsMostlikely =3monthsMaximum =6monthsMean =3.33monthsP*<Impact> =0.7months
BasisofEstimate: 12.5/25/50%FPGAcostincreaseon(36+48=)84FPGAsat12k/chipbasecost..Min/likely/maxcost=126/252/504k$.1/3/6monthsdelayforprocurement.TheaverageL3burnrateduetothedelayofdownstreamactivitiesis$10k/month(CMS-doc-13481).Mincost=$126k+1month*$10kburnrate=$136k.Likelycost=$252k+3months*$10kburnrate=$282k.Maxcost=$504k+6months*$10kburnrate=$564k.
CauseorTrigger: Increasinglyrealistic,betterperformingalgorithmsorincreasednumberofalgorithmsperFPGAmightrequiremorethanthelimitof50%utilizationofFPGAlogicresourcesormightrequireclockspeedsthatarehigherthanavailableforthebaselineFPGA,whichmaynecessitatetheuseofahigherperformanceFPGAoranincreasednumberofFPGAs.Alternatively,lowerlatencyrequirementsplacedonthetriggersubsystemsmightrequirefastercalculationsandhencehigherperformanceFPGAs.
ImpactedActivities: ProbabilityissharedbetweenCaloTriggerandCorrelatorTrigger.
Startdate: 1/Jan/2020 Enddate: 31/Mar/2021RiskMitigations: ThebaselineFPGAsupportstheprocessingofsomeprototypealgorithms,whichhavebeendemonstratedtoutilizenotmorethan40%ofthelogic
resourcesatnominalclockspeeds,providingabout10%headroom.Theprototypealgorithmshavebeendemonstratedtocompletewithinonemicrosecondoflatency(consistentwiththecurrentlatencyallocation),potentiallyprovidingsomelimitedabilitytotradelogicresourcesformoreclock-cycles(i.e.longerlatency).Thetotallatencybudgetsareonlyguidelineestimates,andaresubjecttochange.Thecurrentguidelinesforthecalorimetertriggeris1.5microsecondsoflatency(includingRXandTX)andforthecorrelatortrigger2.5microsecondsoftotallatency,split(roughlyevenly)betweenlayer-1andlayer-2(includingRXandTX).
RiskResponses: (1)attempttooptimisethefirmwareimplementationofthealgorithmtoreducethelogicutilizationtofitwithinatotalof50%,(2)increasethelatency(numberofclockcycles)ofthecalculationinanattempttoreducethelogicutilization,whileremainingwithintheallocatedlatencybudget,(3)useahigherperformanceFPGA(suchasaVU13P)oruseahigher-speedgradeFPGAorbuymorebaselineFPGAs(i.e.moreboards)
Moredetails: CMS-doc-13481
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 49 Updated: 2019-03-06
RT-402-6-90-DTD-KeyTriggerorDAQpersonnelneedtobereplaced(DOE)
RiskRank: 1(Low)Scores:Probability:3(M);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: Keyengineerorseniortechnicianwithspecialknowledgeleavestheprojectandneedstobereplaced.Ifthetransitioncanbemanagedsuchthattheincomingand
outgoingpersonneloverlapandexchangeknowledge,thenthereismainlyalaborcostimpact.Ifthetransitionismoreabrupt,thenthereisnocostimpactofoverlappingpersonnelbuttherecanbeadelaytotheprojectactivitiesastheincomingpersongetsfullyuptospeed.Thisriskdoesnotincludetheriskoflosingkeymanagersbecausetheprojectensuresthateachmanagerhasawell-traineddeputy.
RiskType: Threat Owner: JeffreyWBerryhillWBS: 402.6TD-TriggerandDAQ(DOE) RiskArea: ManagementRisk/FundingorResourcesProbability(P): 25% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular
Minimum =30k$Mostlikely =90k$Maximum =150k$Mean =90k$P*<Impact> =23k$
ScheduleImpact: PDF =3-point-triangularMinimum =0monthsMostlikely =0monthsMaximum =3monthsMean =1monthsP*<Impact> =0.3months
BasisofEstimate: ThisL2areahas2seniorengineerawhosefunctionsareparticularlyhardtoreplaceonashorttimescalebyexistingteammembers.ExperienceintheCMSPhase1Upgradeprojectsuggeststheprobabilityperkeypersonisroughly25%(probabilitytheyleaveduringtheentireproject).Someinputnumberstothecostimpactanalysis:--Laborcostperperson=$15k/month(fullyburdenedseniorenginer=$185k/year).--AverageburnrateperL3areais=$10k/month(CMS-doc-13481).Minimumscenerio:personleavingisreplacedbyanexistingskilledpersonintheteam.1monthoffulloverlapcosting$15koflabor.Noscheduledelay.Likelyscenario:personleavingisreplacedbyapersonlessfamiliarwiththespecificwork.3monthsmanagedoverlapoftheoutgoingandincomingpeoplecosts3*$15k=$45koflabor.Noscheduledelay.Maximumscenario:personleavesunexpectedlywithnotransferofknowledge.Ittakesabout6monthstofindanewpersonandgetthemfullyuptospeed.Thereislossofproductivityduringthisdifficulttransitionperiodresultinginanet3monthdelaytoL3activities.Costoframp-upeffortofthenewperson(learningbutnotfullycontributingtodeliverables)=3FTE-months*$15k=$45k.Theburnratecostis3*$10k=$30k.Totalcostimpact=$75kTotalfor2persons:Min/Likely/Maxcostimpact=2*$(15/45/75)k=$(30/90/150)$.
CauseorTrigger: ImpactedActivities: EachkeypersonsisindependentlymodeledintheMC(eachatP=25%andwithcorrespondingshareofthetotalcostimpact).Typicallyhooktherisktothepersonsmainactivityduringtheproductionphase(maximumconsequences).
Startdate: 1/Jan/2018 Enddate: RiskMitigations: Anumberofengineersandtechnicianshaveoverlappingskills,bothwithinagiveninstituteandacrossUS-CMSinstitutes.Thisprovidessomebackupincaseof
non-availabilityofakeyperson.Pro-activecross-trainingofengineersandtechnicianshelpsensurekeyskillsarenotcompletelylostifakeypersonisnolongeravailable.
RiskResponses: Aimforoutgoingandincomingpersonneltooverlapifpossibletoensureasmoothtransition.Hirefromskilledmembersoftheteamifappropriate.Interimsupportmaybepossibleusingengineersortechnicainsfromelsewhereintheinstituteorproject.
Moredetails: CMS-doc-13481
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 50 Updated: 2019-03-06
RT-402-6-91-DTD-ShortfallinTriggerorDAQscientificlabor(DOE)
RiskRank: 2(Medium)Scores:Probability:3(M);Cost:2(M)Schedule:0(N)) RiskStatus: OpenSummary: Ifasignificantamountofthe(uncosted)scientificlaborisunavailable,thentheprojectwouldthenneedtofundadditional(costed)personnelto
performthework.Itisassumedthattheriskistriggeredbyaseriouslyunfavorableoverallbaseprogramfundingsituation.RiskType: Threat Owner: JeffreyWBerryhillWBS: 402.6TD-TriggerandDAQ(DOE) RiskArea: ManagementRisk/FundingorResourcesProbability(P): 30% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular
Minimum =0k$Mostlikely =0k$Maximum =292k$Mean =97k$P*<Impact> =29k$
ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =0monthsMaximum =monthsMean =0monthsP*<Impact> =0.0months
BasisofEstimate: InthepastUS-CMShasnotexperiencedasignificantlackofscientificlabor(postdocsandgraduatestudents).WhenshortfallsoccuredtheywereusuallyresolvedbycollaboratorsatUS-CMSinstitutesorsometimesfromiCMS.Weassigna30%probabilitythatasignificantshortfalloccursinfuture,duetounfavorablefundingconditionsinthebaseprogram.ThecontributedlaborforthescopeofthisL2areais16.8FTE-yearsspreadoverabout5years.ThisdoesnotincludethemoresecureL2andL3managerswhoaretenuredfacultyandseniorFermilabscientists.Weestimatethelosscouldbeupto20%ofthetotalcontributedlaboror3.4FTE-years(e.g.thisisalossof50%ofallcontributedlaborforatwoyearperiodoralossof1/3ofthecontributedlaborfor3years).Themissinglaborcouldbereplacedbycostedpersonnel:amixtureofmid-rangetechnicians,juniortechnicians,orundergraduatescostingrespectively62$/hr,50$/hrand18$/hrfully-burdened(43$/houronaverage).Allowingforfouryearsofescalationat3.1%perannumyieldsanaveragecostof49$/hror86k$/FTE-year(1768hrsworkedperyear).The(min/likely/max)costimpactistherefore:86k$perFTE-year*(0/0/3.4)FTE-years=$(0/0/292)k.
CauseorTrigger: Theriskistriggeredbyanunfavorablebaseprogramfundingsituation. ImpactedActivities: Startdate: 1/Oct/2018 Enddate: RiskMitigations: Workwithinstitutesandagenciestoensuretheanticipatedamountofscientificlaborwillbeavailable.Whereshortfallslooklikelytooccur,seek
alternativesamongstotherUS-CMSinstitutesorevenfromiCMSinstitutes.RiskResponses: Seekreplacementscientificlaborinotherinstitutes.Ifthislaborcannotbefoundthencontingencywillneedtobespenttosupplementtheeffort
withcostedlabor(e.g.technicians).Moredetails: CMS-doc-13509(FTEdataatCD1)
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 51 Updated: 2019-03-06
RT-402-8-01-DETL-AdditionalFEASICprototypecycleisrequired
RiskRank: 3(High)Scores:Probability:4(H);Cost:2(M)Schedule:2(M)) RiskStatus: OpenSummary: Thisriskcanhavemultiplecauses.
1)Ifthenecessaryperformance(precisionvspowerconsumption)isnotachievedduringthelastprototypecycle,anadditionalprototypecyclemaybenecessarycausingadelayandincurringacostincrease.2)ASICspecificationhaschangedduetoexternalreasons,suchaslpGBTclockdistribtutiondoesnotmeetspecifications.
RiskType: Threat Owner: TiehuiLiuWBS: 402.8.4ETL-EndcapTimingLayer RiskArea: TechnicalRisk/ReliabilityorPerformanceProbability(P): 50% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular
Minimum =500k$Mostlikely =600k$Maximum =700k$Mean =600k$P*<Impact> =300k$
ScheduleImpact: PDF =3-point-triangularMinimum =4monthsMostlikely =5monthsMaximum =6monthsMean =5monthsP*<Impact> =2.5months
BasisofEstimate: ThecostforaMPWrunis400k,andweinclude100-300kforadditionalengineering.
CauseorTrigger: Prototype2doesnotmeetrequirementsandwecannotincludechangesinthepreproductionsubmission.
ImpactedActivities: RiskcoulddelaycompletionofASICprototyping
Startdate: 1/Mar/2020 Enddate: 1/Jan/2022RiskMitigations: Wehaveincreasedtheeffortforthedesignsimulationandverification.
RiskResponses: Weincludeanadditionalprototypecycle.
Moredetails:
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 52 Updated: 2019-03-06
RT-402-8-02-DETL-ProblemswithETLmoduleassemblyfacility
RiskRank: 2(Medium)Scores:Probability:4(H);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: Thisisageneralcatagorythatincludesmultipleissuesthatcanleadtoaproductionslowdownorstopage.IfaModuleAssemblyfacilitytemporarily
becomesinoperableduetolossofcriticalequipmentoractofgod,thentheresultantdipinmodulethroughputmayjeopardizetimelycompletionoftheproject.
RiskType: Threat Owner: SlawomirMTkaczykWBS: 402.8.4ETL-EndcapTimingLayer RiskArea: ExternalRisk/FacilitiesProbability(P): 50% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =1-point-singlevalue
Minimum =k$Mostlikely =30k$Maximum =k$Mean =30k$P*<Impact> =15k$
ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =1monthsMaximum =monthsMean =1monthsP*<Impact> =0.5months
BasisofEstimate: Ifonecenterhasamajorequipmentfailurethesecondcentercanpickuptheadditionalloadwithinthe100%cushion.Itisassumedthatin1monththeequipmentwouldbereplacedandassemblycouldthenberesumedattheaffectedsite.Anexpecteddelayof1monthduetoaneedtotransfercomponentstothesecondproductionsiteandtoorganizesecondshiftmoduleproduction.
CauseorTrigger: ImpactedActivities: RiskcoulddelayanyoneofthebatchesduringETLAssembly
Startdate: 1/Jan/2023 Enddate: 1/Jul/2024RiskMitigations: HavingatleasttwoUSsitesisariskmitigation,andshouldonesitebecometemporarilyinoperable,componentscouldberedirectedtotheother
sitetemporarilytomitigatetheimpact,whiletheaffectedproductionsiteisbeingrepaired.RiskResponses: Modulecomponentscanbedivertedtotheunaffectedsitetoutilizeitsfullthroughput,andadditionalresourcesaddedtoincreasemodule
productionthroughputatbothsites(oncetheaffectedoneisre-established)toregaintimeintheschedule.Moredetails:
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 53 Updated: 2019-03-06
RT-402-8-03-DETL-FEASICdoesnotmeetspecs-needsanotherpre-prodrun
RiskRank: 2(Medium)Scores:Probability:2(L);Cost:3(H)Schedule:3(H)) RiskStatus: OpenSummary: IfthepreproductionFrontEndASICdoesnotmeetspecificationsafterthemasksareproducedthenanadditionalsetofmaskswillneedtobe
producedandadditionalengineeringeffortisrequiredtoadjustthedesignsothattheASICmeetsspecification.RiskType: Threat Owner: TiehuiLiuWBS: 402.8.4ETL-EndcapTimingLayer RiskArea: TechnicalRisk/QualityProbability(P): 10% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular
Minimum =914k$Mostlikely =970k$Maximum =1026k$Mean =970k$P*<Impact> =97k$
ScheduleImpact: PDF =3-point-triangularMinimum =6monthsMostlikely =7.5monthsMaximum =9monthsMean =7.5monthsP*<Impact> =0.8months
BasisofEstimate: TheM&Scostis$760kfornewmasksandqualificationwafers.Thelaborcostfortheadditionalengineeringeffortisestimatedasa3-pttriangularPDF:0.25-0.375-0.5FTE.Theestimatedscheduledelayis3-pttriangularPDF:6-7.5-9monthsandweusethestandingarmycostestimatefortheMTD,asdescribedinDocDB#13481,fortheburnrate(14k$/month).Mincost=760k$+0.25*280k$+6month*14k$burnrate=$914kLikelycost=760k$+0.375*280k$+7.5months*14k$burnrate=$970kMaxcost=760k$+0.5*280k$+9months*14k$burnrate=$1026k
CauseorTrigger: ETLFrontEndASICasreturnedfromproductiondoesnotmeetspecificationdespitethepreviousMPWs.
ImpactedActivities: RiskcoulddelaycompletionofETLpreproduction
Startdate: 1/Jan/2022 Enddate: 1/Jan/2023RiskMitigations: Thisriskismitigatedbycarryingoutseveralprototypecyclesandincreaseddesignsimulationandverificationeffort.
RiskResponses: Ifthefirstpre-productionrundoesnotmeetourrequirementswewillhavetofixtheproblemsandproduceanewmaskset.IftheASICisdelayed
andwecannotmeetthemilestoneofinstallingtheETLduringLS3wemayhavetoinstalltheETLduringoneoftheYETS.Moredetails:
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 54 Updated: 2019-03-06
RT-402-8-04-DBTL-LYSOmatricesnotmeetingspecifications
RiskRank: 1(Low)Scores:Probability:2(L);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: IfdeliveredLYSOcrystalsdonotmeetgeometricoropticalspecificationsthentherewillbeadelayinproduction.RiskType: Threat Owner: AdolfBornheimWBS: 402.8.3BTL-BarrelTimingLayer RiskArea: ExternalRisk/VendorsProbability(P): 10% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =1-point-singlevalue
Minimum =k$Mostlikely =100k$Maximum =k$Mean =100k$P*<Impact> =10k$
ScheduleImpact: PDF =3-point-triangularMinimum =1monthsMostlikely =2monthsMaximum =3monthsMean =2monthsP*<Impact> =0.2months
BasisofEstimate: IfthereisadelayofanyofthesenecessarycomponentsthatistranslatedtoBTLAssembly,thentherewillbeknock-onstandingarmycoststhere.Thisestimateisfor1FTEofonetechnician.
CauseorTrigger: Measurementsofsensorperformanceduringmoduleassembly.
ImpactedActivities: RiskcoulddelayanyoneofthebatchesduringBTLAssembly
Startdate: 1/Jan/2020 Enddate: 18/Jan/2022RiskMitigations: QA/QCprocedureintheLYSOprocurement,strictdefinitionofQAcriteriawithvendor.RiskResponses: Oncedeliveryofqualifiedcomponentsrestarts,runx2shiftsofBTLAssemblyandQCtesting,runx2shiftsofBTLIntegration.Moredetails:
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 55 Updated: 2019-03-06
RT-402-8-05-DBTL-Changeininterfacesoftrayassemblycomponents
RiskRank: 2(Medium)Scores:Probability:2(L);Cost:2(M)Schedule:1(L)) RiskStatus: OpenSummary: Ifthedimensions,connectionsorassemblysequencedetailsofthetraychangethentheproceduresintheintegrationcenterhavetobeadjusted
incurringdelaysandjeopardizingcompletionoftheBTLbytheCMSneed-bydate.RiskType: Threat Owner: AdolfBornheimWBS: 402.8.3BTL-BarrelTimingLayer RiskArea: ExternalRisk/CollaboratorsProbability(P): 20% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular
Minimum =150k$Mostlikely =250k$Maximum =350k$Mean =250k$P*<Impact> =50k$
ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =3monthsMaximum =monthsMean =3monthsP*<Impact> =0.6months
BasisofEstimate: Iftheassemblyofthetraysfromthethreeprincipalcomponents,coolingbar,sensormoulesandfrontendcards,ischangedthemanpowerneedsmayincreaseandadditionaltoolingandskilledlaborneedtobeaaddedtothetask.Mostlikelyimpactis1FTE-yearofaddtionalengineeringatanestimatedcostof$250k.
CauseorTrigger: Changeoftheinterfacesononeorseveralofthecomponentsenteringintothetrayassemblyprocedure.
ImpactedActivities: RiskcoulddelaythecompletionoftheBTLAssemblyR&Dandprototyping
Startdate: 1/Oct/2019 Enddate: 10/Feb/2023RiskMitigations: WorkwithcollboratorsduringtheR&Dphasetoensuredesignsarecoherent,agreeduponandproperlycommunicatedtoallstakeholders.Further,
ensuredetailedtestingofallprototypesandpreproductionitemstoensureperformanceandtechnicalspecsaremetatalltimes.RiskResponses: Ifinterfacechangesarenecessary,ensurethatdesignchangesremainascompatibleaspossiblewiththeassemblycenterprocedures.Update
procedurespursuanttoupdatedinterfaces.Losttimemadeupthroughincreasedlabor.Moredetails:
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 56 Updated: 2019-03-06
RT-402-8-07-DBTL-ConcentratorCarddelayinexternalcomponentdeliveries
RiskRank: 2(Medium)Scores:Probability:2(L);Cost:0(N)Schedule:2(M)) RiskStatus: OpenSummary: Ifexternalcomponentsaredelayedthenthedeliverydeadlineobjectiveisjeopardized.RiskType: Threat Owner: YuriiYMaravinWBS: 402.8.3BTL-BarrelTimingLayer RiskArea: ExternalRisk/VendorsProbability(P): 20% TechnicalImpact: 3(H)-extremelysubstandardorKPPin
jeopardyCostImpact: PDF =1-point-singlevalue
Minimum =k$Mostlikely =0k$Maximum =k$Mean =0k$P*<Impact> =0k$
ScheduleImpact: PDF =3-point-triangularMinimum =1monthsMostlikely =3monthsMaximum =6monthsMean =3.33monthsP*<Impact> =0.7months
BasisofEstimate: AconservativeestimatebasedonthefactthatthelpGBTprototypewasdelayedbyabouthalfayear.GiventhefactthatthelpGBTpinoutisdelayedby6months,thisestimateisratherreasonablethatthedelaywillbeontheorderof6months.
CauseorTrigger: Delaywiththeprototypesorthefinaldesigncomponents ImpactedActivities: RiskcoulddelayanybatchofCCsduringCCproduction
Startdate: 1/Oct/2019 Enddate: 15/Apr/2020RiskMitigations: Itisnotpossibletomitigatethisrisk.RiskResponses: Moredetails:
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 57 Updated: 2019-03-06
RT-402-8-08-DBTL-Delayincoolingplatedelivery
RiskRank: 1(Low)Scores:Probability:2(L);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: Ifthedeliveryofcoolingplatesbyinternationalpartnerstotheassemblycentersisdelayed,theassemblyofBTLtraysandsubsequentlythedelivery
oftraystoCERNgetsdelayed.RiskType: Threat Owner: AdolfBornheimWBS: 402.8.3BTL-BarrelTimingLayer RiskArea: ExternalRisk/CollaboratorsProbability(P): 10% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular
Minimum =10k$Mostlikely =20k$Maximum =30k$Mean =20k$P*<Impact> =2k$
ScheduleImpact: PDF =3-point-triangularMinimum =1monthsMostlikely =2monthsMaximum =3monthsMean =2monthsP*<Impact> =0.2months
BasisofEstimate: Ifcoolingplatedeliveryisdelayedtheundergraduteworkerswillnotbeabletoassembletrays.ExtratimeneedstobeworkedlatertomakeupthedelaytokeepthedeliveryscheduletoCERN.TheassemblyoftheBTLtraysisdesignedsuchthatadditionalFTEcanworkinparallelandincreasedthroughputatthecenter.CostreflectscostforaddtionalFTEstomitigatethescheduleimpact.
CauseorTrigger: Delayofcoolingplatedelivery.
ImpactedActivities: RiskcoulddelayanyoneofthebatchesduringBTLAssembly
Startdate: 28/Jan/2022 Enddate: 31/Jan/2024RiskMitigations: CommunicatecloselywithiCMSpartnerstomaintainscheduleawareness.RiskResponses: Acceleratetrayassembly.Hireadditionalundergratuatestudentstoworkovertimeandweekend.Moredetails:
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 58 Updated: 2019-03-06
RT-402-8-10-DETL-Sensorqualityproblemduringproduction
RiskRank: 2(Medium)Scores:Probability:2(L);Cost:1(L)Schedule:2(M)) RiskStatus: OpenSummary: IfthesensorvendordeliverssensorsthatdonotmeetspecificationsthenthedegradedperformanceoftheETLjeopardizesthephysicsperformance
oftheMTDdetector.RiskType: Threat Owner: ChristopherS.RoganWBS: 402.8.4ETL-EndcapTimingLayer RiskArea: ExternalRisk/VendorsProbability(P): 15% TechnicalImpact: 2(M)-significantlysubstandardCostImpact: PDF =3-point-triangular
Minimum =28k$Mostlikely =52k$Maximum =109k$Mean =63k$P*<Impact> =9k$
ScheduleImpact: PDF =3-point-triangularMinimum =2monthsMostlikely =3monthsMaximum =6monthsMean =3.67monthsP*<Impact> =0.6months
BasisofEstimate: ThecontractwillbewrittenforthevendortodeliveraspecifiednumberofgoodsensorsthatsatisfyCMSspecifications.Thuswedonothavetopayforsensorsthatdonotsatisfythespecificationsandthereisnoimpactonsensorcost.TheonlycostimpactisthatwewillhavetorepeattheQCtestingofthereplacementsensors.Minimalimpact:thishappensduringproductionandiscorrectedquicklyafterfeedbackfromsensorQCleadingtoadelayofabout2monthsandnegligibledirectcost.Maximalscheduleimpact:thishappensduringpreproductionandthepreproductioncyclehastoberepeated,leadingtoadelayofabout6monthsandextralaborcostofabout$25k.TheL3burnrateduetothedelayofdownstreamactivitiesis$14k/month(CMS-doc-13481).Mincost=$0k+2months*$14kburnrate=$28k.Likelycost=$10k+3month*$14kburnrate=$52k.Maxcost=$25k+6months*$14kburnrate=$109k.TheproblemhastoeitherpersistovermanybatchesornotbenoticedduringQCatthevendor(forexampleadegradationofperformanceoversometime).Problemsthataffectasinglebatchofsensors(egbecauseofsomecontaminationorprocessingmistake)willnotleadtoasignificantdelaybecausereprocessingabatchwillonlyaddaweekortwototheproductionperiod.Therefore,theprobabilityofthisthreattooccurismoderate–15%.
CauseorTrigger: Sensorsarrivingfromvendordonotsatisfyspecifications ImpactedActivities: RiskcoulddelayanyoneofthebatchesduringETLsensorQC
Startdate: 1/Jan/2022 Enddate: 1/Apr/2024RiskMitigations: Wecarryoutextensiveprototypingworkwiththevendorspriortoplacingthecontractforsensorproductiontomakesurethatvensorsunderstand
ourspecificationsandcanmeetthem.ThevendorwillcarryoutafirstsetofQCmeasurementsbeforethesensorsareshippedtoCERNanddistributedtoQCcenters.Thisensuresthatmostproblemswillbecaughtquicklyanddonotleadtosignificantimpactontheproject.Thecostofthesemeasurementsisfacturedintothesensorcost.
RiskResponses: Ifamodestproblemoccurs,workcloselywithvendortosolveit(e.g.testing).Replacetheflawedsensors.Moredetails:
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 59 Updated: 2019-03-06
RT-402-8-14-DBTL-ProblemswithSiPMvendor
RiskRank: 2(Medium)Scores:Probability:2(L);Cost:1(L)Schedule:3(H)) RiskStatus: OpenSummary: ThisisageneralcatagorythatincludesmultipleproblemsassociatedwiththeSiPMvendor.
1)IftheproductionSiPMsdonotmeetspecificationsduetoaprocessingerroratthevendor,thentherewillbeasixmonthdelayintheschedulefortestingandcharacterizingtheSiPMs.Thismayimpacttheoverallprojectscheduleandcost.2)IftheproductionofSiPMsisdelayedatthevendor,thatdelaywillpropogatetotheSiPMtestingandcharacterizationschedule,andinturntheoverallprojectschedule.
RiskType: Threat Owner: MitchellWayneWBS: 402.8.3BTL-BarrelTimingLayer RiskArea: ExternalRisk/VendorsProbability(P): 20% TechnicalImpact: 2(M)-significantlysubstandardCostImpact: PDF =3-point-triangular
Minimum =32k$Mostlikely =96k$Maximum =128k$Mean =85k$P*<Impact> =17k$
ScheduleImpact: PDF =3-point-triangularMinimum =2monthsMostlikely =6monthsMaximum =8monthsMean =5.33monthsP*<Impact> =1.1months
BasisofEstimate: 1)IntheeventtheproductionSiPMsdonotmeetspecifications,weestimatea6monthscheduledelaytoproduceanewsetofproductionSiPMS.Thisdelayincludesfabricationofnewwafers,testingandpackagingoftheSiPMs.Thecostimpactwillbe6monthsofhalf-timesupportfortwoengineeringphysicists.2)IntheeventdeliveryoftheproductionSiPMsisdelayedbysomeamountoftime,weestimatetheimpactfromaminimaldelayof2monthsuptoamaximumdelayof8months.Thecostimpactwillbeadditionalsupportfortwoengineeringphysicistsforeachmonthofdelay.
CauseorTrigger: ImpactedActivities: RiskcoulddelayanybatchofSiPMQC
Startdate: 1/Oct/2019 Enddate: 18/Jan/2022RiskMitigations: WorkwithvendortoestablishqualityandgoodyieldthroughprototypeandpreproductionSiPMruns.Couldusemultiple(currentlythereare
2)vendorsifthequalityiscomparable.
RiskResponses: 1)IntheeventofSiPMsnotmeetingspecifications,workwithvendortoidentifythesourceoftheproblem,thenrestarttheproductionprocess.2)IntheeventofadelayinSiPMdelivery,worktotheextentpossibletospeeduptestingandcharacterizationoncetheyaredelivered.
Moredetails:
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 60 Updated: 2019-03-06
RT-402-8-15-DBTL-BatchshipmentofSiPMslostintransport
RiskRank: 1(Low)Scores:Probability:1(VL);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: IfabatchoftestSiPMsislostinshipmenttoaproductionsite,thentherewillbeaonemonthdelayoftheSiPMcompletiondate.Thismayimpact
theoverallscheduleandcost.RiskType: Threat Owner: MitchellWayneWBS: 402.8.3BTL-BarrelTimingLayer RiskArea: ManagementRisk/LogisticsProbability(P): 5% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =1-point-singlevalue
Minimum =k$Mostlikely =224k$Maximum =k$Mean =224k$P*<Impact> =11k$
ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =1monthsMaximum =monthsMean =1monthsP*<Impact> =0.1months
BasisofEstimate: Weestimatea1monthdelaytoQCanewbatchofproductionSiPMs.ThismayinterruptthemoduleassemblybyonemonthanddelaythedeliveryofthefinalSiPMbatchbyonemonth.Thecostimpactwillbe$224k,basedon28,000channelsandperchannelcostof$8.TheimpactisthesameforanyUSbatch,astheUSdeliversthefirsthalfoftheSiPMs.
CauseorTrigger: ImpactedActivities: RiskcoulddelayanyoneofthebatchesduringBTLAssembly
Startdate: 1/Jan/2021 Enddate: 18/Jan/2022RiskMitigations: Divideupthebatchshipmentintoanumberofsmallershipments,thusmitigatingtheimpactofalostshipment.Ofcoursethisimpliesmore
shipments,increasingtheprobabilityofonebeinglost.Weneedtooptimizethisforminimalrisk.RiskResponses: Ifashipmentislost,wewillneedtopurchaseadditionalproductionSiPMsfromthevendor,havethemdelivered,andthenrunthroughthestandard
QCandcharacterization.Moredetails:
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 61 Updated: 2019-03-06
RT-402-8-16-DBTL-ProblemswithSiPMQCtestsite
RiskRank: 1(Low)Scores:Probability:2(L);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: ThisisageneralcategorythatincludesmultipleissuesthatcanleadtoreducedorstoppedthoughputattheSiPMQCsite.
1)IfthereisafailureofoneormoreoftheSiPMproductionteststands,thentherewillbeadelayintestingandcharacterizingtheSiPMs.Thismayimpacttheoverallscheduleandcost2)Ifthereisaproblemwiththelabinfrastructure(construction,lossofpower,flooding)thentherewillbeadelayintestingandcharacterizingtheSiPMs.Thismayimpacttheoveralscheduleandcost.
RiskType: Threat Owner: MitchellWayneWBS: 402.8.3BTL-BarrelTimingLayer RiskArea: TechnicalRisk/RequirementsProbability(P): 20% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular
Minimum =2k$Mostlikely =5k$Maximum =10k$Mean =6k$P*<Impact> =1k$
ScheduleImpact: PDF =3-point-triangularMinimum =0.25monthsMostlikely =0.5monthsMaximum =1monthsMean =0.58monthsP*<Impact> =0.1months
BasisofEstimate: 1)WeestimatethatafailureofoneoftheSiPMQCteststandswilldelaytheSiPMQCandcharacterizationbyahalf-month,includingtimetoidentifythecausesofthefailureandmakethenecessaryfixes.2)WeestimatethataproblemwiththelabinfrastructurewilleitherbesolvedorresultinmovingtoanewlocationatCERN.Eitherwayweestimateamaximumscheduleimpactofonemonth.
CauseorTrigger: ImpactedActivities: RiskcoulddelayanybatchofSiPMQCStartdate: 1/Jan/2021 Enddate: 18/Jan/2022RiskMitigations: 1)TheteststandswillbeusedtoQCandcharacterizeasignificantnumberofpreproductionSiPMsbeforewedothefullproductionrun.Identify
criticalequipmentandverifyavailability(e.g.CERNpool).2)Wewillidentifyotherpotentiallocationsasabackupforthetesting.
RiskResponses: 1)Identifythecauseofthefailure,andrepairasneeded.2)Eitherwaituntillabproblemisresolved,ormovetestingtoanotherlocation.
Moredetails:
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 62 Updated: 2019-03-06
RT-402-8-18-DBTL-Concentratorcardproduction&testingfacilityproblem
RiskRank: 1(Low)Scores:Probability:2(L);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: Thisisageneralcatagorythatincludesmultipleissuesthatcanleadtoaslowdownorstoppage.Forexample,ifateststandfailstotestconcentrator
cards,itimpactsthescheduleforCCdeliverytotheassemblycentersRiskType: Threat Owner: YuriiYMaravinWBS: 402.8.3BTL-BarrelTimingLayer RiskArea: TechnicalRisk/RequirementsProbability(P): 20% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =1-point-singlevalue
Minimum =k$Mostlikely =10k$Maximum =k$Mean =10k$P*<Impact> =2k$
ScheduleImpact: PDF =3-point-triangularMinimum =0.5monthsMostlikely =1monthsMaximum =2monthsMean =1.17monthsP*<Impact> =0.2months
BasisofEstimate: PreviousexperienceintestingMu2eFEBcardsindicatedtheneedforaspecialcard($1k)andmodificationstothetestingjig($9k)componentsduetochangesintheFEBfunctionalityandfeaturesthatprohibitedusingexistingcomponentsatKSU.Totalcostimpactis$10k.Orderingnewparts+installingthem+testingthemcausedadelayofupto2months,includingsoftwareadjustments.Burnrateiszeroastheaffectedpersonnelcanworkonothertasks.
CauseorTrigger: FailureofthetestingjigtoprovideathoroughtestoftheassembledCC ImpactedActivities: RiskcoulddelayanybatchofCCsduringCCproduction
Startdate: 15/Mar/2021 Enddate: 15/Jul/2022RiskMitigations: Keepthecomponentsmaintainedandorderspareitemsforcriticalcomponentstobeavailableonashortnoticeincaseoffailure.RiskResponses: Implementmanualtestingprotocolforundergraduateswhiletestingjigissuesareunderstoodandfixed.Hireextraundergraduatestudentsto
performmanualtestingwhilethetestingjigisbeingrepaired.Moredetails:
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 63 Updated: 2019-03-06
RT-402-8-30-DBTL-ConcentratorCardrequiressignificantdesignchanges
RiskRank: 2(Medium)Scores:Probability:2(L);Cost:1(L)Schedule:2(M)) RiskStatus: OpenSummary: Thisisageneralcatagorythatincludesmultipleissuesthatcanleadtoadesignorlayoutchange.
1)IfscopefortheBTLCCchanges,itwillimpactthecostoftheprojectduetoare-designaswellasdelaytheschedule.2)TheconcentratorcarddesignwilldependonthefinalformfactoroftheDC/DCconvertor
RiskType: Threat Owner: YuriiYMaravinWBS: 402.8.3BTL-BarrelTimingLayer RiskArea: ManagementRisk/ControllingProbability(P): 10% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular
Minimum =1k$Mostlikely =50k$Maximum =100k$Mean =50k$P*<Impact> =5k$
ScheduleImpact: PDF =3-point-triangularMinimum =1monthsMostlikely =3monthsMaximum =6monthsMean =3.33monthsP*<Impact> =0.3months
BasisofEstimate: Inthepast,changesindesigncauseddelaysfrom1-2weekstoamonth,incurringadditionalcostsduetoengineertime.However,amajorredesignoftheboard,forexample:adecisiontodoublethebandwidthbyrequiringusingtwolpGBTsinsteadofonewouldrequiredoublingthecomplexityofrouting,essentiallyrequiringanincreaseofthedesigntimebyengineerbyafactoroftwo,includingincreaseintimefortesting.Giventheesitmateof400hoursofengineeringtodesign&layout,doublingofthistimeisagoodestimateofimpactincostandtime.Estimatingthattheengineerissplitbetweentwodifferenttasks,anaverageestimateof5monthsofdelayisextracted.
CauseorTrigger: NewchangestothescopeoftheCCafteritisdesigned ImpactedActivities: RiskcouldimpactthecompletionofCCR&Dandprototyping
Startdate: 1/Oct/2019 Enddate: 27/Jan/2022RiskMitigations: KeepaclosecontacttotheinternationalBTLefforttoensuretimelyresponsetopotentialchangesinthescopeoftheBTLelectronicsandkeep
internationalCMSinformedofimpactsofdesignchangestotheUSscope.
RiskResponses: Delegateafractionofthetaskathandtoanotheravailableengineer,attheKSUElectronicDesginLaboratoryifavailable,oratanotherinstitution.Moredetails:
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 64 Updated: 2019-03-06
RT-402-8-31-DETL-Storage-relateddegradationofLGADs
RiskRank: 1(Low)Scores:Probability:2(L);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: IftheLGADinventoryisleftaroundfortoolongduringtheassemblyprocessthensomeageingmayoccurthatdegradestheperformanceorrenders
theLGADsunusable.ThiseffectwasobservedinproductionofsensorsforthecurrentCMSTracker.RiskType: Threat Owner: SlawomirMTkaczykWBS: 402.8.4ETL-EndcapTimingLayer RiskArea: TechnicalRisk/QualityProbability(P): 10% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =1-point-singlevalue
Minimum =k$Mostlikely =18k$Maximum =k$Mean =18k$P*<Impact> =2k$
ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =3monthsMaximum =monthsMean =3monthsP*<Impact> =0.3months
BasisofEstimate: On-shelfdegradationofsensorqualitywasobservedforonevendorduringprocurmentofsensorsforthepresentCMStracker.Theproductionwasputonholdforabout3monthstounderstandtheproblemandfindasolution.Provideregularmaintenancetothehumiditycontrolingdevicesforpropoercomponentstorage.Incaseofsmallerstoragecabinetsensurethatdryairiscirculated.
CauseorTrigger: ImpactedActivities: RiskcoulddelayanyoneofthebatchesduringETLAssembly
Startdate: 1/Jan/2023 Enddate: 1/Oct/2025RiskMitigations: CarefulQCofthedeliveredLGADsuponreceptionandrepeatQCpriortoassemblyofmoduleswillaloowtoidentifiedproblematicsensors.Proper
storageofcomponentsandhandlinginacontrolledenvironmente.g.cleanroomwithcontrolledhumidityandfilteredaircirculation.RiskResponses: Approved.Changesstartandexpirationdate.Updatedmitigationandresponsetext.Moredetails:
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 65 Updated: 2019-03-06
RT-402-8-33-DBTL-DifficultiesprocuringLYSOfrominternationalsuppliers
RiskRank: 2(Medium)Scores:Probability:2(L);Cost:2(M)Schedule:3(H)) RiskStatus: OpenSummary: IfLYSObecomesthesubjectofhightariffsfromaspecificcountry,thentheincreasedcostjeopardizesprojectbudgetand/ordelaysjeopardizeQC
testingscheduleandallsubsequentdependentactivities.RiskType: Threat Owner: ChristopherCarlNeuWBS: 402.8.3BTL-BarrelTimingLayer RiskArea: ExternalRisk/MarketProbability(P): 10% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular
Minimum =200k$Mostlikely =450k$Maximum =700k$Mean =450k$P*<Impact> =45k$
ScheduleImpact: PDF =3-point-triangularMinimum =3monthsMostlikely =6monthsMaximum =9monthsMean =6monthsP*<Impact> =0.6months
BasisofEstimate: ConsideringanoverallinvestmentfromtheUSof~$900k.Assuminga50%tariffmeansacostimpactof450k,withsomeguessedrange±250k.CauseorTrigger: Example:TheUSgov'tappliesatariffonrare-earthmetalsortheir
relatedcommodities.Internationalsuppliersapplyexportcontrolsoranembargoonthesematerials.
ImpactedActivities: Riskcoulddelaythedeliveryofthepre-productionLYSOcrystals
Startdate: 1/Oct/2019 Enddate: 18/Jan/2022RiskMitigations: Weshouldhavequalifiedvendorsinmultiplecountriessothatweareinsulatedfromeffectsassociatedwithpurchasingfromasingle
foreigncountry.RiskResponses: WorkwithatleasttwoprovidersthatcanincreasetheirLYSOprovisioninthecaseoftheapplicationofatariffaffectingone.Makesurethecontract
containsappropriatelanguage.HavethepurchasegothroughCERNsothatitisinsulatedfromtheseUS-specificeffects.IncreasetheprovisionofLYSOfromalternativevendorthatisnotsubjecttothepolitical/economicobstacleconfronted.
Moredetails:
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 66 Updated: 2019-03-06
RT-402-8-34-DBTL-DelayindeliveryofcomponentsfromiCMS
RiskRank: 1(Low)Scores:Probability:2(L);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: IfthedeliveryofLYSO,SiPMsorelectronicscomponentsforthemodulesisdelayed,themoduleandtrayproductionandsubsequenttray
integrationisdelayed.RiskType: Threat Owner: AdolfBornheimWBS: 402.8.3BTL-BarrelTimingLayer RiskArea: ExternalRisk/CollaboratorsProbability(P): 20% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular
Minimum =10k$Mostlikely =20k$Maximum =30k$Mean =20k$P*<Impact> =4k$
ScheduleImpact: PDF =3-point-triangularMinimum =1monthsMostlikely =2monthsMaximum =3monthsMean =2monthsP*<Impact> =0.4months
BasisofEstimate: Ifthedeliveryofanyofthecomponentsenteringintothetrayintegrationisdelayedwewouldneedtocatchupproductionbyincreasingavaiablemanpower.Themoduleandtrayassemblyisverygranularandcanbescaledeasilybyaddingmanpower.Multipleteamscanworkonasingletraytointegratecomponents.Thecostimpactreflectsthecostofhiringadditionallabor.Thescheduleimpactaccountsforthefactthathiringadditionallaborandtrainingnewpeoplemaycausedelays.
CauseorTrigger: Deliveryofcomponentsisdelayed.
ImpactedActivities: RiskcoulddelayanyoneofthebatchesduringBTLAssembly
Startdate: 28/Jan/2022 Enddate: 31/Jan/2024RiskMitigations: Ensurestrictdefinitionofdeliveryscheduleandintegrateinthecontractualagreementwithvendorsandcollaborators.RiskResponses: Hireadditionalunskilledlabortoworkonthemoduleassemblyandtrayintegrationwithmultipleteamsononetray.Moredetails:
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 67 Updated: 2019-03-06
RT-402-8-35-DBTL-DelaysordamageoftrayintransporttoCERN
RiskRank: 1(Low)Scores:Probability:1(VL);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: IfatrayisdelayedordamagedintransportofBTLtoCERNthentheintegrationandcommissioningtaskmaybedelayed,withapotentialcost
increase.RiskType: Threat Owner: AdolfBornheimWBS: 402.8.3BTL-BarrelTimingLayer RiskArea: ManagementRisk/LogisticsProbability(P): 5% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =1-point-singlevalue
Minimum =k$Mostlikely =220k$Maximum =k$Mean =220k$P*<Impact> =11k$
ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =1monthsMaximum =monthsMean =1monthsP*<Impact> =0.1months
BasisofEstimate: Oneassemblysiteproduces2trayspermonth.Weplantoshipinboxesoftwotrayssincewewillmaketwosparetrays.Alossofoneboxcouldbecompensated.Thecostestimatecorrespondstothecomponentcostof1trayplusextralabortomakethetray.Afterintegrationisfinishedwewanttokeepatleastoneextratrayforfurthertestingandstudiesofthedetectorcomponentseg.intestbeamsoutsideoftheoperatingdetector.
CauseorTrigger: Lossofoneshipment.
ImpactedActivities: RiskcoulddelayanyoneofthebatchesduringBTLAssembl
Startdate: 10/Feb/2023 Enddate: 31/Jan/2024RiskMitigations: Scheduleshipmentsinsmallquantities-2perboxinourcase.RiskResponses: ThereissufficientbufferintheschedulesothatthelossononeboxwillhavenoimpactonthescheduleMoredetails:
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 68 Updated: 2019-03-06
RT-402-8-36-DBTL-InterfacetoiCMSchanges
RiskRank: 1(Low)Scores:Probability:2(L);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: Ifthemechanical,thermalorserviceconnectionsoftheBTLtotherestofCMSchange,therespectivedesignshavetobeadjusted.RiskType: Threat Owner: AdolfBornheimWBS: 402.8.3BTL-BarrelTimingLayer RiskArea: TechnicalRisk/InterfacesProbability(P): 20% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =1-point-singlevalue
Minimum =k$Mostlikely =30k$Maximum =k$Mean =30k$P*<Impact> =6k$
ScheduleImpact: PDF =3-point-triangularMinimum =1monthsMostlikely =2monthsMaximum =3monthsMean =2monthsP*<Impact> =0.4months
BasisofEstimate: Ifeg.thedetailsoftheserviceroutingstoBTLgetchangedweneedtomodifytherespectivedesign.Thismaydelaystartofassemblytoallowadjustmentsontrayconnections.FortheintegrationintoCMSthismaycomplicateintegrationprocedure.Thecostimpactreflectsadditionaldesignworkandadditionallaborfortheassemblyprocedure.Scheduleimpactaccountsforpossibledelayinassemblyorneedtoaccelerateassemblytocompensatetimeittakestoimplementchanges.
CauseorTrigger: ChangeofservicesandmechanicalinterfacesbyCMSTRKorCMSTC
ImpactedActivities: RiskcoulddelaythecompletionoftheBTLAssemblyR&Dandprototyping
Startdate: 1/Apr/2019 Enddate: 3/Jan/2024RiskMitigations: Keepdesignflexibleenoughtoallowmodificationstoservicesandmechanicalinterfaces.Ensurepropercommunication,reviewofdesignsandtight
collaborationinparticularwiththeCMStrackergroup.RiskResponses: Adjustserviceandmechanicalinterfacedesign.MayhaveimpactonreadinessofcomponentsorcomplexitiyoftheassemblyprocessMoredetails:
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 69 Updated: 2019-03-06
RT-402-8-42-DBTL-Problemswithmoduleassemblysite
RiskRank: 1(Low)Scores:Probability:2(L);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: Thisisageneralcategorythatincludesmultipleissuesthatcanleadtoaslowdownorstoppageattheintegrationcenter.Ifsuchaneventoccurs,a
delayisincurredwhiletheissueisbeingresolved.RiskType: Threat Owner: AdolfBornheimWBS: 402.8.3BTL-BarrelTimingLayer RiskArea: ExternalRisk/FacilitiesProbability(P): 10% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular
Minimum =10k$Mostlikely =20k$Maximum =30k$Mean =20k$P*<Impact> =2k$
ScheduleImpact: PDF =3-point-triangularMinimum =1monthsMostlikely =2monthsMaximum =3monthsMean =2monthsP*<Impact> =0.2months
BasisofEstimate: Ifinanyoftheproceduralstepsofthetrayintegrationaproblemoccurs,theassemblyprocedurewillbedelayed.Possibleproblemsarebreakageoftoolsortestequipment,accidentsinhandlingcomponents,qualityproblemseg.ingluingormatingofcomponentscausedbyexternalfactorssuchasimproperenviromentalconditionsortemporarydisruptioninthefunctinalityoftheintegrationcenterinfrastructure.
CauseorTrigger: Stopageofseriousslowdownofassemblyprocedure.
ImpactedActivities: RiskcoulddelayanyoneofthebatchesduringBTLAssembly
Startdate: 28/Jan/2022 Enddate: 31/Jan/2024RiskMitigations: Productionscheduleisdesignedwithsufficientfloattooffsetpossibledelays.Theuseofasecondproductioncentercanoffsetproductioncapacity
issuesinonecenter.RiskResponses: Hireadditionalunskilledlabortointegratetrayswithtwopeopleworkinginparallel.Moredetails:
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 70 Updated: 2019-03-06
RT-402-8-43-DTL-SystemTesting-componentslateforsystemtest
RiskRank: 2(Medium)Scores:Probability:3(M);Cost:1(L)Schedule:2(M)) RiskStatus: OpenSummary: Componentsneededforintegrationintoalargersystemarenotavailableintimeforschedulebeamtests.Theresultsofthesystemtestmaybe
neededfortheEngineeringDesignReviews.Ifcomponentsaredelayedwewilleitherdelaythesystemtestorwillneedtohaveanadditionalsystemtestwhnethecomponentsareavailable.
RiskType: Threat Owner: LindseyA.GrayWBS: 402.8TL-TimingLayer(generalrisks) RiskArea: ExternalRisk/CollaboratorsProbability(P): 30% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular
Minimum =0k$Mostlikely =10k$Maximum =20k$Mean =10k$P*<Impact> =3k$
ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =4monthsMaximum =monthsMean =4monthsP*<Impact> =1.2months
BasisofEstimate: Ifwedelaythetestingwewouldnothaveanyadditionalcost,butmayincuradelayintheschedule.Ifweneedtoincludeanadditionaltestforwhenthecomponentisavailable,wewouldincuradditionalcostsandpossiblyascheduledelay.Probabilityisestimatedtobe30%(15%forBTLand15%forETL).
CauseorTrigger: Delayincomponentneededforasystemtest.
ImpactedActivities: TheresultsoftheBTLandETLsystemtestsmaybeneededduringprototypedevelopmentandasinputtotheEngineeringDesignReviews.
Startdate: 1/Jan/2019 Enddate: 1/Mar/2022RiskMitigations: Monitorthecomponentavailabilityandadjustthetestscheduleifneeded.RiskResponses: Ifacomponentisdelayedwecanstilldoapartialsystemtest.Wecanalsotestthecomponentseparatelyordelaythetestuntilthecomponentis
available.Moredetails:
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 71 Updated: 2019-03-06
RT-402-8-44-DBTL-ConcentratorCardbatchshipmentlost/damaged/delayed
RiskRank: 1(Low)Scores:Probability:1(VL);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: Ifbatchislostordamaged,areplacementneedstobeprocuredthatwouldincreasetheoverallcostoftheproject.Adelayindeliverywouldresultin
thedelayofmeetingthedeadlines.RiskType: Threat Owner: YuriiYMaravinWBS: 402.8.3BTL-BarrelTimingLayer RiskArea: ExternalRisk/VendorsProbability(P): 5% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular
Minimum =0k$Mostlikely =3k$Maximum =9k$Mean =4k$P*<Impact> =0k$
ScheduleImpact: PDF =3-point-triangularMinimum =0monthsMostlikely =0.5monthsMaximum =1monthsMean =0.5monthsP*<Impact> =0.0months
BasisofEstimate: Incaseofacompletelossof22CCsinasingleshipment,theseboardsneedtobeprocured,withanestimateofabout1monthofproductiontime,estimatedfrompreviousexperienceofproducingbatchesofsimilarcomplexityforotherHEPexperiments.Thedelayisnotlikelytobesignificantifitisjustonebatchofmany,asthespeedofdeliveryofCCswilllikelyoutpacetheassemblyspeed.Giventhefactthatwewillproduce10%extracomponents,whichcorrespondstoabout71extraboards,alossofthreeshipmentswillbearzeroimpact.
CauseorTrigger: Lossoftheshipment ImpactedActivities: RiskcoulddelayanyoneofthebatchesduringBTLAssembly
Startdate: 15/Mar/2021 Enddate: 15/Jul/2022RiskMitigations: EnsurespeedydeliveryofalltheCCstoassemblycenters,sothatapotentiallossofashipmentwillnotaffectthetimingoftheassembly.Perform
fastQA/QCofthefirstbatchoftheCCstoestimatethefractionofthespareCCsthatwouldsatisfyqualityrequirementandcouldbeusedifashipmentbatchislost.
RiskResponses: AttempttorepairboardsthatfailQC/QAtestsifthefixesaresimple.Moredetails:
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 72 Updated: 2019-03-06
RT-402-8-48-DETL-DelayindeliveryofpartsfromiCMS
RiskRank: 1(Low)Scores:Probability:2(L);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: Theassemblyofmodulesgetsdelayedifthecomponentsofpartsarenotdeliveredontime.ThismayjeopardizethedeliveryofETLmoduleson
time.RiskType: Threat Owner: SlawomirMTkaczykWBS: 402.8.4ETL-EndcapTimingLayer RiskArea: ExternalRisk/CollaboratorsProbability(P): 20% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular
Minimum =10k$Mostlikely =20k$Maximum =30k$Mean =20k$P*<Impact> =4k$
ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =1monthsMaximum =monthsMean =1monthsP*<Impact> =0.2months
BasisofEstimate: Wewillrecoverthescheduledelaybyaddingadditionalshiftswhichleadstoanimpactonthecosts.Weallowforapossible1monthdelay.CauseorTrigger: ImpactedActivities: Riskcoulddelayanyoneofthebatchesduring
ETLAssemblyStartdate: 1/Jan/2023 Enddate: 1/Oct/2024RiskMitigations: Theprototypeexpereinceshouldinformthemoduleassemblytimeestimate.RiskResponses: Ifthedeliveryofcomponentsisdelayedwewouldneedtoincreaseproductionbyaddingadditionalshifts.Moredetails:
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 73 Updated: 2019-03-06
RT-402-8-49-DETL-DelaysordamageintransportofETLmodulestoCERN
RiskRank: 1(Low)Scores:Probability:1(VL);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: Ifmodulesaredamagedinshippingtheywillneedtobereplacedandthisriskmayimpacttheboththecostandscheduleandmayjeperdizethe
completionoftheprojectonbudgetandontime.RiskType: Threat Owner: SlawomirMTkaczykWBS: 402.8.4ETL-EndcapTimingLayer RiskArea: ManagementRisk/LogisticsProbability(P): 5% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =1-point-singlevalue
Minimum =k$Mostlikely =10k$Maximum =k$Mean =10k$P*<Impact> =1k$
ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =1monthsMaximum =monthsMean =1monthsP*<Impact> =0.1months
BasisofEstimate: CostestimatedforthelossofabatchofmodulestoCERNandtoincreasemoduleproductiontoreplacedamagedmodules.CauseorTrigger: ImpactedActivities: Riskcoulddelayanyoneofthebatchesduring
ETLIntegrationandCommissioningStartdate: 1/Jan/2023 Enddate: 1/Oct/2024RiskMitigations: Carefuldesignofmodulecarrierstoreducetheriskofdamage.Selectionofashippercompanyafterqualification.Quantityofmodulesinone
shipmentshouldnotexceedthenumebrofsparemodulesintheproject.RiskResponses: Increasemoduleproductiontoreplacethedamagedmodulesinordertoreducetheimpactontheschedule.Moredetails:
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 74 Updated: 2019-03-06
RT-402-8-50-DETL-Moduleassemblyyieldislow
RiskRank: 1(Low)Scores:Probability:2(L);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: IftheyieldfromModuleAssemblyislowerthanexpected,thentheadditionalresourcesneededtocompensatefordelaysinprojectscheduleand
budget.RiskType: Threat Owner: SlawomirMTkaczykWBS: 402.8.4ETL-EndcapTimingLayer RiskArea: TechnicalRisk/QualityProbability(P): 10% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular
Minimum =0k$Mostlikely =5k$Maximum =15k$Mean =7k$P*<Impact> =1k$
ScheduleImpact: PDF =3-point-triangularMinimum =0monthsMostlikely =0monthsMaximum =1monthsMean =0.33monthsP*<Impact> =0.0months
BasisofEstimate: Minimum:nocostimpact,nodelay.Likely:nodelay.5k$isthecostofadditionallabourandadditionalcomponents.Maximum:15k$isthecostofadditionallabourandadditionalcomponents.1monthdelay:timetodiagnosetheproblemandimprovethemoduleassemblyprocedure.
CauseorTrigger: ImpactedActivities: RiskcoulddelayanyoneofthebatchesduringETLAssembly
Startdate: 1/Oct/2023 Enddate: 1/Oct/2024RiskMitigations: ExtensivemoduleprototypingandgoodunderstandingoftheassemblyprocessshouldmitigatetheriskofloweryieldsduringModuleassembly.
RigidQCshouldbeabletoidentifymoduleswithlowerqualityataproductionfacilityandtriggerastudyleadingtoundertandtheoriginoftheproblem.
RiskResponses: Additionalresourcesbothtocompensatefortheloweryieldandfurtherparallelizetheassemblyproceduretoregainschedulewouldbeneeded.Moredetails:
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 75 Updated: 2019-03-06
RT-402-8-51-DETL-ProblemwithAlNvendor
RiskRank: 1(Low)Scores:Probability:1(VL);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: IfthereisaproblemwiththeAlNvendorthentherecouldbedelaysorcostincreases.Examplesofproblemsinclude:vendorgoingoutofbusiness,
vendordeliveringsubstandardcarrierAlN,orthevendorcannotdeliveraccordingtotheagreedschedule.RiskType: Threat Owner: SlawomirMTkaczykWBS: 402.8.4ETL-EndcapTimingLayer RiskArea: ExternalRisk/VendorsProbability(P): 5% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular
Minimum =0k$Mostlikely =15k$Maximum =30k$Mean =15k$P*<Impact> =1k$
ScheduleImpact: PDF =3-point-triangularMinimum =1monthsMostlikely =2monthsMaximum =3monthsMean =2monthsP*<Impact> =0.1months
BasisofEstimate: Minimumimpact:theproblemisrelativelymodestandissolvedbythevendorwithin1monthswithoutanydirectcostimpacttotheproject.Likelyimpacts:theproblemismoresignificantandrequiresadditionalprototypeswiththevendorresultinginadelayof2monthsandacostof$15kforAlNandmachining.Maximumimpact:ifweneedtoswitchvendorwewouldneedtoqualifyascondvendorresultingina3monthdelay.Thecostincreasecouldbe$30kforprototypingandsetupcostswiththenewvendor.
CauseorTrigger: ImpactedActivities: RiskcoulddelayanyoneofthebatchesduringETLAssembly
Startdate: 1/Oct/2022 Enddate: 1/Oct/2024RiskMitigations: PrototypingexperienceinthenexttwoyearsshouldresultinareliablecostestimateofthenecessaryAlNformoduleassemblies.
BuyAlNsupportstructuresasearlyaspossible.RiskResponses: Ifweneedtoswitchvendorwewouldneedtoqualifyasecondvendorresultinginamonthdelay.Moredetails:
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 76 Updated: 2019-03-06
RT-402-8-52-DETL-ModuleRadiationTolerance
RiskRank: 1(Low)Scores:Probability:2(L);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: IfModulematerialssuchasadhesives,cables,fibers,etcaresusceptibletodegradationfromradiationexposure,thenresultingdelaystofindrobust
alternativesjeopardizetimelycompletionoftheproject.Thisisariskduringmoduleproductionaftertheprototypingqualificationofmodulecomponents.
RiskType: Threat Owner: SlawomirMTkaczykWBS: 402.8.4ETL-EndcapTimingLayer RiskArea: TechnicalRisk/ReliabilityorPerformanceProbability(P): 10% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =1-point-singlevalue
Minimum =k$Mostlikely =15k$Maximum =k$Mean =15k$P*<Impact> =2k$
ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =1monthsMaximum =monthsMean =1monthsP*<Impact> =0.1months
BasisofEstimate: IncreasedresourcesforlaborandinfrastructuretoparallelizedownstreamactivitiesinModuleAssemblytorecoupdelaysfromradiationeffectsdiscoveredlate.Estimated1monthdelaytorecoupthedelays.
CauseorTrigger: ImpactedActivities: RiskcoulddelayanyoneofthebatchesduringETLAssembly
Startdate: 1/Oct/2022 Enddate: 1/Oct/2024RiskMitigations: Irradiationofdummyandprototypemodulesshouldinformhowlikelyariskthisis.RiskResponses: IncreasedresourcesforlaborandinfrastructuretoparallelizedownstreamactivitiesinModuleAssemblytorecoupdelaysfromradiationeffects
discoveredlate.Moredetails:
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 77 Updated: 2019-03-06
RT-402-8-53-DETL-IntegrationfacilityatCERNrunsoutofcomponents
RiskRank: 1(Low)Scores:Probability:3(M);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: IfETLcomponentdeliverytoCERNisdelayedthenthesubsequentdelayinintegrationjeopardizesthecompletionoftheprojectintimefor
scheduledstartofETLdatataking.RiskType: Threat Owner: SlawomirMTkaczykWBS: 402.8.4ETL-EndcapTimingLayer RiskArea: ExternalRisk/FacilitiesProbability(P): 25% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =1-point-singlevalue
Minimum =k$Mostlikely =21k$Maximum =k$Mean =21k$P*<Impact> =5k$
ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =3monthsMaximum =monthsMean =3monthsP*<Impact> =0.8months
BasisofEstimate: TheI&CactivitieswillbedelayedandadditionalCOLAsupportforUSpersonnelneedstobeprovided.Weassume4studentsand2postdocsneedCOLAsupportat1k$/student/monthand$1.5k/postdoc/month(totalCOLAcost=$7k/month).
CauseorTrigger: ComponentsforETLmoduleassemblyontheDees/wedgesaredelayed. ImpactedActivities: RiskcoulddelayanyoneofthebatchesduringETLIntegrationandCommissioning
Startdate: 1/Jan/2025 Enddate: 1/Jun/2026RiskMitigations: Frontloaddeliveryscheduleofcomponentstogetaheadofneedsofassemblyschedule.RiskResponses: USwillprovidepersonnelatCERNtoassistinintegrationandcommissioningoftheETLdetector.Moredetails:
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 78 Updated: 2019-03-06
RT-402-8-90-DTL-KeyTimingLayerpersonnelneedtobereplaced
RiskRank: 2(Medium)Scores:Probability:3(M);Cost:2(M)Schedule:1(L)) RiskStatus: OpenSummary: Keyengineerorseniortechnicianwithspecialknowledgeleavestheprojectandneedstobereplaced.Ifthetransitioncanbemanagedsuchthattheincomingand
outgoingpersonneloverlapandexchangeknowledge,thenthereismainlyalaborcostimpact.Ifthetransitionismoreabrupt,thenthereisnocostimpactofoverlappingpersonnelbuttherecanbeadelaytotheprojectactivitiesastheincomingpersongetsfullyuptospeed.Thisriskdoesnotincludetheriskoflosingkeymanagersbecausetheprojectensuresthateachmanagerhasawell-traineddeputy.
RiskType: Threat Owner: ChristopherCarlNeuWBS: 402.8TL-TimingLayer(generalrisks) RiskArea: ManagementRisk/FundingorResourcesProbability(P): 25% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular
Minimum =45k$Mostlikely =135k$Maximum =261k$Mean =147k$P*<Impact> =37k$
ScheduleImpact: PDF =3-point-triangularMinimum =0monthsMostlikely =0monthsMaximum =3monthsMean =1monthsP*<Impact> =0.3months
BasisofEstimate: ThisL2areahas3seniorengineerswhosefunctionsareparticularlyhardtoreplaceonashorttimescalebyexistingteammembers.Theyworkinthefollowingareas:(1)SiPMR&Dandqualityassurance;(2)EEforBTLCC;and(3)ASICengineering.ExperienceintheCMSPhase1Upgradeprojectsuggeststheprobabilityperkeypersonisroughly25%(probabilitytheyleaveduringtheentireproject).Someinputnumberstothecostimpactanalysis:--Laborcostperperson=$15k/month(fullyburdenedseniorenginer=$185k/year).--AverageburnrateperL3areais=$14k/month(CMS-doc-13481).Minimumscenerio:personleavingisreplacedbyanexistingskilledpersonintheteam.1monthoffulloverlapcosting$15koflabor.Noscheduledelay.Likelyscenario:personleavingisreplacedbyapersonlessfamiliarwiththespecificwork.3monthsmanagedoverlapoftheoutgoingandincomingpeoplecosts3*$15k=$45koflabor.Noscheduledelay.Maximumscenario:personleavesunexpectedlywithnotransferofknowledge.Ittakesabout6monthstofindanewpersonandgetthemfullyuptospeed.Thereislossofproductivityduringthisdifficulttransitionperiodresultinginanet3monthdelaytoL3activities.Costoframp-upeffortofthenewperson(learningbutnotfullycontributingtodeliverables)=3FTE-months*$15k=$45k.Theburnratecostis3*$14k=$42k.Totalcostimpact=$87kTotalforallkeypersons:Min/Likely/Maxcostimpact=3*$(15/45/87)k=$(45/135/261)$.
CauseorTrigger: ImpactedActivities: EachkeypersonsisindependentlymodeledintheMC(eachatP=25%andwithcorrespondingshareofthetotalcostimpact).Typicallyhooktherisktothepersonsmainactivityduringtheproductionphase(maximumconsequences).
Startdate: 1/Jan/2018 Enddate: RiskMitigations: Anumberofengineersandtechnicianshaveoverlappingskills,bothwithinagiveninstituteandacrossUS-CMSinstitutes.Thisprovidessomebackupincaseof
non-availabilityofakeyperson.Pro-activecross-trainingofengineersandtechnicianshelpsensurekeyskillsarenotcompletelylostifakeypersonisnolongeravailable.
RiskResponses: Aimforoutgoingandincomingpersonneltooverlapifpossibletoensureasmoothtransition.Hirefromskilledmembersoftheteamifappropriate.Interimsupportmaybepossibleusingengineersortechnicainsfromelsewhereintheinstituteorproject.
Moredetails: CMS-doc-13481
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 79 Updated: 2019-03-06
RT-402-8-91-DTL-ShortfallinTimingLayerscientificlabor
RiskRank: 2(Medium)Scores:Probability:3(M);Cost:2(M)Schedule:0(N)) RiskStatus: OpenSummary: Ifasignificantamountofthe(uncosted)scientificlaborisunavailable,thentheprojectwouldthenneedtofundadditional(costed)personnelto
performthework.Itisassumedthattheriskistriggeredbyaseriouslyunfavorableoverallbaseprogramfundingsituation.RiskType: Threat Owner: ChristopherCarlNeuWBS: 402.8TL-TimingLayer(generalrisks) RiskArea: ManagementRisk/FundingorResourcesProbability(P): 30% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular
Minimum =0k$Mostlikely =0k$Maximum =421k$Mean =140k$P*<Impact> =42k$
ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =0monthsMaximum =monthsMean =0monthsP*<Impact> =0.0months
BasisofEstimate: InthepastUS-CMShasnotexperiencedasignificantlackofscientificlabor(postdocsandgraduatestudents).WhenshortfallsoccuredtheywereusuallyresolvedbycollaboratorsatUS-CMSinstitutesorsometimesfromiCMS.Weassigna30%probabilitythatasignificantshortfalloccursinfuture,duetounfavorablefundingconditionsinthebaseprogram.ThecontributedlaborforthescopeofthisL2areais24.4FTE-yearsspreadoverabout5years.ThisdoesnotincludethemoresecureL2andL3managerswhoaretenuredfacultyandseniorFermilabscientists.Weestimatethelosscouldbeupto20%ofthetotalcontributedlaboror4.9FTE-years(e.g.thisisalossof50%ofallcontributedlaborforatwoyearperiodoralossof1/3ofthecontributedlaborfor3years).Themissinglaborcouldbereplacedbycostedpersonnel:amixtureofmid-rangetechnicians,juniortechnicians,orundergraduatescostingrespectively62$/hr,50$/hrand18$/hrfully-burdened(43$/houronaverage).Allowingforfouryearsofescalationat3.1%perannumyieldsanaveragecostof49$/hror86k$/FTE-year(1768hrsworkedperyear).The(min/likely/max)costimpactistherefore:86k$perFTE-year*(0/0/4.9)FTE-years=$(0/0/421)k.
CauseorTrigger: Theriskistriggeredbyanunfavorablebaseprogramfundingsituation. ImpactedActivities: Startdate: 1/Oct/2018 Enddate: RiskMitigations: Workwithinstitutesandagenciestoensuretheanticipatedamountofscientificlaborwillbeavailable.Whereshortfallslooklikelytooccur,seek
alternativesamongstotherUS-CMSinstitutesorevenfromiCMSinstitutes.RiskResponses: Seekreplacementscientificlaborinotherinstitutes.Ifthislaborcannotbefoundthencontingencywillneedtobespenttosupplementtheeffort
withcostedlabor(e.g.technicians).Moredetails:
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 80 Updated: 2019-03-06
RU-402-1-01-DPM-Foreignexchangeratesareuncertain(DOE)
RiskRank: 3(High)Scores:Probability:5(VH);Cost:3(H)Schedule:0(N)) RiskStatus: OpenSummary: Futureexchangeratesaremoreorlessfavorablethanthecanonicalvalueinthebaselineplanresultinginachangeinthecosttotheproject.RiskType: Uncertainty Owner: VivianO'DellWBS: 402.1PM-ProjectManagement(DOE) RiskArea: ExternalRisk/MarketProbability(P): 100% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular
Minimum =-7260k$Mostlikely =1780k$Maximum =10420k$Mean =1647k$P*<Impact> =1647k$
ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =0monthsMaximum =monthsMean =0monthsP*<Impact> =0.0months
BasisofEstimate: Thecostimpactparametersaretheresultofadetailedscenarioanalysis.Historicalexchangeratedatawereusedtoreplaytypicalratevariations(scenarios)andhencemodelpotentialfuturechanges(typicalpercentagechangesfromquartertoquarter--notabsoluterates),asdescribedinCMS-doc-13481andcalculatedinCMS-doc-11825.Theriskuncertaintyisappliedtotheamountofforeigncostsperfiscalyear(determinedfromP6)throughouttheprojectallowingforthefactthattheuncertaintyincreasesfurtherintothefuture.
CauseorTrigger: ImpactedActivities: Startdate: 1/Jan/2018 Enddate: RiskMitigations: Theexchangerateuncertaintyincreasesthefurtherinthefuturethecostisincurred.Thereforetheprojectwillseektoadvanceforeignexchange
transactionswherepossibleandconsistentwiththeavailabilityoffunds.Forexample,itmaybepossibletofront-loadthepaymentoftheCMScommonfund(inSwissFrancs)orprocuresilicon(inYen)earlierthanisabsolutelyneeded(ontechnicalgrounds).
RiskResponses: Ifthecostimpactsareacceptabletheriskresponseistousecontingencytocovertheincreasedcosts.Forextremeexchangeratechangesthatcannotbecoveredbycontingency,workwiththeagenciestoaddresstheissue.Inextremis,donotcompletetheobjectiveKPPsifthereareinsufficientfund.
Moredetails: CMS-doc-13481
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 81 Updated: 2019-03-06
RU-402-1-02-DPM-Futureescalationratesareuncertain(DOE)
RiskRank: 3(High)Scores:Probability:5(VH);Cost:3(H)Schedule:0(N)) RiskStatus: OpenSummary: ThecostestimatesintheBasisofEstimate(BoE)documentsareexpressedintermsofthebase-yeardollarwhentheestimatewasmade(typically
FY17).Forbudgetingpurposes[GAO-09-3SP,page102],projectedescalationratesarethenappliedtoescalatethebase-yeardollarstothen-yeardollarsandhencereflecttheexpectedcostintheyearinwhichitwillbeincurred.Ifactualinflationinfuturediffersfromtheassumedescalationrates,thentheprojectcostmayincreaseordecrease.
RiskType: Uncertainty Owner: VivianO'DellWBS: 402.1PM-ProjectManagement(DOE) RiskArea: ExternalRisk/MarketProbability(P): 100% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular
Minimum =-1850k$Mostlikely =1290k$Maximum =2870k$Mean =770k$P*<Impact> =770k$
ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =0monthsMaximum =monthsMean =0monthsP*<Impact> =0.0months
BasisofEstimate: Thecostimpactisestimatedbyapplyingvarioussetsofhistoricalescalationratetimeprofiles(seperatelyforbothM&Sandlabor)totheprojectcostprofiletoseehowmuchisthecostimpactcomparedtothedefaultassumptioninthebaseplan.Themainanalysisisdescribedin[CMS-doc-13481].Thehistoricaldatasetsassessedare[CMS-doc-12866]:--generalinflation(CPI);--theUSBureauofLaborStatistics(BLS)ProducerPriceIndex(PPI)forallmanufacturingindustries;--theBLSEmploymentCostIndexforUniversityLaborcosts;and--asampleofactualandprojectlaborratesfromUS-CMSinstitutes.
CauseorTrigger: ImpactedActivities: Startdate: 1/Jan/2018 Enddate: RiskMitigations: Monitoractualescalationannuallyandupdateprojectplanaccordingly.RiskResponses: Ifthecostimpactsareacceptabletheriskresponseistousecontingencytocovertheincreasedcosts.Forextremeinflationratechangesthat
cannotbecoveredbycontingency,workwiththeagenciestoaddresstheissue.Ifneedbe,descopeoptions(thathavealreadybeenidentified)mayneedtobeexercisedtoensuretheoverallprojectcostremainswithintheagencyguidance.
Moredetails: CMS-doc-13481
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 82 Updated: 2019-03-06
RU-402-1-03-DPM-FutureFermilaboverheadratesareuncertain(DOE)
RiskRank: 3(High)Scores:Probability:5(VH);Cost:3(H)Schedule:0(N)) RiskStatus: OpenSummary: Fermilabchargesindirectcostsonfundsspentatthelabandalso--subjecttothresholdsandatdifferentrates--onfundsthataretransferredtoUS-
CMSinstitutesthroughsub-awards.Theoverheadratesareadjustedeachyearandmaybecomemoreorlessfavorablecomparedtothecanonicalvalueinthebaselineplan.Note:giventhenumberofUS-CMSinstitutesinvolved,weassumethatincreasesanddecreasesintheirratesroughlycancelonaverage.Inaddition,institutesgenerallydonotchargeanyindirectcostsforcapitalM&Sexpenditures(e.g.largeprocurementsofequipment).
RiskType: Uncertainty Owner: VivianO'DellWBS: 402.1PM-ProjectManagement(DOE) RiskArea: ManagementRisk/FundingorResourcesProbability(P): 100% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular
Minimum =-2050k$Mostlikely =-100k$Maximum =2200k$Mean =17k$P*<Impact> =17k$
ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =0monthsMaximum =monthsMean =0monthsP*<Impact> =0.0months
BasisofEstimate: SinceFY08,theFermilabLaborandM&Sstandardrateshavevariedbytypically3.6%peryearforMIEfunds.Inthisperiodtheratestrendedsystematicallyupwardsbutwedonotbelievesuchatrendwillcontinueunabated.Infuturetheremightevenbeareversiontoalowerlong-termmean.Infutureyears,wethereforeassumethattheindirectratescouldincrease,decreaseorremainthesamewithapproximatelyequalprobabilities.TomodeltheimpactsofthisweconstructatoyMC.TheProjectspendsabout$57.8MatFermilab(directcosts+indirectcosts+escalation+estimateuncertainty).Wetaketheprofileofthesecostsandassessthecostimpactiftheindirectratesineachfutureyeararevariedrandomlybybetween±3.6%comparedtothepreviousyear.Thiscoversarangeofscenariosinwhichratescantrendupwards,trenddownwards,orfluctuaterandomlyfromyeartoyear.Themin/likely/maxcostimpactsaredeterminedfromthistoyMC.See:CMS-doc-13501andCMS-doc-13481.
CauseorTrigger: ImpactedActivities: Startdate: 1/Oct/2016 Enddate: 30/Sep/2025RiskMitigations: RiskResponses: Updateprojectplanstoaccountforindirectratechangesatleastannuallyandcarefullymonitorchangesintheremainingcontingency.
Moredetails: CMS-doc-13501(seealsoCMS-doc-13481)
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 83 Updated: 2019-03-06
RU-402-2-01-DOT-UncertainperformanceofHybridsvendor
RiskRank: 3(High)Scores:Probability:5(VH);Cost:2(M)Schedule:3(H)) RiskStatus: OpenSummary: Thereisuncertaintyintheperformanceofthevendorofhybridsformoduleswhichcouldhavearangeofimpactsfromnegligibletosignificantcost
impactanddelay.
RiskType: Uncertainty Owner: LeonardGSpiegelWBS: 402.2OT-OuterTracker RiskArea: ExternalRisk/VendorsProbability(P): 100% TechnicalImpact: 1(L)-somewhatsubstandardCostImpact: PDF =3-point-triangular
Minimum =0k$Mostlikely =168k$Maximum =648k$Mean =272k$P*<Impact> =272k$
ScheduleImpact: PDF =3-point-triangularMinimum =0monthsMostlikely =2monthsMaximum =12monthsMean =4.67monthsP*<Impact> =4.7months
BasisofEstimate: Minimumimpact:noextracostnordelayLikelyimpact:issueswiththevendorcausingadelayatdeliveryof2months.Useamoduleassemblylaborburnrateof$54k/monthwithoutescallation.Maximumimpact:weneedtofindanalternativevendororsignificantreworkisneededattheselectedvendor.Thisimpliesacostimpactof$648k(12*54k/month).
CauseorTrigger: ImpactedActivities: Moduleassembly.Thisriskshouldbeimplementedinserieswiththemoduleriskonreceivingthemechanicalparts.
Startdate: 1/Apr/2020 Enddate: 31/Dec/2024RiskMitigations: TheprototypingexperienceshouldinformthehybridfabricationtimeestimatesRiskResponses: IncreasedresourcesforlaborandinfrastructuretoparallelizedownstreamactivitiesinModuleAssemblyandPlank/LayerAssemblytorecoup
delaysfromHybridsMoredetails: CMS-doc-13481
Risk Register for the HL-LHC CMS Detector Upgrades Project
Lucas Taylor Page 84 Updated: 2019-03-06
RU-402-6-07-DTD-DAQSTMSI/Operformancedoesnotmeetrequirements(DOE)
RiskRank: 2(Medium)Scores:Probability:2(L);Cost:2(M)Schedule:0(N)) RiskStatus: OpenSummary: TheI/OrequirementsoftheDAQSTMScouldchangeinresponsetochangesineventsizeorHLToutputeventrate.Thisevent,inturn,
requiresmore(orless)STMSresourcesinresponse.RiskType: Uncertainty Owner: JeffreyWBerryhillWBS: 402.6TD-TriggerandDAQ(DOE) RiskArea: TechnicalRisk/RequirementsProbability(P): 20% TechnicalImpact: 2(M)-significantlysubstandardCostImpact: PDF =3-point-triangular
Minimum =-192k$Mostlikely =384k$Maximum =768k$Mean =320k$P*<Impact> =64k$
ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =0monthsMaximum =monthsMean =0monthsP*<Impact> =0.0months
BasisofEstimate: PurchasecostofDAQSTMS(768k$)scaleddown/upby-25%/50%/100%CauseorTrigger: SpecifyingHLToutputeventrateoreventsizedifferentfromTDRvalues. ImpactedActivities: ProcurementcostofDAQSTMSStartdate: 1/Oct/2018 Enddate: 3/Jun/2025RiskMitigations: OngoingevaluationofDAQconfigurationinresponsetoHLTdesignandT0computing.RiskResponses: NegotiationwithCMSDAQonre-configurationofprojecttomeetstartupgoals.Moredetails: