risk register - fermilab

89
HL-LHC CMS Detector Upgrades Project CMS-doc-13480 Version 3.0 Lucas Taylor, 06-Mar-2019 1 Risk Register For DOE Scope This document is a snapshot of the Web-based risk register that is used to help identify, assess, and manage risks. The register includes threats (with negative impacts); opportunities (with positive impacts); and project-wide uncertainties (with either negative or positive impacts, such as exchange rate uncertainties). These risks are described herein. The general procedures and techniques used to identify and assess risks, perform quantitative analysis, develop mitigation and response plans, and control and report on risks are described here: [1] Fermilab Risk Management Procedure for Projects”, PPP-doc-65. http://ppp-docdb.fnal.gov/cgi-bin/ShowDocument?docid=65. The modest amount of project-specific tailoring of this process is described here: [2] HL-LHC CMS Upgrades “Risk Management Plan”, CMS-doc-13749. https://cms-docdb.cern.ch/cgi-bin/DocDB/ShowDocument?docid=13749 The Monte Carlo methodologies that we use to: analyze the consequences of risks; aggregate cost and schedule impacts, including cost estimate uncertainties; and establish the cost and schedule contingency needs of the project are documented elsewhere: [3] HL-LHC CMS Upgrades “Risk and Contingency Analysis (DOE Scope)”, CMS-doc-13481. https://cms-docdb.cern.ch/cgi-bin/DocDB/ShowDocument?docid=13481 The risks herein are listed in order of Risk ID, which has the syntax Rx-402-n-iii. Rx = RT (for threats), RO (for opportunities), and RU (for uncertainties); 402 denotes the P6 ID of the project (the risk register is common across Fermilab); n denotes the L2 WBS ID: o 0=Project Management (NSF scope – not shown here); o 1=Project Management (DOE scope); o 2=Outer Tracker (DOE scope); o 3=Barrel Calorimeter (NSF scope – not shown here); o 4=Endcap Calorimeter (DOE scope); o 5=Endcap Muons (NSF scope – not shown here); o 6=Trigger and DAQ (DOE scope); o 7=Forward Pixels (NSF scope – not shown here); o 8=Timing Layer (DOE scope); and o 9=Trigger and DAQ (NSF scope – not shown here). iii = 001, 002, etc. is a numerical risk identifier.

Upload: others

Post on 22-Dec-2021

10 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Risk Register - Fermilab

HL-LHC CMS Detector Upgrades Project CMS-doc-13480 Version 3.0

Lucas Taylor, 06-Mar-2019 1

Risk Register For DOE Scope

ThisdocumentisasnapshotoftheWeb-basedriskregisterthatisusedtohelpidentify,assess,andmanage risks. The register includes threats (with negative impacts); opportunities (with positiveimpacts);andproject-wideuncertainties(witheithernegativeorpositiveimpacts,suchasexchangerateuncertainties).Theserisksaredescribedherein.

The general procedures and techniques used to identify and assess risks, perform quantitativeanalysis,developmitigationandresponseplans,andcontrolandreportonrisksaredescribedhere:

[1]“FermilabRiskManagementProcedureforProjects”,PPP-doc-65.

http://ppp-docdb.fnal.gov/cgi-bin/ShowDocument?docid=65.

Themodestamountofproject-specifictailoringofthisprocessisdescribedhere:

[2]HL-LHCCMSUpgrades“RiskManagementPlan”,CMS-doc-13749.https://cms-docdb.cern.ch/cgi-bin/DocDB/ShowDocument?docid=13749

TheMonteCarlomethodologies thatweuse to:analyze theconsequencesof risks;aggregatecostand schedule impacts, including cost estimate uncertainties; and establish the cost and schedulecontingencyneedsoftheprojectaredocumentedelsewhere:

[3]HL-LHCCMSUpgrades“RiskandContingencyAnalysis(DOEScope)”,CMS-doc-13481.https://cms-docdb.cern.ch/cgi-bin/DocDB/ShowDocument?docid=13481

TheriskshereinarelistedinorderofRiskID,whichhasthesyntaxRx-402-n-iii.• Rx=RT(forthreats),RO(foropportunities),andRU(foruncertainties);• 402denotestheP6IDoftheproject(theriskregisteriscommonacrossFermilab);• ndenotestheL2WBSID:

o 0=ProjectManagement (NSFscope–notshownhere);o 1=ProjectManagement (DOEscope);o 2=OuterTracker (DOEscope);o 3=BarrelCalorimeter (NSFscope–notshownhere);o 4=EndcapCalorimeter (DOEscope);o 5=EndcapMuons (NSFscope–notshownhere);o 6=TriggerandDAQ (DOEscope);o 7=ForwardPixels (NSFscope–notshownhere);o 8=TimingLayer (DOEscope);ando 9=TriggerandDAQ (NSFscope–notshownhere).

iii=001,002,etc.isanumericalriskidentifier.

Page 2: Risk Register - Fermilab

HL-LHC CMS Detector Upgrades Project CMS-doc-13480 Version 3.0

Lucas Taylor, 06-Mar-2019 2

Document Change Log

Version Date Changedby ChangesV-0.1 2018-01-18 LucasTaylor FirsttemplateversioncreatedbasedonNSFPDRversion.V-1.0 2018-03-15 LucasTaylor VersioncreatedforDOECD-1Director’sReview.V-2.0 2018-05-08 LucasTaylor VersioncreatedforDOECD-1Review,includingnewsummarytableofrisks.

V-2.1 2018-05-15 LucasTaylor Updatecontributedlaborrisk:newTimingLayerdataandcostedlaborrate.

V-3.0 2019-03-06 LucasTaylor VersioncreatedforDOECD-1(v2)Director’sReview,March2019.

Admin note: how to update this document:

(1) ConnecttotheWindowsTerminalServer(youneedWindowsExplorerfortheSharePointexport).(2) Openthewebbasedriskregister1.(3) Openview=ADMIN--Exceldumpforproject=XXX.(4) Checkthefiltersareappropriate(e.g.project,fundingsource,riskstatus,etc.)(5) Exportregister toExcel.TheExcel filehashiddencharactersand“dataconnections” thatcorrupt the

restoftheprocess.SoopentheExcelfile,selectallthetextanddo“ClearFormats”.Thencopy-pastethecellsofinterestBYVALUEintoafreshExceldocumentandsaveit.

(6) InthefreshExceldocument,searchfordouble-quotesintheExcelfile.ReplacethemwithsinglequotesinthewebRiskRegisterbecausetheycorruptthemailmerge.ThenredotheexporttoExcel.

(7) Adjust the Excel formatting (e.g. set date format, set percentage format and precision, set P*Impactprecisions).Wherepossiblefixtherootcauseintheriskregisterandredotheexport.

(8) DeletespuriouscolumnsintherightoftheExceltable(theextraonesthatSharePointexportsforyouwithoutyourasking).Theysometimeshangthemailmerge.

(9) Update theMSWordmail merge template (e.g.2019-03-06---CMS-HL-LHC---Risk-Register---mail-merge-template.docx,storedinCMS-doc-13480)withthenewExceldata.

(10) CheckWordformattingandtidyupifnecessary(preferablyfixtherootcauseintheriskregisterandredotheexport).Forsomereasonrandomtextfieldscangetcorrupted(eventhoughtheylookfineinSharePointandExcel).Ifyoucannotfixtherootcause,copypastetheuncorruptedtextbyhand.

(11) InWordriskregister:usespecialmenuofReplacemenutochangeallSectionbreakstoPagebreaks.(12) Addheaderandfooterwithpagenumbers,ifdesired.(13) Pastethefinalresultsfromthemailmergeintothisdocument(e.g.2019-03-06---CMS-HL-LHC---Risk-

Register---cover-page.docx, stored in CMS-doc-13480), check it formats OK, and generate a PDF.AlternativelymaketwoPDFsandmergethemwithe.g.AdobeAcrobatPro.

(14) UploadeverythingtoDocDB:CMS-doc-13480.

1https://fermipoint.fnal.gov/organization/ocoo/ippm/SitePages/ProjectRiskManagement.aspx

Page 3: Risk Register - Fermilab

HL-LHC CMS Detector Upgrades Project CMS-doc-13480 Version 3.0

Lucas Taylor, 06-Mar-2019 3

Summary table of risks The table below shows the risks that are included in the MC model, including their estimatedprobabilities, and cost and schedule impacts. A risk identifier (RI-ID)with a prefix RT denotes athreat, RO denotes an opportunity, and RU denotes a risk uncertainty. The cost and scheduleimpactshaveeitherone,twoor(usually)threevaluesdependingonwhetheritisa1-pointestimate(singlevalue),a2-pointestimate(i.e.aflatrangep.d.f.),ora3-pointestimate(i.e.atrianglep.d.f.).

Page 4: Risk Register - Fermilab

HL-LHC CMS Detector Upgrades Project CMS-doc-13480 Version 3.0

Lucas Taylor, 06-Mar-2019 4

Page 5: Risk Register - Fermilab

HL-LHC CMS Detector Upgrades Project CMS-doc-13480 Version 3.0

Lucas Taylor, 06-Mar-2019 5

TheremainderofthisdocumentlistsallrisksorderedfirstbyWBS,thenbyriskID.

Page 6: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 1 Updated: 2019-03-06

RO-402-2-03-DOT-Moduleassemblycanbeautomated

RiskRank: 3(High)Scores:Probability:5(VH);Cost:2(M)Schedule:1(L)) RiskStatus: OpenSummary: Ifautomationinmoduleassemblycomestofruition,thenlaborcostsandscheduledurationsbothdecreaseRiskType: Opportunity Owner: LeonardGSpiegelWBS: 402.2OT-OuterTracker RiskArea: ExternalRisk/FacilitiesProbability(P): 66% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =1-point-singlevalue

Minimum =k$Mostlikely =-1000k$Maximum =k$Mean =-1000k$P*<Impact> =-660k$

ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =-2monthsMaximum =monthsMean =-2monthsP*<Impact> =-1.3months

BasisofEstimate: Areductionof$1.6Minfullyburdenedlaborcoststhroughouttheproductionperiodisdeterminedassumingthat2techniciansperassemblysitearereplacedwith2students(uncostedgraduatestudentsatFNALandmoderatelycostedundergraduatesatBrown=$30/hr).Thecostoftwogantryrobots($150keach)andengineering+programmingdevelopmentcosts($300k)leadstoa$1.0Mopportunity.Theimpactonthescheduleisexpectedtobe2monthsofsavingsbasedonthecurrentestimateofthelaborrequiredtocarryourthestepsbeingconsideredforautomation.

CauseorTrigger: ImpactedActivities: AllPSand2SModuleassemblyactivitiesinaggregate.Implementedasariskhookbetweenstartandcompletionofproductionof2Smodules.

Startdate: 1/Oct/2019 Enddate: 1/Oct/2022RiskMitigations: IntheR&Dphasewewillpro-activelyexploreautomatingcertainstepsinthemoduleassemblyprocesswiththeaimofrealisingtheassociatedcost

andschedulesavings.RiskResponses: Accepttherisk,producemodulesmoreefficientlyMoredetails:

Page 7: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 2 Updated: 2019-03-06

RO-402-2-08-DOT-CanusecheaperCarbonFoamforrings

RiskRank: 2(Medium)Scores:Probability:4(H);Cost:1(L)Schedule:0(N)) RiskStatus: OpenSummary: IfcheaperCarbonFoamshowsacceptableperformancefortherings,thepotentialdecreaseinthecostofthecarbonfoammaybenefittheproject

budgetRiskType: Opportunity Owner: StefanGruenendahlWBS: 402.2OT-OuterTracker RiskArea: ExternalRisk/MarketProbability(P): 50% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =1-point-singlevalue

Minimum =k$Mostlikely =-60k$Maximum =k$Mean =-60k$P*<Impact> =-30k$

ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =0monthsMaximum =monthsMean =0monthsP*<Impact> =0.0months

BasisofEstimate: Estimatethatthecostcouldbehalved(reducedfrom$120kto$60k).CauseorTrigger: Ringprototypesbuiltwithmechanicalgrade(cheaper)carbonfoammeet

QCcriteria(precision,stiffness)

ImpactedActivities: Carbonfoamprocurements

Startdate: 1/Jan/2020 Enddate: 31/Dec/2024RiskMitigations: Weareevaluatingringprototypesbuiltwithmechanicalgrade(cheaper)carbonfoam.Resultsobtainedsofar(firstpairofprototypes)look

promising.RiskResponses: AcceptopportunityMoredetails:

Page 8: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 3 Updated: 2019-03-06

RO-402-8-01-DETL-UseAltiROC

RiskRank: 2(Medium)Scores:Probability:2(L);Cost:2(M)Schedule:3(H)) RiskStatus: OpenSummary: IftheAltiROCmeetsourspecification,wewillbeabletouseitandwewillnothavetopayforthecostoftheproductionmaskset.

RiskType: Opportunity Owner: TiehuiLiuWBS: 402.8.4ETL-EndcapTimingLayer RiskArea: ManagementRisk/PlanningProbability(P): 10% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =1-point-singlevalue

Minimum =k$Mostlikely =-760k$Maximum =k$Mean =-760k$P*<Impact> =-76k$

ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =-8monthsMaximum =monthsMean =-8monthsP*<Impact> =-0.8months

BasisofEstimate: Theprobabilityisinitiallysetto10%sincetheinitialtestresultsoftheAlitROCv0doesnotmeetourspecification.WewillevaluatetheperformanceoftheAltiROCasnewversionsbecomeavailableandwewillremaininclosecontactwiththeAltiROCdevelopers.Thecostsavingscorrespoondstothecostoftheproductionmaskset.

CauseorTrigger: ImpactedActivities: Startdate: 1/Jan/2022 Enddate: 1/Jan/2023RiskMitigations: RiskResponses: Moredetails:

Page 9: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 4 Updated: 2019-03-06

RT-402-1-05-DPM-Significantfundingdelayduringprojectexecution(DOE)

RiskRank: 3(High)Scores:Probability:3(M);Cost:3(H)Schedule:3(H)) RiskStatus: OpenSummary: ItisassumedthatDOEfundswillnotnecessarilyarrivepromptlyatthestartofeachfiscalyearforvariousadministrativeorotherreasons.In

addition,theProjectmaytemporarilyreceivesomewhatlessthantheexpectedleveloffundingifthereisaContinuingResolutioninplace.TheProjectexpectstobeabletocontinueworkingthroughsuch(non-catastrophic)scenarioswiththehelpofcarry-overfunds,suchasunspentcontingency.Thereis,however,amoresignificantfundingdelayrisk:ifthedecisiontoapprovetheProject’sinitialuseofMIEfunds(i.e.CD-3A)coincideswithaContinuingResolution(CR)–thatprohibitsany'newstarts'–thenallMIE-supportedworkwouldbeheldupforthedurationoftheCR.Thisdelaysmanyactivitiesandhasancostimpactduetoescalationandstandingarmycosts.

RiskType: Threat Owner: VivianO'DellWBS: 402.1PM-ProjectManagement(DOE) RiskArea: ManagementRisk/FundingorResourcesProbability(P): 21% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular

Minimum =0k$Mostlikely =0k$Maximum =3339k$Mean =1113k$P*<Impact> =234k$

ScheduleImpact: PDF =3-point-triangularMinimum =0monthsMostlikely =0monthsMaximum =9monthsMean =3monthsP*<Impact> =0.6months

BasisofEstimate: CongressionalResearchServicedataonCRdurations(https://fas.org/sgp/crs/misc/R42647.pdf)indicatethatinthelasttwodecadestherehavebeenmorethanfiveCRsperyearlastinganaverageof138daysperyear(4.5monthsor38%ofthewholeyear)and,inthreecasesforanentirefiscalyear.WeassumethatwewillseekCD-3Aapprovalatleast3monthsbeforethefundsareactuallyneeded.ThereforethefirstthreemonthsofaCRwouldnothaveanimpact.Wefindthaton21%ofthedaysfromFY1998–FY2016aCRwoulddelayCD-3Aapprovalbybetween0monthsand9months(i.e.21%ofthedaysanalyzedhaveaCRofdurationbetween3monthsand12months).Wemodelthisriskasatrianglep.d.f.withP=21%andmin/likely/max=0/0/9months.Thefullprojectburnrateis$371kpermonth(CMS-doc-13481)comprising$249kforstandingarmycostsand$122kforescalation.Thisestimatedstandingarmycostincludesthepossibilitythatsome,butnotall,personnelcanchargetheirtimetootheroff-projectactivitiesduringthedelay.9monthsat$371k/monthis$3,339k.

CauseorTrigger: Changesinfundingagencypriorities,orinCongressionalappropriationsduetopoliticalorotherreasons.

ImpactedActivities:

Startdate: 1/Jan/2018 Enddate: RiskMitigations: Itisassumedthatmodestfundingdelaysareinevitable.Theprojectplanwillmitigatethepotentialimpactsbyavoidinglargeprocurementsearly

ineachfiscalyearandusingcarry-overfunds(e.g.unspentcontingency).TheProjectwillaimtoseekapprovaltospendMIEfunds(i.e.'newstart'approval)inthesecondhalfofthefiscalyearifthisispossiblegiventhevariouspre-requisites(e.g.R&Dresults,vendorinformation,etc.).ThiswillreducethelikelihoodofcoincidingwithaCR.

RiskResponses: Ifsignificantdelaysduetolimitedavailabilityoffundsstarttolookinevitable,theprojectwillpro-activelyre-prioritiseworktominimisetheconsequencesforcriticalpathactivities.Theprojectwillworkcloselywiththeagenciestooptimizetheflowofevenlimitedfundstoensuretheprojectdoesnotgrindtoahalt.

Moredetails: CMS-doc-13481

Page 10: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 5 Updated: 2019-03-06

RT-402-1-09-DPM-ES&HincidentatUS-CMSinstitute(DOE)

RiskRank: 1(Low)Scores:Probability:1(VL);Cost:2(M)Schedule:1(L)) RiskStatus: OpenSummary: Ifacontractororlaboratory/universityemployeeisinjuredorifmaterialisseverelydamagedinansafetyincident,orifanunplannedenvironmentalreleaseoccurs

duringtheconstructionorinstallationphases,theresultinginvestigationandimplementationofcorrectiveactionscouldshutdowntheprojectandcausescheduledelays,dependingontheseverityoftheincidentanditsimpactoncriticalpathwork.

RiskType: Threat Owner: GregoryRSnowWBS: 402.1PM-ProjectManagement(DOE) RiskArea: TechnicalRisk/ES&HProbability(P): 5% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular

Minimum =4k$Mostlikely =52k$Maximum =471k$Mean =176k$P*<Impact> =9k$

ScheduleImpact: PDF =3-point-triangularMinimum =0.25monthsMostlikely =0.5monthsMaximum =2monthsMean =0.92monthsP*<Impact> =0.0months

BasisofEstimate: ThisriskassessmentisbasedonexperiencewithCMSPhase1Upgrades(only1minorES&Hincidentin5years)andexpertinputfromAmberKenney,DeputyHeadofFermilabESH&Q.Theapparentlyhighvalueoftheprobability(forasafety-relatedrisk)reflectstheinvolvementormorethan30institutesforaperiodofabout5years.Ifasafetyincidentoccurs,theinstitute,hostLab,DOE,NSForotherstakeholdersmayrequireaworkstoppageuntiltheyaresatisfiedtheissuesareresolvedandworkcanresume.Historicalexperienceindicatesthisisusuallyafewdaysorweeksbutinexceptionalcasesitcouldbemonths,particularlyifthereisafatality.Thecostimpactmayincludereworkorrepairs(beyondwhatiscoveredbyinsurance)andburnrates(asdescribedinCMS-doc-13481).Thescheduleimpactincludesdelaysduetoworkstoporderswhiletheincidentisanalysedandcorrectiveactionistaken.Tocapturearangeofpossibleconsequencesweconsiderthreescenarios:Minimpact:OneL3areaisimpactedfor1weekwithoutanydamage.TheaverageL3burnrate($18k/month)resultsinacostimpactofabout$4k(for1week).Likelyimpact:OneL2areaisimpactedfor2weeks.Thereis$20kofcorrectiveactioncost.TheaverageL2burnrate($68k/month)resultsinacostimpactof$32k(for2weeks).Totalcostimpact=$52k.Maximpact:Aseriousincidentresultsin$100kofcorrectiveactioncost.Aworkstoporderhaltsrelatedactivitiesproject-widefor2monthsasfailingsareaddressed.Someactivitiescontinue(e.g.design,firmware,software,etc.).Weassumethemonthlyburnrateinthisscenariois50%ofthetotalprojectburnrateof$371k/month.Thetotalcostimpact=$100k+2months*50%*$371k=$471k.

CauseorTrigger: DespiterigorousES&Hprogramatcollaboratinginstitutes,anaccidentcouldhappenduetounsafepractices,beingdistractedwhileworking,beinglaxaboutattentiontoenvironmentalregulatoryrequirements,orpoorenforcementofcontractES&Hprovisions.

ImpactedActivities:

Startdate: 1/Jan/2018 Enddate: RiskMitigations: ComprehensiveuseofISMprogramacrossentireprojectincludingcompleteHAprocess;goodESHcontractualprovisionsaswellasenforcement;workingupfront

duringconstructioncontractstogainbuy-inofconstructioncontractormanagementtoESHissueprevention.Oversightrolesandresponsibilitiesforcontractorandprojectteamtoincludeensuringadequatetrainingandattentiontositespecificrisks,andcompliancewiththeirESHprogram.ThehostLabsandagenciesprovidereview/oversightonregularbasis.

RiskResponses: Moredetails: CMS-doc-13481

Page 11: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 6 Updated: 2019-03-06

RT-402-1-12-DPM-Majorimportorexportissue(DOE)

RiskRank: 2(Medium)Scores:Probability:2(L);Cost:1(L)Schedule:2(M)) RiskStatus: OpenSummary: Thereisamajorunforeseenimport/exportissue.Forexample,ruleschangeinsuchawayastodelayorincreasethecostofimportingorexporting

certainitems(e.g.radhardcomponents).Ifinsufficientinteragencyandinternationalregulatorycoordinationhasoccurred,thenitemsmovedbetweencountriesmaybedelayedorsubjecttoadditionalcosts.

RiskType: Threat Owner: VivianO'DellWBS: 402.1PM-ProjectManagement(DOE) RiskArea: ExternalRisk/RegulatoryProbability(P): 20% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular

Minimum =18k$Mostlikely =56k$Maximum =172k$Mean =82k$P*<Impact> =16k$

ScheduleImpact: PDF =3-point-triangularMinimum =1monthsMostlikely =2monthsMaximum =4monthsMean =2.33monthsP*<Impact> =0.5months

BasisofEstimate: Themainimpactisadelaywhileissuesareresolved.Unexpectedcustomsdelaysofseveralmonthshavehappenedinthepast.Allowforalongtail(butunlikely)toevenhighervaluesif,forexample,thereisanissuethatalsorequirestheinvolvementofthehostLabs,fundingagencyortheStateDepartment.Directcostimpactcouldbesay10%ofextradutyongoodsworthupto$1Moruptoabout$100k.Directlaborcostsarealreadycoveredon-project(e.g.management)orbyindirectcosts(e.g.Labprocurementofficers).Theburnratecosts(standingarmyandescalation)aredescribedinCMS-doc-13481.WeassumethedelayimpactsoneL3systemwithatypicalburnrateof$18kpermonth.Minscenario:1monthdelay.Nodirectcostimpact.$18kofburnrate.Likelyscenario:2monthsdelay.$20kofdirectcosts.$36kofburnrate.Maxscenario:4monthsdelay.$100kofdirectcosts.$72kofburnrate.

CauseorTrigger: ImpactedActivities: Startdate: 1/Jan/2018 Enddate: RiskMitigations: Pro-activelyworkwithhostlabandfundingagencyprocurement,shipping,andimport/exportexpertstoidentifypotentialissues.Prepareplans

andworkinadvancetoensurethatallrulesandregulationsarecompliedwith.Monitorshippingactivitiesclosely,talktogovermentagencies,putassumptionsaboutwhopaysdutiesintocontracts.Considerholdinglogisticsreviews.Talktootherprojectstobetterunderstandprobabilities&impactsandmitigationstrategies.

RiskResponses: Workwiththehostlabandfundingagenciesprocurement,shipping,andimport/exportexpertstoaddresstheissues.Moredetails: CMS-doc-13481

Page 12: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 7 Updated: 2019-03-06

RT-402-1-90-DPM-KeyProjectOfficepersonnelneedtobereplaced(DOE)

RiskRank: 1(Low)Scores:Probability:3(M);Cost:1(L)Schedule:0(N)) RiskStatus: OpenSummary: Keyengineerorseniortechnicianwithspecialknowledgeleavestheprojectandneedstobereplaced.Ifthetransitioncanbemanagedsuchthat

theincomingandoutgoingpersonneloverlapandexchangeknowledge,thenthereismainlyalaborcostimpact.Ifthetransitionismoreabrupt,thenthereisnocostimpactofoverlappingpersonnelbuttherecanbeadelaytotheprojectactivitiesastheincomingpersongetsfullyuptospeed.Thisriskdoesnotincludetheriskoflosingkeymanagersbecausetheprojectensuresthateachmanagerhasawell-traineddeputy.

RiskType: Threat Owner: VivianO'DellWBS: 402.1PM-ProjectManagement(DOE) RiskArea: ManagementRisk/FundingorResourcesProbability(P): 25% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular

Minimum =30k$Mostlikely =60k$Maximum =90k$Mean =60k$P*<Impact> =15k$

ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =0monthsMaximum =monthsMean =0monthsP*<Impact> =0.0months

BasisofEstimate: Thehas2staffwhosefunctionsareparticularlyhardtoreplaceonashorttimescale,inparticulartheAssociatePMforCost,ScheduleandRisk,andtheProjectControlsLead.ExperienceintheCMSPhase1Upgradeprojectsuggeststheprobabilityperkeypersonisroughly25%(probabilitytheyleaveduringtheentireproject).Thelaborcostperperson(acanonicalrate)is$15k/month(fullyburdenedrate=$185k/year).Minimumscenerio:personleavingisreplacedbyanexistingskilledperson.1monthoffulloverlapcosting$15k.Noscheduledelay.Likelyscenario:personleavingisreplacedbyanexistingpersonlessfamiliarwiththespecificwork.2monthsoverlaptotrainnewpersoncosting$30k.Maximumscenario:personleavesunexpectedlywithnotransferofknowledge.Significanttimehireanewpersonplus3monthsoflearningtimetobecomefullyproductiveatacostof3*$15k=$45kextralaborcost.Totalfor2persons:Min/Likely/Maxcostimpact=2*$(15/30/45)k=$(30/60/90)k.Ineachscenario,itisassumedthatothersintheteam(projectmanagersandprojectcontrols)cancovercriticaltaskssuchthatthereisnonetscheduleimpact.Inefficienciesinthistransitionarecoveredbythecostimpactsabove.

CauseorTrigger: ImpactedActivities: IndividualpersonsareindependentlymodeledintheMC(eachwithcorrespondingfractionofthetotalcostimpact).Typicallyhooktherisktothepersonsmainactivityduringtheproductionphase(maximumconsequences).

Startdate: 1/Oct/2018 Enddate: RiskMitigations: KeyprojectofficestaffarematrixedatFermilabandcanbereplaced(withsomeinitiallossofefficiency)byotherLabstaff,contractors,ornew

hires.Intheveryshortterm,theprojectmanagerscanstepintosustainkeyfunctionsoftheprojectoffice.RiskResponses: Aimforoutgoingandincomingpersonneltooverlapifpossibletoensureasmoothtransition.Hirefromskilledmembersoftheexisitngteamif

appropriate.InterimsupportmaybepossibleusingstaffintheFermilabOfficeofProjectSupportServices.Moredetails:

Page 13: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 8 Updated: 2019-03-06

RT-402-2-01-DOT-Sensorqualityproblemduringproduction

RiskRank: 3(High)Scores:Probability:4(H);Cost:1(L)Schedule:2(M)) RiskStatus: OpenSummary: Ifthesensorvendordeliverssensorsthatdonotmeetspecificationsthenthedegradedperformanceofthetrackerjeopardizesthephysics

performanceoftheupgradeddetector.RiskType: Threat Owner: UlrichHeintzWBS: 402.2OT-OuterTracker RiskArea: ExternalRisk/VendorsProbability(P): 50% TechnicalImpact: 2(M)-significantlysubstandardCostImpact: PDF =3-point-triangular

Minimum =46k$Mostlikely =79k$Maximum =163k$Mean =96k$P*<Impact> =48k$

ScheduleImpact: PDF =3-point-triangularMinimum =2monthsMostlikely =3monthsMaximum =6monthsMean =3.67monthsP*<Impact> =1.8months

BasisofEstimate: ThecontractwillbewrittenforthevendortodeliveraspecifiednumberofgoodsensorsthatsatisfyCMSspecifications.Thuswedonothavetopayforsensorsthatdonotsatisfythespecificationsandthereisnoimpactonsensorcost.TheonlycostimpactisthatwewillhavetorepeattheQCtestingofthereplacementsensors.Minimalimpact:thishappensduringproductionandiscorrectedquicklyafterfeedbackfromsensorQCleadingtoadelayofabout2monthsandnegligibledirectcost.Maximalscheduleimpact:thishappensduringpreproductionandthepreproductioncyclehastoberepeated,leadingtoadelayofabout6monthsandextralaborcostofabout$25k(costforpreproductioncycleofonesensortype).TheL3burnrateduetothedelayofdownstreamactivitiesis$23k/month(CMS-doc-13481).Mincost=$0k+2months*$23kburnrate=$46k.Likelycost=$10k+3month*$23kburnrate=$79k.Maxcost=$25k+6months*$23kburnrate=$163k.TheproblemhastoeitherpersistovermanybatchesornotbenoticedduringQCatthevendor(forexampleadegradationofperformanceoversometime).Problemsthataffectasinglebatchofsensors(egbecauseofsomecontaminationorprocessingmistake)willnotleadtoasignificantdelaybecausereprocessingabatchwillonlyaddaweekortwototheproductionperiod.Basedonpastexperiencewiththevendorweexpectthistohappenatleastonceduringproductionandweassign50%probabilityforeachsensortype.

CauseorTrigger: Sensorsdonotsatisfyspecifications ImpactedActivities: Sensorprocurementactivitiesanddownstreamactivities.Thisappliestoeachtypeofsensor,buttheprobabilityshouldbe5%pertype(PS-s,PS-p,2S)

Startdate: 1/Apr/2020 Enddate: 31/Dec/2024RiskMitigations: Wecarryoutextensiveprototypingworkwiththevendorspriortoplacingthecontractforsensorproductiontomakesurethatvensorsunderstand

ourspecificationsandcanmeetthem.ThevendorwillcarryoutafirstsetofQCmeasurementsbeforethesensorsareshippedtoCERNanddistributedtoQCcenters.Thisensuresthatmostproblemswillbecaughtquicklyanddonotleadtosignificantimpactontheproject.Thecostofthesemeasurementsisfacturedintothesensorcost.

RiskResponses: Ifamodestproblemoccurs,workcloselywithvendortosolveit(e.g.testing).Replacetheflawedsensors.Moredetails: CMS-doc-13481

Page 14: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 9 Updated: 2019-03-06

RT-402-2-06-DOT-TemporarylossofSensorQCSite

RiskRank: 2(Medium)Scores:Probability:2(L);Cost:1(L)Schedule:2(M)) RiskStatus: OpenSummary: IfaSensorQCfacilitytemporarilybecomesinoperableduetolossordamageofcriticalequipment(e.g.duetoawaterleak)thentheresultantdip

insensorthroughputmayjeopardizetimelycompletionoftheproject.RiskType: Threat Owner: UlrichHeintzWBS: 402.2OT-OuterTracker RiskArea: TechnicalRisk/ES&HProbability(P): 20% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular

Minimum =22k$Mostlikely =48k$Maximum =86k$Mean =52k$P*<Impact> =10k$

ScheduleImpact: PDF =3-point-triangularMinimum =1monthsMostlikely =2monthsMaximum =4monthsMean =2.33monthsP*<Impact> =0.5months

BasisofEstimate: Probability=20%isapproximatelyestimatedfrom2sites*10%persite.ThisisbasedonexperiencefromoriginalCMStracker,originalpixel,andPhase1pixelwhereoneincidentoccurredinO(10)sites.Ifonecenterhasamajorequipmentfailurethesecondcentercanpickuptheadditionalloadwithinthe100%cushion.Min/likely/maxdelay=1/2/4monthsdelayfortheinefficiencyinthelogisticstotransfermaterialsandpeoplebackandforth.Min/likely/maxrepairestimateis10/25/40k$.Thisassumesinsurancewillcoverloss/damageofmajorequipment.TheL3burnrateduetothedelayofdownstreamactivitiesis$23k/month(CMS-doc-13481).Mincost=$10k+1month*$23kburnrate=$33k.Likelycost=$25k+2months*$23kburnrate=$71k.Maxcost=$40k+4months*$23kburnrate=$132k.

CauseorTrigger: ImpactedActivities: ThisisimplementedastwoindependentriskeventsforthetwoQCsites(BrownandRochester).Ateachsite,3tasksareimpactedinacorrelatedway,representingtheQCworkonthe3sensortypes.TheimpactismodeledinthemiddleoftheQCwork(Lot5).

Startdate: 1/Apr/2020 Enddate: 31/Dec/2024RiskMitigations: Havingtwositesisalreadyahedgeagainstthecompletestoppageofsensortesting,andshouldonesitebecometemporarilyinoperable,sensorsw

ouldberedirectedtotheothersitetemporarilytomitigatetheimpact.RiskResponses: Sensorscanbedivertedtotheunaffectedsitetoutilizeitsfullthroughput,andadditionalresourcesaddedtoincreasemoduleproduction

throughputatbothsites(oncetheaffectedoneisre-established)toregaintimeintheschedule.Moredetails: CMS-doc-13481

Page 15: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 10 Updated: 2019-03-06

RT-402-2-09-DOT-MaPSAyieldislowerthanexpected

RiskRank: 2(Medium)Scores:Probability:2(L);Cost:2(M)Schedule:0(N)) RiskStatus: OpenSummary: IfMaPSAyieldislowerthanexpected,theadditionalwastagealsosacrificestheassociatedsensorsandMPAchips,whichwouldneedtobereplaced

attheproject'scost.RiskType: Threat Owner: RonLiptonWBS: 402.2OT-OuterTracker RiskArea: TechnicalRisk/QualityProbability(P): 15% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =2-point-flatrange

Minimum =370k$Mostlikely =k$Maximum =640k$Mean =505k$P*<Impact> =76k$

ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =0monthsMaximum =monthsMean =0monthsP*<Impact> =0.0months

BasisofEstimate: Foreachlossof10%inyield,wewouldneed10%moresensors,estimatedat245k,and10%moreMPAchips,estimatedat125k.Therangecoverswastagebetween10and20%.ImplementedbetweenTestingbatch3andVendorproducingbatch5.

CauseorTrigger: Amyriadnumberofproblemsatthebumpbondingstagemightreducetheyield,orhandlingduringtheassembly.

ImpactedActivities: IncreasedwastageduringtheMaPSAassemblywouldrequireadditionalcomponents,namelyPSsensorsandMPAchips

Startdate: 2/Sep/2021 Enddate: 20/Sep/2023RiskMitigations: RiskResponses: Moredetails:

Page 16: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 11 Updated: 2019-03-06

RT-402-2-10-DOT-VendorcannotperformMaPSAqualificationtests

RiskRank: 2(Medium)Scores:Probability:3(M);Cost:2(M)Schedule:0(N)) RiskStatus: OpenSummary: MaPSAqualificationisdoneatthevendorsite.Thecurrentcostestimatemayincreaseconsiderablyifthevendorsdonothavetheproper

infrastructuretoqualifytheparts.RiskType: Threat Owner: RonLiptonWBS: 402.2OT-OuterTracker RiskArea: TechnicalRisk/ComplexityProbability(P): 33% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular

Minimum =200k$Mostlikely =400k$Maximum =600k$Mean =400k$P*<Impact> =132k$

ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =0monthsMaximum =monthsMean =0monthsP*<Impact> =0.0months

BasisofEstimate: QualificationofMaPSAsmayrequiresophisticatedprobingequipment,whichcancostupbetween200-600kforprocurement,installation,andcomissioningoftherequisiteequipment,potentiallyatseveralvendors.

CauseorTrigger: ImpactedActivities: MaPSAprocurementcostswouldincrease.Implementedasacostincreaseafterround2ofMaPSAprototyping.

Startdate: 1/Jan/2019 Enddate: 14/Jun/2023RiskMitigations: RiskResponses: Workwithvendortoimprovetheirinfrastructureormovetestingtodifferentsite(othervendororcollaborator)Moredetails:

Page 17: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 12 Updated: 2019-03-06

RT-402-2-11-DOT-MaPSAbumpbondingcostincreases

RiskRank: 2(Medium)Scores:Probability:2(L);Cost:3(H)Schedule:0(N)) RiskStatus: OpenSummary: CurrentlywehaveseveralMaPSAestimates,withaverybroadrangebetweenhighandlow,indicatingtheindustrydoesnotgiveaclearindication

oftheactualcost.Thisriskistocoverthepossibilitythatthishighcostitemexceedsthenominalestimateuncertainty,M5atthemoment.RiskType: Threat Owner: RonLiptonWBS: 402.2OT-OuterTracker RiskArea: ExternalRisk/VendorsProbability(P): 20% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular

Minimum =500k$Mostlikely =1000k$Maximum =1500k$Mean =1000k$P*<Impact> =200k$

ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =0monthsMaximum =monthsMean =0monthsP*<Impact> =0.0months

BasisofEstimate: CurrentlywehaveseveralMaPSAestimates,withaverybroadrangebetweenhighandlow,indicatingtheindustrydoesnotgiveaclearindicationoftheactualcost.Thisriskistocoverthepossibilitythatthishighcostitemexceedsthenominalestimateuncertainty,M5atthemoment.

CauseorTrigger: ImpactedActivities: ThecostsofMaPSAbumpbondingwouldincrease,increasingthecostsofPSmodulefabrication.

Startdate: 1/Jan/2019 Enddate: 14/Jun/2023RiskMitigations: Prototypeswillbeusedtovalidatelowbidders,forwhichthereisnotyetconfidenceofdeliveringwithrequisitequality.Thereismoreconfidence

forhighcostbidders,whichwillalsobevalidatedintheprototypingphase,buteventherethequotesarestillpreliminary.RiskResponses: Moredetails:

Page 18: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 13 Updated: 2019-03-06

RT-402-2-14-DOT-Systemtesthardwarehasinsufficientcapacity

RiskRank: 2(Medium)Scores:Probability:2(L);Cost:2(M)Schedule:2(M)) RiskStatus: OpenSummary: Ifunforeseenproblemsoccurduringassemblyandtesting,thenthebaselinetestingsystemsmaynotbesufficienttomaintaintherequired

throughput.Thiswouldnecessitatetheprocurementandcommissioningofadditionaltestsystemsandadditionallabourfortesting.

RiskType: Threat Owner: AnadiCanepaWBS: 402.2OT-OuterTracker RiskArea: TechnicalRisk/ReliabilityorPerformanceProbability(P): 10% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular

Minimum =71k$Mostlikely =169k$Maximum =292k$Mean =177k$P*<Impact> =18k$

ScheduleImpact: PDF =3-point-triangularMinimum =2monthsMostlikely =3monthsMaximum =4monthsMean =3monthsP*<Impact> =0.3months

BasisofEstimate: Themaximumcostimpactcorrespondstoascenarioinwhichthecapacityofalltheproductiontestsystems(totalcostabout$200k)needstobedoubled,i.e.acostimpactof$200k.Theminimumandlikelycostsreflecttheneedtoduplicatepartsofthetestsystems.Themin/likely/maxscheduleimpactof2/3/4monthsisestimatedassumingthattheproblembecomesapparentduringproduction.TheL3burnrateduetothedelayofdownstreamactivitiesis$23k/month(CMS-doc-13481).Minimpact=$25k+2months*$23k/month=$71k.Likelyimpact=$100k+3months*$23k/month=$169k.Maximpact=$200k+4months*$23k=$292k.

CauseorTrigger: ImpactedActivities: Moremoduletestingequipmentwouldberequired,possiblymorecoldboxes,singlemoduletesting,orhybridtestingequipment.ImplementedasacostimpactonFNALPSmoduleproduction,A,EastCoastPSModuleproduction(B),FNAL2SModuleProduction(C),andEastCoast2Smoduleproduction(D).Sitesshouldbedelayedthesameamount,buttheprobabilityshouldbesplitevenlybetweenPS(A,C)and2S(B,D)

Startdate: 1/Jan/2022 Enddate: 31/Dec/2024RiskMitigations: Thetestinghardwareisanexternaldeliverable.WewillmonitortheprogressoftheUSCMSmoduleproductionandensurethatnewtesting

equipmentispurchasedwhennecessaryanddeliveredwhentheproductionrateisincreasedtomeettheschedule.Labourisincreasedaccordinlytosupportthehigherproductionrate.

RiskResponses: Moredetails: CMS-doc-13481

Page 19: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 14 Updated: 2019-03-06

RT-402-2-23-DOT-Vendorisunabletoproducesensorstospecifications

RiskRank: 2(Medium)Scores:Probability:1(VL);Cost:3(H)Schedule:3(H)) RiskStatus: OpenSummary: IfvendorisunabletoproducesensorsthatmeetCMSSpecificationthentheadditonalcostanddelayofidentifyinganewvendorjeopardizesthetimelyandon-

budgetcompletionoftheprojectRiskType: Threat Owner: UlrichHeintzWBS: 402.2OT-OuterTracker RiskArea: ExternalRisk/VendorsProbability(P): 5% TechnicalImpact: 3(H)-extremelysubstandardorKPPinjeopardyCostImpact: PDF =3-point-triangular

Minimum =210k$Mostlikely =315k$Maximum =2720k$Mean =1082k$P*<Impact> =54k$

ScheduleImpact: PDF =3-point-triangularMinimum =6monthsMostlikely =9monthsMaximum =12monthsMean =9monthsP*<Impact> =0.5months

BasisofEstimate: Iftheselectedvendorisunabletoproducesensorstospecificationsanewvendorhastobedeveloped.Ataminimumthiswillrequireanotherpreproductionrun(6monthdelay).Atamaximumonetotwoprototyperunsmayalsoberequired(12monthsdelay).TheburnratefortheentireOuterTrackeris$70k/month(CMS-doc-13481).WeassumehalfoftheOTscopeisimpactedbyadelayfromthisrisk,therebyincurringaburnrateof$35k/month.Minimpact=nodirectcostincrease.Burnrate=6*$35k=$210k.Likelyimpact:costincreaseiscoveredbythe30%sensorestimateuncertainty.Burnrate=9*$35k=$315k.Maximpact:theworstcasescenariobasedoninformalcostinformationreceivedduringthemarketsurveyisanincreaseinthecostofthesensorsby2/3=66%.30%arecoveredbythecostuncertainty.Theadditionalcostof36%ofthe$6.5Msensorpurchaseis$2.3M.Burnrate=12*$35k=$420k.Total=$2,720k.Wehaveidentifiedavendor(HPK)whohasalreadyproducedsensorsofalltypesthatsatisfyourspecifications.TogetherwiththehistoricallyreliableperformanceofHPKitisveryunlikelythatthisthreatwilloccur.WearenotawarethatHPKhaseverfailedtoproducesensorstospecificationsafterapurchasewasnegotiated.Hencetheprobabilityisconsideredtobelow.

CauseorTrigger: Sensorsdeliveredbyvendoraresubstandardandvendorisunabletofixtheproblem.

ImpactedActivities: SensorproductionandQC.Costriskisimplementedasasinglerisk.Scheduleriskisimplementedasthreeseperaterisks(probabilitydependsonsensortype).Therearethreeriskhooksforthethreesensortypes,butbecause2SandPS-saresimilar,wouldsplittheprobability:1%forPS-s(hookA),2%for2S(hookC),2%forPS-p(hookB).Note:PRAdoesnotsupportfractionsofpercent.

Startdate: 1/Apr/2020 Enddate: 3/Dec/2024RiskMitigations: CERNiscarryingoutamarketsurveytoidentifypossiblevendors.CompaniesareselectedbasedontheircapabilitytoproducesensorsthatsatisfyCMS

specificationsandtoproduceallthesensorsneededbyCMSandATLASwithinatwo-yearperiod.CompanieshavetobequalifiedbyproducingprototypesensorstoCMSspecifications.Thisminimizestheprobabilitythattheselectedcompanycannotdelivertheorder.

RiskResponses: Anewvendorhastobeidentifiedandproductionrestarted.Moredetails: CMS-doc-13481

Page 20: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 15 Updated: 2019-03-06

RT-402-2-24-DOT-Problemwithmodulemechanicalpartsvendor

RiskRank: 2(Medium)Scores:Probability:2(L);Cost:2(M)Schedule:2(M)) RiskStatus: OpenSummary: Majorproblemswiththevendorofmechanical(bridges,spacers,etc)partsformodulesRiskType: Threat Owner: LeonardGSpiegelWBS: 402.2OT-OuterTracker RiskArea: Probability(P): 20% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular

Minimum =0k$Mostlikely =0k$Maximum =324k$Mean =108k$P*<Impact> =22k$

ScheduleImpact: PDF =3-point-triangularMinimum =0monthsMostlikely =0monthsMaximum =6monthsMean =2monthsP*<Impact> =0.4months

BasisofEstimate: Minimumimpact:noextracostnordelayMaximumimpact:significantdelayscausingastandingarmycostof$324k(54k/monthmoduleassemblylaborburnratefor6months).Theburnrateisassumedtobe1/3timesthetotalmoduleassemblylaborcostof5Mduringtheproductiondividedbytheproductionintervalof31months.The1/3assumesthatacertainfractionofthelaborforceiseithermatrixedouttootherareasorcanadvancesomeoftheotheractivitieswhilewaitingforparts.

CauseorTrigger: ImpactedActivities: Implementedasadelaybetweenbatch3and4ofAL-CFspacerproductionfor2S(A)andPS(B),probabilityshouldbesplitbetweenthetwo.

Startdate: 1/Apr/2020 Enddate: 31/Dec/2024RiskMitigations: Vendorqualificationwillprovidesomeexperiencewithreliability,andthecontractwillincludescheduleexpectations.RiskResponses: IncreasedresourcesforlaborandinfrastructuretoparallelizedownstreamactivitiesinModuleAssemblyandPlank/LayerAssemblytorecoup

delaysMoredetails: CMS-doc-13481

Page 21: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 16 Updated: 2019-03-06

RT-402-2-25-DOT-Moduleassemblyyieldislow

RiskRank: 2(Medium)Scores:Probability:2(L);Cost:1(L)Schedule:2(M)) RiskStatus: OpenSummary: IftheyieldfromModuleAssemblyislowerthanexpected,thentheadditionalresourcesneededtocompensatejeopardizeprojectscheduleand

budgetRiskType: Threat Owner: LeonardGSpiegelWBS: 402.2OT-OuterTracker RiskArea: Probability(P): 10% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular

Minimum =0k$Mostlikely =40k$Maximum =240k$Mean =93k$P*<Impact> =9k$

ScheduleImpact: PDF =3-point-triangularMinimum =0monthsMostlikely =0monthsMaximum =6monthsMean =2monthsP*<Impact> =0.2months

BasisofEstimate: Minimum:nocostimpact,nodelay.Likely:nodelay.40k$isthecostofadditionallabourandadditionalcomponents.Maximum:240k$isthecostofadditionallabourandadditionalcomponents.6monthdelay:timetodiagnosetheproblemandimprovethemoduleassemblyprocedure.

CauseorTrigger: ImpactedActivities: ImplementedasaRiskHookbetweencompletionof2S/PSModulesBatch3(assumingthatiswhenyouassesstheyield)andMechanicalAssemblyofBatch5(Batch4isinprogress).ShareprobabilityequallybetweenPS(hookA)and2S(hookB).

Startdate: 1/Apr/2020 Enddate: 31/Dec/2024RiskMitigations: ExtensiveprototypingshouldmitigatetheriskofoverestimatedyieldsinModuleassembly.RiskResponses: AdditionalresourcesbothtocompensatefortheloweryieldandfurtherparallelizetheassemblyproceduretoregainschedulewouldbeneededMoredetails: CMS-doc-13481

Page 22: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 17 Updated: 2019-03-06

RT-402-2-33-DOT-Morepreproductionmodulesneeded

RiskRank: 2(Medium)Scores:Probability:3(M);Cost:2(M)Schedule:2(M)) RiskStatus: OpenSummary: Ifmorepre-productionmodulesareneededtoqualifytheproductioncomponentsandassembly,thentheremaybecostincreasesanddelays.RiskType: Threat Owner: MeenakshiNarainWBS: 402.2OT-OuterTracker RiskArea: Probability(P): 25% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular

Minimum =0k$Mostlikely =0k$Maximum =330k$Mean =110k$P*<Impact> =28k$

ScheduleImpact: PDF =3-point-triangularMinimum =0monthsMostlikely =0monthsMaximum =6monthsMean =2monthsP*<Impact> =0.5months

BasisofEstimate: Additionallabour:22k/monthandadditionalmechanicalcomponents10k/month.Total=6months*($22k+$10k)=$192k.TheL3burnrateduetothedelayofdownstreamactivitiesis$23k/month(CMS-doc-13481).Totalmaximpact=$192k+6*$23k=$330k.

CauseorTrigger: ImpactedActivities: Modulepreproductionactivitieswouldstretchanadditional3months.Implementedasahooksbetween2S(hookB)orPS(hookA)PreproductionModuleAssemblyactivitiescompleteand2S/PSReadyforproduction.Shareprobabilityequally.

Startdate: 1/Apr/2020 Enddate: 31/Dec/2020RiskMitigations: Preproductionactivitiesareadequatelyestimatedassumingnomajorproblemsarise,afterextensiveprototypingRiskResponses: Moredetails: CMS-doc-13481

Page 23: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 18 Updated: 2019-03-06

RT-402-2-35-DOT-Temporarylossofmoduleassemblyfacility

RiskRank: 2(Medium)Scores:Probability:4(H);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: IfaModuleAssemblyfacilitytemporarilybecomesinoperableduetolossofcriticalequipmentorActofGod,thentheresultantdipinmodule

throughputmayjeopardizetimelycompletionoftheprojectRiskType: Threat Owner: MeenakshiNarainWBS: 402.2OT-OuterTracker RiskArea: Probability(P): 50% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =1-point-singlevalue

Minimum =k$Mostlikely =50k$Maximum =k$Mean =50k$P*<Impact> =25k$

ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =1monthsMaximum =monthsMean =1monthsP*<Impact> =0.5months

BasisofEstimate: Itisassumedthatin1monththeequipmentwouldbereplacedandassemblycouldthenproceedattheaffectedsite.Riskimpactisestimatedfrom1/2ofthemodulelaborburnrate($84k/month)plusanestimateof$8ktomakeemergencyrepairs.

CauseorTrigger: ImpactedActivities: Allassemblyactivitiesattheaffectedsite.ImjplementedasahookbetweenPS/2Smechanicalassemblyofsubsequentbatches(notethoseshouldbecorrelated-ifyoucannotbuildPS,youalsocannotbuild2S)atFermilab(B,D)andBrown(A,C)

Startdate: 1/Apr/2020 Enddate: 31/Dec/2024RiskMitigations: Havingtwositesisalreadyahedgeagainstthecompletestoppageofmoduleproduction,andshouldonesitebecometemporarilyinoperable,

componentscouldberedirectedtotheothersitetemporarilytomitigatetheimpactRiskResponses: Modulecomponentscanbedivertedtotheunaffectedsitetoutilizeitsfullthroughput,andadditionalresourcesaddedtoincreasemodule

productionthroughputatbothsites(oncetheaffectedoneisre-established)toregaintimeinthescheduleMoredetails: CMS-doc-13481

Page 24: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 19 Updated: 2019-03-06

RT-402-2-43-DOT-Problemwithcarbonfibervendor

RiskRank: 2(Medium)Scores:Probability:3(M);Cost:1(L)Schedule:2(M)) RiskStatus: OpenSummary: IfthereisaproblemwiththeCarbonFibervendorthentherecouldbedelaysorcostincreases.Examplesofproblemsinclude:vendorgoingoutof

business,vendordeliveringsubstandardCF,orthevendorcannotdeliveraccordingtotheagreedschedule.

RiskType: Threat Owner: StefanGruenendahlWBS: 402.2OT-OuterTracker RiskArea: ExternalRisk/VendorsProbability(P): 25% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular

Minimum =23k$Mostlikely =79k$Maximum =158k$Mean =87k$P*<Impact> =22k$

ScheduleImpact: PDF =3-point-triangularMinimum =1monthsMostlikely =3monthsMaximum =6monthsMean =3.33monthsP*<Impact> =0.8months

BasisofEstimate: Minimumimpact:theproblemisrelativelymodestandissolvedbythevendorwithin1monthswithoutanydirectcostimpacttotheproject.Likelyimpacts:theproblemismoresignificantandrequiresadditionalprototypeswiththevendorresultinginadelayof3monthsandacostof$10kforCFandmachining.Maximumimpact:ifweneedtoswitchvendorwewouldneedtoqualifyascondvendorresultingina6monthdelay.Thecostincreasecouldbe$20kforprototypingandsetupcostswiththenewvendor.TheL3burnrateduetothedelayofdownstreamactivitiesis$23k/month(CMS-doc-13481).Mincost=$0k+1month*$23kburnrate=$23k.Likelycost=$10k+3month*$23kburnrate=$79k.Maxcost=$20k+6months*$23kburnrate=$158k.

CauseorTrigger: ImpactedActivities: Carbonfiberbasedstructuresinbothmodulemechanicsandflatbarrelmechanics.Implementedasadelayinprocurementforproductionplanks(hookA)andspacers(hookB).

Startdate: 1/Jan/2019 Enddate: 31/Dec/2024RiskMitigations: Prototypingexperienceinthenexttwoyearsshouldresultinareliablecostestimateofthenecessarycarbonfiberforbothmoduleandflatbarrel

structures.RiskResponses: Moredetails: CMS-doc-13481

Page 25: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 20 Updated: 2019-03-06

RT-402-2-46-DOT-Problemwithcarbonfoamvendor

RiskRank: 3(High)Scores:Probability:3(M);Cost:2(M)Schedule:3(H)) RiskStatus: OpenSummary: IfthereisaproblemwiththeCarbonFoamvendorthentherecouldbedelaysorcostincreases.Examplesofproblemsinclude:vendorgoingoutof

business,vendordeliveringsubstandardCarbonFoam,orthevendorcannotdeliveraccordingtotheagreedschedule.RiskType: Threat Owner: StefanGruenendahlWBS: 402.2OT-OuterTracker RiskArea: ExternalRisk/VendorsProbability(P): 25% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular

Minimum =23k$Mostlikely =158k$Maximum =396k$Mean =192k$P*<Impact> =48k$

ScheduleImpact: PDF =3-point-triangularMinimum =1monthsMostlikely =6monthsMaximum =12monthsMean =6.33monthsP*<Impact> =1.6months

BasisofEstimate: TheL3burnrateduetothedelayofdownstreamactivitiesis$23k/month(CMS-doc-13481).Minimumimpact:theproblemisrelativelymodestandissolvedbythevendorwithin1monthswithoutanydirectcostimpacttotheproject.Costimpact=1month*$23k/month(burnrate).Likelyimpacts:theproblemismoresignificantandrequiresadditionalprototypeswiththevendorresultinginadelayof6monthsandacostof$20kforCarbonFoamandtechnicalwork.Totallikelyimpact=$20k+6*$23k=$158k.Maximumimpact:ifweneedtoswitchvendorwewouldneedtoqualifyasecondvendorresultingina12monthdelay.Thecostincreasecouldbe$120kforincreasedvendorcosts,prototypingandsetupcostswiththenewvendor.Totallikelyimpact=$120k+12*$23k=$396k.

CauseorTrigger: ImpactedActivities: FabricationofthecarbonfoamstructuresinMechanics-implementedasadelayinthecarbonfoamprocurementforproductionplanks.

Startdate: 1/Jan/2019 Enddate: 31/Dec/2024RiskMitigations: RiskResponses: JoinwiththerestoftheLHCcommunityinseekingoutanewvendorofCarbonFoamMoredetails: CMS-doc-13481

Page 26: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 21 Updated: 2019-03-06

RT-402-2-54-DOT-Mechanicsmaterialsdegradedbyradiation

RiskRank: 2(Medium)Scores:Probability:2(L);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: Ifthemechanicalmaterialsaresusceptibletointegritydegradationduetoradiationexposure,theresultingmaterialmodificationsanddesign

changesjeopardizetimelycompletionoftheflatbarrel.RiskType: Threat Owner: StefanGruenendahlWBS: 402.2OT-OuterTracker RiskArea: TechnicalRisk/QualityProbability(P): 10% TechnicalImpact: 2(M)-significantlysubstandardCostImpact: PDF =3-point-triangular

Minimum =48k$Mostlikely =96k$Maximum =144k$Mean =96k$P*<Impact> =10k$

ScheduleImpact: PDF =3-point-triangularMinimum =1monthsMostlikely =2monthsMaximum =3monthsMean =2monthsP*<Impact> =0.2months

BasisofEstimate: Totalcostofthematerialsinvolvedisabout$300k(MandS)and$220kinlabortomaketheproductionplanks(x80)andrings(x6).Itisassumedthatanyproblemsarefoundearlyinthemechanicsproductionsuchthatthelikelycostimpactisabout10oftheMandSandlabor:min/likely/max=25/50/75k$andadelayof1/2/3months.TheL3burnrateduetothedelayofdownstreamactivitiesis$23k/month(CMS-doc-13481).Mincost=$25k+1months*$23kburnrate=$48k.Likelycost=$50k+2month*$23kburnrate=$96k.Maxcost=$75k+3months*$23kburnrate=$144k.

CauseorTrigger: ImpactedActivities: Plankandringfabricationwouldbedelayed.Startdate: 1/Jan/2019 Enddate: 1/Apr/2021RiskMitigations: AllmaterialsandassemblieswillberadiationtestedintheprototypingphaseRiskResponses: Moredetails: CMS-doc-13481

Page 27: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 22 Updated: 2019-03-06

RT-402-2-57-DOT-Majorfailureoflayerassemblyinfrastructure

RiskRank: 1(Low)Scores:Probability:1(VL);Cost:1(L)Schedule:2(M)) RiskStatus: OpenSummary: Ifthereisalongertermnon-availabilityofFNALCO2system,survey/alignmentequipment,autoclave/oven,etc,thenthesubsequentdelayuntila

alternativeisinoperationjeopardizestimelycompletionoftheflatbarrelRiskType: Threat Owner: StefanGruenendahlWBS: 402.2OT-OuterTracker RiskArea: ExternalRisk/FacilitiesProbability(P): 5% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular

Minimum =56k$Mostlikely =112k$Maximum =178k$Mean =115k$P*<Impact> =6k$

ScheduleImpact: PDF =3-point-triangularMinimum =2monthsMostlikely =4monthsMaximum =6monthsMean =4monthsP*<Impact> =0.2months

BasisofEstimate: Estimateweneedmin/likely/max=2/4/6monthstocompleterepairsorprocurereplacementatacostofmin/likely/max=10/20/30k$.TheL3burnrateduetothedelayofdownstreamactivitiesis$23k/month(CMS-doc-13481).Mincost=$10k+2months*$23kburnrate=$56k.Likelycost=$20k+4month*$23kburnrate=$112k.Maxcost=$40k+6months*$23kburnrate=$178k.

CauseorTrigger: majorcomponentfailure(e.g.pump,accumulator) ImpactedActivities: Implementedasthreeriskhooks,oneeachforInner(hookA),Middle(hookB),andOuter(hookc)Layerassembly,delaybetweenstartmilestoneandactualassembly.Suggestsplittingtheprobabilityequally.

Startdate: 13/Jun/2017 Enddate: 31/Dec/2025RiskMitigations: monitorsystemperformance&performregularmaintenance

RiskResponses: repairsystem;buyreplacement(mobile)systemMoredetails: CMS-doc-13481

Page 28: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 23 Updated: 2019-03-06

RT-402-2-58-DOT-DamagetoFlatBarrelPlanks

RiskRank: 2(Medium)Scores:Probability:1(VL);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: Ifanaccidentdamagesaflatbarrelcomponent(e.g.aplankoranendring)thenrepairworkwouldcauseadelayandcostincrease.Thedamage

couldbepurelyphysical(e.g.thecomponentsarecrushed),chemical(e.g.watercontamination),orotherhazard.RiskType: Threat Owner: StefanGruenendahlWBS: 402.2OT-OuterTracker RiskArea: ManagementRisk/LogisticsProbability(P): 5% TechnicalImpact: 3(H)-extremelysubstandardorKPPin

jeopardyCostImpact: PDF =3-point-triangular

Minimum =30k$Mostlikely =91k$Maximum =141k$Mean =87k$P*<Impact> =4k$

ScheduleImpact: PDF =3-point-triangularMinimum =1monthsMostlikely =1monthsMaximum =2monthsMean =1.33monthsP*<Impact> =0.1months

BasisofEstimate: MinimumImpact:thephysicaldamageisnotpermanentandonlyreworkisneededtoeffectarepair.Theestimateddelayis1monthatacostof2FTE-monthsofengineeringorabout$30k.Minimpacttotal=$30k.Likelyimpact:OneentireLayer2plank(plusassociatedmodules)needstobereplacedincludingabout11modulesmaximum(11*$5k=$55k).Tothisisaddedthemechanicscostandlabor(about10hoursperplank)foratotalofabout$6k.Thedelayis1month(assumingthatmodulesareavailable).Likelyimpacttotal=$30k+$55k+$6k=$91k.Maximumimpact:thedamageiscatastrophicforaLayer3plank,includingabout15modules(15*$5k=$75k).Tothisisaddedthemechanicscostandlabor(about10hoursperplank)foratotalofabout$6k.Thedelayis2month(assumingthatmodulesareavailable).Likelyimpacttotal=$60k+$75k+$6k=$141k.Theriskoccurslateinthescheduleandhencedoesnothaveasignificantburnratefromescalation.Thestandingarmycosts(theyarenotidle!)areincludedinthelaborcostsabove.

CauseorTrigger: Damagetoaplankduringplankassemblyorduringlayerassemblyortransport.

ImpactedActivities: ImplementedasadelaybetweenQCtestingofInner,Middle,andOuterLayer(whicheverislast)andmilestoneofcompletionofFlatBarrel

Startdate: 1/Jan/2021 Enddate: 31/Dec/2024RiskMitigations: Plankhandlingproceduresaredesignedtoreducedamageduetohandling.Transportenclosuresaredesginedtominimizerisktodetector.RiskResponses: Thedamagedplankwouldneedtobereplaced,increasingbothtimeandlabordependingontheextentofthedamageMoredetails: CMS-doc-13481

Page 29: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 24 Updated: 2019-03-06

RT-402-2-59-DOT-DamagetoFlatBarrelLayer

RiskRank: 2(Medium)Scores:Probability:1(VL);Cost:3(H)Schedule:3(H)) RiskStatus: OpenSummary: Ifanaccidentdestroysamajorflatbarrelcomponent(i.e.awholelayer)thenrepairworkwouldcauseadelayandcostincrease.Thedamagecould

bepurelyphysical(e.g.thecomponentsarecrushed),chemical(e.g.watercontamination),orotherhazard.RiskType: Threat Owner: StefanGruenendahlWBS: 402.2OT-OuterTracker RiskArea: Probability(P): 1% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular

Minimum =930k$Mostlikely =1880k$Maximum =3150k$Mean =1987k$P*<Impact> =20k$

ScheduleImpact: PDF =3-point-triangularMinimum =6monthsMostlikely =9monthsMaximum =12monthsMean =9monthsP*<Impact> =0.1months

BasisofEstimate: Costtorebuildanentirelayerusingplanksat$6klaborperplankandmodulesat$5k/module,foreitherLayer1(minimalrisk),Layer2(likelyrisk)orLayer3(maximumrisk).ThecosttoreplacethemodulesforLayer3(assumingthetotalnumberofsparesneededbyCMSremainsthesame)=540*$5k=$2.7M(includingM&Sandmoduleassemblylabor).Thecostforthemechanicsofalayerisabout$150k(1/3ofthetotal).Laborcostestimate(plankconstruction+modulemounting)rangesfrom200k$(layer1)to400k$(layer3)or$300konaverage.Maximumdirectcostimpact=$3.15M$.Layers1and2scaledaccordingly.

CauseorTrigger: catastrophicdamagetoamajorityofcomponentsofaFlatBarrellayer.

ImpactedActivities: ImplementedasadelaybetweenQCtestingofInner,Middle,andOuterLayer(whicheverislast)andmilestoneofcompletionofFlatBarrel

Startdate: 1/Jan/2021 Enddate: 31/Dec/2024RiskMitigations: DesignandenforcementofFlatBarrelhandlingandtransportprocedures,anddesignofFlatBarrellayertranportenclosures.

RiskResponses: rebuildplanksusingsparemodules,andreassemblelayer.

Moredetails:

Page 30: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 25 Updated: 2019-03-06

RT-402-2-60-DOT-Problemswithwirebonding

RiskRank: 2(Medium)Scores:Probability:5(VH);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: TemporarylossofabondingmachineatFNAL,Princeton,orRutgers.Whiletherepaircostsarecoveredtherewouldbeadditionalexpensesinthe

hold-upofmoduleassemblyuntilthemachineisrepairedortheworkloadcouldberebalance.RiskType: Threat Owner: LeonardGSpiegelWBS: 402.2OT-OuterTracker RiskArea: TechnicalRisk/ReliabilityorPerformanceProbability(P): 80% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =2-point-flatrange

Minimum =13.5k$Mostlikely =k$Maximum =27k$Mean =20k$P*<Impact> =16k$

ScheduleImpact: PDF =2-point-flatrangeMinimum =1monthsMostlikely =monthsMaximum =2monthsMean =1.5monthsP*<Impact> =1.2months

BasisofEstimate: Probabilityisassumedtobe20%permachine(fourmachines:twoatFNAL,oneatPrinceton,oneatRutgers)soroughly80%probabilityintotal(thenumberthatisusedinriskranking).IntheMCtheprobabilityistreatedas4independenteventseachwithP=20%.BasedonaModuleAssemblymonthlylaborburnrateof$54kandtheassumptionthatasinglewirebonderfailsoverthecourseoftheproductionperiod,thusaffecting1/4ofthemoduleassembly(burnrate=$54k/4=$13.5kpermonth).Thisismodeledasadelayof1-2monthsinwirebondingateachof3sitesindependently,bothforPSand2Smodules(correlated).

CauseorTrigger: Failureofawirebonder,forexamplefromthecrashofabondinghead. ImpactedActivities: WirebondingandallOTmoduleassemblyactivitiesthataredownstreamofbonding.IntheMCtheprobabilityistreatedas4independentriskeventswithP=20%each--oneperbondingmachine.

Startdate: 20/Dec/2021 Enddate: 2/Aug/2024RiskMitigations: Withtimethebondingloadcanberebalancedamongstthe3bondingfacilities.Itmaybepossibletopayforexpeditedservicefromthebonding

machinevendors.RiskResponses: Understandhowquicklyamachinecanbebroughtbackintoservice.Moredetails:

Page 31: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 26 Updated: 2019-03-06

RT-402-2-90-DOT-KeyOuterTrackerpersonnelneedtobereplaced

RiskRank: 2(Medium)Scores:Probability:3(M);Cost:2(M)Schedule:1(L)) RiskStatus: OpenSummary: Keyengineerorseniortechnicianwithspecialknowledgeleavestheprojectandneedstobereplaced.Ifthetransitioncanbemanagedsuchthattheincomingand

outgoingpersonneloverlapandexchangeknowledge,thenthereismainlyalaborcostimpact.Ifthetransitionismoreabrupt,thenthereisnocostimpactofoverlappingpersonnelbuttherecanbeadelaytotheprojectactivitiesastheincomingpersongetsfullyuptospeed.Thisriskdoesnotincludetheriskoflosingkeymanagersbecausetheprojectensuresthateachmanagerhasawell-traineddeputy.

RiskType: Threat Owner: StevenC.NahnWBS: 402.2OT-OuterTracker RiskArea: ManagementRisk/FundingorResourcesProbability(P): 25% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular

Minimum =75k$Mostlikely =225k$Maximum =570k$Mean =290k$P*<Impact> =73k$

ScheduleImpact: PDF =3-point-triangularMinimum =0monthsMostlikely =0monthsMaximum =3monthsMean =1monthsP*<Impact> =0.3months

BasisofEstimate: hisL2areahas5seniortechnicalstaffwhosefunctionsareparticularlyhardtoreplaceonashorttimescalebyexistingteammembers:4seniorengineersand1seniortechnician.ExperienceintheCMSPhase1Upgradeprojectsuggeststheprobabilityperkeypersonisroughly25%(probabilitytheyleaveduringtheentireproject).Someinputnumberstothecostimpactanalysis:--Laborcostperperson=$15k/month(fullyburdenedseniorenginer=$185k/year).--AverageburnrateperL3areais=$23k/month(CMS-doc-13481).Minimumscenerio:personleavingisreplacedbyanexistingskilledpersonintheteam.1monthoffulloverlapcosting$15koflabor.Noscheduledelay.Likelyscenario:personleavingisreplacedbyapersonlessfamiliarwiththespecificwork.3monthsmanagedoverlapoftheoutgoingandincomingpeoplecosts3*$15k=$45koflabor.Noscheduledelay.Maximumscenario:personleavesunexpectedlywithnotransferofknowledge.Ittakesabout6monthstofindanewpersonandgetthemfullyuptospeed.Thereislossofproductivityduringthisdifficulttransitionperiodresultinginanet3monthdelaytoL3activities.Costoframp-upeffortofthenewperson(learningbutnotfullycontributingtodeliverables)=3FTE-months*$15k=$45k.Theburnratecostis3*$23k=$69k.Totalcostimpact=$114kTotalfor5persons:Min/Likely/Maxcostimpact=5*$(15/45/114)k=$(75/225/570)$.

CauseorTrigger: ImpactedActivities: EachkeypersonsisindependentlymodeledintheMC(eachatP=25%andwithcorrespondingshareofthetotalcostimpact).Typicallyhooktherisktothepersonsmainactivityduringtheproductionphase(maximumconsequences).

Startdate: 1/Jan/2018 Enddate: RiskMitigations: Anumberofengineersandtechnicianshaveoverlappingskills,bothwithinagiveninstituteandacrossUS-CMSinstitutes.Thisprovidessomebackupincaseof

non-availabilityofakeyperson.Pro-activecross-trainingofengineersandtechnicianshelpsensurekeyskillsarenotcompletelylostifakeypersonisnolongeravailable.

RiskResponses: Aimforoutgoingandincomingpersonneltooverlapifpossibletoensureasmoothtransition.Hirefromskilledmembersoftheteamifappropriate.Interimsupportmaybepossibleusingengineersortechnicainsfromelsewhereintheinstituteorproject.

Moredetails: CMS-doc-13481

Page 32: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 27 Updated: 2019-03-06

RT-402-2-91-DOT-ShortfallinOuterTrackerscientificlabor

RiskRank: 3(High)Scores:Probability:3(M);Cost:3(H)Schedule:0(N)) RiskStatus: OpenSummary: Ifasignificantamountofthe(uncosted)scientificlaborisunavailable,thentheprojectwouldthenneedtofundadditional(costed)personnelto

performthework.Itisassumedthattheriskistriggeredbyaseriouslyunfavorableoverallbaseprogramfundingsituation.RiskType: Threat Owner: StevenC.NahnWBS: 402.2OT-OuterTracker RiskArea: ManagementRisk/FundingorResourcesProbability(P): 30% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular

Minimum =0k$Mostlikely =0k$Maximum =1049k$Mean =350k$P*<Impact> =105k$

ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =0monthsMaximum =monthsMean =0monthsP*<Impact> =0.0months

BasisofEstimate: InthepastUS-CMShasnotexperiencedasignificantlackofscientificlabor(postdocsandgraduatestudents).WhenshortfallsoccuredtheywereusuallyresolvedbycollaboratorsatUS-CMSinstitutesorsometimesfromiCMS.Weassigna30%probabilitythatasignificantshortfalloccursinfuture,duetounfavorablefundingconditionsinthebaseprogram.ThecontributedlaborforthescopeofthisL2areais60.8FTE-yearsspreadoverabout5years.ThisdoesnotincludethemoresecureL2andL3managerswhoaretenuredfacultyandseniorFermilabscientists.Weestimatethelosscouldbeupto20%ofthetotalcontributedlaboror12.2FTE-years(e.g.thisisalossof50%ofallcontributedlaborforatwoyearperiodoralossof1/3ofthecontributedlaborfor3years).Themissinglaborcouldbereplacedbycostedpersonnel:amixtureofmid-rangetechnicians,juniortechnicians,orundergraduatescostingrespectively62$/hr,50$/hrand18$/hrfully-burdened(43$/houronaverage).Allowingforfouryearsofescalationat3.1%perannumyieldsanaveragecostof49$/hror86k$/FTE-year(1768hrsworkedperyear).The(min/likely/max)costimpactistherefore:86k$perFTE-year*(0/0/12.2)FTE-years=$(0/0/1049)k.

CauseorTrigger: Theriskistriggeredbyanunfavorablebaseprogramfundingsituation. ImpactedActivities: Startdate: 1/Oct/2018 Enddate: RiskMitigations: Workwithinstitutesandagenciestoensuretheanticipatedamountofscientificlaborwillbeavailable.Whereshortfallslooklikelytooccur,seek

alternativesamongstotherUS-CMSinstitutesorevenfromiCMSinstitutes.RiskResponses: Seekreplacementscientificlaborinotherinstitutes.Ifthislaborcannotbefoundthencontingencywillneedtobespenttosupplementtheeffort

withcostedlabor(e.g.technicians).Moredetails: CMS-doc-13509(FTEdataatCD1)

Page 33: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 28 Updated: 2019-03-06

RT-402-4-01-DCE-AdditionalFEASICengineeringrunrequired

RiskRank: 3(High)Scores:Probability:3(M);Cost:2(M)Schedule:3(H)) RiskStatus: OpenSummary: TheFEASICisnotaUSCMSdeliverable.TherequiredFEASICareexpectedtobereceivedfreeofchargefromtheinternationalprojectinexchange

fortheconcentratorASICsprovidedfortheelectromagneticsectionoftheCE(CE-E).However,theUS,alongwiththerestofthecollaboration,wouldshareaportionofthecostofanadditionalengineeringrunfortheFEASICbeyondthecurrentplannedtwoengineeringruns.Ifasignificantflawisobservedafterthesecondengineeringrun,athirdcyclemayberequired.Inthiscase,USASICengineeringresourcesmayberequiredtoconsult,validate,orfinalizesomespecificpartofthedesign.ThelikelyriskisaradiationeffectinthedigitalpartoftheASIC,sincethatpartofthechipwilllikelybecompletedlastandtheirradiationcampaignwillnotbecompleteduntilaftertheengineeringrun.

RiskType: Threat Owner: JeremiahMichaelMansWBS: 402.4CE-CalorimeterEndcap RiskArea: TechnicalRisk/QualityProbability(P): 25% TechnicalImpact: 2(M)-significantlysubstandardCostImpact: PDF =1-point-singlevalue

Minimum =k$Mostlikely =336k$Maximum =k$Mean =336k$P*<Impact> =84k$

ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =8monthsMaximum =monthsMean =8monthsP*<Impact> =2.0months

BasisofEstimate: Cost:AssumefourmonthsofASICengineering($108k)and$60kMandSbasedona27%cost-sharefortheASIC=$168k.The27%fractionisthestandardUSCMScontributiontointernationalCMScommoncosts.Schedule:Aneight-monthdelaywouldbeincurred.TheL3burnrateduetothedelayofdownstreamactivitiesis$21k/month(CMS-doc-13481).Somedownstreamactivitiesmayneedtobeacceleratedifdeemednecessarytocompleteproductiononschedule.Thecostassociatedwithsuchanaccelerationisnotincludedherebutincludedasaseparaterisk.Totalimpact=$168k+8months*$21kburnrate=$336k.

CauseorTrigger: ImpactedActivities: SiliconmodulePCBcompletion,siliconmoduleassembly,scintillatortile-moduleassembly,andcassetteassembly.

Startdate: 1/Jun/2020 Enddate: 1/Mar/2021RiskMitigations:

ManagerswillworkwithengineeringanddevelopmentteamtoprovidefullqualityassurancefortheASICdesignbeforeengineeringrunsoccur.Wehave1FTEofextraengineeringover2yearstoworkwithCERNandFEASICengineerstohelpvalidatetheFEASIC,inadditiontotheworkthatisneededtointerfacethechipintoUSCMSdeliverables.

RiskResponses: Somedownstreamactivitiesmayneedtobeacceleratedifdeemednecessarytocompleteproductiononschedule.Thecostassociatedwithsuchanaccelerationisnotincludedherebutincludedasaseparaterisk.

Moredetails: CMS-doc-13481

Page 34: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 29 Updated: 2019-03-06

RT-402-4-02-DCE-Infrastructurefailureatmoduleassemblyfacility

RiskRank: 2(Medium)Scores:Probability:3(M);Cost:2(M)Schedule:2(M)) RiskStatus: OpenSummary: Asignificantfailurecouldoccurinamoduleassemblyfacilitywhichrendersitunabletooperateforsomeperiodoftime.Exampleswouldinclude

floodingofthefacilityorseveredamagetothegantrysystemusedforthegluingofmodules.RiskType: Threat Owner: ManfredPauliniWBS: 402.4CE-CalorimeterEndcap RiskArea: ExternalRisk/FacilitiesProbability(P): 30% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =2-point-flatrange

Minimum =100k$Mostlikely =k$Maximum =336k$Mean =218k$P*<Impact> =65k$

ScheduleImpact: PDF =2-point-flatrangeMinimum =1monthsMostlikely =monthsMaximum =4monthsMean =2.5monthsP*<Impact> =0.8months

BasisofEstimate: Probability=30%for3sites.Probabilitypersite=10%,basedonexperiencefromoriginalCMStracker,originalpixel,andPhase1pixelwhereoneincidentoccurredinO(10)sites.Cost:Costforsufficientmodulestorecommissionthesystemandapossiblestanding-armycosttomaintaintheteamwhilerepairsarecarriedout.Assume40modulesforrecommissioningatacost(includinglabor)of$40k.Standingarmycostsareestimatedat$58k/month.Costsforrepairoffloodingareassumedtobecoveredbyinsurance,whilecostsforadamagedgantryareestimatedas25%oftotalreplacementcostor$64k.Totalof$100kto$336kdependingondamageanddelay.Schedule:delayof1to4months.

CauseorTrigger: ImpactedActivities: IntheriskMCthisriskneedstobeduplicatedonceforeachcentre

Startdate: 1/Jan/2022 Enddate: 30/Sep/2023RiskMitigations: Mitigatescheduledelaysandstandingarmycostsbyusinguniformproceduresatallmoduleassemblysites.Theplanisconstructedtoallowsome

excesscapacityateachsite.RiskResponses: Inthecaseofaseriousfailure,somestaffandproductionworkcanbetransferredtotheothersites,possiblyaddingaweekenddaytotheshiftcycle.

Costsfortravelwouldbeincurred,assumedtobeatthelevelof$4k/month/technican.Moredetails:

Page 35: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 30 Updated: 2019-03-06

RT-402-4-04-DCE-Concentratordoesnotmeetspecifications

RiskRank: 2(Medium)Scores:Probability:2(L);Cost:3(H)Schedule:3(H)) RiskStatus: OpenSummary: IftheconcentratorASICdoesnotmeetspecificationsaftertheproductionmasksareproducedthenanadditionalsetofmaskswillneedtobe

producedandadditionalengineeringeffortisrequiredtoadjustthedesignsothattheASICmeetspecifications.Theriskprobabilitydependsonhowmanyengineering/multi-projectwaferrunsaredoneandtestedbeforetheproductionmasksareproduced,andalsoontheexactmaturityofthedesign(progressmade),andontheschedule,e.g.ifallirradiationtestscanbedonebeforetheproductionmasksarefabricated.Thefullpackagingcostsarerealizedonlyoncesincewewouldfindtheproblembeforepackaging.ThemaskswillincludebothTRGandDAQchips.

RiskType: Threat Owner: JamesFHirschauerWBS: 402.4CE-CalorimeterEndcap RiskArea: TechnicalRisk/RequirementsProbability(P): 10% TechnicalImpact: 2(M)-significantlysubstandardCostImpact: PDF =3-point-triangular

Minimum =907k$Mostlikely =971k$Maximum =1035k$Mean =971k$P*<Impact> =97k$

ScheduleImpact: PDF =3-point-triangularMinimum =6monthsMostlikely =7.5monthsMaximum =9monthsMean =7.5monthsP*<Impact> =0.8months

BasisofEstimate: TheM&Scostis$690kformasksandwaferand$25kforasinglesetuptopackageafewchips.Thelaborcostfortheadditionalengineeringeffortisa3-pttriangularPDF:0.25-0.375-0.5FTE.Themin/likely/maxcostishence781/813/846k$.TheL3burnrateduetothedelayofdownstreamactivitiesis$21k/month(CMS-doc-13481).Mincost=$781k+6month*$21kburnrate=$907k.Likelycost=$813k+7.5months*$21kburnrate=$971k.Maxcost=$846k+9months*$21kburnrate=$1035k.

CauseorTrigger: ConcentratorASICasreturnedfromproductiondoesnotmeetspecificationdespitethetwopreviousmulti-projectwaferruns(MPWs)

ImpactedActivities:

Startdate: 30/Aug/2021 Enddate: 29/Aug/2022RiskMitigations: Thisriskismitigatedbyprototypingandtestingthefulldesignthroughtwomulti-projectwafersubmissions,andbydesignandproductionreviews.

TheASICsfromthesewillbeusedinfullverticalslicetests,aswellasbeamtestsandirradiationtests.RiskResponses: ToreducetheimpactduetothedelayofthisASICwewillneedtoimplementacombinationofthefollowing:acceleratetheproductionandtestingof

motherboards(MB)oncetheASICisavailable,andalsotheassemblyofcassettes;domoreassemblyandtestinginparallelforpiecesthatdonotrequiretheASIC(fortheMB),orthatdonotrequiretheMB,tominimizethetimetocompleteassemblyoftheMBandcassettes,andtominimizethetimeneededtofortests.

Moredetails: CMS-doc-13481

Page 36: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 31 Updated: 2019-03-06

RT-402-4-09-DCE-ModulePCBbatchfailure

RiskRank: 1(Low)Scores:Probability:1(VL);Cost:1(L)Schedule:2(M)) RiskStatus: OpenSummary: Onebatchworth1000modulePCB'shaslowqualityyieldandisunusable.Thisriskcoversanumberofscenarioswheretheassembledmodule

PCB'sareunusable.ThiscouldincludeproblemsintheperformanceofthepartsorinthemodulePCBsorintheassemblyprocess,orinshippingorhandling.WeassumeallPCBswillberemadeandthepartsontheassembledPCBarelost.

RiskType: Threat Owner: ManfredPauliniWBS: 402.4CE-CalorimeterEndcap RiskArea: ExternalRisk/VendorsProbability(P): 5% TechnicalImpact: 1(L)-somewhatsubstandardCostImpact: PDF =2-point-flatrange

Minimum =144k$Mostlikely =k$Maximum =186k$Mean =165k$P*<Impact> =8k$

ScheduleImpact: PDF =2-point-flatrangeMinimum =2monthsMostlikely =monthsMaximum =4monthsMean =3monthsP*<Impact> =0.2months

BasisofEstimate: Onebatchworth1000modulePCB'shaslowqualityyieldandisunusable.Costimpactis$102kand2-4monthdelay.Costincludestotalcostofpartsandassembly.TheL3burnrateduetothedelayofdownstreamactivitiesis$21k/month(CMS-doc-13481).Mincost=$102k+2months*$21kburnrate=$144k.Maxcost=$102k+4months*$21kburnrate=$186k.

CauseorTrigger: ImpactedActivities: Moduleassemblyatmodulesassemblysites.

Startdate: 1/Oct/2019 Enddate: 2/Oct/2022RiskMitigations: Thedesign,andmodulePCBwillgothroughtwoprototypestagesandverticalslidetests,aswellasbeamtests.Theproductionpartswillgothrough

QC,andthevendorswillbequalifiedthroughtheprototypecycles.Costswillbemitigatedbywritingthepurchasecontractssoweonlypayforgoodparts.

RiskResponses: WewillneedtorequestadditionalfundsfromcontingencytopayforthereplacementassembledPCBsandwemayneedtoacceleratemoduleproduction,and/orcassetteproductionintheseareonthecriticalpath.

Moredetails: CMS-doc-13481

Page 37: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 32 Updated: 2019-03-06

RT-402-4-10-DCE-Siliconsensorhaslowyield

RiskRank: 1(Low)Scores:Probability:1(VL);Cost:2(M)Schedule:2(M)) RiskStatus: OpenSummary: Siliconsensorsshowlessthananticipatedquality,orareunsableinsomeway.Thisriskismeanttocoveranumberofissuesthatcouldaffectthe

usabilityofthesensor,likehandling,QCprocessissues,radiationeffects,post-assemblyeffectsonleakagecurrents,andsingle-vendor-relatedproblems.Weuseaneffective10%yieldforthepurposeofquantifyingthisrisk.

RiskType: Threat Owner: ManfredPauliniWBS: 402.4CE-CalorimeterEndcap RiskArea: ExternalRisk/VendorsProbability(P): 1% TechnicalImpact: 2(M)-significantlysubstandardCostImpact: PDF =2-point-flatrange

Minimum =542k$Mostlikely =k$Maximum =784k$Mean =663k$P*<Impact> =7k$

ScheduleImpact: PDF =2-point-flatrangeMinimum =2monthsMostlikely =monthsMaximum =4monthsMean =3monthsP*<Impact> =0.0months

BasisofEstimate: Usea10%yieldissuetoquantifythisrisk.Costimpactis500-700k$andadelayof2-4months.TheL3burnrateduetothedelayofdownstreamactivitiesis$21k/month(CMS-doc-13481).Mincost=$500k+2months*$21kburnrate=$542k.Maxcost=$700k+4months*$21kburnrate=$784k.

CauseorTrigger: ImpactedActivities: Implementedbetween'deliveryofproductionsensorsbatch3'and'fabricationofproductionsensorsbatch4'

Startdate: 1/Oct/2019 Enddate: 1/Oct/2021RiskMitigations: Thedesignwillbeprototypestartingwithteststructuresandproceedingtofull-sizedsesnorsoffinaldesign.Testsofthesewithincludeirradiation

studiesandtestbeamstudies,aswellasassemblyintoprototypemodulesthatwillundergotwomajorsystemprototypesthatserveasverticalslicetests.Thecontractforthesensorswillhavefullspecificationstoensurewepayonlyforeventualgoodsensors,andwewillqualifyasecondvendorthathasthecapacitytofabricateatleastsomefractionofthesensors.Orderthesensorsasearlyaspossible.

RiskResponses: Requestadditionalfundingfromcontingencytopurchaseadditionalsensors.Activateasecondvendorifneeded.Acceleratedownstreamproductionifneeded.Uselessthanidealsensorsifneededinregionssotheydon'toverlapandthatwouldaffectlesstheperformanceofthecalorimeter.

Moredetails:

Page 38: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 33 Updated: 2019-03-06

RT-402-4-13-DCE-HGCROCfrontendchipisdelayed

RiskRank: 2(Medium)Scores:Probability:2(L);Cost:2(M)Schedule:3(H)) RiskStatus: OpenSummary: IftheHGCROCfrontendchipdeliverydateisdelayedthentheassemblyofmoduleboardsandthereforetheassemblyofthemodulesgetsdelayed.

Thismayjeopardizethedeliveryofcassettesontime.ThisriskcanincludedelaysinanyoftheprototypeHGCROCchipsubmissionsbesidesthefinalproductionrun.

RiskType: Threat Owner: JeremiahMichaelMansWBS: 402.4CE-CalorimeterEndcap RiskArea: ExternalRisk/CollaboratorsProbability(P): 20% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular

Minimum =21k$Mostlikely =126k$Maximum =252k$Mean =133k$P*<Impact> =27k$

ScheduleImpact: PDF =3-point-triangularMinimum =1monthsMostlikely =6monthsMaximum =12monthsMean =6.33monthsP*<Impact> =1.3months

BasisofEstimate: Themin/likely/maxdelaysareestimatedtobe1/6/12months.TheL3burnrateduetothedelayofdownstreamactivitiesis$21k/month(CMS-doc-13481).Noadditioncostshavebeenassumeinthisriskentryforapossibleaccelerationofmoduleandcassetteproduction.TheseareincluidedasaseparateriskeventMincost=1month*$21kburnrate=$21k.Likelycost=6months*$21kburnrate=$126k.Maxcost=12months*$21kburnrate=$252k.

CauseorTrigger: ImpactedActivities: Assemblyofmoduleboards.JM:Implementedbetween:(A)'vendorfabricatesHGROC'and'vendordeliversHGROC'forsilicon,and(B)'prepareandplaceorders'and'deliveryofHGROC-SiPM'forscintillator.LT:assumeburnratecostisequallysharedbetweenthesetwoitems.

Startdate: 1/Dec/2019 Enddate: 1/Dec/2020RiskMitigations: DevelopteststandssothatcomponentsthatdependontheHGCROCcanbetestedasmuchaspossibleusingemulatorsandolder/prototype/non-

productionversionsoftheHGCROCchip.IncludesignificantengineeringsothatwecanhelpverifyandvalidatetheprototypeandproductionHGCROCchipfunctionalityandqualityasquicklyaspossible.Useprototypeandpre-productionHGCROCchipsinthe2majorsystemprototypesforverticalslicetestsandinbeamtests,andirradiationstudies.Ensurethemoduleandcassetteproductionfacilitiesaresetuptobeabletoaccelerateproductionifneeded.

RiskResponses: HelpvalidateortesttheHGCROCchipsasquicklyaspossibletoreducethetestingtime.Acceleratedownstreamproduction(modulesandcassette)asneeded.

Page 39: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 34 Updated: 2019-03-06

RT-402-4-14-DCE-Cassettecoolingplatefabricationfailure

RiskRank: 1(Low)Scores:Probability:2(L);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: Ifcassettecoolingplatefabricationfailsspecificationintermsofflatness,locationoffeatures,orcoolingtubeperformancethentheassemblyofthe

cassettescannotproceedandmayjeopardizethedeliveryofassembledcassettestoCMS.RiskType: Threat Owner: ZoltanGecseWBS: 402.4CE-CalorimeterEndcap RiskArea: ManagementRisk/ExperienceorCapabilityProbability(P): 10% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =2-point-flatrange

Minimum =73k$Mostlikely =k$Maximum =213k$Mean =143k$P*<Impact> =14k$

ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =3monthsMaximum =monthsMean =3monthsP*<Impact> =0.3months

BasisofEstimate: Thecoolingplatemanufacturingprocesscontainsanon-standardoperationofembeddingacoolingtube.Thisisthereasonforanon-negligiblefailureduringproductionevenifthevendorqualificationwassuccessful.The3monthdelaywasestimatedasatimeneededtoshiftproductiontoanalternativevendororin-houseatFNAL.Thecostimpact($10k-$150k)isbasedontheassumptionofafailedbatchofcoolingplates,andusesFNALlaborratesforfabricationasifitisdonein-house.WehavenotyetqualifiedanexternalvendorsoFNALratesareassumedforcostestimes.Theentirecostofabatchisusedbecausethisriskcoversarangeofsourcesoffailuresthatmaynotbecoveredbythevendoror/andinatimelymanner.TheL3burnrateduetothedelayofdownstreamactivitiesis$21k/month(CMS-doc-13481).Mincost=$10k+3months*$21kburnrate=$73k.Maxcost=$150k+3months*$21kburnrate=$213k.Aburnrateisusedtocovertheriskthatwemaynotbeabletoscheduleproductionofcoolingplatesearlyenoughtoavoidbeingonthecriticalpath,orsomeotherdelaypushesthisintothecriticalpath.

CauseorTrigger: ImpactedActivities: CassetteAssembly.Insertedbetweendeliveryofbatch1andfabricationofbatch2

Startdate: 1/Jan/2021 Enddate: 12/Dec/2023RiskMitigations: Thecassetteprototypingprogramalreadyunderwayaddressesissuesofmanufacturabilityofthecoolingplatesandexperiencegainedwillfeed

backtothedesign.Threeroundsofprototypecassettesareplannedpriortothestartofproduction.Qualifyseveralvendorsforcoolingplatefabricationsuchthattheycanpick-upproductionifonevendorfails.RequirearigorousQA/QCprogrambythevendor.Begincoolingplateproductioninadvanceofcassetteassemblytoensureabufferofcomponentssothatinterruptionofthesupplydoesnotaffectcassetteproduction.

RiskResponses: Workwithvendortofixproductionproblemsordistributeproductiontoothervendors,orfabricationin-houseatFNAL.Increasethecassetteassemblyrate,utilizingplannedbackupcapacityoftheassemblysites,toreducethescheduleimpact.Thecostofacceleratedcassetteproductioniscoveredbyaseparaterisk.

Moredetails: CMS-doc-13481

Page 40: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 35 Updated: 2019-03-06

RT-402-4-15-DCE-Motherboardandinterfaceboardfabricationfailure

RiskRank: 1(Low)Scores:Probability:2(L);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: Iflargerthanexpectedfailurerateisidentifiedduringmotherboardandinterfaceboardproductionthenthecassetteassemblywillslowdownand

mayjeopardizethedeliveryofcassettestoCMSontime.RiskType: Threat Owner: ZoltanGecseWBS: 402.4CE-CalorimeterEndcap RiskArea: TechnicalRisk/QualityProbability(P): 10% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =2-point-flatrange

Minimum =73k$Mostlikely =k$Maximum =193k$Mean =133k$P*<Impact> =13k$

ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =3monthsMaximum =monthsMean =3monthsP*<Impact> =0.3months

BasisofEstimate: Thecostimpact(10-130k$)isbasedontheassumptionthatatmaximum10%ofthemotherboardsforthewholedetectorhavetobeproducedduetoMotherboardfailures.Thebasecostincludesanadditional10%sparemotherboardsontopofthatneededforthedetector.Thesesparespartlycovermoreminorfailuresandyieldofindividualcomponents,ormotherboardvariants,andmaynotcoveramoresignificantfailurecoveredbythisriskwheretheproductionmotherboardsaredoneintwobatches.Weassumeafailurescenariowhere10%ofthemotherboardswouldneedtobeproducedandassembledtoensurethatwehavesufficientsparestocoverlostedmotherboardsofanyofthevariantsbytheendoftheproduction.Thedurationimpact(3months)isbasedontheassumptionthatanothervendormightbeneededtobequalifiedtotakeovertheproduction,orsomespecificvariantshavetobemadeagainbeyondwhatwasorderedforspares.TheL3burnrateduetothedelayofdownstreamactivitiesis$21k/month(CMS-doc-13481).Mincost=$10k+3months*$21kburnrate=$73k.Maxcost=$130k+3months*$21kburnrate=$193k.

CauseorTrigger: ImpactedActivities: Startdate: 1/Jan/2021 Enddate: 12/Dec/2023RiskMitigations: Thoroughlyqualifythedesignthroughthecassetteprototypeprogramand,asappropriate,focusedmockups.

Qualifyseveralvendorsforboardfabricationsuchthattheycanpick-upproductionifonevendorfails.RequirearigorousQA/QCprogrambythevendor.Beginmotherboardandinterfaceboardproductioninadvanceofcassetteassemblytoensureabufferofcomponentssothatinterruptionofthesupplydoesnotaffectcassetteproduction.Designcassetteassemblysitestoallowreworkwithoutinterruptingproductionflow.

RiskResponses: Dependingonthescheduleandwhichmotherboardvariantsneedtobereplacedwemayneedtoorderanewroundofmotherboardsandassembly.Ifotrherdelaysorthisdelaycausesthistasktobeonthecritcialpathwewillacceleratethecassetteproductionandtesting.Thepossibilityofneedingtoacceleratethecassetteproductioniscoveredbyaseparaterisk.

Moredetails: CMS-doc-13481

Page 41: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 36 Updated: 2019-03-06

RT-402-4-16-DCE-Cassettesdamagedorlostinassembly,testingorshipping

RiskRank: 1(Low)Scores:Probability:1(VL);Cost:2(M)Schedule:1(L)) RiskStatus: OpenSummary: Ifacassettegetsdamagedduringassemblyorabatchof15cassettesgetdamagedduringcoldtestingorabatchof15cassettesgetlostduring

shipping,thenthelostcassettesneedtobefabricatedandassembledagain,whichmayjeopardizethedeliveryofcassettestoCMSontime.RiskType: Threat Owner: ZoltanGecseWBS: 402.4CE-CalorimeterEndcap RiskArea: ManagementRisk/ExperienceorCapabilityProbability(P): 5% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =2-point-flatrange

Minimum =100k$Mostlikely =k$Maximum =1000k$Mean =550k$P*<Impact> =28k$

ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =3monthsMaximum =monthsMean =3monthsP*<Impact> =0.2months

BasisofEstimate: Thecostestimateisbasedonthecostoflosingupto15cassettes.Wewillbeassemblyingandtestinginbatchesof15,andshippingthemtoCERNinbatchesof15.Wecalculatethecostbysimplyrollingupthecostofproducing375cassettes,andtaking(15/375)ofthiscostforthemaximumcostimpact.Thecostofproducingandtestingallcomponents,includingsiliconmodulesandscintillatortile-modules,andthecostofcassetteassembly,testing,andshipping,areincluded.WedonotincludethecostofHGCROCandECONASICasenoughsparesareexpectedtobepurchasedsowedonotneedanadditionproductionrunofthesechips.Thedelaysisbasedonthetimeneededtoreplacethelostcassettes.

CauseorTrigger: ImpactedActivities: LinkedtoShipcassettes331-360

Startdate: 1/Jan/2021 Enddate: 12/Dec/2023RiskMitigations:

Setinplacecarefullydesignedtoolingandsafehandlingprocedures.Donothandlemanycassettesatthesametime;limitnumberofcassettesinashipmentto15andnomorethanoneofeachtypepershipment.Plannedproductionincludes1sparecassetteofeachtype(forthetestbeamwedge).Ensureadequatequantityofsparepartstoallowrapidassemblyofreplacementcassettes.Ensurethatallshipmentsareadequatelyinsured.Contractstoincludeoptionsforlaterdeliveryofadditionalcomponents.

RiskResponses: Responsedependsontheexactlost,iflosinganentirebatchof15cassettesinshipping,ordamagingsomecassettesinabatch(es)duringhandlingintheassembly,testing,orshipping.Intheworstcaseoflosingall15cassettesinabatch,wewillorderadditionalpartsasneededandmaketheadditional15cassettes.The15testbeamwedgecassettesmaybeusedinthedetectorifthisisneededtgoavoidsignificantdelays.Asneededwewillacceleratethecassetteassemblyandtesting.Thecostofacceleratingthecassetteproductionisincludedinaseparateriskentry.

Moredetails:

Page 42: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 37 Updated: 2019-03-06

RT-402-4-17-DCE-Cassetteassemblysitefailure

RiskRank: 1(Low)Scores:Probability:2(L);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: IfthecleanroomareaofthecassetteassemblysitegetsdamagedoriftheCO2coolingplantfailsthentheassemblyandtestingprocedurewillstop

untiltheproblemsarefixedanditmayjeopardizethedeliveryofcassettestoCMSontime.RiskType: Threat Owner: ZoltanGecseWBS: 402.4CE-CalorimeterEndcap RiskArea: ExternalRisk/FacilitiesProbability(P): 10% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =2-point-flatrange

Minimum =73k$Mostlikely =k$Maximum =163k$Mean =118k$P*<Impact> =12k$

ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =3monthsMaximum =monthsMean =3monthsP*<Impact> =0.3months

BasisofEstimate: Theestimateisbasedontherangeofcostsneededtoreplacethedamagedequipment=10-100k$.The3monthdelayisestimatedbasedonthetimeitmaytaketofixtheproblems.TheL3burnrateduetothedelayofdownstreamactivitiesis$21k/month(CMS-doc-13481).Mincost=$10k+3months*$21kburnrate=$73k.Maxcost=$100k+3months*$21kburnrate=$163k.

CauseorTrigger: ImpactedActivities: JM:Insertedintoassemblybetween150and151

Startdate: 1/Jan/2021 Enddate: 12/Dec/2023RiskMitigations: Tomitigatetheimpactontheschedule,thecapacityoftheassemblyandtestingfacilityisplannedtotwicelargerthanrequiredfornormal

operations.RiskResponses: Moredetails: CMS-doc-13481

Page 43: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 38 Updated: 2019-03-06

RT-402-4-18-DCE-AdditionalconcentratorASICengineering(MPW)runisrequired

RiskRank: 3(High)Scores:Probability:4(H);Cost:2(M)Schedule:3(H)) RiskStatus: OpenSummary: CurrentplanningincludesthecostoftwoMulti-ProjectWafer(MPW)prototyperuns.Ifasignificantflawisobservedafterthesecondrun,athird

cyclemayberequired.Thisriskaddressestheneedanadditionalsinglerun(eitherDAQorTRG).

RiskType: Threat Owner: JamesFHirschauerWBS: 402.4CE-CalorimeterEndcap RiskArea: TechnicalRisk/QualityProbability(P): 50% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular

Minimum =164k$Mostlikely =241k$Maximum =385k$Mean =263k$P*<Impact> =132k$

ScheduleImpact: PDF =3-point-triangularMinimum =6monthsMostlikely =7.5monthsMaximum =9monthsMean =7.5monthsP*<Impact> =3.8months

BasisofEstimate: CurrentplanningincludesthecostoftwoMulti-ProjectWafer(MPW)prototyperuns.Ifasignificantflawisobservedafterthesecondrun,athirdcyclemayberequired.Thisriskaddressestheneedanadditionalsinglerun(eitherDAQorTRG).TheM&Scostsare$136kfortheMPWrunand$28kforpackaging.Thelaborcostforadditionaldesignworktofixtheobservedflawisa3-pttriangularPDF:0/0.17/0.5FTE.Itisassumedtheflawisfairlyeasytofixandtheexacttimeneededdependsonwhenwhentheissueisdiscovered,ifinthebeginning,middle,orendofthecycle.TheadditionaltestingisassumedtobemodestandcoveredbythelaborincludedforASICtestinginthebaseline.Themin/likely/maxcostishence164/209/295k$.TheL3burnrateduetothedelayofdownstreamactivitiesis$21k/month(CMS-doc-13481),butthereare6monthsoffloatavailablebeforecassetteassemblyisaffected.therefore:Mincost=$164k+0=$164k.Likelycost=$209k+1.5months*$21kburnrate=$241k.Maxcost=$295k+3months*$21kburnrate=$385k.

CauseorTrigger: ImpactedActivities: JM:ImplementedinP6between'ECONp2testparts'and'ECONv1finalizedesign'

Startdate: 31/Aug/2020 Enddate: 30/Aug/2021RiskMitigations: Wewillworkcloselywithengineerstomakesurethattwoplannedengineering(MPW)runsaresufficient.Wewillbeusingsomecodeblocksfrom

thelpGBTthatwouldhaveundergonesometestingbeforethetwoMPWruns,andwewillbeincludingtheprototypeECONASICinamajorsystemprototypeverticalslicetestbeforetheproductionrun.

RiskResponses: RequestadditionalfundingforathirdMPWrunandfordesignchanges.Reducetheneededtestingtime.Accelerateorparallelizetheproductionofthemotherboardsifpossibleandifneededacceleratethecassetteassemblyandtesting.Thecostofacceleratingtheassemblyofcassettesiscoveredbyaseparaterisk.

Moredetails: CMS-doc-13481

Page 44: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 39 Updated: 2019-03-06

RT-402-4-20-DCE-BoundarybetweenSiandscintillatorsectionsismoved

RiskRank: 2(Medium)Scores:Probability:2(L);Cost:2(M)Schedule:0(N)) RiskStatus: OpenSummary: Ifthecombinedperformanceofthescintillator+SiPMsystemcannotmeettheperformancerequirementsoftheproject,wemaybeforcedtomove

backtheboundarybetweenthesiliconandscintillatorsections.Weassumeheretheimpactisequivalenttomovingthefirstmixedlayerbackbyonelayer.

RiskType: Threat Owner: JeremiahMichaelMansWBS: 402.4CE-CalorimeterEndcap RiskArea: TechnicalRisk/RequirementsProbability(P): 10% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =1-point-singlevalue

Minimum =k$Mostlikely =252k$Maximum =k$Mean =252k$P*<Impact> =25k$

ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =0monthsMaximum =monthsMean =0monthsP*<Impact> =0.0months

BasisofEstimate: Estimatedasthecostoftwoadditionalall-siliconlayersminusthe(small)costofonelessscintillatorlayer.Weassumethatthedecisionismadeearlyenoughthatitwouldnothaveascheduleimpactsincetheassemblycenterswillhavesufficientcapacityforanyforeseenchange.,andwecanmakemodificationstothecoolingplatesintheshadowofothertasks.

CauseorTrigger: ImpactedActivities: Startdate: 29/Jan/2018 Enddate: 1/May/2020RiskMitigations: Performdetailedsimulationstudiestoensurewehaveanaccurateradiationmap.Doirradiationstudiesoftheperformanceofprototypesiliconand

scintillatordetectormodulestomakesuretheymeetrequirements.Dofulldetectorphysicsstudiestoensurefullunderstandingofthephysicsimpactoftheboundarychange.Setupmoduleassemblycenterstohavesufficientcapacityifextrasiliconmoduleshavetobeconstructed.Wedesignthecoolingplateandcoolingtuberouting,tile-modules,andmodule-to-motherboardconnectionsthatareflexibletohandleavariableboundaryfromonelayertothenext.

RiskResponses: Adjustbudget,andschedulestoproducemore/lesssiliconmodules,andcorrespondingless/morescintillatortile-modiules.Modifytheaffectedcoolingplateswiththecorrectmountingholes,andproducethemodifiednumberofeachtypeofmotherboards.

Moredetails:

Page 45: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 40 Updated: 2019-03-06

RT-402-4-22-DCE-Additionalproductionaccelerationrequired

RiskRank: 2(Medium)Scores:Probability:2(L);Cost:2(M)Schedule:1(L)) RiskStatus: OpenSummary: Ifupstreamprojectdelaysabsorballfloatbeforeseriesproductionbeginsthenadditionalfundswouldberequiredtoaccelerateproductiontoavoid

jeoparizingtheinternationalneed-bydateRiskType: Threat Owner: JeremiahMichaelMansWBS: 402.4CE-CalorimeterEndcap RiskArea: TechnicalRisk/RequirementsProbability(P): 20% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular

Minimum =564k$Mostlikely =564k$Maximum =777k$Mean =635k$P*<Impact> =127k$

ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =1monthsMaximum =monthsMean =1monthsP*<Impact> =0.2months

BasisofEstimate: Minimalcostimpactisbasedonaccelerationofmoduleproductionfrom120modules/weekto160modules/week.Thisrequiresastaffingincreaseof33%duringthisperiodwhichisbalancedbyadecreasedproductionperiod.Duetothelimitationsinhowquicklystaffcanbetrained,however,weassumethatthelaborlevelduringtheramp-upofproductionwouldbe100%insteadoftherampedprocessinthebaselineplan.(costsdeterminedfromP6tobe$564k)Maximalcostimpactincludesthemoduleproductionaccelerationandalsoincludeslabor(1FNALtechnicianfull-time=$113k)andinfrastructure(CO2plantenhancements=$100k)toacceleratethecassettetestingcapacityfrom2/dayto3/day.

CauseorTrigger: Primarycausesareadelayononeofthecriticalinputsforproduction:HGCROC,concentrator,siliconsensor,modulePCB,motherboards

ImpactedActivities:

Startdate: 15/Oct/2021 Enddate: 15/Jan/2023RiskMitigations: Activemanagementandcarefultrackingofmilestonesandacompleteprototype2validationofthesystemtoavoidsurprisesatthestartof

production.RiskResponses: Moredetails:

Page 46: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 41 Updated: 2019-03-06

RT-402-4-23-DCE-SiMotherboardcomplexityismuchhigherthanexpected

RiskRank: 2(Medium)Scores:Probability:2(L);Cost:2(M)Schedule:0(N)) RiskStatus: OpenSummary: Thesiliconmotherboardisexpectedtoexistinseveralvarients(~5).Dependingondecisionsmadeelsewhereintheproject,thenumberofdesigns

couldincreasebyafactoroftwooreventen.Ifthisweretooccur,theunitcostformotherboardswouldincreaseandthedesigntimewouldalsosignificantlyincrease.

RiskType: Threat Owner: ZoltanGecseWBS: 402.4CE-CalorimeterEndcap RiskArea: TechnicalRisk/InterfacesProbability(P): 20% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular

Minimum =383k$Mostlikely =575k$Maximum =767k$Mean =575k$P*<Impact> =115k$

ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =0monthsMaximum =monthsMean =0monthsP*<Impact> =0.0months

BasisofEstimate: Lowendofcostrangeassumes6monthsofadditionalengineering($120k)andafactorof50%inunitcostincreaseforPCBsandassemblyonly($263k),whilehighendassumes12monthsofadditionalengineering($240k)andafactorof100%increaseinunitcostforPCBsandassembly($527k).Thescheduledelayassumesthattheissuesareidentifiedsufficientlyinadvancethatadditionalengineeringcanbebroughtontotheprojecttoavoiddelays.

CauseorTrigger: Triggeredbydesignchoiceswhichrequiresignificantadditionalmotherboarddesigns,separatefromcostuncertaintyinasingledesign.

ImpactedActivities:

Startdate: 1/Jan/2019 Enddate: 11/Dec/2020RiskMitigations: WorkingwithinternationalcollaboratorstoadjustdesignsofcassetteandmodulePCBtoavoidrequiringadditionaldesignstomeetoveralldesign

constraints.RiskResponses: Moredetails:

Page 47: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 42 Updated: 2019-03-06

RT-402-4-90-DCE-KeyCalorimeterEndcappersonnelneedtobereplaced

RiskRank: 2(Medium)Scores:Probability:3(M);Cost:2(M)Schedule:1(L)) RiskStatus: OpenSummary: Keyengineerorseniortechnicianwithspecialknowledgeleavestheprojectandneedstobereplaced.Ifthetransitioncanbemanagedsuchthattheincomingand

outgoingpersonneloverlapandexchangeknowledge,thenthereismainlyalaborcostimpact.Ifthetransitionismoreabrupt,thenthereisnocostimpactofoverlappingpersonnelbuttherecanbeadelaytotheprojectactivitiesastheincomingpersongetsfullyuptospeed.Thisriskdoesnotincludetheriskoflosingkeymanagersbecausetheprojectensuresthateachmanagerhasawell-traineddeputy.

RiskType: Threat Owner: JeremiahMichaelMansWBS: 402.4CE-CalorimeterEndcap RiskArea: ManagementRisk/FundingorResourcesProbability(P): 25% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular

Minimum =75k$Mostlikely =225k$Maximum =555k$Mean =285k$P*<Impact> =71k$

ScheduleImpact: PDF =3-point-triangularMinimum =0monthsMostlikely =0monthsMaximum =3monthsMean =1monthsP*<Impact> =0.3months

BasisofEstimate: ThisL2area5seniortechnicalstaffwhosefunctionsareparticularlyhardtoreplaceonashorttimescalebyexistingteammembers.ExperienceintheCMSPhase1Upgradeprojectsuggeststheprobabilityperkeypersonisroughly25%(probabilitytheyleaveduringtheentireproject).Someinputnumberstothecostimpactanalysis:Laborcostperperson=$15k/month(fullyburdenedseniorenginer=$185k/year).AverageburnrateperL3areais=$21k/month(CMS-doc-13481).Minimumscenerio:personleavingisreplacedbyanexistingskilledpersonintheteam.1monthoffulloverlapcosting$15koflabor.Noscheduledelay.Likelyscenario:personleavingisreplacedbyapersonlessfamiliarwiththespecificwork.3monthsmanagedoverlapoftheoutgoingandincomingpeoplecosts3*$15k=$45koflabor.Noscheduledelay.Maximumscenario:personleavesunexpectedlywithnotransferofknowledge.Ittakesabout6monthstofindanewpersonandgetthemfullyuptospeed.Thereislossofproductivityduringthisdifficulttransitionperiodresultinginanet3monthdelaytoL3activities.Costoframp-upeffortofthenewperson(learningbutnotfullycontributingtodeliverables)=3FTE-months*$15k=$45k.Theburnratecostis3*$21k=$66k.Totalcostimpact=$111kTotalfor5persons:Min/Likely/Maxcostimpact=5*$(15/45/111)k=$(75/225/555)$.

CauseorTrigger: ImpactedActivities: EachkeypersonsisindependentlymodeledintheMC(eachatP=25%andwithcorrespondingshareofthetotalcostimpact).Typicallyhooktherisktothepersonsmainactivityduringtheproductionphase(maximumconsequences).Note:currentMCmodeltakescareofthetotalprobabilityandcostimpact.Butweshouldultimatelymodelthe5engineersindividuallyatappropriatepointsoftheschedule.

Startdate: 1/Jan/2018 Enddate: RiskMitigations: Anumberofengineersandtechnicianshaveoverlappingskills,bothwithinagiveninstituteandacrossUS-CMSinstitutes.Thisprovidessomebackupincaseof

non-availabilityofakeyperson.Pro-activecross-trainingofengineersandtechnicianshelpsensurekeyskillsarenotcompletelylostifakeypersonisnolongeravailable.

RiskResponses: Aimforoutgoingandincomingpersonneltooverlapifpossibletoensureasmoothtransition.Hirefromskilledmembersoftheteamifappropriate.Interimsupportmaybepossibleusingengineersortechnicainsfromelsewhereintheinstituteorproject.

Moredetails: CMS-doc-13481

Page 48: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 43 Updated: 2019-03-06

RT-402-4-91-DCE-ShortfallinCalorimeterEndcapscientificlabor

RiskRank: 2(Medium)Scores:Probability:3(M);Cost:2(M)Schedule:0(N)) RiskStatus: OpenSummary: Ifasignificantamountofthe(uncosted)scientificlaborisunavailable,thentheprojectwouldthenneedtofundadditional(costed)personnelto

performthework.Itisassumedthattheriskistriggeredbyaseriouslyunfavorableoverallbaseprogramfundingsituation.RiskType: Threat Owner: JeremiahMichaelMansWBS: 402.4CE-CalorimeterEndcap RiskArea: ManagementRisk/FundingorResourcesProbability(P): 30% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular

Minimum =0k$Mostlikely =0k$Maximum =982k$Mean =327k$P*<Impact> =98k$

ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =0monthsMaximum =monthsMean =0monthsP*<Impact> =0.0months

BasisofEstimate: InthepastUS-CMShasnotexperiencedasignificantlackofscientificlabor(postdocsandgraduatestudents).WhenshortfallsoccuredtheywereusuallyresolvedbycollaboratorsatUS-CMSinstitutesorsometimesfromiCMS.Weassigna30%probabilitythatasignificantshortfalloccursinfuture,duetounfavorablefundingconditionsinthebaseprogram.ThecontributedlaborforthescopeofthisL2areais57.1FTE-yearsspreadoverabout5years.ThisdoesnotincludethemoresecureL2andL3managerswhoaretenuredfacultyandseniorFermilabscientists.Weestimatethelosscouldbeupto20%ofthetotalcontributedlaboror11.4FTE-years(e.g.thisisalossof50%ofallcontributedlaborforatwoyearperiodoralossof1/3ofthecontributedlaborfor3years).Themissinglaborcouldbereplacedbycostedpersonnel:amixtureofmid-rangetechnicians,juniortechnicians,orundergraduatescostingrespectively62$/hr,50$/hrand18$/hrfully-burdened(43$/houronaverage).Allowingforfouryearsofescalationat3.1%perannumyieldsanaveragecostof49$/hror86k$/FTE-year(1768hrsworkedperyear).The(min/likely/max)costimpactistherefore:86k$perFTE-year*(0/0/11.4)FTE-years=$(0/0/982)k.

CauseorTrigger: Theriskistriggeredbyanunfavorablebaseprogramfundingsituation. ImpactedActivities: Startdate: 1/Oct/2018 Enddate: RiskMitigations: Workwithinstitutesandagenciestoensuretheanticipatedamountofscientificlaborwillbeavailable.Whereshortfallslooklikelytooccur,seek

alternativesamongstotherUS-CMSinstitutesorevenfromiCMSinstitutes.RiskResponses: Seekreplacementscientificlaborinotherinstitutes.Ifthislaborcannotbefoundthencontingencywillneedtobespenttosupplementtheeffort

withcostedlabor(e.g.technicians).Moredetails: CMS-doc-13509(FTEdataatCD1)

Page 49: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 44 Updated: 2019-03-06

RT-402-6-02-DTD-Boardorpartsvendornon-performance(DOE)

RiskRank: 2(Medium)Scores:Probability:2(L);Cost:2(M)Schedule:2(M)) RiskStatus: OpenSummary: Ifthevendorhasintermittentproblemsduringtheproductioncausingsub-standardbatchestobedeliveredthenpartsfrombatchNdonotmeet

electricalspecscausingadelayoforderonebatchlengthoranothervendorissought.Similarlyifthereissignificantvendordelaytoduetointernaltechnicalorinventoryproblems.TheriskisrepeatedperL3area.

RiskType: Threat Owner: JeffreyWBerryhillWBS: 402.6TD-TriggerandDAQ(DOE) RiskArea: ExternalRisk/VendorsProbability(P): 20% TechnicalImpact: 2(M)-significantlysubstandardCostImpact: PDF =3-point-triangular

Minimum =60k$Mostlikely =180k$Maximum =310k$Mean =183k$P*<Impact> =37k$

ScheduleImpact: PDF =3-point-triangularMinimum =1monthsMostlikely =3monthsMaximum =6monthsMean =3.33monthsP*<Impact> =0.7months

BasisofEstimate: Basedonpreviousdealingswithsimilarvendorsandeffectsfromproductionissues.Possiblecostincreasesduetonewvendorchoiceandadditionalprocurementofboardsandparts:Min/likely/maxcost=50/150/250k$for2/6/10newboardequivalents.Min/likely/maxdelay=1/3/6monthsforanotherproductionbatchorvendorselection.TheaverageL3burnrateduetothedelayofdownstreamactivitiesis$10k/month(CMS-doc-13481).Mincost=$50k+1month*$10kburnrate=$60k.Likelycost=$150k+3months*$10kburnrate=$180k.Maxcost=$250k+6months*$10kburnrate=$310k.

CauseorTrigger: ImpactedActivities: Procurementofproductionelectronics.ProbabilityissharedbetweenCaloTriggerandCorrelatorTrigger.

Startdate: 1/Jan/2021 Enddate: 1/Jan/2024RiskMitigations: Baselineboardproductionproceedsinwelldefinedsteps:(1)apreproductionofafewboardsensuresthatthequalityofthepartsprovidedbythe

vendormeetsthespecifications,(2)apilotproductionoffurtherensuresthatthequalityofthepartsprovidedbythevendormeetsspecifications,beforethefinalproductionisstarted.Thisphasedapproachprovidesreasonableassurancethatanyqualityperformanceissueswiththevendorwillbediscoveredandmitigatedbeforefullproductionbegins.

RiskResponses: Diagnosetheproblem.Ifasimpletechnicalfixisavailable,considerreordering.Ifnot,findanothervendor.Moredetails: CMS-doc-13481

Page 50: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 45 Updated: 2019-03-06

RT-402-6-03-DTD-I/Operformancedoesnotmeetrequirements(DOE)

RiskRank: 2(Medium)Scores:Probability:2(L);Cost:2(M)Schedule:3(H)) RiskStatus: OpenSummary: IftheI/Orequirementsofatriggersubsystemchange,thusjeopardizingtheabilityofthetriggertoreceive/transmitalltriggerprimitivedata

from/tothesource/sinksystem.RiskType: Threat Owner: JeffreyWBerryhillWBS: 402.6TD-TriggerandDAQ(DOE) RiskArea: TechnicalRisk/RequirementsProbability(P): 20% TechnicalImpact: 2(M)-significantlysubstandardCostImpact: PDF =3-point-triangular

Minimum =280k$Mostlikely =550k$Maximum =820k$Mean =550k$P*<Impact> =110k$

ScheduleImpact: PDF =3-point-triangularMinimum =3monthsMostlikely =5monthsMaximum =7monthsMean =5monthsP*<Impact> =1.0months

BasisofEstimate: 10-30%upscopeofa100-boardproductiontomeetchangedrequirements,withmin/likely/maxcostof250/500/750k$.Scheduledelaysof3/5/7monthsduetotheacquisitionofmorebatches.TheaverageL3burnrateduetothedelayofdownstreamactivitiesis$10k/month(CMS-doc-13481).Mincost=$250k+3month*$10kburnrate=$280k.Likelycost=$500k+5months*$10kburnrate=$550k.Maxcost=$750k+7months*$10kburnrate=$820k.

CauseorTrigger: Theinterfacebetweenthedetectorbackendelectronicsandthetriggersubsystemscouldchangetoaccomodateincreasedbandwidthrequirements.Thiscantaketheformofeithermorelinksorhigherlinkspeeds.Ifmorelinksarerequired,thiscouldcausethenumberofboardsrequiredtoabsorbthenumberoflinkstoincrease.Alternatively,ifthelinkspeedschange,thiscouldnecessitatenewopticaltransceiversontheboards,tomeetthenewlinkspeedrequirements.

ImpactedActivities: Productionprocurement,assembly,andtesting.ProbabilityissharedbetweenCaloTriggerandCorrelatorTrigger.

Startdate: 1/Jan/2021 Enddate: 1/Jan/2024RiskMitigations: Thebaselineincludesextralinkcapacityforalltriggersubsystems,toreasonabllyaccountforpossiblechangesinthenumberofinputlinkstothe

triggersubsystem.Thebaselinealsoincludesopticaltransceiversthatoperateacrossarangeof10-25Gb/slinkspeeds,allowingforreasonablechangesinlinkspeedinterfaces.

RiskResponses: (1)Attempttooptimise(reduce)thewordbitdefinitionofthereceivedortransmittedtriggerprimitivedata,sothatthetotalbandwidthin/outisconsistentwiththeI/Ocapacityoftheboard,whileattemptingtopreservethesciencegoals.(2)Increasethenumberofelectronicboardstomatchthenumberofinputlinks.

Moredetails: CMS-doc-13481

Page 51: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 46 Updated: 2019-03-06

RT-402-6-04-DTD-Additionalboardredesignisrequired(DOE)

RiskRank: 2(Medium)Scores:Probability:2(L);Cost:2(M)Schedule:2(M)) RiskStatus: OpenSummary: Aprototypeorproductionbatchhasdesignflaws.RiskType: Threat Owner: JeffreyWBerryhillWBS: 402.6TD-TriggerandDAQ(DOE) RiskArea: TechnicalRisk/ReliabilityorPerformanceProbability(P): 10% TechnicalImpact: 2(M)-significantlysubstandardCostImpact: PDF =3-point-triangular

Minimum =60k$Mostlikely =180k$Maximum =310k$Mean =183k$P*<Impact> =18k$

ScheduleImpact: PDF =3-point-triangularMinimum =1monthsMostlikely =3monthsMaximum =6monthsMean =3.33monthsP*<Impact> =0.3months

BasisofEstimate: 2-10boardsat25keacharefoundtorequireatechnicalredesign.Min/likely/maxcost=50/150/250k$for2/6/10newboardprocurements.Min/likely/maxdelay=1/3/6monthsforredesignandprocurement.TheaverageL3burnrateduetothedelayofdownstreamactivitiesis$10k/month(CMS-doc-13481).Mincost=$50k+1month*$10kburnrate=$60k.Likelycost=$150k+3months*$10kburnrate=$180k.Maxcost=$250k+6months*$10kburnrate=$310k.

CauseorTrigger: ImpactedActivities: Designofnextprotoype/batch.ProbabilityissharedbetweenCaloTriggerandCorrelatorTrigger.

Startdate: 1/Jul/2017 Enddate: 1/Jan/2024RiskMitigations: BaselineboardR&Dandproductionproceedsinwelldefinedsteps:TherearetwoprototypedesignsduringtheR&Dphase.Ifthefirstprototype

doesnotmeettherequirements,thesecondprototypewillberedesignedtomeettherequirements,basedontheexperiencelearned.Ifthesecondprototypealsodoesnotmeettherequirements,aredesigncanbeincorporatedintothepreproductioncycles(involvingafewboards),whichprovidesreasonableassurancethattheproductionboardwillmeettherequirements.Ifthepreproductionboarditselfalsodoesnotcompletelymeetallrequirements,thepilot-productioncyclecanbeusedtotweakthedesignsothatallrequirementsaremetbeforegoingintofinalproduction.

RiskResponses: Ifaprototypeorpreproductionboarddoesnotmeettherequirements,redesigntheboardinthenextcyclesothatitdoesmeettherequirements.Moredetails: CMS-doc-13481

Page 52: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 47 Updated: 2019-03-06

RT-402-6-05-DTD-Additionalfirmwaredevelopmentisrequired(DOE)

RiskRank: 1(Low)Scores:Probability:2(L);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: Firmwaredoesnotmeettechnicalorscientificrequirementsattimeofmilestonereleases.RiskType: Threat Owner: JeffreyWBerryhillWBS: 402.6TD-TriggerandDAQ(DOE) RiskArea: TechnicalRisk/ReliabilityorPerformanceProbability(P): 20% TechnicalImpact: 1(L)-somewhatsubstandardCostImpact: PDF =3-point-triangular

Minimum =10k$Mostlikely =30k$Maximum =60k$Mean =33k$P*<Impact> =7k$

ScheduleImpact: PDF =3-point-triangularMinimum =1monthsMostlikely =2monthsMaximum =3monthsMean =2monthsP*<Impact> =0.4months

BasisofEstimate: Anecessaryfirmwareblockrequires1/3/6monthsofreworkorredesign,withafirmwareengineerat0.5FTE.Nostandingarmycost(thereisnoidleworkforceintheeventofadelay).TheaverageL3burnrateduetothedelayofdownstreamactivitiesis$10k/month(CMS-doc-13481).Mincost=1month*$10kburnrate=$10k.Likelycost=3months*$10kburnrate=$30k.Maxcost=6months*$10kburnrate=$60k.

CauseorTrigger: ImpactedActivities: Prototypeandproductionfirmwarereleases.ProbabilityissharedbetweenCaloTriggerandCorrelatorTrigger.

Startdate: 1/Jul/2017 Enddate: 1/Jan/2024RiskMitigations: Infrastructureandalgorithmfirmwarearedevelopedincyclesandprogresstracked.Ifthetechnicalorsciencerequirementsarenotmetfora

particularmilestone,thisallowsengineerstomodifythefirmwaresothatitmeetsthetechnicalandsciencerequirementsinthenextcycle.Theuseofhigh-levelsynthesistoolsenablescientiststotranslatehigh-levelalgorithmsintofirmware,reducingthepotentialforinconsistenciesbetweenfirmwareandsoftwareemulation,aswellasreducingthepotentialtointroducebugsinthefirmwareimplementation,comparedwiththescientificintentofthealgorithm.

RiskResponses: (1)Ifthetechnicalorsciencerequirementsarenotmetforaparticularmilestone,thenallowtheengineerstomodifythefirmwaresothatitmeetsthetechnicalandsciencerequirementsinthenextcycle.(2)hireadditionalfirmwareengineering,eitherbyallocatingmoretimetoon-projectengineers,orhiringadditionalpersons.

Moredetails:

Page 53: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 48 Updated: 2019-03-06

RT-402-6-06-DTD-BaselineFPGAdoesnotsatisfyrequirements(DOE)

RiskRank: 2(Medium)Scores:Probability:2(L);Cost:2(M)Schedule:2(M)) RiskStatus: OpenSummary: IfthecomputationrequirementsofcalorimetertriggerandcorrelatorarenotsatisfiedbythebaselineFPGA,thenthesciencerequirementsare

jeopardized.RiskType: Threat Owner: JeffreyWBerryhillWBS: 402.6TD-TriggerandDAQ(DOE) RiskArea: TechnicalRisk/ReliabilityorPerformanceProbability(P): 20% TechnicalImpact: 2(M)-significantlysubstandardCostImpact: PDF =3-point-triangular

Minimum =136k$Mostlikely =282k$Maximum =564k$Mean =327k$P*<Impact> =65k$

ScheduleImpact: PDF =3-point-triangularMinimum =1monthsMostlikely =3monthsMaximum =6monthsMean =3.33monthsP*<Impact> =0.7months

BasisofEstimate: 12.5/25/50%FPGAcostincreaseon(36+48=)84FPGAsat12k/chipbasecost..Min/likely/maxcost=126/252/504k$.1/3/6monthsdelayforprocurement.TheaverageL3burnrateduetothedelayofdownstreamactivitiesis$10k/month(CMS-doc-13481).Mincost=$126k+1month*$10kburnrate=$136k.Likelycost=$252k+3months*$10kburnrate=$282k.Maxcost=$504k+6months*$10kburnrate=$564k.

CauseorTrigger: Increasinglyrealistic,betterperformingalgorithmsorincreasednumberofalgorithmsperFPGAmightrequiremorethanthelimitof50%utilizationofFPGAlogicresourcesormightrequireclockspeedsthatarehigherthanavailableforthebaselineFPGA,whichmaynecessitatetheuseofahigherperformanceFPGAoranincreasednumberofFPGAs.Alternatively,lowerlatencyrequirementsplacedonthetriggersubsystemsmightrequirefastercalculationsandhencehigherperformanceFPGAs.

ImpactedActivities: ProbabilityissharedbetweenCaloTriggerandCorrelatorTrigger.

Startdate: 1/Jan/2020 Enddate: 31/Mar/2021RiskMitigations: ThebaselineFPGAsupportstheprocessingofsomeprototypealgorithms,whichhavebeendemonstratedtoutilizenotmorethan40%ofthelogic

resourcesatnominalclockspeeds,providingabout10%headroom.Theprototypealgorithmshavebeendemonstratedtocompletewithinonemicrosecondoflatency(consistentwiththecurrentlatencyallocation),potentiallyprovidingsomelimitedabilitytotradelogicresourcesformoreclock-cycles(i.e.longerlatency).Thetotallatencybudgetsareonlyguidelineestimates,andaresubjecttochange.Thecurrentguidelinesforthecalorimetertriggeris1.5microsecondsoflatency(includingRXandTX)andforthecorrelatortrigger2.5microsecondsoftotallatency,split(roughlyevenly)betweenlayer-1andlayer-2(includingRXandTX).

RiskResponses: (1)attempttooptimisethefirmwareimplementationofthealgorithmtoreducethelogicutilizationtofitwithinatotalof50%,(2)increasethelatency(numberofclockcycles)ofthecalculationinanattempttoreducethelogicutilization,whileremainingwithintheallocatedlatencybudget,(3)useahigherperformanceFPGA(suchasaVU13P)oruseahigher-speedgradeFPGAorbuymorebaselineFPGAs(i.e.moreboards)

Moredetails: CMS-doc-13481

Page 54: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 49 Updated: 2019-03-06

RT-402-6-90-DTD-KeyTriggerorDAQpersonnelneedtobereplaced(DOE)

RiskRank: 1(Low)Scores:Probability:3(M);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: Keyengineerorseniortechnicianwithspecialknowledgeleavestheprojectandneedstobereplaced.Ifthetransitioncanbemanagedsuchthattheincomingand

outgoingpersonneloverlapandexchangeknowledge,thenthereismainlyalaborcostimpact.Ifthetransitionismoreabrupt,thenthereisnocostimpactofoverlappingpersonnelbuttherecanbeadelaytotheprojectactivitiesastheincomingpersongetsfullyuptospeed.Thisriskdoesnotincludetheriskoflosingkeymanagersbecausetheprojectensuresthateachmanagerhasawell-traineddeputy.

RiskType: Threat Owner: JeffreyWBerryhillWBS: 402.6TD-TriggerandDAQ(DOE) RiskArea: ManagementRisk/FundingorResourcesProbability(P): 25% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular

Minimum =30k$Mostlikely =90k$Maximum =150k$Mean =90k$P*<Impact> =23k$

ScheduleImpact: PDF =3-point-triangularMinimum =0monthsMostlikely =0monthsMaximum =3monthsMean =1monthsP*<Impact> =0.3months

BasisofEstimate: ThisL2areahas2seniorengineerawhosefunctionsareparticularlyhardtoreplaceonashorttimescalebyexistingteammembers.ExperienceintheCMSPhase1Upgradeprojectsuggeststheprobabilityperkeypersonisroughly25%(probabilitytheyleaveduringtheentireproject).Someinputnumberstothecostimpactanalysis:--Laborcostperperson=$15k/month(fullyburdenedseniorenginer=$185k/year).--AverageburnrateperL3areais=$10k/month(CMS-doc-13481).Minimumscenerio:personleavingisreplacedbyanexistingskilledpersonintheteam.1monthoffulloverlapcosting$15koflabor.Noscheduledelay.Likelyscenario:personleavingisreplacedbyapersonlessfamiliarwiththespecificwork.3monthsmanagedoverlapoftheoutgoingandincomingpeoplecosts3*$15k=$45koflabor.Noscheduledelay.Maximumscenario:personleavesunexpectedlywithnotransferofknowledge.Ittakesabout6monthstofindanewpersonandgetthemfullyuptospeed.Thereislossofproductivityduringthisdifficulttransitionperiodresultinginanet3monthdelaytoL3activities.Costoframp-upeffortofthenewperson(learningbutnotfullycontributingtodeliverables)=3FTE-months*$15k=$45k.Theburnratecostis3*$10k=$30k.Totalcostimpact=$75kTotalfor2persons:Min/Likely/Maxcostimpact=2*$(15/45/75)k=$(30/90/150)$.

CauseorTrigger: ImpactedActivities: EachkeypersonsisindependentlymodeledintheMC(eachatP=25%andwithcorrespondingshareofthetotalcostimpact).Typicallyhooktherisktothepersonsmainactivityduringtheproductionphase(maximumconsequences).

Startdate: 1/Jan/2018 Enddate: RiskMitigations: Anumberofengineersandtechnicianshaveoverlappingskills,bothwithinagiveninstituteandacrossUS-CMSinstitutes.Thisprovidessomebackupincaseof

non-availabilityofakeyperson.Pro-activecross-trainingofengineersandtechnicianshelpsensurekeyskillsarenotcompletelylostifakeypersonisnolongeravailable.

RiskResponses: Aimforoutgoingandincomingpersonneltooverlapifpossibletoensureasmoothtransition.Hirefromskilledmembersoftheteamifappropriate.Interimsupportmaybepossibleusingengineersortechnicainsfromelsewhereintheinstituteorproject.

Moredetails: CMS-doc-13481

Page 55: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 50 Updated: 2019-03-06

RT-402-6-91-DTD-ShortfallinTriggerorDAQscientificlabor(DOE)

RiskRank: 2(Medium)Scores:Probability:3(M);Cost:2(M)Schedule:0(N)) RiskStatus: OpenSummary: Ifasignificantamountofthe(uncosted)scientificlaborisunavailable,thentheprojectwouldthenneedtofundadditional(costed)personnelto

performthework.Itisassumedthattheriskistriggeredbyaseriouslyunfavorableoverallbaseprogramfundingsituation.RiskType: Threat Owner: JeffreyWBerryhillWBS: 402.6TD-TriggerandDAQ(DOE) RiskArea: ManagementRisk/FundingorResourcesProbability(P): 30% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular

Minimum =0k$Mostlikely =0k$Maximum =292k$Mean =97k$P*<Impact> =29k$

ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =0monthsMaximum =monthsMean =0monthsP*<Impact> =0.0months

BasisofEstimate: InthepastUS-CMShasnotexperiencedasignificantlackofscientificlabor(postdocsandgraduatestudents).WhenshortfallsoccuredtheywereusuallyresolvedbycollaboratorsatUS-CMSinstitutesorsometimesfromiCMS.Weassigna30%probabilitythatasignificantshortfalloccursinfuture,duetounfavorablefundingconditionsinthebaseprogram.ThecontributedlaborforthescopeofthisL2areais16.8FTE-yearsspreadoverabout5years.ThisdoesnotincludethemoresecureL2andL3managerswhoaretenuredfacultyandseniorFermilabscientists.Weestimatethelosscouldbeupto20%ofthetotalcontributedlaboror3.4FTE-years(e.g.thisisalossof50%ofallcontributedlaborforatwoyearperiodoralossof1/3ofthecontributedlaborfor3years).Themissinglaborcouldbereplacedbycostedpersonnel:amixtureofmid-rangetechnicians,juniortechnicians,orundergraduatescostingrespectively62$/hr,50$/hrand18$/hrfully-burdened(43$/houronaverage).Allowingforfouryearsofescalationat3.1%perannumyieldsanaveragecostof49$/hror86k$/FTE-year(1768hrsworkedperyear).The(min/likely/max)costimpactistherefore:86k$perFTE-year*(0/0/3.4)FTE-years=$(0/0/292)k.

CauseorTrigger: Theriskistriggeredbyanunfavorablebaseprogramfundingsituation. ImpactedActivities: Startdate: 1/Oct/2018 Enddate: RiskMitigations: Workwithinstitutesandagenciestoensuretheanticipatedamountofscientificlaborwillbeavailable.Whereshortfallslooklikelytooccur,seek

alternativesamongstotherUS-CMSinstitutesorevenfromiCMSinstitutes.RiskResponses: Seekreplacementscientificlaborinotherinstitutes.Ifthislaborcannotbefoundthencontingencywillneedtobespenttosupplementtheeffort

withcostedlabor(e.g.technicians).Moredetails: CMS-doc-13509(FTEdataatCD1)

Page 56: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 51 Updated: 2019-03-06

RT-402-8-01-DETL-AdditionalFEASICprototypecycleisrequired

RiskRank: 3(High)Scores:Probability:4(H);Cost:2(M)Schedule:2(M)) RiskStatus: OpenSummary: Thisriskcanhavemultiplecauses.

1)Ifthenecessaryperformance(precisionvspowerconsumption)isnotachievedduringthelastprototypecycle,anadditionalprototypecyclemaybenecessarycausingadelayandincurringacostincrease.2)ASICspecificationhaschangedduetoexternalreasons,suchaslpGBTclockdistribtutiondoesnotmeetspecifications.

RiskType: Threat Owner: TiehuiLiuWBS: 402.8.4ETL-EndcapTimingLayer RiskArea: TechnicalRisk/ReliabilityorPerformanceProbability(P): 50% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular

Minimum =500k$Mostlikely =600k$Maximum =700k$Mean =600k$P*<Impact> =300k$

ScheduleImpact: PDF =3-point-triangularMinimum =4monthsMostlikely =5monthsMaximum =6monthsMean =5monthsP*<Impact> =2.5months

BasisofEstimate: ThecostforaMPWrunis400k,andweinclude100-300kforadditionalengineering.

CauseorTrigger: Prototype2doesnotmeetrequirementsandwecannotincludechangesinthepreproductionsubmission.

ImpactedActivities: RiskcoulddelaycompletionofASICprototyping

Startdate: 1/Mar/2020 Enddate: 1/Jan/2022RiskMitigations: Wehaveincreasedtheeffortforthedesignsimulationandverification.

RiskResponses: Weincludeanadditionalprototypecycle.

Moredetails:

Page 57: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 52 Updated: 2019-03-06

RT-402-8-02-DETL-ProblemswithETLmoduleassemblyfacility

RiskRank: 2(Medium)Scores:Probability:4(H);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: Thisisageneralcatagorythatincludesmultipleissuesthatcanleadtoaproductionslowdownorstopage.IfaModuleAssemblyfacilitytemporarily

becomesinoperableduetolossofcriticalequipmentoractofgod,thentheresultantdipinmodulethroughputmayjeopardizetimelycompletionoftheproject.

RiskType: Threat Owner: SlawomirMTkaczykWBS: 402.8.4ETL-EndcapTimingLayer RiskArea: ExternalRisk/FacilitiesProbability(P): 50% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =1-point-singlevalue

Minimum =k$Mostlikely =30k$Maximum =k$Mean =30k$P*<Impact> =15k$

ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =1monthsMaximum =monthsMean =1monthsP*<Impact> =0.5months

BasisofEstimate: Ifonecenterhasamajorequipmentfailurethesecondcentercanpickuptheadditionalloadwithinthe100%cushion.Itisassumedthatin1monththeequipmentwouldbereplacedandassemblycouldthenberesumedattheaffectedsite.Anexpecteddelayof1monthduetoaneedtotransfercomponentstothesecondproductionsiteandtoorganizesecondshiftmoduleproduction.

CauseorTrigger: ImpactedActivities: RiskcoulddelayanyoneofthebatchesduringETLAssembly

Startdate: 1/Jan/2023 Enddate: 1/Jul/2024RiskMitigations: HavingatleasttwoUSsitesisariskmitigation,andshouldonesitebecometemporarilyinoperable,componentscouldberedirectedtotheother

sitetemporarilytomitigatetheimpact,whiletheaffectedproductionsiteisbeingrepaired.RiskResponses: Modulecomponentscanbedivertedtotheunaffectedsitetoutilizeitsfullthroughput,andadditionalresourcesaddedtoincreasemodule

productionthroughputatbothsites(oncetheaffectedoneisre-established)toregaintimeintheschedule.Moredetails:

Page 58: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 53 Updated: 2019-03-06

RT-402-8-03-DETL-FEASICdoesnotmeetspecs-needsanotherpre-prodrun

RiskRank: 2(Medium)Scores:Probability:2(L);Cost:3(H)Schedule:3(H)) RiskStatus: OpenSummary: IfthepreproductionFrontEndASICdoesnotmeetspecificationsafterthemasksareproducedthenanadditionalsetofmaskswillneedtobe

producedandadditionalengineeringeffortisrequiredtoadjustthedesignsothattheASICmeetsspecification.RiskType: Threat Owner: TiehuiLiuWBS: 402.8.4ETL-EndcapTimingLayer RiskArea: TechnicalRisk/QualityProbability(P): 10% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular

Minimum =914k$Mostlikely =970k$Maximum =1026k$Mean =970k$P*<Impact> =97k$

ScheduleImpact: PDF =3-point-triangularMinimum =6monthsMostlikely =7.5monthsMaximum =9monthsMean =7.5monthsP*<Impact> =0.8months

BasisofEstimate: TheM&Scostis$760kfornewmasksandqualificationwafers.Thelaborcostfortheadditionalengineeringeffortisestimatedasa3-pttriangularPDF:0.25-0.375-0.5FTE.Theestimatedscheduledelayis3-pttriangularPDF:6-7.5-9monthsandweusethestandingarmycostestimatefortheMTD,asdescribedinDocDB#13481,fortheburnrate(14k$/month).Mincost=760k$+0.25*280k$+6month*14k$burnrate=$914kLikelycost=760k$+0.375*280k$+7.5months*14k$burnrate=$970kMaxcost=760k$+0.5*280k$+9months*14k$burnrate=$1026k

CauseorTrigger: ETLFrontEndASICasreturnedfromproductiondoesnotmeetspecificationdespitethepreviousMPWs.

ImpactedActivities: RiskcoulddelaycompletionofETLpreproduction

Startdate: 1/Jan/2022 Enddate: 1/Jan/2023RiskMitigations: Thisriskismitigatedbycarryingoutseveralprototypecyclesandincreaseddesignsimulationandverificationeffort.

RiskResponses: Ifthefirstpre-productionrundoesnotmeetourrequirementswewillhavetofixtheproblemsandproduceanewmaskset.IftheASICisdelayed

andwecannotmeetthemilestoneofinstallingtheETLduringLS3wemayhavetoinstalltheETLduringoneoftheYETS.Moredetails:

Page 59: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 54 Updated: 2019-03-06

RT-402-8-04-DBTL-LYSOmatricesnotmeetingspecifications

RiskRank: 1(Low)Scores:Probability:2(L);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: IfdeliveredLYSOcrystalsdonotmeetgeometricoropticalspecificationsthentherewillbeadelayinproduction.RiskType: Threat Owner: AdolfBornheimWBS: 402.8.3BTL-BarrelTimingLayer RiskArea: ExternalRisk/VendorsProbability(P): 10% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =1-point-singlevalue

Minimum =k$Mostlikely =100k$Maximum =k$Mean =100k$P*<Impact> =10k$

ScheduleImpact: PDF =3-point-triangularMinimum =1monthsMostlikely =2monthsMaximum =3monthsMean =2monthsP*<Impact> =0.2months

BasisofEstimate: IfthereisadelayofanyofthesenecessarycomponentsthatistranslatedtoBTLAssembly,thentherewillbeknock-onstandingarmycoststhere.Thisestimateisfor1FTEofonetechnician.

CauseorTrigger: Measurementsofsensorperformanceduringmoduleassembly.

ImpactedActivities: RiskcoulddelayanyoneofthebatchesduringBTLAssembly

Startdate: 1/Jan/2020 Enddate: 18/Jan/2022RiskMitigations: QA/QCprocedureintheLYSOprocurement,strictdefinitionofQAcriteriawithvendor.RiskResponses: Oncedeliveryofqualifiedcomponentsrestarts,runx2shiftsofBTLAssemblyandQCtesting,runx2shiftsofBTLIntegration.Moredetails:

Page 60: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 55 Updated: 2019-03-06

RT-402-8-05-DBTL-Changeininterfacesoftrayassemblycomponents

RiskRank: 2(Medium)Scores:Probability:2(L);Cost:2(M)Schedule:1(L)) RiskStatus: OpenSummary: Ifthedimensions,connectionsorassemblysequencedetailsofthetraychangethentheproceduresintheintegrationcenterhavetobeadjusted

incurringdelaysandjeopardizingcompletionoftheBTLbytheCMSneed-bydate.RiskType: Threat Owner: AdolfBornheimWBS: 402.8.3BTL-BarrelTimingLayer RiskArea: ExternalRisk/CollaboratorsProbability(P): 20% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular

Minimum =150k$Mostlikely =250k$Maximum =350k$Mean =250k$P*<Impact> =50k$

ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =3monthsMaximum =monthsMean =3monthsP*<Impact> =0.6months

BasisofEstimate: Iftheassemblyofthetraysfromthethreeprincipalcomponents,coolingbar,sensormoulesandfrontendcards,ischangedthemanpowerneedsmayincreaseandadditionaltoolingandskilledlaborneedtobeaaddedtothetask.Mostlikelyimpactis1FTE-yearofaddtionalengineeringatanestimatedcostof$250k.

CauseorTrigger: Changeoftheinterfacesononeorseveralofthecomponentsenteringintothetrayassemblyprocedure.

ImpactedActivities: RiskcoulddelaythecompletionoftheBTLAssemblyR&Dandprototyping

Startdate: 1/Oct/2019 Enddate: 10/Feb/2023RiskMitigations: WorkwithcollboratorsduringtheR&Dphasetoensuredesignsarecoherent,agreeduponandproperlycommunicatedtoallstakeholders.Further,

ensuredetailedtestingofallprototypesandpreproductionitemstoensureperformanceandtechnicalspecsaremetatalltimes.RiskResponses: Ifinterfacechangesarenecessary,ensurethatdesignchangesremainascompatibleaspossiblewiththeassemblycenterprocedures.Update

procedurespursuanttoupdatedinterfaces.Losttimemadeupthroughincreasedlabor.Moredetails:

Page 61: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 56 Updated: 2019-03-06

RT-402-8-07-DBTL-ConcentratorCarddelayinexternalcomponentdeliveries

RiskRank: 2(Medium)Scores:Probability:2(L);Cost:0(N)Schedule:2(M)) RiskStatus: OpenSummary: Ifexternalcomponentsaredelayedthenthedeliverydeadlineobjectiveisjeopardized.RiskType: Threat Owner: YuriiYMaravinWBS: 402.8.3BTL-BarrelTimingLayer RiskArea: ExternalRisk/VendorsProbability(P): 20% TechnicalImpact: 3(H)-extremelysubstandardorKPPin

jeopardyCostImpact: PDF =1-point-singlevalue

Minimum =k$Mostlikely =0k$Maximum =k$Mean =0k$P*<Impact> =0k$

ScheduleImpact: PDF =3-point-triangularMinimum =1monthsMostlikely =3monthsMaximum =6monthsMean =3.33monthsP*<Impact> =0.7months

BasisofEstimate: AconservativeestimatebasedonthefactthatthelpGBTprototypewasdelayedbyabouthalfayear.GiventhefactthatthelpGBTpinoutisdelayedby6months,thisestimateisratherreasonablethatthedelaywillbeontheorderof6months.

CauseorTrigger: Delaywiththeprototypesorthefinaldesigncomponents ImpactedActivities: RiskcoulddelayanybatchofCCsduringCCproduction

Startdate: 1/Oct/2019 Enddate: 15/Apr/2020RiskMitigations: Itisnotpossibletomitigatethisrisk.RiskResponses: Moredetails:

Page 62: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 57 Updated: 2019-03-06

RT-402-8-08-DBTL-Delayincoolingplatedelivery

RiskRank: 1(Low)Scores:Probability:2(L);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: Ifthedeliveryofcoolingplatesbyinternationalpartnerstotheassemblycentersisdelayed,theassemblyofBTLtraysandsubsequentlythedelivery

oftraystoCERNgetsdelayed.RiskType: Threat Owner: AdolfBornheimWBS: 402.8.3BTL-BarrelTimingLayer RiskArea: ExternalRisk/CollaboratorsProbability(P): 10% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular

Minimum =10k$Mostlikely =20k$Maximum =30k$Mean =20k$P*<Impact> =2k$

ScheduleImpact: PDF =3-point-triangularMinimum =1monthsMostlikely =2monthsMaximum =3monthsMean =2monthsP*<Impact> =0.2months

BasisofEstimate: Ifcoolingplatedeliveryisdelayedtheundergraduteworkerswillnotbeabletoassembletrays.ExtratimeneedstobeworkedlatertomakeupthedelaytokeepthedeliveryscheduletoCERN.TheassemblyoftheBTLtraysisdesignedsuchthatadditionalFTEcanworkinparallelandincreasedthroughputatthecenter.CostreflectscostforaddtionalFTEstomitigatethescheduleimpact.

CauseorTrigger: Delayofcoolingplatedelivery.

ImpactedActivities: RiskcoulddelayanyoneofthebatchesduringBTLAssembly

Startdate: 28/Jan/2022 Enddate: 31/Jan/2024RiskMitigations: CommunicatecloselywithiCMSpartnerstomaintainscheduleawareness.RiskResponses: Acceleratetrayassembly.Hireadditionalundergratuatestudentstoworkovertimeandweekend.Moredetails:

Page 63: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 58 Updated: 2019-03-06

RT-402-8-10-DETL-Sensorqualityproblemduringproduction

RiskRank: 2(Medium)Scores:Probability:2(L);Cost:1(L)Schedule:2(M)) RiskStatus: OpenSummary: IfthesensorvendordeliverssensorsthatdonotmeetspecificationsthenthedegradedperformanceoftheETLjeopardizesthephysicsperformance

oftheMTDdetector.RiskType: Threat Owner: ChristopherS.RoganWBS: 402.8.4ETL-EndcapTimingLayer RiskArea: ExternalRisk/VendorsProbability(P): 15% TechnicalImpact: 2(M)-significantlysubstandardCostImpact: PDF =3-point-triangular

Minimum =28k$Mostlikely =52k$Maximum =109k$Mean =63k$P*<Impact> =9k$

ScheduleImpact: PDF =3-point-triangularMinimum =2monthsMostlikely =3monthsMaximum =6monthsMean =3.67monthsP*<Impact> =0.6months

BasisofEstimate: ThecontractwillbewrittenforthevendortodeliveraspecifiednumberofgoodsensorsthatsatisfyCMSspecifications.Thuswedonothavetopayforsensorsthatdonotsatisfythespecificationsandthereisnoimpactonsensorcost.TheonlycostimpactisthatwewillhavetorepeattheQCtestingofthereplacementsensors.Minimalimpact:thishappensduringproductionandiscorrectedquicklyafterfeedbackfromsensorQCleadingtoadelayofabout2monthsandnegligibledirectcost.Maximalscheduleimpact:thishappensduringpreproductionandthepreproductioncyclehastoberepeated,leadingtoadelayofabout6monthsandextralaborcostofabout$25k.TheL3burnrateduetothedelayofdownstreamactivitiesis$14k/month(CMS-doc-13481).Mincost=$0k+2months*$14kburnrate=$28k.Likelycost=$10k+3month*$14kburnrate=$52k.Maxcost=$25k+6months*$14kburnrate=$109k.TheproblemhastoeitherpersistovermanybatchesornotbenoticedduringQCatthevendor(forexampleadegradationofperformanceoversometime).Problemsthataffectasinglebatchofsensors(egbecauseofsomecontaminationorprocessingmistake)willnotleadtoasignificantdelaybecausereprocessingabatchwillonlyaddaweekortwototheproductionperiod.Therefore,theprobabilityofthisthreattooccurismoderate–15%.

CauseorTrigger: Sensorsarrivingfromvendordonotsatisfyspecifications ImpactedActivities: RiskcoulddelayanyoneofthebatchesduringETLsensorQC

Startdate: 1/Jan/2022 Enddate: 1/Apr/2024RiskMitigations: Wecarryoutextensiveprototypingworkwiththevendorspriortoplacingthecontractforsensorproductiontomakesurethatvensorsunderstand

ourspecificationsandcanmeetthem.ThevendorwillcarryoutafirstsetofQCmeasurementsbeforethesensorsareshippedtoCERNanddistributedtoQCcenters.Thisensuresthatmostproblemswillbecaughtquicklyanddonotleadtosignificantimpactontheproject.Thecostofthesemeasurementsisfacturedintothesensorcost.

RiskResponses: Ifamodestproblemoccurs,workcloselywithvendortosolveit(e.g.testing).Replacetheflawedsensors.Moredetails:

Page 64: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 59 Updated: 2019-03-06

RT-402-8-14-DBTL-ProblemswithSiPMvendor

RiskRank: 2(Medium)Scores:Probability:2(L);Cost:1(L)Schedule:3(H)) RiskStatus: OpenSummary: ThisisageneralcatagorythatincludesmultipleproblemsassociatedwiththeSiPMvendor.

1)IftheproductionSiPMsdonotmeetspecificationsduetoaprocessingerroratthevendor,thentherewillbeasixmonthdelayintheschedulefortestingandcharacterizingtheSiPMs.Thismayimpacttheoverallprojectscheduleandcost.2)IftheproductionofSiPMsisdelayedatthevendor,thatdelaywillpropogatetotheSiPMtestingandcharacterizationschedule,andinturntheoverallprojectschedule.

RiskType: Threat Owner: MitchellWayneWBS: 402.8.3BTL-BarrelTimingLayer RiskArea: ExternalRisk/VendorsProbability(P): 20% TechnicalImpact: 2(M)-significantlysubstandardCostImpact: PDF =3-point-triangular

Minimum =32k$Mostlikely =96k$Maximum =128k$Mean =85k$P*<Impact> =17k$

ScheduleImpact: PDF =3-point-triangularMinimum =2monthsMostlikely =6monthsMaximum =8monthsMean =5.33monthsP*<Impact> =1.1months

BasisofEstimate: 1)IntheeventtheproductionSiPMsdonotmeetspecifications,weestimatea6monthscheduledelaytoproduceanewsetofproductionSiPMS.Thisdelayincludesfabricationofnewwafers,testingandpackagingoftheSiPMs.Thecostimpactwillbe6monthsofhalf-timesupportfortwoengineeringphysicists.2)IntheeventdeliveryoftheproductionSiPMsisdelayedbysomeamountoftime,weestimatetheimpactfromaminimaldelayof2monthsuptoamaximumdelayof8months.Thecostimpactwillbeadditionalsupportfortwoengineeringphysicistsforeachmonthofdelay.

CauseorTrigger: ImpactedActivities: RiskcoulddelayanybatchofSiPMQC

Startdate: 1/Oct/2019 Enddate: 18/Jan/2022RiskMitigations: WorkwithvendortoestablishqualityandgoodyieldthroughprototypeandpreproductionSiPMruns.Couldusemultiple(currentlythereare

2)vendorsifthequalityiscomparable.

RiskResponses: 1)IntheeventofSiPMsnotmeetingspecifications,workwithvendortoidentifythesourceoftheproblem,thenrestarttheproductionprocess.2)IntheeventofadelayinSiPMdelivery,worktotheextentpossibletospeeduptestingandcharacterizationoncetheyaredelivered.

Moredetails:

Page 65: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 60 Updated: 2019-03-06

RT-402-8-15-DBTL-BatchshipmentofSiPMslostintransport

RiskRank: 1(Low)Scores:Probability:1(VL);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: IfabatchoftestSiPMsislostinshipmenttoaproductionsite,thentherewillbeaonemonthdelayoftheSiPMcompletiondate.Thismayimpact

theoverallscheduleandcost.RiskType: Threat Owner: MitchellWayneWBS: 402.8.3BTL-BarrelTimingLayer RiskArea: ManagementRisk/LogisticsProbability(P): 5% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =1-point-singlevalue

Minimum =k$Mostlikely =224k$Maximum =k$Mean =224k$P*<Impact> =11k$

ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =1monthsMaximum =monthsMean =1monthsP*<Impact> =0.1months

BasisofEstimate: Weestimatea1monthdelaytoQCanewbatchofproductionSiPMs.ThismayinterruptthemoduleassemblybyonemonthanddelaythedeliveryofthefinalSiPMbatchbyonemonth.Thecostimpactwillbe$224k,basedon28,000channelsandperchannelcostof$8.TheimpactisthesameforanyUSbatch,astheUSdeliversthefirsthalfoftheSiPMs.

CauseorTrigger: ImpactedActivities: RiskcoulddelayanyoneofthebatchesduringBTLAssembly

Startdate: 1/Jan/2021 Enddate: 18/Jan/2022RiskMitigations: Divideupthebatchshipmentintoanumberofsmallershipments,thusmitigatingtheimpactofalostshipment.Ofcoursethisimpliesmore

shipments,increasingtheprobabilityofonebeinglost.Weneedtooptimizethisforminimalrisk.RiskResponses: Ifashipmentislost,wewillneedtopurchaseadditionalproductionSiPMsfromthevendor,havethemdelivered,andthenrunthroughthestandard

QCandcharacterization.Moredetails:

Page 66: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 61 Updated: 2019-03-06

RT-402-8-16-DBTL-ProblemswithSiPMQCtestsite

RiskRank: 1(Low)Scores:Probability:2(L);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: ThisisageneralcategorythatincludesmultipleissuesthatcanleadtoreducedorstoppedthoughputattheSiPMQCsite.

1)IfthereisafailureofoneormoreoftheSiPMproductionteststands,thentherewillbeadelayintestingandcharacterizingtheSiPMs.Thismayimpacttheoverallscheduleandcost2)Ifthereisaproblemwiththelabinfrastructure(construction,lossofpower,flooding)thentherewillbeadelayintestingandcharacterizingtheSiPMs.Thismayimpacttheoveralscheduleandcost.

RiskType: Threat Owner: MitchellWayneWBS: 402.8.3BTL-BarrelTimingLayer RiskArea: TechnicalRisk/RequirementsProbability(P): 20% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular

Minimum =2k$Mostlikely =5k$Maximum =10k$Mean =6k$P*<Impact> =1k$

ScheduleImpact: PDF =3-point-triangularMinimum =0.25monthsMostlikely =0.5monthsMaximum =1monthsMean =0.58monthsP*<Impact> =0.1months

BasisofEstimate: 1)WeestimatethatafailureofoneoftheSiPMQCteststandswilldelaytheSiPMQCandcharacterizationbyahalf-month,includingtimetoidentifythecausesofthefailureandmakethenecessaryfixes.2)WeestimatethataproblemwiththelabinfrastructurewilleitherbesolvedorresultinmovingtoanewlocationatCERN.Eitherwayweestimateamaximumscheduleimpactofonemonth.

CauseorTrigger: ImpactedActivities: RiskcoulddelayanybatchofSiPMQCStartdate: 1/Jan/2021 Enddate: 18/Jan/2022RiskMitigations: 1)TheteststandswillbeusedtoQCandcharacterizeasignificantnumberofpreproductionSiPMsbeforewedothefullproductionrun.Identify

criticalequipmentandverifyavailability(e.g.CERNpool).2)Wewillidentifyotherpotentiallocationsasabackupforthetesting.

RiskResponses: 1)Identifythecauseofthefailure,andrepairasneeded.2)Eitherwaituntillabproblemisresolved,ormovetestingtoanotherlocation.

Moredetails:

Page 67: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 62 Updated: 2019-03-06

RT-402-8-18-DBTL-Concentratorcardproduction&testingfacilityproblem

RiskRank: 1(Low)Scores:Probability:2(L);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: Thisisageneralcatagorythatincludesmultipleissuesthatcanleadtoaslowdownorstoppage.Forexample,ifateststandfailstotestconcentrator

cards,itimpactsthescheduleforCCdeliverytotheassemblycentersRiskType: Threat Owner: YuriiYMaravinWBS: 402.8.3BTL-BarrelTimingLayer RiskArea: TechnicalRisk/RequirementsProbability(P): 20% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =1-point-singlevalue

Minimum =k$Mostlikely =10k$Maximum =k$Mean =10k$P*<Impact> =2k$

ScheduleImpact: PDF =3-point-triangularMinimum =0.5monthsMostlikely =1monthsMaximum =2monthsMean =1.17monthsP*<Impact> =0.2months

BasisofEstimate: PreviousexperienceintestingMu2eFEBcardsindicatedtheneedforaspecialcard($1k)andmodificationstothetestingjig($9k)componentsduetochangesintheFEBfunctionalityandfeaturesthatprohibitedusingexistingcomponentsatKSU.Totalcostimpactis$10k.Orderingnewparts+installingthem+testingthemcausedadelayofupto2months,includingsoftwareadjustments.Burnrateiszeroastheaffectedpersonnelcanworkonothertasks.

CauseorTrigger: FailureofthetestingjigtoprovideathoroughtestoftheassembledCC ImpactedActivities: RiskcoulddelayanybatchofCCsduringCCproduction

Startdate: 15/Mar/2021 Enddate: 15/Jul/2022RiskMitigations: Keepthecomponentsmaintainedandorderspareitemsforcriticalcomponentstobeavailableonashortnoticeincaseoffailure.RiskResponses: Implementmanualtestingprotocolforundergraduateswhiletestingjigissuesareunderstoodandfixed.Hireextraundergraduatestudentsto

performmanualtestingwhilethetestingjigisbeingrepaired.Moredetails:

Page 68: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 63 Updated: 2019-03-06

RT-402-8-30-DBTL-ConcentratorCardrequiressignificantdesignchanges

RiskRank: 2(Medium)Scores:Probability:2(L);Cost:1(L)Schedule:2(M)) RiskStatus: OpenSummary: Thisisageneralcatagorythatincludesmultipleissuesthatcanleadtoadesignorlayoutchange.

1)IfscopefortheBTLCCchanges,itwillimpactthecostoftheprojectduetoare-designaswellasdelaytheschedule.2)TheconcentratorcarddesignwilldependonthefinalformfactoroftheDC/DCconvertor

RiskType: Threat Owner: YuriiYMaravinWBS: 402.8.3BTL-BarrelTimingLayer RiskArea: ManagementRisk/ControllingProbability(P): 10% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular

Minimum =1k$Mostlikely =50k$Maximum =100k$Mean =50k$P*<Impact> =5k$

ScheduleImpact: PDF =3-point-triangularMinimum =1monthsMostlikely =3monthsMaximum =6monthsMean =3.33monthsP*<Impact> =0.3months

BasisofEstimate: Inthepast,changesindesigncauseddelaysfrom1-2weekstoamonth,incurringadditionalcostsduetoengineertime.However,amajorredesignoftheboard,forexample:adecisiontodoublethebandwidthbyrequiringusingtwolpGBTsinsteadofonewouldrequiredoublingthecomplexityofrouting,essentiallyrequiringanincreaseofthedesigntimebyengineerbyafactoroftwo,includingincreaseintimefortesting.Giventheesitmateof400hoursofengineeringtodesign&layout,doublingofthistimeisagoodestimateofimpactincostandtime.Estimatingthattheengineerissplitbetweentwodifferenttasks,anaverageestimateof5monthsofdelayisextracted.

CauseorTrigger: NewchangestothescopeoftheCCafteritisdesigned ImpactedActivities: RiskcouldimpactthecompletionofCCR&Dandprototyping

Startdate: 1/Oct/2019 Enddate: 27/Jan/2022RiskMitigations: KeepaclosecontacttotheinternationalBTLefforttoensuretimelyresponsetopotentialchangesinthescopeoftheBTLelectronicsandkeep

internationalCMSinformedofimpactsofdesignchangestotheUSscope.

RiskResponses: Delegateafractionofthetaskathandtoanotheravailableengineer,attheKSUElectronicDesginLaboratoryifavailable,oratanotherinstitution.Moredetails:

Page 69: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 64 Updated: 2019-03-06

RT-402-8-31-DETL-Storage-relateddegradationofLGADs

RiskRank: 1(Low)Scores:Probability:2(L);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: IftheLGADinventoryisleftaroundfortoolongduringtheassemblyprocessthensomeageingmayoccurthatdegradestheperformanceorrenders

theLGADsunusable.ThiseffectwasobservedinproductionofsensorsforthecurrentCMSTracker.RiskType: Threat Owner: SlawomirMTkaczykWBS: 402.8.4ETL-EndcapTimingLayer RiskArea: TechnicalRisk/QualityProbability(P): 10% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =1-point-singlevalue

Minimum =k$Mostlikely =18k$Maximum =k$Mean =18k$P*<Impact> =2k$

ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =3monthsMaximum =monthsMean =3monthsP*<Impact> =0.3months

BasisofEstimate: On-shelfdegradationofsensorqualitywasobservedforonevendorduringprocurmentofsensorsforthepresentCMStracker.Theproductionwasputonholdforabout3monthstounderstandtheproblemandfindasolution.Provideregularmaintenancetothehumiditycontrolingdevicesforpropoercomponentstorage.Incaseofsmallerstoragecabinetsensurethatdryairiscirculated.

CauseorTrigger: ImpactedActivities: RiskcoulddelayanyoneofthebatchesduringETLAssembly

Startdate: 1/Jan/2023 Enddate: 1/Oct/2025RiskMitigations: CarefulQCofthedeliveredLGADsuponreceptionandrepeatQCpriortoassemblyofmoduleswillaloowtoidentifiedproblematicsensors.Proper

storageofcomponentsandhandlinginacontrolledenvironmente.g.cleanroomwithcontrolledhumidityandfilteredaircirculation.RiskResponses: Approved.Changesstartandexpirationdate.Updatedmitigationandresponsetext.Moredetails:

Page 70: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 65 Updated: 2019-03-06

RT-402-8-33-DBTL-DifficultiesprocuringLYSOfrominternationalsuppliers

RiskRank: 2(Medium)Scores:Probability:2(L);Cost:2(M)Schedule:3(H)) RiskStatus: OpenSummary: IfLYSObecomesthesubjectofhightariffsfromaspecificcountry,thentheincreasedcostjeopardizesprojectbudgetand/ordelaysjeopardizeQC

testingscheduleandallsubsequentdependentactivities.RiskType: Threat Owner: ChristopherCarlNeuWBS: 402.8.3BTL-BarrelTimingLayer RiskArea: ExternalRisk/MarketProbability(P): 10% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular

Minimum =200k$Mostlikely =450k$Maximum =700k$Mean =450k$P*<Impact> =45k$

ScheduleImpact: PDF =3-point-triangularMinimum =3monthsMostlikely =6monthsMaximum =9monthsMean =6monthsP*<Impact> =0.6months

BasisofEstimate: ConsideringanoverallinvestmentfromtheUSof~$900k.Assuminga50%tariffmeansacostimpactof450k,withsomeguessedrange±250k.CauseorTrigger: Example:TheUSgov'tappliesatariffonrare-earthmetalsortheir

relatedcommodities.Internationalsuppliersapplyexportcontrolsoranembargoonthesematerials.

ImpactedActivities: Riskcoulddelaythedeliveryofthepre-productionLYSOcrystals

Startdate: 1/Oct/2019 Enddate: 18/Jan/2022RiskMitigations: Weshouldhavequalifiedvendorsinmultiplecountriessothatweareinsulatedfromeffectsassociatedwithpurchasingfromasingle

foreigncountry.RiskResponses: WorkwithatleasttwoprovidersthatcanincreasetheirLYSOprovisioninthecaseoftheapplicationofatariffaffectingone.Makesurethecontract

containsappropriatelanguage.HavethepurchasegothroughCERNsothatitisinsulatedfromtheseUS-specificeffects.IncreasetheprovisionofLYSOfromalternativevendorthatisnotsubjecttothepolitical/economicobstacleconfronted.

Moredetails:

Page 71: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 66 Updated: 2019-03-06

RT-402-8-34-DBTL-DelayindeliveryofcomponentsfromiCMS

RiskRank: 1(Low)Scores:Probability:2(L);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: IfthedeliveryofLYSO,SiPMsorelectronicscomponentsforthemodulesisdelayed,themoduleandtrayproductionandsubsequenttray

integrationisdelayed.RiskType: Threat Owner: AdolfBornheimWBS: 402.8.3BTL-BarrelTimingLayer RiskArea: ExternalRisk/CollaboratorsProbability(P): 20% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular

Minimum =10k$Mostlikely =20k$Maximum =30k$Mean =20k$P*<Impact> =4k$

ScheduleImpact: PDF =3-point-triangularMinimum =1monthsMostlikely =2monthsMaximum =3monthsMean =2monthsP*<Impact> =0.4months

BasisofEstimate: Ifthedeliveryofanyofthecomponentsenteringintothetrayintegrationisdelayedwewouldneedtocatchupproductionbyincreasingavaiablemanpower.Themoduleandtrayassemblyisverygranularandcanbescaledeasilybyaddingmanpower.Multipleteamscanworkonasingletraytointegratecomponents.Thecostimpactreflectsthecostofhiringadditionallabor.Thescheduleimpactaccountsforthefactthathiringadditionallaborandtrainingnewpeoplemaycausedelays.

CauseorTrigger: Deliveryofcomponentsisdelayed.

ImpactedActivities: RiskcoulddelayanyoneofthebatchesduringBTLAssembly

Startdate: 28/Jan/2022 Enddate: 31/Jan/2024RiskMitigations: Ensurestrictdefinitionofdeliveryscheduleandintegrateinthecontractualagreementwithvendorsandcollaborators.RiskResponses: Hireadditionalunskilledlabortoworkonthemoduleassemblyandtrayintegrationwithmultipleteamsononetray.Moredetails:

Page 72: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 67 Updated: 2019-03-06

RT-402-8-35-DBTL-DelaysordamageoftrayintransporttoCERN

RiskRank: 1(Low)Scores:Probability:1(VL);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: IfatrayisdelayedordamagedintransportofBTLtoCERNthentheintegrationandcommissioningtaskmaybedelayed,withapotentialcost

increase.RiskType: Threat Owner: AdolfBornheimWBS: 402.8.3BTL-BarrelTimingLayer RiskArea: ManagementRisk/LogisticsProbability(P): 5% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =1-point-singlevalue

Minimum =k$Mostlikely =220k$Maximum =k$Mean =220k$P*<Impact> =11k$

ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =1monthsMaximum =monthsMean =1monthsP*<Impact> =0.1months

BasisofEstimate: Oneassemblysiteproduces2trayspermonth.Weplantoshipinboxesoftwotrayssincewewillmaketwosparetrays.Alossofoneboxcouldbecompensated.Thecostestimatecorrespondstothecomponentcostof1trayplusextralabortomakethetray.Afterintegrationisfinishedwewanttokeepatleastoneextratrayforfurthertestingandstudiesofthedetectorcomponentseg.intestbeamsoutsideoftheoperatingdetector.

CauseorTrigger: Lossofoneshipment.

ImpactedActivities: RiskcoulddelayanyoneofthebatchesduringBTLAssembl

Startdate: 10/Feb/2023 Enddate: 31/Jan/2024RiskMitigations: Scheduleshipmentsinsmallquantities-2perboxinourcase.RiskResponses: ThereissufficientbufferintheschedulesothatthelossononeboxwillhavenoimpactonthescheduleMoredetails:

Page 73: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 68 Updated: 2019-03-06

RT-402-8-36-DBTL-InterfacetoiCMSchanges

RiskRank: 1(Low)Scores:Probability:2(L);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: Ifthemechanical,thermalorserviceconnectionsoftheBTLtotherestofCMSchange,therespectivedesignshavetobeadjusted.RiskType: Threat Owner: AdolfBornheimWBS: 402.8.3BTL-BarrelTimingLayer RiskArea: TechnicalRisk/InterfacesProbability(P): 20% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =1-point-singlevalue

Minimum =k$Mostlikely =30k$Maximum =k$Mean =30k$P*<Impact> =6k$

ScheduleImpact: PDF =3-point-triangularMinimum =1monthsMostlikely =2monthsMaximum =3monthsMean =2monthsP*<Impact> =0.4months

BasisofEstimate: Ifeg.thedetailsoftheserviceroutingstoBTLgetchangedweneedtomodifytherespectivedesign.Thismaydelaystartofassemblytoallowadjustmentsontrayconnections.FortheintegrationintoCMSthismaycomplicateintegrationprocedure.Thecostimpactreflectsadditionaldesignworkandadditionallaborfortheassemblyprocedure.Scheduleimpactaccountsforpossibledelayinassemblyorneedtoaccelerateassemblytocompensatetimeittakestoimplementchanges.

CauseorTrigger: ChangeofservicesandmechanicalinterfacesbyCMSTRKorCMSTC

ImpactedActivities: RiskcoulddelaythecompletionoftheBTLAssemblyR&Dandprototyping

Startdate: 1/Apr/2019 Enddate: 3/Jan/2024RiskMitigations: Keepdesignflexibleenoughtoallowmodificationstoservicesandmechanicalinterfaces.Ensurepropercommunication,reviewofdesignsandtight

collaborationinparticularwiththeCMStrackergroup.RiskResponses: Adjustserviceandmechanicalinterfacedesign.MayhaveimpactonreadinessofcomponentsorcomplexitiyoftheassemblyprocessMoredetails:

Page 74: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 69 Updated: 2019-03-06

RT-402-8-42-DBTL-Problemswithmoduleassemblysite

RiskRank: 1(Low)Scores:Probability:2(L);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: Thisisageneralcategorythatincludesmultipleissuesthatcanleadtoaslowdownorstoppageattheintegrationcenter.Ifsuchaneventoccurs,a

delayisincurredwhiletheissueisbeingresolved.RiskType: Threat Owner: AdolfBornheimWBS: 402.8.3BTL-BarrelTimingLayer RiskArea: ExternalRisk/FacilitiesProbability(P): 10% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular

Minimum =10k$Mostlikely =20k$Maximum =30k$Mean =20k$P*<Impact> =2k$

ScheduleImpact: PDF =3-point-triangularMinimum =1monthsMostlikely =2monthsMaximum =3monthsMean =2monthsP*<Impact> =0.2months

BasisofEstimate: Ifinanyoftheproceduralstepsofthetrayintegrationaproblemoccurs,theassemblyprocedurewillbedelayed.Possibleproblemsarebreakageoftoolsortestequipment,accidentsinhandlingcomponents,qualityproblemseg.ingluingormatingofcomponentscausedbyexternalfactorssuchasimproperenviromentalconditionsortemporarydisruptioninthefunctinalityoftheintegrationcenterinfrastructure.

CauseorTrigger: Stopageofseriousslowdownofassemblyprocedure.

ImpactedActivities: RiskcoulddelayanyoneofthebatchesduringBTLAssembly

Startdate: 28/Jan/2022 Enddate: 31/Jan/2024RiskMitigations: Productionscheduleisdesignedwithsufficientfloattooffsetpossibledelays.Theuseofasecondproductioncentercanoffsetproductioncapacity

issuesinonecenter.RiskResponses: Hireadditionalunskilledlabortointegratetrayswithtwopeopleworkinginparallel.Moredetails:

Page 75: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 70 Updated: 2019-03-06

RT-402-8-43-DTL-SystemTesting-componentslateforsystemtest

RiskRank: 2(Medium)Scores:Probability:3(M);Cost:1(L)Schedule:2(M)) RiskStatus: OpenSummary: Componentsneededforintegrationintoalargersystemarenotavailableintimeforschedulebeamtests.Theresultsofthesystemtestmaybe

neededfortheEngineeringDesignReviews.Ifcomponentsaredelayedwewilleitherdelaythesystemtestorwillneedtohaveanadditionalsystemtestwhnethecomponentsareavailable.

RiskType: Threat Owner: LindseyA.GrayWBS: 402.8TL-TimingLayer(generalrisks) RiskArea: ExternalRisk/CollaboratorsProbability(P): 30% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular

Minimum =0k$Mostlikely =10k$Maximum =20k$Mean =10k$P*<Impact> =3k$

ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =4monthsMaximum =monthsMean =4monthsP*<Impact> =1.2months

BasisofEstimate: Ifwedelaythetestingwewouldnothaveanyadditionalcost,butmayincuradelayintheschedule.Ifweneedtoincludeanadditionaltestforwhenthecomponentisavailable,wewouldincuradditionalcostsandpossiblyascheduledelay.Probabilityisestimatedtobe30%(15%forBTLand15%forETL).

CauseorTrigger: Delayincomponentneededforasystemtest.

ImpactedActivities: TheresultsoftheBTLandETLsystemtestsmaybeneededduringprototypedevelopmentandasinputtotheEngineeringDesignReviews.

Startdate: 1/Jan/2019 Enddate: 1/Mar/2022RiskMitigations: Monitorthecomponentavailabilityandadjustthetestscheduleifneeded.RiskResponses: Ifacomponentisdelayedwecanstilldoapartialsystemtest.Wecanalsotestthecomponentseparatelyordelaythetestuntilthecomponentis

available.Moredetails:

Page 76: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 71 Updated: 2019-03-06

RT-402-8-44-DBTL-ConcentratorCardbatchshipmentlost/damaged/delayed

RiskRank: 1(Low)Scores:Probability:1(VL);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: Ifbatchislostordamaged,areplacementneedstobeprocuredthatwouldincreasetheoverallcostoftheproject.Adelayindeliverywouldresultin

thedelayofmeetingthedeadlines.RiskType: Threat Owner: YuriiYMaravinWBS: 402.8.3BTL-BarrelTimingLayer RiskArea: ExternalRisk/VendorsProbability(P): 5% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular

Minimum =0k$Mostlikely =3k$Maximum =9k$Mean =4k$P*<Impact> =0k$

ScheduleImpact: PDF =3-point-triangularMinimum =0monthsMostlikely =0.5monthsMaximum =1monthsMean =0.5monthsP*<Impact> =0.0months

BasisofEstimate: Incaseofacompletelossof22CCsinasingleshipment,theseboardsneedtobeprocured,withanestimateofabout1monthofproductiontime,estimatedfrompreviousexperienceofproducingbatchesofsimilarcomplexityforotherHEPexperiments.Thedelayisnotlikelytobesignificantifitisjustonebatchofmany,asthespeedofdeliveryofCCswilllikelyoutpacetheassemblyspeed.Giventhefactthatwewillproduce10%extracomponents,whichcorrespondstoabout71extraboards,alossofthreeshipmentswillbearzeroimpact.

CauseorTrigger: Lossoftheshipment ImpactedActivities: RiskcoulddelayanyoneofthebatchesduringBTLAssembly

Startdate: 15/Mar/2021 Enddate: 15/Jul/2022RiskMitigations: EnsurespeedydeliveryofalltheCCstoassemblycenters,sothatapotentiallossofashipmentwillnotaffectthetimingoftheassembly.Perform

fastQA/QCofthefirstbatchoftheCCstoestimatethefractionofthespareCCsthatwouldsatisfyqualityrequirementandcouldbeusedifashipmentbatchislost.

RiskResponses: AttempttorepairboardsthatfailQC/QAtestsifthefixesaresimple.Moredetails:

Page 77: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 72 Updated: 2019-03-06

RT-402-8-48-DETL-DelayindeliveryofpartsfromiCMS

RiskRank: 1(Low)Scores:Probability:2(L);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: Theassemblyofmodulesgetsdelayedifthecomponentsofpartsarenotdeliveredontime.ThismayjeopardizethedeliveryofETLmoduleson

time.RiskType: Threat Owner: SlawomirMTkaczykWBS: 402.8.4ETL-EndcapTimingLayer RiskArea: ExternalRisk/CollaboratorsProbability(P): 20% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular

Minimum =10k$Mostlikely =20k$Maximum =30k$Mean =20k$P*<Impact> =4k$

ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =1monthsMaximum =monthsMean =1monthsP*<Impact> =0.2months

BasisofEstimate: Wewillrecoverthescheduledelaybyaddingadditionalshiftswhichleadstoanimpactonthecosts.Weallowforapossible1monthdelay.CauseorTrigger: ImpactedActivities: Riskcoulddelayanyoneofthebatchesduring

ETLAssemblyStartdate: 1/Jan/2023 Enddate: 1/Oct/2024RiskMitigations: Theprototypeexpereinceshouldinformthemoduleassemblytimeestimate.RiskResponses: Ifthedeliveryofcomponentsisdelayedwewouldneedtoincreaseproductionbyaddingadditionalshifts.Moredetails:

Page 78: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 73 Updated: 2019-03-06

RT-402-8-49-DETL-DelaysordamageintransportofETLmodulestoCERN

RiskRank: 1(Low)Scores:Probability:1(VL);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: Ifmodulesaredamagedinshippingtheywillneedtobereplacedandthisriskmayimpacttheboththecostandscheduleandmayjeperdizethe

completionoftheprojectonbudgetandontime.RiskType: Threat Owner: SlawomirMTkaczykWBS: 402.8.4ETL-EndcapTimingLayer RiskArea: ManagementRisk/LogisticsProbability(P): 5% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =1-point-singlevalue

Minimum =k$Mostlikely =10k$Maximum =k$Mean =10k$P*<Impact> =1k$

ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =1monthsMaximum =monthsMean =1monthsP*<Impact> =0.1months

BasisofEstimate: CostestimatedforthelossofabatchofmodulestoCERNandtoincreasemoduleproductiontoreplacedamagedmodules.CauseorTrigger: ImpactedActivities: Riskcoulddelayanyoneofthebatchesduring

ETLIntegrationandCommissioningStartdate: 1/Jan/2023 Enddate: 1/Oct/2024RiskMitigations: Carefuldesignofmodulecarrierstoreducetheriskofdamage.Selectionofashippercompanyafterqualification.Quantityofmodulesinone

shipmentshouldnotexceedthenumebrofsparemodulesintheproject.RiskResponses: Increasemoduleproductiontoreplacethedamagedmodulesinordertoreducetheimpactontheschedule.Moredetails:

Page 79: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 74 Updated: 2019-03-06

RT-402-8-50-DETL-Moduleassemblyyieldislow

RiskRank: 1(Low)Scores:Probability:2(L);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: IftheyieldfromModuleAssemblyislowerthanexpected,thentheadditionalresourcesneededtocompensatefordelaysinprojectscheduleand

budget.RiskType: Threat Owner: SlawomirMTkaczykWBS: 402.8.4ETL-EndcapTimingLayer RiskArea: TechnicalRisk/QualityProbability(P): 10% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular

Minimum =0k$Mostlikely =5k$Maximum =15k$Mean =7k$P*<Impact> =1k$

ScheduleImpact: PDF =3-point-triangularMinimum =0monthsMostlikely =0monthsMaximum =1monthsMean =0.33monthsP*<Impact> =0.0months

BasisofEstimate: Minimum:nocostimpact,nodelay.Likely:nodelay.5k$isthecostofadditionallabourandadditionalcomponents.Maximum:15k$isthecostofadditionallabourandadditionalcomponents.1monthdelay:timetodiagnosetheproblemandimprovethemoduleassemblyprocedure.

CauseorTrigger: ImpactedActivities: RiskcoulddelayanyoneofthebatchesduringETLAssembly

Startdate: 1/Oct/2023 Enddate: 1/Oct/2024RiskMitigations: ExtensivemoduleprototypingandgoodunderstandingoftheassemblyprocessshouldmitigatetheriskofloweryieldsduringModuleassembly.

RigidQCshouldbeabletoidentifymoduleswithlowerqualityataproductionfacilityandtriggerastudyleadingtoundertandtheoriginoftheproblem.

RiskResponses: Additionalresourcesbothtocompensatefortheloweryieldandfurtherparallelizetheassemblyproceduretoregainschedulewouldbeneeded.Moredetails:

Page 80: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 75 Updated: 2019-03-06

RT-402-8-51-DETL-ProblemwithAlNvendor

RiskRank: 1(Low)Scores:Probability:1(VL);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: IfthereisaproblemwiththeAlNvendorthentherecouldbedelaysorcostincreases.Examplesofproblemsinclude:vendorgoingoutofbusiness,

vendordeliveringsubstandardcarrierAlN,orthevendorcannotdeliveraccordingtotheagreedschedule.RiskType: Threat Owner: SlawomirMTkaczykWBS: 402.8.4ETL-EndcapTimingLayer RiskArea: ExternalRisk/VendorsProbability(P): 5% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular

Minimum =0k$Mostlikely =15k$Maximum =30k$Mean =15k$P*<Impact> =1k$

ScheduleImpact: PDF =3-point-triangularMinimum =1monthsMostlikely =2monthsMaximum =3monthsMean =2monthsP*<Impact> =0.1months

BasisofEstimate: Minimumimpact:theproblemisrelativelymodestandissolvedbythevendorwithin1monthswithoutanydirectcostimpacttotheproject.Likelyimpacts:theproblemismoresignificantandrequiresadditionalprototypeswiththevendorresultinginadelayof2monthsandacostof$15kforAlNandmachining.Maximumimpact:ifweneedtoswitchvendorwewouldneedtoqualifyascondvendorresultingina3monthdelay.Thecostincreasecouldbe$30kforprototypingandsetupcostswiththenewvendor.

CauseorTrigger: ImpactedActivities: RiskcoulddelayanyoneofthebatchesduringETLAssembly

Startdate: 1/Oct/2022 Enddate: 1/Oct/2024RiskMitigations: PrototypingexperienceinthenexttwoyearsshouldresultinareliablecostestimateofthenecessaryAlNformoduleassemblies.

BuyAlNsupportstructuresasearlyaspossible.RiskResponses: Ifweneedtoswitchvendorwewouldneedtoqualifyasecondvendorresultinginamonthdelay.Moredetails:

Page 81: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 76 Updated: 2019-03-06

RT-402-8-52-DETL-ModuleRadiationTolerance

RiskRank: 1(Low)Scores:Probability:2(L);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: IfModulematerialssuchasadhesives,cables,fibers,etcaresusceptibletodegradationfromradiationexposure,thenresultingdelaystofindrobust

alternativesjeopardizetimelycompletionoftheproject.Thisisariskduringmoduleproductionaftertheprototypingqualificationofmodulecomponents.

RiskType: Threat Owner: SlawomirMTkaczykWBS: 402.8.4ETL-EndcapTimingLayer RiskArea: TechnicalRisk/ReliabilityorPerformanceProbability(P): 10% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =1-point-singlevalue

Minimum =k$Mostlikely =15k$Maximum =k$Mean =15k$P*<Impact> =2k$

ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =1monthsMaximum =monthsMean =1monthsP*<Impact> =0.1months

BasisofEstimate: IncreasedresourcesforlaborandinfrastructuretoparallelizedownstreamactivitiesinModuleAssemblytorecoupdelaysfromradiationeffectsdiscoveredlate.Estimated1monthdelaytorecoupthedelays.

CauseorTrigger: ImpactedActivities: RiskcoulddelayanyoneofthebatchesduringETLAssembly

Startdate: 1/Oct/2022 Enddate: 1/Oct/2024RiskMitigations: Irradiationofdummyandprototypemodulesshouldinformhowlikelyariskthisis.RiskResponses: IncreasedresourcesforlaborandinfrastructuretoparallelizedownstreamactivitiesinModuleAssemblytorecoupdelaysfromradiationeffects

discoveredlate.Moredetails:

Page 82: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 77 Updated: 2019-03-06

RT-402-8-53-DETL-IntegrationfacilityatCERNrunsoutofcomponents

RiskRank: 1(Low)Scores:Probability:3(M);Cost:1(L)Schedule:1(L)) RiskStatus: OpenSummary: IfETLcomponentdeliverytoCERNisdelayedthenthesubsequentdelayinintegrationjeopardizesthecompletionoftheprojectintimefor

scheduledstartofETLdatataking.RiskType: Threat Owner: SlawomirMTkaczykWBS: 402.8.4ETL-EndcapTimingLayer RiskArea: ExternalRisk/FacilitiesProbability(P): 25% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =1-point-singlevalue

Minimum =k$Mostlikely =21k$Maximum =k$Mean =21k$P*<Impact> =5k$

ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =3monthsMaximum =monthsMean =3monthsP*<Impact> =0.8months

BasisofEstimate: TheI&CactivitieswillbedelayedandadditionalCOLAsupportforUSpersonnelneedstobeprovided.Weassume4studentsand2postdocsneedCOLAsupportat1k$/student/monthand$1.5k/postdoc/month(totalCOLAcost=$7k/month).

CauseorTrigger: ComponentsforETLmoduleassemblyontheDees/wedgesaredelayed. ImpactedActivities: RiskcoulddelayanyoneofthebatchesduringETLIntegrationandCommissioning

Startdate: 1/Jan/2025 Enddate: 1/Jun/2026RiskMitigations: Frontloaddeliveryscheduleofcomponentstogetaheadofneedsofassemblyschedule.RiskResponses: USwillprovidepersonnelatCERNtoassistinintegrationandcommissioningoftheETLdetector.Moredetails:

Page 83: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 78 Updated: 2019-03-06

RT-402-8-90-DTL-KeyTimingLayerpersonnelneedtobereplaced

RiskRank: 2(Medium)Scores:Probability:3(M);Cost:2(M)Schedule:1(L)) RiskStatus: OpenSummary: Keyengineerorseniortechnicianwithspecialknowledgeleavestheprojectandneedstobereplaced.Ifthetransitioncanbemanagedsuchthattheincomingand

outgoingpersonneloverlapandexchangeknowledge,thenthereismainlyalaborcostimpact.Ifthetransitionismoreabrupt,thenthereisnocostimpactofoverlappingpersonnelbuttherecanbeadelaytotheprojectactivitiesastheincomingpersongetsfullyuptospeed.Thisriskdoesnotincludetheriskoflosingkeymanagersbecausetheprojectensuresthateachmanagerhasawell-traineddeputy.

RiskType: Threat Owner: ChristopherCarlNeuWBS: 402.8TL-TimingLayer(generalrisks) RiskArea: ManagementRisk/FundingorResourcesProbability(P): 25% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular

Minimum =45k$Mostlikely =135k$Maximum =261k$Mean =147k$P*<Impact> =37k$

ScheduleImpact: PDF =3-point-triangularMinimum =0monthsMostlikely =0monthsMaximum =3monthsMean =1monthsP*<Impact> =0.3months

BasisofEstimate: ThisL2areahas3seniorengineerswhosefunctionsareparticularlyhardtoreplaceonashorttimescalebyexistingteammembers.Theyworkinthefollowingareas:(1)SiPMR&Dandqualityassurance;(2)EEforBTLCC;and(3)ASICengineering.ExperienceintheCMSPhase1Upgradeprojectsuggeststheprobabilityperkeypersonisroughly25%(probabilitytheyleaveduringtheentireproject).Someinputnumberstothecostimpactanalysis:--Laborcostperperson=$15k/month(fullyburdenedseniorenginer=$185k/year).--AverageburnrateperL3areais=$14k/month(CMS-doc-13481).Minimumscenerio:personleavingisreplacedbyanexistingskilledpersonintheteam.1monthoffulloverlapcosting$15koflabor.Noscheduledelay.Likelyscenario:personleavingisreplacedbyapersonlessfamiliarwiththespecificwork.3monthsmanagedoverlapoftheoutgoingandincomingpeoplecosts3*$15k=$45koflabor.Noscheduledelay.Maximumscenario:personleavesunexpectedlywithnotransferofknowledge.Ittakesabout6monthstofindanewpersonandgetthemfullyuptospeed.Thereislossofproductivityduringthisdifficulttransitionperiodresultinginanet3monthdelaytoL3activities.Costoframp-upeffortofthenewperson(learningbutnotfullycontributingtodeliverables)=3FTE-months*$15k=$45k.Theburnratecostis3*$14k=$42k.Totalcostimpact=$87kTotalforallkeypersons:Min/Likely/Maxcostimpact=3*$(15/45/87)k=$(45/135/261)$.

CauseorTrigger: ImpactedActivities: EachkeypersonsisindependentlymodeledintheMC(eachatP=25%andwithcorrespondingshareofthetotalcostimpact).Typicallyhooktherisktothepersonsmainactivityduringtheproductionphase(maximumconsequences).

Startdate: 1/Jan/2018 Enddate: RiskMitigations: Anumberofengineersandtechnicianshaveoverlappingskills,bothwithinagiveninstituteandacrossUS-CMSinstitutes.Thisprovidessomebackupincaseof

non-availabilityofakeyperson.Pro-activecross-trainingofengineersandtechnicianshelpsensurekeyskillsarenotcompletelylostifakeypersonisnolongeravailable.

RiskResponses: Aimforoutgoingandincomingpersonneltooverlapifpossibletoensureasmoothtransition.Hirefromskilledmembersoftheteamifappropriate.Interimsupportmaybepossibleusingengineersortechnicainsfromelsewhereintheinstituteorproject.

Moredetails: CMS-doc-13481

Page 84: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 79 Updated: 2019-03-06

RT-402-8-91-DTL-ShortfallinTimingLayerscientificlabor

RiskRank: 2(Medium)Scores:Probability:3(M);Cost:2(M)Schedule:0(N)) RiskStatus: OpenSummary: Ifasignificantamountofthe(uncosted)scientificlaborisunavailable,thentheprojectwouldthenneedtofundadditional(costed)personnelto

performthework.Itisassumedthattheriskistriggeredbyaseriouslyunfavorableoverallbaseprogramfundingsituation.RiskType: Threat Owner: ChristopherCarlNeuWBS: 402.8TL-TimingLayer(generalrisks) RiskArea: ManagementRisk/FundingorResourcesProbability(P): 30% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular

Minimum =0k$Mostlikely =0k$Maximum =421k$Mean =140k$P*<Impact> =42k$

ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =0monthsMaximum =monthsMean =0monthsP*<Impact> =0.0months

BasisofEstimate: InthepastUS-CMShasnotexperiencedasignificantlackofscientificlabor(postdocsandgraduatestudents).WhenshortfallsoccuredtheywereusuallyresolvedbycollaboratorsatUS-CMSinstitutesorsometimesfromiCMS.Weassigna30%probabilitythatasignificantshortfalloccursinfuture,duetounfavorablefundingconditionsinthebaseprogram.ThecontributedlaborforthescopeofthisL2areais24.4FTE-yearsspreadoverabout5years.ThisdoesnotincludethemoresecureL2andL3managerswhoaretenuredfacultyandseniorFermilabscientists.Weestimatethelosscouldbeupto20%ofthetotalcontributedlaboror4.9FTE-years(e.g.thisisalossof50%ofallcontributedlaborforatwoyearperiodoralossof1/3ofthecontributedlaborfor3years).Themissinglaborcouldbereplacedbycostedpersonnel:amixtureofmid-rangetechnicians,juniortechnicians,orundergraduatescostingrespectively62$/hr,50$/hrand18$/hrfully-burdened(43$/houronaverage).Allowingforfouryearsofescalationat3.1%perannumyieldsanaveragecostof49$/hror86k$/FTE-year(1768hrsworkedperyear).The(min/likely/max)costimpactistherefore:86k$perFTE-year*(0/0/4.9)FTE-years=$(0/0/421)k.

CauseorTrigger: Theriskistriggeredbyanunfavorablebaseprogramfundingsituation. ImpactedActivities: Startdate: 1/Oct/2018 Enddate: RiskMitigations: Workwithinstitutesandagenciestoensuretheanticipatedamountofscientificlaborwillbeavailable.Whereshortfallslooklikelytooccur,seek

alternativesamongstotherUS-CMSinstitutesorevenfromiCMSinstitutes.RiskResponses: Seekreplacementscientificlaborinotherinstitutes.Ifthislaborcannotbefoundthencontingencywillneedtobespenttosupplementtheeffort

withcostedlabor(e.g.technicians).Moredetails:

Page 85: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 80 Updated: 2019-03-06

RU-402-1-01-DPM-Foreignexchangeratesareuncertain(DOE)

RiskRank: 3(High)Scores:Probability:5(VH);Cost:3(H)Schedule:0(N)) RiskStatus: OpenSummary: Futureexchangeratesaremoreorlessfavorablethanthecanonicalvalueinthebaselineplanresultinginachangeinthecosttotheproject.RiskType: Uncertainty Owner: VivianO'DellWBS: 402.1PM-ProjectManagement(DOE) RiskArea: ExternalRisk/MarketProbability(P): 100% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular

Minimum =-7260k$Mostlikely =1780k$Maximum =10420k$Mean =1647k$P*<Impact> =1647k$

ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =0monthsMaximum =monthsMean =0monthsP*<Impact> =0.0months

BasisofEstimate: Thecostimpactparametersaretheresultofadetailedscenarioanalysis.Historicalexchangeratedatawereusedtoreplaytypicalratevariations(scenarios)andhencemodelpotentialfuturechanges(typicalpercentagechangesfromquartertoquarter--notabsoluterates),asdescribedinCMS-doc-13481andcalculatedinCMS-doc-11825.Theriskuncertaintyisappliedtotheamountofforeigncostsperfiscalyear(determinedfromP6)throughouttheprojectallowingforthefactthattheuncertaintyincreasesfurtherintothefuture.

CauseorTrigger: ImpactedActivities: Startdate: 1/Jan/2018 Enddate: RiskMitigations: Theexchangerateuncertaintyincreasesthefurtherinthefuturethecostisincurred.Thereforetheprojectwillseektoadvanceforeignexchange

transactionswherepossibleandconsistentwiththeavailabilityoffunds.Forexample,itmaybepossibletofront-loadthepaymentoftheCMScommonfund(inSwissFrancs)orprocuresilicon(inYen)earlierthanisabsolutelyneeded(ontechnicalgrounds).

RiskResponses: Ifthecostimpactsareacceptabletheriskresponseistousecontingencytocovertheincreasedcosts.Forextremeexchangeratechangesthatcannotbecoveredbycontingency,workwiththeagenciestoaddresstheissue.Inextremis,donotcompletetheobjectiveKPPsifthereareinsufficientfund.

Moredetails: CMS-doc-13481

Page 86: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 81 Updated: 2019-03-06

RU-402-1-02-DPM-Futureescalationratesareuncertain(DOE)

RiskRank: 3(High)Scores:Probability:5(VH);Cost:3(H)Schedule:0(N)) RiskStatus: OpenSummary: ThecostestimatesintheBasisofEstimate(BoE)documentsareexpressedintermsofthebase-yeardollarwhentheestimatewasmade(typically

FY17).Forbudgetingpurposes[GAO-09-3SP,page102],projectedescalationratesarethenappliedtoescalatethebase-yeardollarstothen-yeardollarsandhencereflecttheexpectedcostintheyearinwhichitwillbeincurred.Ifactualinflationinfuturediffersfromtheassumedescalationrates,thentheprojectcostmayincreaseordecrease.

RiskType: Uncertainty Owner: VivianO'DellWBS: 402.1PM-ProjectManagement(DOE) RiskArea: ExternalRisk/MarketProbability(P): 100% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular

Minimum =-1850k$Mostlikely =1290k$Maximum =2870k$Mean =770k$P*<Impact> =770k$

ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =0monthsMaximum =monthsMean =0monthsP*<Impact> =0.0months

BasisofEstimate: Thecostimpactisestimatedbyapplyingvarioussetsofhistoricalescalationratetimeprofiles(seperatelyforbothM&Sandlabor)totheprojectcostprofiletoseehowmuchisthecostimpactcomparedtothedefaultassumptioninthebaseplan.Themainanalysisisdescribedin[CMS-doc-13481].Thehistoricaldatasetsassessedare[CMS-doc-12866]:--generalinflation(CPI);--theUSBureauofLaborStatistics(BLS)ProducerPriceIndex(PPI)forallmanufacturingindustries;--theBLSEmploymentCostIndexforUniversityLaborcosts;and--asampleofactualandprojectlaborratesfromUS-CMSinstitutes.

CauseorTrigger: ImpactedActivities: Startdate: 1/Jan/2018 Enddate: RiskMitigations: Monitoractualescalationannuallyandupdateprojectplanaccordingly.RiskResponses: Ifthecostimpactsareacceptabletheriskresponseistousecontingencytocovertheincreasedcosts.Forextremeinflationratechangesthat

cannotbecoveredbycontingency,workwiththeagenciestoaddresstheissue.Ifneedbe,descopeoptions(thathavealreadybeenidentified)mayneedtobeexercisedtoensuretheoverallprojectcostremainswithintheagencyguidance.

Moredetails: CMS-doc-13481

Page 87: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 82 Updated: 2019-03-06

RU-402-1-03-DPM-FutureFermilaboverheadratesareuncertain(DOE)

RiskRank: 3(High)Scores:Probability:5(VH);Cost:3(H)Schedule:0(N)) RiskStatus: OpenSummary: Fermilabchargesindirectcostsonfundsspentatthelabandalso--subjecttothresholdsandatdifferentrates--onfundsthataretransferredtoUS-

CMSinstitutesthroughsub-awards.Theoverheadratesareadjustedeachyearandmaybecomemoreorlessfavorablecomparedtothecanonicalvalueinthebaselineplan.Note:giventhenumberofUS-CMSinstitutesinvolved,weassumethatincreasesanddecreasesintheirratesroughlycancelonaverage.Inaddition,institutesgenerallydonotchargeanyindirectcostsforcapitalM&Sexpenditures(e.g.largeprocurementsofequipment).

RiskType: Uncertainty Owner: VivianO'DellWBS: 402.1PM-ProjectManagement(DOE) RiskArea: ManagementRisk/FundingorResourcesProbability(P): 100% TechnicalImpact: 0(N)-negligibletechnicalimpactCostImpact: PDF =3-point-triangular

Minimum =-2050k$Mostlikely =-100k$Maximum =2200k$Mean =17k$P*<Impact> =17k$

ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =0monthsMaximum =monthsMean =0monthsP*<Impact> =0.0months

BasisofEstimate: SinceFY08,theFermilabLaborandM&Sstandardrateshavevariedbytypically3.6%peryearforMIEfunds.Inthisperiodtheratestrendedsystematicallyupwardsbutwedonotbelievesuchatrendwillcontinueunabated.Infuturetheremightevenbeareversiontoalowerlong-termmean.Infutureyears,wethereforeassumethattheindirectratescouldincrease,decreaseorremainthesamewithapproximatelyequalprobabilities.TomodeltheimpactsofthisweconstructatoyMC.TheProjectspendsabout$57.8MatFermilab(directcosts+indirectcosts+escalation+estimateuncertainty).Wetaketheprofileofthesecostsandassessthecostimpactiftheindirectratesineachfutureyeararevariedrandomlybybetween±3.6%comparedtothepreviousyear.Thiscoversarangeofscenariosinwhichratescantrendupwards,trenddownwards,orfluctuaterandomlyfromyeartoyear.Themin/likely/maxcostimpactsaredeterminedfromthistoyMC.See:CMS-doc-13501andCMS-doc-13481.

CauseorTrigger: ImpactedActivities: Startdate: 1/Oct/2016 Enddate: 30/Sep/2025RiskMitigations: RiskResponses: Updateprojectplanstoaccountforindirectratechangesatleastannuallyandcarefullymonitorchangesintheremainingcontingency.

Moredetails: CMS-doc-13501(seealsoCMS-doc-13481)

Page 88: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 83 Updated: 2019-03-06

RU-402-2-01-DOT-UncertainperformanceofHybridsvendor

RiskRank: 3(High)Scores:Probability:5(VH);Cost:2(M)Schedule:3(H)) RiskStatus: OpenSummary: Thereisuncertaintyintheperformanceofthevendorofhybridsformoduleswhichcouldhavearangeofimpactsfromnegligibletosignificantcost

impactanddelay.

RiskType: Uncertainty Owner: LeonardGSpiegelWBS: 402.2OT-OuterTracker RiskArea: ExternalRisk/VendorsProbability(P): 100% TechnicalImpact: 1(L)-somewhatsubstandardCostImpact: PDF =3-point-triangular

Minimum =0k$Mostlikely =168k$Maximum =648k$Mean =272k$P*<Impact> =272k$

ScheduleImpact: PDF =3-point-triangularMinimum =0monthsMostlikely =2monthsMaximum =12monthsMean =4.67monthsP*<Impact> =4.7months

BasisofEstimate: Minimumimpact:noextracostnordelayLikelyimpact:issueswiththevendorcausingadelayatdeliveryof2months.Useamoduleassemblylaborburnrateof$54k/monthwithoutescallation.Maximumimpact:weneedtofindanalternativevendororsignificantreworkisneededattheselectedvendor.Thisimpliesacostimpactof$648k(12*54k/month).

CauseorTrigger: ImpactedActivities: Moduleassembly.Thisriskshouldbeimplementedinserieswiththemoduleriskonreceivingthemechanicalparts.

Startdate: 1/Apr/2020 Enddate: 31/Dec/2024RiskMitigations: TheprototypingexperienceshouldinformthehybridfabricationtimeestimatesRiskResponses: IncreasedresourcesforlaborandinfrastructuretoparallelizedownstreamactivitiesinModuleAssemblyandPlank/LayerAssemblytorecoup

delaysfromHybridsMoredetails: CMS-doc-13481

Page 89: Risk Register - Fermilab

Risk Register for the HL-LHC CMS Detector Upgrades Project

Lucas Taylor Page 84 Updated: 2019-03-06

RU-402-6-07-DTD-DAQSTMSI/Operformancedoesnotmeetrequirements(DOE)

RiskRank: 2(Medium)Scores:Probability:2(L);Cost:2(M)Schedule:0(N)) RiskStatus: OpenSummary: TheI/OrequirementsoftheDAQSTMScouldchangeinresponsetochangesineventsizeorHLToutputeventrate.Thisevent,inturn,

requiresmore(orless)STMSresourcesinresponse.RiskType: Uncertainty Owner: JeffreyWBerryhillWBS: 402.6TD-TriggerandDAQ(DOE) RiskArea: TechnicalRisk/RequirementsProbability(P): 20% TechnicalImpact: 2(M)-significantlysubstandardCostImpact: PDF =3-point-triangular

Minimum =-192k$Mostlikely =384k$Maximum =768k$Mean =320k$P*<Impact> =64k$

ScheduleImpact: PDF =1-point-singlevalueMinimum =monthsMostlikely =0monthsMaximum =monthsMean =0monthsP*<Impact> =0.0months

BasisofEstimate: PurchasecostofDAQSTMS(768k$)scaleddown/upby-25%/50%/100%CauseorTrigger: SpecifyingHLToutputeventrateoreventsizedifferentfromTDRvalues. ImpactedActivities: ProcurementcostofDAQSTMSStartdate: 1/Oct/2018 Enddate: 3/Jun/2025RiskMitigations: OngoingevaluationofDAQconfigurationinresponsetoHLTdesignandT0computing.RiskResponses: NegotiationwithCMSDAQonre-configurationofprojecttomeetstartupgoals.Moredetails: