applying the navigation guide systematic review ... · 2 protocol information review authors juleen...

1

ApplyingtheNavigationGuideSystematicReviewMethodologyCaseStudy#6

AssociationbetweenFormaldehydeExposureand

Asthma

ASystematicReviewoftheEvidenceProtocolApril2016

2

PROTOCOLINFORMATIONReviewAuthorsJuleenLam,PhD,MHS,MSAssociateResearchScientistDepartmentofObstetrics,Gynecology&ReproductiveSciencesProgramonReproductiveHealthandtheEnvironmentUniversityofCalifornia,SanFranciscoPatriceSutton,MPHAcademicCoordinatorDepartmentofObstetrics,Gynecology&ReproductiveSciencesProgramonReproductiveHealthandtheEnvironmentUniversityofCalifornia,SanFranciscoAmyM.Padula,PhD,MScNovember12015–February29,2016: Instructor Dept.ofPediatrics,DivisionofNeonatology StanfordUniversity March1-October31,2016:AssistantProfessorDepartmentofObstetrics,Gynecology&ReproductiveSciencesProgramonReproductiveHealthandtheEnvironmentUniversityofCalifornia,SanFranciscoMichaelD.Cabana,MD,MPHProfessorofPediatrics,Epidemiology&BiostatisticsPhilipR.LeeInstituteforHealthPolicyStudiesUniversityofCalifornia,SanFranciscoEricaKoustas,PhDScientificconsultanttoUCSFHannaVesterinen,PhDScientificconsultanttoUCSFEvansWhitaker,MD,MLISEducationConsultantfortheSchoolsofMedicineandPharmacyUniversityofCalifornia,SanFranciscoLesleySkalla,PhD,MSLSInformationSpecialistMDB,Inc.NatalynDanielsResearchAssistantDepartmentofObstetrics,Gynecology&ReproductiveSciencesProgramonReproductiveHealthandtheEnvironmentUniversityofCalifornia,SanFrancisco

3

TraceyJ.Woodruff,PhD,MPHProfessorandDirectorDepartmentofObstetrics,Gynecology&ReproductiveSciencesProgramonReproductiveHealthandtheEnvironmentUniversityofCalifornia,SanFranciscoContact:JuleenLamUniversityofCaliforniaatSanFranciscoDepartmentofOB/GYN&RSMailStop0132,55016thSt,7thFloor,SanFrancisco,[email protected];(415)476-3219

4

TABLEOFCONTENTS

PROTOCOLINFORMATION...............................................................................................................................2

BACKGROUND.......................................................................................................................................................5NavigationGuideSystematicReviewMethodology............................................................................................5RationaleforReview:FormaldehydeandAsthma................................................................................................6

AIM...........................................................................................................................................................................8

METHODS...............................................................................................................................................................8ReviewTeam...........................................................................................................................................................................8CriteriaforSelectingStudies..........................................................................................................................................9PECOStatement...................................................................................................................................................................9SearchMethods...................................................................................................................................................................10StudySelectionCriteria...................................................................................................................................................14DataCollection....................................................................................................................................................................17RiskofBiasDetermination............................................................................................................................................17DataAnalysis.......................................................................................................................................................................18QualityandStrengthofEvidenceRatings.............................................................................................................20

SUPPLEMENTARYINFORMATION................................................................................................................22AppendixI.Coauthor/LibrarianBiosketches,ProjectContributionStatements,andConflictofInterestStatements...........................................................................................................................................................22AppendixII.SearchTerms.............................................................................................................................................68AppendixIII.OtherResourcesforLiteratureSearch........................................................................................73

APPENDIXIV.EXCLUSIONCRITERIASCREENINGFORMSANDAMENDMENTSTOCLARIFYSCREENING

PROCESS...................................................................................................................................................................................75AppendixV.DataCollectionForms..........................................................................................................................78AppendixVI.InstructionsforMakingRiskofBiasDeterminations.........................................................82AppendixVII.InstructionsforGradingtheQualityandStrengthofEvidence.........................................96

ABOUTTHEARTICLE.....................................................................................................................................109

ACKNOWLEDGEMENTS.................................................................................................................................109

REFERENCES.....................................................................................................................................................110

5

BACKGROUND

NavigationGuideSystematicReviewMethodology

Robustmethodstoevaluateavailablescientificevidencetoreachconclusionsregardingthestrengthofevidencearefundamentaltospeedingthetranslationofthescienceintopoliciesanddecisionstoimprovehealthoutcomes.Intheclinicalsciences,methodsofevidenceintegrationhaveplayedatransformativeroleinthetimelyincorporationofscienceintotherapeutic,preventiveandcosteffectiveactionattheindividualandsocietallevel(Fox2010).Beginningin2009,researchersbegantoexploretheapplicationofsystematicandrobustmethodsofevidenceintegrationinenvironmentalhealthsciences(Woodruffetal.2011,Rooneyetal.2014).In2014,tworeportsbytheNationalAcademyofSciences(NAS)stronglyendorsedtheuptakeofsuchimprovedmethodsofevidenceintegrationinenvironmentalhealthsciences,andspecificallyencouragedtheirusebytheU.S.EnvironmentalProtectionAgency(EPA)indeterminationsofwhetherenvironmentalchemicalsareharmfultohumanhealth(NationalResearchCouncil2014,2014).TheUSEPAisworkingtoinitiatestepstoincorporateprinciplesofsystematicreviewintoitsIRISprocess(NationalResearchCouncil2014,USEnvironmentalProtectionAgency2014),whiletheNationalInstituteforEnvironmentalHealthSciences’(NIEHS)NationalToxicologyProgram(NTP)OfficeofHealthAssessmentandTranslation(OHAT)hasbeendevelopingthetools,expertise,casestudies,andotherinfrastructurethatwillfacilitateincreasedutilizationofsystematicreviewmethodologies(Rooneyetal.2014,NationalToxicologyProgram2015).

TheNavigationGuidesystematicreviewmethodology(NavigationGuide)wasdevelopedin2011aspartofaninterdisciplinarycollaborationbetweenclinicians,academicians,andpractitionersinanattempttoharmonizetheapproachesforassessingevidenceintheclinicalscienceswithenvironmentalhealthsciences(Woodruffetal.2011).TheNavigationGuideisasystematicandtransparentmethodologythatproceedsfrombestpracticesintheclinicalarena,i.e.,Cochrane(HigginsandGreen2011)andtheGradingofRecommendationsAssessmentDevelopmentandEvaluation(GRADEWorkingGroup2014),butaccountsforthedifferencesinevidenceanddecisioncontextinvolvedinenvironmentalhealthriskassessments,suchastherelianceonanimaltoxicologyandhumanobservationalstudiesintheabsenceofrandomizedcontrolledtrials(RCTs)(Mandrioli2016).Todate,theNavigationGuidehasbeenappliedinfiveproof-of-conceptstudies:1.Toevaluatethehumanandnon-humanevidenceofperfluorooctanoicacid(PFOA)onfetalgrowth(Johnsonetal.2014,Koustasetal.2014,Lametal.2014).FromthisapplicationoftheNavigationGuide,reviewauthorsconcludedthatPFOAwas“knowntobetoxic”tohumanreproductionanddevelopment,basedonafindingof“moderate”qualityand“sufficient”strengthofboththehumanandnon-humanmammalianevidence.

6

2.Toevaluatethehumanandnon-humanevidenceoftheassociationbetweenfetalgrowthandglomerularfiltrationrate(GFR)inordertoassessthestrengthoftheevidenceofa‘reversecausality’hypothesis:thatthesizeofadevelopingfetusmayaffectmaternalGFRsuchthatasmallfetusleadstoreducedplasmavolumeexpansion,reducedGFR,andsubsequentlyhigherconcentrationsofbiomarkersinmaternalserum.Thishadbeenproposedasapotentialalternateexplanationforobservationalstudiesdocumentinganinverseassociationbetweenprenatalexposuretochemicalsclearedrenallyandfetalgrowth(Savitz2007,Whitworthetal.2012,Loccisanoetal.2013).Theauthorsofthisreviewfoundinsufficientevidencetosupporttheplausibilityofthereversecausalityhypothesisandrecommendedfurtherhighqualityresearch(Vesterinenetal.2014).

3.Toevaluatethehumanandnon-humanevidenceoftriclosanonreproductiveand/ordevelopmentaltoxicity.Thisreviewhasbeencompletedandthemanuscriptiscurrentlyinpress(Johnsonetal.2015).

4.Toevaluatethehumanevidenceoftherelationshipbetweenairpollutionandautismspectrumdisorder.Thiscasestudyiscompletedandthemanuscriptiscurrentlybeingdraftedforpublication(Lametal.2015).

5.ToevaluatethehumanevidenceoftherelationshipbetweenPBDEexposureandintelligenceorattention-relatedbehaviorconditions.Thiscasestudyiscompletedandthemanuscriptiscurrentlybeingdraftedforpublication(Lametal.2015).

TheresultsofthesecasestudiestodatedemonstratethatthemethodsunderdevelopmentbytheUSEPAandtheNTParefullyachievable(Johnsonetal.2014,Koustasetal.2014,Lametal.2014,WoodruffandSutton2014).

This6thcasestudyoftheNavigationGuidesystematicreviewmethodinenvironmentalhealthwillassessthehumanevidenceforeffectsofexposuretoformaldehydeandasthma-relatedoutcomes.ThehumanhealthrationaleforthisreviewrelatestotherecentattentiontopopulationexposuretoformaldehydebytheUSEPAthroughitsattemptstoregulateexposuretoformaldehydewithitsauthorityundertheToxicSubstancesControlAct(TSCA).

RationaleforReview:FormaldehydeandAsthmaFormaldehydeisthesimplestofallaldehydesandexistsatroomtemperatureasanearlycolorlessgaswithapungent,suffocatingodor(AgencyforToxicSubstancesandDiseaseRegistry1999).Ithasnumerousindustrialandcommercialusesasasolutionortoproduceindustrialresinsinturnusedtomanufactureadhesivesandbindersinnumerouscommercialproducts(NationalToxicologyProgram2010).FormaldehydewasrecognizedasaknownhumancarcinogenbytheInternationalAgencyforResearchonCancer(IARC)in2004andbytheNationalToxicologyProgram(NTP)withintheNationalInstitutesforEnvironmentalHealth(NIEHS)in2011(InternationalAgencyforResearchonCancer2006,NationalToxicologyProgram2010).Formaldehydeisacommonindoorairpollutantfoundinvirtuallyallhomesandbuildings.Itis

7

presentinmanyhouseholdproducts,suchasfoaminsulationandcleaningandpersonalcareproductsand,mostcommonly,intheformofpressedwoodproductssuchasparticleboardandplywood(NationalToxicologyProgram2010).Low-incomepopulationsaredisproportionatelyatriskofexposuretoformaldehydefromcompositewoodproductsinhomesbuiltusingcheaperbuildingmaterials.However,newerhomesgenerallyhavebeenfoundtoraiseconcernsregardinglevelsofformaldehydeoffgassedfromnewhousingmaterials,withavailabilityandratesofventilationhavingminimalimpactonexposurelevels(ParkandIkeda2006).Furthermore,formaldehydeexposurecanalsobeofconcerninbuildingsotherthanresidentialhomes—forinstance,formaldehydehasbeenmeasuredatlevelsexceedingexposurelimitsinchildcaresettingsinCalifornia(Bradmanetal.2016),thusindicatingthatwidespreadexposureinmanysettingsispotentiallyofconcern.Currentlythereisnoregulationofthelevelofformaldehydeallowedincompositewoodproducts.Evaluatingthescienceforhealthimpactsofformaldehydewillprovidetheevidencebaseformakingdecisionsaboutitsuseinbuildingproductsandassuchwillhelptoprotectthehealthofvulnerablepopulations.

Therelationshipbetweenasthmaandexposuretoformaldehydehasbeenunderevaluationforatleast25years.In1990,USEPAreleaseditsIntegratedRiskInformationSystem(IRIS)non-cancerassessmentofformaldehyde;in1998theAgencybeganareassessmentofthetopic;andinJune2010USEPAreleasedadraftIRISassessment.(NationalResearchCouncil2011)In2011theNationalAcademyofSciencespublishedanindependentreviewoftheUSEPA’sIRISassessmentofformaldehyde,to“providecandidandcriticalcommentsthatwillassisttheinstitution…[in]makingits[USEPA’s]publishedreportassoundaspossibleandtoensurethatthereportmeetsinstitutionalstandardsofobjectivity,evidence,andresponsivenesstothestudycharge”(NationalResearchCouncil2011).

Ingeneral,theNAS’reviewoftheIRISformaldehydeassessmentwashighlycriticalofUSEPA’snon-systematic,descriptive,andunspecifiedformatforstudyevaluation;intermsofasthma,theNASwasadditionallycriticalofthefactthatUSEPAdidnotanalyzetheevidenceaccordingtospecificasthmaoutcomes,i.e.,incidentasthma(theoccurrenceofnewcases),prevalentasthma(thepresenceofasthmaatthetimeofstudy),orexacerbationofestablishedasthma(NationalResearchCouncil2011).YettheUSEPAdidnotdirectlyaddressthesecriticismsinitsformaldehydeandcompositewoodproductsrulemaking.Rather,theAgencyjudgeda2010meta-analysisofformaldehyde-associatedchildhoodasthmaconductedbyMcGwinetal.(McGwinetal.2011)tobethemostrelevantstudytouseinitsbenefitsassessmentforasthma.Itappearsthatbasedonthedocumentedchangesintherule’scostbenefitanalysisbetween2012and2013,theOfficeofManagementandBudget(OMB)lackedconfidenceinUSEPA’suseoftheMcGwinmeta-analysisofasthmaandformaldehyde.

ThissystematicreviewoftheevidencelinkingexposuretoformaldehydeandasthmawillbeconductedusingtheNavigationGuide,amethodologythatspecificallyaccountsfortheweaknessesidentifiedbytheNASintheIRISformaldehydeassessment.

8

Aim

StudyQuestion

Ouraimistoanswerthequestion:“Isexposuretoformaldehydeassociatedwithdiagnosis,signs,symptoms,exacerbation,orothermeasuresofasthmainhumans?”

Objectives:

• Identifystudiesorexperimentsconductedinhumansconcerningtheassociationofexposuretoformaldehydewithasthmaoutcomes;

• Evaluatetheevidenceforaneffectacrossstudiesandifappropriate,conductameta-analysisoftheeffectsofexposuretoformaldehydeandasthma,andassessforpotentialsourcesofheterogeneity;

• Assesstheriskofbiasofindividualstudiesand,whereappropriate,assesstheirimpact(includingdirection)onmeasuresofestimatedeffectsize;

• Ratethequalityoftheoverallbodyofhumanevidence;and• Ratethestrengthofthehumanevidenceontheeffectofexposuretoformaldehydeon

asthmaoutcomesaccordingtooneofthefollowingfourstatements:1.Sufficientevidenceoftoxicity;2.Limitedevidenceoftoxicity;3.Inadequateevidenceoftoxicity;or4.Evidenceoflackoftoxicity.

Methods

ReviewTeamAtthebeginningofthecasestudy,UCSFwillassembleareviewteamconsistingofexpertsfromavarietyofresearchfieldsrelevanttothestudyquestionathand(i.e.,epidemiology,clinicalsciences,specificallyasthmaoutcomeassessment,exposureassessment,biostatistics,librarysciences,andsystematicreviewmethodology).Thefirstauthor(JL),ProjectDirector(PS)andseniorauthor(TW)willcollectivelyidentifypotentialreviewteammembersbasedontheirresearchinterests,expertise,availability,capacitytomeetprojectdeadlines,andtheabsenceofanyrealorpotentialconflictofinterestandinvitereviewteammembers.

Thelistofcoauthors,theirbiographicalsketches,proposedcontributionstotheproject,andacompletedconflictofinterestformaredocumentedinAppendixI.Specificrolesandresponsibilitiesforreviewauthorswillbedocumentedthroughouttheprotocol,i.e.,thedesignandconductofthesearch,applyinginclusion/exclusioncriteria,assessingriskofbiasforincludedstudies,dataextractiondataanalysis,andratingthequalityandstrengthoftheevidence.Theconductofthecasestudy,itsconclusionsandpublicationsarethesoleresponsibilityofthereviewteammembers.

9

Allreviewteammemberswillactivelyparticipateindevelopingtheprotocolandreviewingandapprovingthefinalmanuscript.Intheeventthatamemberofthereviewteamwasacoauthorofastudyunderreview,thatmembermustrecusethemselvesfromtheevaluatingthequalityandstrengthofthatstudy.

Throughoutthecourseofthereviewwewillalsoengagetopicexpertswithabroadersetofinterestsandexpertise.Topicexpertswillprovideconsultationasneeded.Wewilldocumentandacknowledgethecontributionofallindividualswhoparticipatedastopicexperts.Thecontributionoftopicexpertsislimitedtoadvisingthereviewteamanddoesnotconstituteauthorshiporagreementordisagreementwiththereviewteam’sfindings.

CriteriaforSelectingStudiesStudiesthatareeligibleforreviewwilladdressthestudyquestionaboveandthefollowingcharacteristicsasoutlinedinthefollowing“PECO”aid.

PECOStatement

“PECO”isanaidusedtoformulateananswerablequestioninasystematicreviewofhealthstudies.Theacronymstandsfor“Population/Participants”,“Exposure,”“Comparator”and“Outcomes.”

Population:Humans

Exposure:Anyindoororoutdoorsourcesofairborneinhalationexposuretoformaldehyde,includingbutnotlimitedtooccupational,outdoorambient,indoorhouseholdsettings,and/orexposuretohouseholdproductsthatoccurredpriororconcurrenttodiagnosis,exacerbation,orothermeasureofasthma.Exposurepriortodiagnosis,exacerbation,orothermeasureofasthmaisdefinedasmaternal/paternalpriortoconception,maternalexposureduringpregnancy,orexposuretotheindividualduringanylifestageEXCLUDINGactiveandpassivesmoking.Householdproductsthatmayserveassourcesofairborneinhalationexposuretoformaldehydeincludemedium-densityfiberboard,particleboard,compositewood,pressedwood,polyurethanefoam,adhesives,mobilehomes,trailerhomes,customkitchencabinetry,etc.Comparator:Humansexposedtolowerlevelsofformaldehydethanthemorehighlyexposedhumans.

Thisdefinitionisintendedtocaptureonlystudieswithsomecomparatorgroupasabaselineforcomparisonwhenexaminingeffectsofformaldehydeexposure.Thisisintendedtoincludegroups

10

definedbycase-controlstudydesigns;forinstancecomparingtheformaldehydeexposurelevelsforpeoplewithasthmaversusthosewithout.

Outcomes:Anyofthefollowingasthma-relatedoutcomes:diagnosisofasthma,asthmasignsorsymptoms,asthmaexacerbation(requiringsystemictreatment),orindirectmeasuresofasthma.Diagnosisofasthmaisdefinedasaclinicalasthmadiagnosisbyahealthcareprovider,whichmayincludeparentalreportofaclinicalasthmadiagnosisfortheirchild.

Directmeasuresofasthmasignsorsymptomsorexacerbationaredefinedas:asthmasymptoms(cough,wheeze,chestpain,etc.),asthmasymptomsleadingtobronchodilatoruse(albuterolonly),asthma-specificemergencydepartmentvisits,outpatientvisits,orhospitaladmissions,asthma-specificICUadmissions/intubations,asthma-specificdetailedmedicationuse,AsthmaSyndromeUtilityIndex(ASUI)retrospectivequestionnaire,useofsystemiccorticosteroidsforasthma,methacholinechallengetest,and/orchangesinspirometrymeasurese.g.,FEV1,FVC,FEV1/FVC,PEForPEFR,etc.overtime.

Thetermasthma“exacerbation’(e.g.,asthmaflare-up,asthmaattack,etc.)hasmanydifferentdefinitionsintheclinicalliterature.Forthisanalysis,wewilldefineanasthmaexacerbationasanyincreaseinsymptomsthatrequiretheuseofsystemiccorticosteroid(Fuhlbrigge2012).

Indirectmeasuresofasthmaimpactsaredefinedasbeinginindividualswithnorestrictiononage,measuredas:schoolabsences,lossofworkerdays,etc.impactingthequalityoflifeasaresultofasthmaeffects.

SearchMethods Oursearchtermsandsearchstrategyweredevelopedbyalibrariantrainedinsystematicreviewmethodology(LS)(seeAppendixIforLS’sbiosketch),incollaborationwithasecondInformationSpecialist(EW),whohastrainingandexpertisewithsearchingtheclinicalliteraturebutwillbelearningtheNavigationGuidesystematicreviewmethodologybyworkingwithLSduringthiscasestudy(seeAppendixIforEW’sbiosketch).Collectivelyourlibrarianteamwilldesignandimplementthesearchforrelevantstudies.

Wewillperformelectronicsearchesofonlinedatabases(PubMed,WebofScienceCoreCollection,BiosisPreviews,Embase,GoogleScholar,andToxline/DART)usingthesearchtermsoutlinedinAppendixII.Forpracticalityreasons,GoogleScholarwillbesearchedwithasimplifiedsearchstringdictatedbythelimitationsofthesearchengine(AppendixII),andthefirst100relevance-rankedsearchresultswillbeevaluated.Oursearchwillnotbelimitedbylanguageorpublicationdate.

Tofacilitatedevelopmentofthesearchstrategy,wewillbeginbyperformingapreliminaryscopingassessmentinPubMedofthenatureandextentoftheavailableliteraturerelevanttoourPECOstatement.Wewillaccomplishthreethingswiththisreview:wewillidentifycontrolledvocabulary(MedicalSubjectHeadingsorMeSH)andsynonymsforourtwosearchconceptsof

11

formaldehydeandasthma,wewillidentifyproxytermsforbothconcepts,andwewillestimatethetypeandamountofinformationextantaboutourtopic.Inaddition,weplantoestimatethenumberofhumanstudiesavailableandidentifydifferenttypesofasthmaoutcomesavailableintheliterature.Thescopingsearchtermsandstrategywillbedocumentedandusedtoinformouroverallsearchstrategy.ThescopingstrategyandtermsarelistinAppendixII.

Toassistinthedevelopmentofalistoftermsrelevanttooursearchstrategy,wewillusetheMedicalSubjectHeadings(MeSH)databasetocompilesynonymsforformaldehydeandoutcomesrelatedtoasthma.WewillusethefollowingMeSHtermstoguideoursearch:

Asthma

RespiratoryFunctionTests

Respiratorysounds

ForcedExpiratoryVolume

PeakExpiratoryFlowRate

Bronchoconstriction

BronchialSpasm

Formaldehyde

Floorsorfloorcoverings

Adhesives

Airpollution

Constructionmaterials

Housing

Thefollowingarticleswereidentifiedinthescopingsearchorwereknowntotheauthorsasrelevanttoourstudy.Eachwasexaminedforterminologyandreferences.DuringdevelopmentofafinalsearchstrategyinPubMedthissetofarticleswasusedtotestforrecall.Inadditionwewillidentifyfurthersynonymsfromthefollowingknownresearcharticlesonformaldehydeandasthma:

1) RumchevKB,SpickettJT,BulsaraMK,etal.2002.Domesticexposuretoformaldehydesignificantlyincreasesriskofasthmainyoungchildren.EurRepirJ.20(2):403-408.

2) SchachterEN,WitekTJJr.,BrodyDJ,etal.1987.Astudyofrespiratoryeffectsfromexposureto2.0ppmformaldehydeinoccupationallyexposedworkers.EnvironRes.44(2):188-205.

3) G.Smedje,D.Norbäck.Incidenceofasthmadiagnosisandself-reportedallergyinrelationtotheschoolenvironment—afouryearfollow-upinschoolchildren.IntJTubercLungDis.5(2001),pps.1059–1066.

12

4) GarrettMH,HooperBM,RaymentPR,AbramsonMJ.2001.Increasedriskofallergyinchildrenduetoformaldehydeexposureinhomes.Allergy.54:330-337.

5) ZhaoZZhangZ,WangZ,FermM,LiangY,NorbäckD.2008.AsthmaticsymptomsamongpupilsinrelationtowinterindoorandoutdoorairpollutioninschoolsinTaiyuan,China.EnvironHealthPerspect.116(1):90-7.

6) KrzyzanowskiM,QuackenbossJJ,LebowitzMD.1990.Chronicrespiratoryeffectsofindoorformaldehydeexposure.EnvironRes.52(2):117-25.

7) HulinM,CaillaudD,Annesi-MaesanoI.2010.Indoorairpollutionandchildhoodasthma:variationsbetweenurbanandruralareas.IndoorAir.20(6):502-14.

8) DriscollTR,CareyRN,PetersS,GlassDC,BenkeG,ReidA,FritschiL.2016.TheAustralianWorkExposuresStudy:Prevalenceofoccupationalexposuretoformaldehyde.AnnOccupHyg.60(1):132-8.

Theseeightselectedpaperswererelevanttothestudyquestion,werepublishedindifferentyearsandjournals,authoredbyavarietyofresearchers,andtheyexaminedavarietyoftopicsrelevanttothestudyquestion.

Additionally,weusedthefollowinglistofbroadsurveysandsynthesestocompileoutcomesearchterms.Furthermore,weselectedbroadsurveysrelatedtoasthmatoreviewforcompilingoutcomesearchterms.Theseincluded:

1. McGwinG,LienertJ,KennedyJI.2010.Formaldehydeexposureandasthmainchildren:asystematicreview.EnvironHealthPerspect.118(3):313-7.doi:10.1289/ehp.0901143.

2. YaoY,LiangW,ZhuL,DuanY,JinY,HeL.2015.Relationshipbetweentheconcentrationofformaldehydeintheairandasthmainchildren:ameta-analysis.IntJClinExpMed.8(6):8358-62.

3. ReddelHK,TaylorR,BatemanED,etal.2009.AnofficialAmericanThoracicSociety/EuropeanRespiratorySocietyStatement:Asthmacontrolandexacerbations.AmJRespirCritCareMed180:59-99.DOI:10.1164/rccm.200801-060ST

4. FuhlbriggeA,PedenD,ApterAJ,etal.2012.Asthmaoutcomes:Exacerbations.JAllergyClinImmunol.129:S34-S48.DOI:10.1016/j.jaci.2011.12.983

5. KrishnanJA,LemanskeRF,CaninoGJ,etal.2012.Asthmaoutcomes:Symptoms.JAllergyClinImmunol.129:S124-S135.DOI:10.1016/j.jaci.2011.12.981

PubMedFortheexposurecomponentofthePECOstatement,wewillcombinetermsrepresentingformaldehyde,itssynonymsandnotableenvironmental/householdsourcesusingtheBoolean“OR”operator.Wewillnotincludeexposuretoformaldehydeasaproductofcombustion;homeheating,cigarettesmoking,exhaustexposureswereexcluded.Fortheoutcomes,wewillsearchusingtermsrepresentingasthma,itssynonyms,anditsproxiescombinedwiththeBoolean“OR”operator.Proxytermswillincludemeasuresoflungfunctionandsignsorsymptomsofasthma.WewillincluderelevantMeSHheadings(usingthe[mesh]fieldtag)andtitleand/orabstract

13

keywordsidentifiedinthescopingphaseofthesearch(usingthe[ti]or[tiab]fieldtags).Wewilladdtheformaldehyderegistrynumber[rn]totheformaldehydesearchstring.

WewillcombinetheexposureandoutcomesearchesusingtheBoolean“AND”operatortoidentifyarticleswithbothconcepts.

PubMedwillserveasourprimaryonlinedatabase.OnceourPubMedsearchstrategyisfinalized,thesearchwillbetranslatedtoperformoptimallyineachadditionaldatabase.

WebofScienceandBiosisPreviewsWewillmodifythePubMedsearchtoworkinWebofScienceCoreCollectionandBiosisPreviewsasbothdatabasesarehousedinThomsonReutersWebofScienceplatform.Todoso,wewillremovethePubMed-specific[mesh],[tiab],and[ti]fieldtagsandwhileretainingthekeywordsandphrases.WewillperformatopicsearchinWebofSciencewiththistranslatedsearchstring;atopicsearchexaminesthetitle,abstract,author-definedkeywords,and“KeywordsPlus”fieldsofWebofScience.

Embase

WewilldevelopourEmbasesearchfilterusingthesamemethodasdescribedaboveforWebofScienceCoreCollectionandBiosisPreviews.WewilltranslateMeSHtermstothenearestEmtreeThesaurusequivalentandusethe“:ab,ti.”functiontolimitthesearchtoabstractsandtitles.Wewillusethe:rnfunctiontosearchregistrynumbersforformaldehyde.

Toxnetdatabases(ToxlineandDART)Duetothesearchinterfacelimitationsweusedasimplifiedsearchstring(“formaldehydeandasthma”)forToxlineandDart,thetwotoxicologicliteraturedatabasesofToxnet.SeeAppendixIIfordetails.

SearchresultsfromeachdatabasewillbestoredinitsownEndNotecollectionforaccountingpurposes.WewillthencreateacombinedEndNotecollectionforalldatabases.The“findduplicates”functioninEndNotewillbeusedtofindandremoveduplicatesinthecombinedfolder.PubMedwillbeconsideredourprimaryonlinedatabasesothatduplicatesfromnon-PubMeddatabaseswillberemoved.

SearchingOtherResources

Wewilluseavarietyofmethodstofindadditionalinformationnotidentifiedthroughsearchesofonlinebibliographicdatabases.Theso-calledgreyliteratureincludestechnicalreportsfromgovernmentagenciesorscientificresearchgroups,workingpapersfromresearchgroupsorcommittees,whitepapers,preprints,conferenceproceedings,personalcommunications,dissertationsandtheses,etc.

Thesemethodsinclude:• SearchingthewebsitesanddatabaseslistedinAppendixIII.• LocatingconferenceabstractsfromWebofScienceCoreCollection,BIOSISPreviews,

Embase,ProceedingsFirst,andPapersFirst.

14

• Handsearchingthereferencelistsofallstudiesincludedafterfulltextreview(priortostudyauthorcontact,ifapplicable)usingWebofSciencetosearchforarticlescitedbyandthosethatcitedtheincludedstudies.

• Handsearchingthereferencelistsofallexcludedstudiesidentifiedduringourliteraturesearchasreviewsorsynthesesrelevanttoourstudyquestion.

• AGoogleandGoogleScholarsearchusingdomainoperatorsof.eduand.gov.Thefirst100resultsfromeachsearchwillbeevaluated.

• Personalcommunicationwithauthorstorequestunpublisheddataoriftheyhaveknowledgeofadditionaldatafromotherauthors.

• Havingexpertsinthefieldofformaldehydeandasthmareviewourlistofincludedstudiesforcompleteness.

StudySelectionCriteria

AllsearchresultswillbeimportedormanuallyenteredintoEndNote(Versionx7)referencemanagementsoftware.WewilluseEndNotetoeliminateanyduplicatereferencesbeforewebeginevaluatingtheeligibilityofthestudiesidentified.

TitleandabstractscreeningEachreferencewillbescreenedinduplicate.Fourreviewers(EK,ND,AP,HV)willindependentlyconductatitleandabstractreviewofeachreferencefromtheliteraturesearchresultstodeterminewhetheritmeetstheselectioncriteriaforinclusion.Eachauthorwillbeassignedanon-randomsubsetofreferencestoscreen,toensurethatallreferencesarescreenedinduplicateandtoensurethatthesametwoauthorsdonotalwaysscreenthesamereferences(i.e.HVwillbeassignedthefirstthreequarters(75%)ofthereferences;EKthelastthreequarters(75%);NDthe1stquarter(25%);APthelastquarter(25%)).

Referenceswhichareincludedatthetitle/abstractscreeninglevelwillbesubjecttoafulltextreviewbythesamefourauthors(moredetailfollowsinthenextsection).

Intheeventthatthereisadiscrepancybetweenreviewers,thedefaultwillbetopushthereferenceforwardtothenextstepintheprocess(i.e.,ifthetworeviewersdisagreeonwhetherthestudyisrelevantatthetitleandabstractscreeninglevel,thereferencewillbeincludedbydefaultforfull-textscreening).

Toensurequalitycontrol,oneauthor(JL)willperformtitleandabstractscreeningofarandomselection(usingarandomnumbergeneratorassignment)offivepercentofthesearchresultsor5papers,whicheverisgreater.Thesedeterminationswillbecomparedtotheotherreviewers’determinationsforthesestudies.

Thereviewofarticlesagainstthepre-specifiedinclusionandexclusioncriteriawillbeperformedusingastructuredforminDistillerSR(EvidencePartners;availableat:http://www.systematic-review.net),anonlineapplicationdesignedspecificallyforthescreeninganddataextractionphasesofasystematicreview(seeAppendixIVfortitleandabstractinclusion/exclusionform).

15

Reportsinanylanguage,fromanyyear,willbeeligibleforinclusion.Allreportsthatcomparehumansexposedtoformaldehydetoappropriatecomparatorsandevaluatethemfortheasthma-relatedhealthoutcomesasdescribedinthePECOstatementsabovewillbeeligibleforinclusion.

Thetitle/abstractscreeningformwillbeusedtoscreenandEXCLUDEreferencesifoneormoreofthefollowingcriteriaaremet:

1. Articleisareviewofformaldehydeexposureandasthma;2. Articlecontainsnooriginaldata(e.g.,editorial,reviewpapernotrelevanttostudy

question,etc.)3. Articledidnotinvolvehumansubjects(i.e.,animalevidenceorcelllinesonly,case

reportofsinglehuman,etc.);4. Articledidnotreportformaldehydeexposure,asdefinedbyPECOstatement;5. Articledidnotreportoutcomesofasthma,asdefinedbyPECOstatement;6. Otherreason(explanationrequired).

Thecriteriaforanarticlebeingareviewarticleisseparatelycategorizedfromothertypesofnon-originaldatasothatreviewarticlesmayberetainedandsearchedincaseanyofitsreferencesmaybeidentifiedforinclusionandtoultimatelycomparetheresultsofourreviewtothemethodsandconclusionsofpreviousreviews.Thefollowinginstructionswillbeprovidedtoreviewauthorsconductingthetitleandabstractscreening:

“Whenexcludingareference,pleaseselectonlyONE(1)exclusionreason.PleasereviewtheexclusionreasonsinorderandselecttheFIRSTexclusionreasonrelevanttothereferencebeingscreened.Pleaseaddinanyadditionalnotesinthecommentboxtoexplainyourselectionifnecessary.”

ThefollowingtypesofrecordswillbeINCLUDEDatthetitle/abstractlevel:

• Studiesofanydesignconductedinhumansinvestigatingassociationsbetweenformaldehydeexposureandasthma,asdefinedbythePECOstatement.

Forcitationswherethedatabasecontainsnoabstract,authorswillattempttoobtaintheabstractsfromanInternetsearch.ArticlesforwhichtheabstractremainsunavailablewillbescreenedbasedontitlesandPubMedMeSHheadings.Anystudynotexcludedbasedonthestatedcriteriaabovewillbeincludedforfull-textreview.Updateddetailstoinstructionsandinterpretationsfortitleandabstractscreening(additionaltowhatisprovidedhereintheprotocol)maybeaddedtoAppendixIVtothoroughlydocumenttheprocessofthereviewteamduringthescreeningprocess,ifnecessary.

Full-TextScreeningReferenceswhichareincludedatthetitle/abstractscreeninglevelwillbesubjecttoafulltextreviewbythesamefourauthorsinvolvedintitleandabstractscreening(EK,ND,AP,HV).Eachreferencewillbescreenedindependentlyandinduplicate.Eachauthorwillbeassignedanon-randomsubsetofreferencestoscreen,toensurethatallreferencesarescreenedinduplicateandtoensurethatthesametwoauthorsdonotalwaysscreenthesamereferences(i.e.HVwillbe

16

assignedthefirstthreequarters(75%)ofthereferences;EKthelastthreequarters(75%);NDthe1stquarter(25%);APthelastquarter(25%)).

Oneauthor(JL)willbebroughtintosettleanydiscrepanciesbetweenthereviewersresultingfromeachstepofthereviewprocessifnecessary.Intheeventthatthediscrepancyultimatelycannotberesolved,thedefaultwillbetoincludethestudy.

Toensurequalitycontrol,oneauthor(JL)willperformfulltextscreeningofarandomselection(usingarandomnumbergeneratorassignment)offivepercentorfivepapers,whicheverisgreater,ofsearchresultseligibleforfulltextreview.Thesedeterminationswillbecomparedtotheotherreviewers’determinationsforthesestudies.

Thereviewofarticlesagainstthepre-specifiedinclusionandexclusioncriteriawillbeperformedusingastructuredforminDistillerSR(EvidencePartners;availableat:http://www.systematic-review.net),anonlineapplicationdesignedspecificallyforthescreeninganddataextractionphasesofasystematicreview(seeAppendixIVforfulltextinclusion/exclusionform).

CitationseligibleforfulltextreviewwillbescreenedandEXCLUDEDifoneormoreofthefollowingcriteriaaremet:

1. Articleisareviewofformaldehydeexposureandasthma;2. Articlecontainsnooriginaldata(e.g.,editorial,reviewpapernotrelevanttostudy

question,etc.);3. Articledidnotinvolvehumansubjects(i.e.,animalevidenceorcelllineonly,casereport

ofsinglehuman,etc.);4. Articledoesnotreportexposuretoformaldehyde,asdefinedbythePECOstatement;5. Articledoesnotreportdiagnosisofasthma,asthmasignsorsymptoms,asthma

exacerbation,orindirectmeasuresofasthma,asdefinedbythePECOstatement;6. Therewasnocomparatorgroup;7. Duplicatestudy;8. Otherreasons(explanationrequired).

9. POSSIBLYInclude—otherlanguage(specifywhichlanguagebelowincomments);

Thefollowinginstructionswillbeprovidedtoreviewauthorsconductingfulltextscreening:

“Whenexcludingareference,pleaseselectonlyONE(1)exclusionreason.PleasereviewtheexclusionreasonsinorderandselecttheFIRSTexclusionreasonrelevanttothereferencebeingscreened.Pleaseaddinanyadditionalnotesinthecommentboxtoexplainyourselectionifnecessary.”

CitationswillbeINCLUDEDiftheymeetthePECOstatementcriteria,thatis,theirsubjectsrepresenthumans,theyincludeexposurecomparisonsforrelevantperiods,andtheyreporttheoutcomeofasthmaasdefinedinthePECOstatement.

Forarticles(includingnon-Englisharticles)thatarenotavailableinthedatabase,wewillattempttoobtainarticlesfromabroadInternetsearch.Potentiallyrelevantnon-EnglisharticleswillbetranslatedintoEnglishtodetermineeligibility.Alistofallexcludedstudiesandabriefreasonfor

17

exclusionwillbedocumentedfortransparencyandreportedinthestudyfindings(HigginsandGreen2011).

DataCollectionDatawillbeextractedfromeachincludedstudytoevaluatereportingquality,riskofbias,and/ortoconductstatisticalanalyses.Datawillbeextractedfromeachstudyinduplicate(byHVandEK)andcomparedunderthedirectionofathirdco-author(JL)forqualityassurance/qualitycontrolandtoresolveanydiscrepancies.DatawillbeextractedusinganAccessdatabaseformthathasbeenmodifiedfrompreviousapplications(seeAppendixVforthestudycharacteristicsdatacollectionform).Thedataextractionformwillbepilot-testedonasmallnumberofstudiestoidentifyanyissuesandmaximizeconsistencyacrossdataextractorsbeforeextractingthefullsetofincludedstudies.

Foreverystudythatdoesnotreportallthedataneededfordataanalysisorriskofbiasassessment,wewillrequestthesedatafromthestudycontactauthorbyemail.Ifstudyauthorsdonotrespondtorequestsafterbeingcontactedthroughatleast2emailmessagesoverthecourseof1month,reviewauthorswillnotethatattemptstocontactstudyresearcherswereunsuccessful.

RiskofBiasDeterminationRiskofbiaswillbeassessedforhumanstudiesusingdomainsfromtheCochraneCollaboration’s“RiskofBias”toolandtheAgencyforHealthcareResearchandQuality’s(AHRQ)criteria(HigginsandDeeks2011,Viswanathanetal.2012).Thesetoolshavebeenmodifiedtomakethemappropriateforhumanobservationalstudies,andincludedomainsthataddressrecruitmentstrategy,blinding,confounding,exposureassessment,outcomeassessment,incompleteoutcomedata,selectiveoutcomereporting,andconflictofinterest(AppendixVI).WehavemodifiedthesetoolsandappliedthemtoevaluateriskofbiasinfivepreviouscasestudiesapplyingtheNavigationGuidesystematicreviewmethodology(Johnsonetal.2014,Vesterinenetal.2014,Johnsonetal.2015,Lametal.2015,Lametal.2015).

AlthoughCochraneandAHRQdonotcurrentlyincorporateaconflictofinterestdomainasapotentialsourceofbias,theNavigationGuidetooldoesincludethisdomain,representedasfundingsourceordeclaredconflictsofinterest.Thisdecisionwasinformedbyempiricaldatafrommeta-analysesconductedonpharmacologicaltreatmentsandstudiesofriskofbiasandsponsorship(Rosemanetal.2011,Lundhetal.2012,Krauthetal.2013).WewillalsosearchforeachstudyinPubMedandnoteiftherehasbeenaretractionofthepublishedarticleinordertodetermineifthestudymaybefraudulentorifanycorrectionshavebeenpublished.

Reviewauthorswithrelevanttopicmatterexpertise(AP,TW,PS,HV,EK,MC)willassessriskofbias.Eachincludedstudywillberatedinduplicate.Reviewauthorswillbeassignedasmallsampleofstudies(e.g.,2)topilottheriskofbiasassessmenttool.Allreviewauthors(ratersaswellasothercoauthors)willthenmeettodiscusstheseratings,identifypotentialissuesorconfusion,andthenestablishconsistentrulestoensureconsistencyinratings.Anydecisionsorrulesestablishedduringthismeetingwillbedocumentedandappendedtotheprotocolfortransparency.Reviewauthorswillthenratetheremainingstudies,applyinganyrelevantdecisionsorrulesdiscussedsubsequenttothepilotrating.

AnydiscrepanciesbetweenROBraterswillbeinitiallyreviewedbyJLandthetworaterstoattempttoreachconsensusontheratings.Anyremainingdiscrepancieswillthenbereviewedby

18

JLandallotherriskofbiasraters.If,uponfurtherdiscussiontheraterscannotreachagreementonanappropriateriskofbiasdeterminationforaparticulardomain,theratingjudgmentwillbeselectedasfollows:ifonereviewermakesajudgmentof‘high’riskofbiasandtheothermakesajudgmentof‘probablyhigh’riskofbias,the‘high’riskofbiasjudgmentwillbeused,etc.Ifadditionaldataorinformationisacquiredfromstudyauthors,riskofbiasjudgmentswillbemodifiedtoreflecttheupdatedstudyinformation.Thediscussionamongraterswillbeusedtoinformratingsacrosssimilarstudies.Allfinalratingswillbereviewedtoensureconsistencyacrossratingsofstudieswithsimilarexposureoroutcomeassessmentmethods,studydesigns,etc.

Toensurequalitycontrol,JLwillalsomakeriskofbiasdeterminationsforarandomselection(usingarandomnumbergeneratorassignment)offivepercentofor5includedstudies,whicheverisgreaterandthesewillbecomparedtootherreviewers’determinationsforthesestudies.

Wewillattempttominimizetheimpactofpublicationbiasby:(1)implementingacomprehensivesearchoftheliteratureusingmultiplesourcesandmethodsinordertoidentifypublishedaswellasunpublishedstudiesthatmeettheeligibilitycriteria;and(2)ifpossible,usingfunnelplotanalysisand/orotherstatisticalanalyses(e.g.,Eggerregression(LightandPillemer1984)and“trimandfill”(DuvalandTweedie2000)ofthestudiesincludedinthesystematicreview,asappropriate.Thesestatisticalapproacheshavebeenrecommendedonlywhenthenumberofstudiesincludedinthemeta-analysisissufficientlylarge(Sterneetal.2011);andsotheseanalyseswillonlybeperformedwhen>10studiesareincludedinthemeta-analysis.Intheeventthatthenumberofstudiesincludedinthemeta-analysisistoosmalltheseanalyticalapproacheswillnotbepursued.Furthermore,intheeventofsubstantialbetween-studyheterogeneity,thesemethodsareknowntoperformpoorlyandsowewilltestforbetween-studyheterogeneityaswelltomakethedeterminationofwhetherthismethodwouldbeappropriateforthecollectionofincludedstudies(Higgins2011).

DataAnalysisWeintendtoperformameta-analysistosummarizetheeffectsofexposuretoformaldehydeonasthma,andtoassesstheimpactofstudydesigncharacteristicsonfindings.Todothis,firstwewillcompileandreviewimportantcharacteristicsfromeachstudytoestablishcomparabilitybetweenstudiesortoidentifydatatransformationsnecessarytoensuresuchcomparability.Keycharacteristicsinclude:

• Studydesign(cross-sectional,case-control,etc.)• Populationstudied(includinggeographicregion,sexand/orageofindividualswhen

assessed)• Exposurelevels,methodofmeasurement,andtimingofmeasurement• Healthoutcomeassessed,thetest/assessmenttoolused,andtimingofassessmentrelative

toexposure• Typeofdata/summarystatisticavailable

Thefollowingareexamplesofcharacteristicsthatwouldpotentiallybeconsiderednon-combinable:

19

• Non-quantitativeexposure(i.e.,whereexposureisnotquantifiedbutclassifiedashigh/loworothercategoriesofexposure)inanoccupationalsetting(i.e.,byjobclassification)wouldnotbecombinablewithnon-quantitativeexposureincommunitysettingsintheabsenceofdatatovalidatethatthesequalitativemeasureswerecomparable.

• Exposuretoambientairwouldpossiblynotbecombinablewithindoorairwithoutknowledgethattheairpollutantlevelsand/orcompositionofeachwassimilar.

• Moredirectmeasuresofasthmahealthoutcomes(i.e.,diagnosisofasthma,asthmasignsandsymptoms,asthmaexacerbation)wouldnotbecombinablewithindirectmeasuresofschoolabsences,lossofworkerdays,etc.impactingqualityoflifeasaresultofasthmaeffects.

• Effectsonchildrenandadultsmaybeaddressedseparately.

Summariesofthesecharacteristicsforeachincludedstudywillbeassessedbytwoormorereviewerauthors(JL,EK,HV,AP,TW)todeterminecomparabilitybetweenstudiesandtoidentifyanyheterogeneityconcerns.JLwillthenidentifystudieswithsufficientmethodologicalhomogeneitywithrespecttopopulation,studydesign,studyduration,exposurelevelandhealthoutcomeamongotherconsiderationsthatcanbecombinedinameta-analysis.Iftransformationstoreportedeffectestimatesarenecessarytoacommonscaleacrossdifferenttests/assessmentsofasthmaormeasuresofformaldehydeexposure,thesewillbedocumented.Thestatistician(TBD)andseniorauthor(TW)willreviewstudycharacteristicsandrecommendationsofJLregardingmeta-analysis.

Ifameta-analysisisdeemedappropriate,JLincollaborationwiththestatisticianwillidentifyappropriatestatisticalmethodstoanalyzethedata,andtodeterminewhetherfurthermodificationsarerequiredpriortoperformingthemeta-analysis.Ourinitialproposedapproachistoextractfromeachstudydeemedcombinabletheadjustedeffectestimates(oddsratios,relativerisks,linearregressionbetaestimates,etc.)aswellastherelevantscalereportedforeach(log-transformed,percontinuousunitincreaseofexposure,etc.).Wewillthenestablishacommonscaleandeffectestimateandeitherperformthenecessarycalculationstotransformeacheffectestimatetoacommonscaleifpossible,andcontactstudyauthorstoobtainthenecessarytransformedeffectestimatesifnecessary.Wewillalsotesttheseestimates(calculatingandinterpretingtheI2estimateaswellasachi-squaredtestforheterogeneity)toinvestigatewhetherstatisticalheterogeneityispresent.Furthermore,wewillattempttodeterminethecausesofpotentialheterogeneityamongresultsforstudiestodetermineifafixedeffectorrandomeffectsmodelisappropriate.Theseestimateswillthenbecombinedacrosscomparablestudies,usingeitherthefixedorrandomeffectsmodeltoaccountforpotentialheterogeneityacrossstudies.Thefinalquantitativeresultwillbethecombinedadjustedeffectestimatesonthesamescalewithanassociatedconfidenceinterval.OuranalysisplanwillberefinedbyJLincollaborationwiththestatisticianasneededbasedonthedatathatenterthereview.

Intheeventthattheseproposedmethodsfordataanalysisarealteredtotailortotheevidencebasefromincludedstudies,theprotocolwillbeamendedaccordinglyandthereasonsforchangewillbejustifiedinthedocumentation.

Toteststatisticalheterogeneityacrossthestudyestimates,wewillestimatethevariancecomponentcorrespondingtobetween-studyvariability(“Cochran’sQ”),andusealikelihoodratiotestforthenullhypothesisthatbetween-studyvariabilityisabsent.Ap-valueof0.05orlesswillbeconsideredstatisticallysignificant.Furthermore,toassesstheimpactofbetween-study

20

heterogeneityonthemeta-analysis,theI2teststatisticwillbecalculatedandevaluatedbyconsideringthemagnitude/directionofeffects,strengthofevidenceforheterogeneity(e.g.,p-valuefromachisquaredtestoraconfidenceintervalforI2),andtheCochrane’sguidetointerpretationasfollows:

• 0%-40%:mightnotbeimportant;• 30%-60%:mayrepresentmoderateheterogeneity;• 50%-90%:mayrepresentsubstantialheterogeneity;• 75%-100%:considerableheterogeneity.

Wewillalsoperformsensitivityanalysesbyexaminingtheeffectsofexcludingstudieswithparticularlyheterogeneousresultsaswellasperformingsubgroupanalysesbasedonexcludingsubsetsofstudieswithsharedcharacteristicsthatmightbeinfluential.Ifasufficientnumberofstudiesareavailable,analternativeapproachwouldbetoperformasubgroupanalysis,clusteringstudiesbyspecifiedcharacteristicstodeterminetheimpactonstatisticalheterogeneity.

Asdiscussedaboveinthe“RiskofBiasDetermination”section,ifpossible,i.e.thereareenoughstudies,wewillassessforthepresenceofpublicationbiasbyfunnelplottingandEggerregressionontheestimatesofeffectsize(LightandPillemer1984)andpredicttheimpactofhypothetical“missing”studiesusing“TrimandFill”(DuvalandTweedie2000).

QualityandStrengthofEvidenceRatingsUponcompletionofthedatacollection,riskofbiasdeterminations,anddataanalysis,eachofthereviewauthorswillindependentlycomparetheresultsofthesystematicreviewtothecriteriaoutlinedintheNavigationGuidesystematicreviewmethodologyforratingthequalityandstrengthoftheevidence.Allreviewauthorswillbegivenexplicitdirectionsbeforerating(seeAppendixVII,“InstructionsforRatingtheQualityandStrengthofEvidence”).

Theinitialqualitylevelofhumanobservationaldatawillstartatmoderate,ashasbeenassignedinpriorcasestudiesofapplyingtheNavigationGuidemethodology(WoodruffandSutton2014).

Factorsthatmaydecreasethequalitylevelofthebodyofevidenceinclude:1. RiskofBiasAcrossStudies:Studylimitations–asubstantialriskofbiasacrossbodyof

evidence;2. Indirectness:Evidencewasnotdirectlycomparabletothequestionofinterest(i.e.,

population,exposure,comparator,andoutcome).3. Inconsistency:Widelydifferentestimatesofeffect(heterogeneityorvariabilityinresults);4. Imprecision:Studieshadfewparticipantsandfewevents(wideconfidenceintervals);and5. PublicationBias:Studiesmissingfrombodyofevidence,resultinginanoverestimateor

underestimateoftrueeffectsfromexposure.

Factorsthatmayincreasethequalitylevelofthebodyofevidenceinclude:

1. Largemagnitudeofeffect:Upgradedifmodelingsuggestedconfoundingaloneunlikelytoexplainassociationswithlargemagnitudeofeffect.

2. Dose-response:Upgradedifconsistentdoseresponsegradientinoneormultiplestudies,and/ordoseresponseacrossstudies.

3. Residualconfoundingincreasesconfidence(NationalToxicologyProgram2015):Upgradedifconsiderationofallplausibleresidualconfounders,biases,oreffectmodificationwould

21

underestimatetheeffectorsuggestaspuriouseffectwhenresultsshownoeffect.Ifastudyreportsanassociationdespitethepresenceofresidualconfounding,biasesoreffectmodificationthatwoulddiminishtheassociation,confidenceintheassociationisincreased.GRADEprovidesanillustrativeexamplerelatedtobias:ratingupobservationalevidencefindinglackofassociationbetweenvaccinationandautism,whichoccurreddespiteempiricallyconfirmedbiasthatparentsofautisticchildrenmaybemorelikelytoremembertheirvaccineexperience.Thenegativefindingsdespitethisformofrecallbiassuggestratingupthequalityofevidence(Guyattetal.2011).

Possibleratingsforqualityofevidenceare“high,”“moderate,”or“low.”Possibledowngradesorupgradesare:0(nochange),-1(1leveldowngrade),–2(2leveldowngrade),+1(1levelupgrade)or+2(2levelupgrade).Theratingsoftheseparatefactorsarenotaddedtogetherintoascore,e.g.a-1downgradeforinconsistencyanda-1downgradeforimprecisiondoesnotautomaticallydictateanoverall-2downgradeforthebodyofevidence.Judgmentisexercisedtodetermineiftherationalebehindeachdowngradewarrantsanoveralldowngradeof1or2levels.Thesameappliestoupgradingtheoverallbodyofevidence.Likewise,a-1downgradeforonefactoranda+1upgradeforanotherfactordonotautomaticallycanceloutanddeterminen0downgradesorupgradesfortheoverallbodyofevidence.

Authorswhodecidetoratequalitydownorupneedtospecifythe1or2criteriamostresponsiblefortheirdecisionwhiledocumentingallfactorsthatcontributedtothefinaldecision.Afterindependentlyevaluatingthequalityoftheevidence,reviewauthorswillcomparetheirevaluationsandanydiscrepanciesbetweenthereviewers’decisionswillberesolvedthroughdiscussionuntilconsensusisreached,ifpossible.Therationaleforeachdecisiononeachofthefivefactorswillberecorded.Alackofconsensusonanyspecificfactordoesnotprecludeconsensusontheoverallqualityoftheevidence.

Subsequenttoratingthequalityoftheevidence,thereviewauthorswillratethestrengthofevidence.Theoverallstrengthofthebodyofhumanevidenceisbasedonacombinationoffourcriteria:(1)Qualityofbodyofevidence(i.e.,theratingfromthepreviousstep);(2)Directionofeffect;(3)Confidenceineffect;and(4)Othercompellingattributesofthedatathatmayinfluencecertainty.TheresultsofratingofthestrengthofthehumanevidencewillthenbecomparedtothecriteriaspecifiedintheNavigationGuidesystematicreviewmethodologyanddescribedaccordingtooneofthefollowingfourconcludingstatements:1.Sufficient;2.Limited;3.Inadequate;or4.Evidenceoflackoftoxicity(Table1)(Woodruffetal.2011,Johnsonetal.2015).Anydiscrepanciesbetweenthereviewers’decisionswillberesolvedthroughdiscussion.Theseniorauthor(TW)willbetheultimatearbiterofthediscrepanciesthatcannotberesolvedthroughconsensusamongthereviewauthors.Theresultsofthereview,includingimplicationsforpublichealth,willbecompiledinamanuscriptforsubmissiontothepeer-reviewliterature.

22

SUPPLEMENTARYINFORMATION

AppendixI.Coauthor/LibrarianBiosketches,ProjectContributionStatements,andConflictofInterestStatements

JULEENLAM

JuleenLamisanAssociateResearchScientistattheUniversityofCaliforniaatSanFrancisco(UCSF).ShehasbeeninvolvedwiththeNavigationGuidesince2011whileemployedattheEnvironmentalProtectionAgency’sOfficeofPolicyattheNationalCenterforEnvironmentalEconomicsasanOakRidgeInstituteforScienceandEducation(ORISE)postdoctoralfellowandasaresearcherattheJohnsHopkinsUniversityBloombergSchoolofPublicHealthintheDepartmentofHealth,PolicyandManagement.ShehasbeeninvolvedinfourcasestudiestodateapplyingtheNavigationGuidetoaddressproblemsinthefieldofenvironmentalhealth.JuleenreceivedherPhDfromJohnsHopkinsUniversityinEnvironmentalHealthPolicy,MHSfromJohnsHopkinsUniversityinBiostatistics,MSfromGeorgeWashingtonUniversityinEnvironmentalEngineeringManagement,andtwoBSdegreesfromtheUniversityofCaliforniaatDavisinMathandEnvironmentalToxicology.Shehasoveradecadeofexperienceinenvironmentalhealthresearchandpolicy,holdingpositionsatstateandfederalgovernmentagencies,academicinstitutions,andintheconsultingandnonprofitsectors.Shespecializesinanalysisofenvironmentalhealthdataandfocusesherresearchonthetranslationofscientificfindingsintomakinginformeddecisionsandpolicies.

Contributionstothereview:

• Conceivingthereview.• Designingthereview.• Coordinatingthereview.• Datacollectionforthereview.• Designingsearchstrategies.• Screeningsearchresults.• Organizingretrievalofpapers.• Screeningretrievedpapersagainsteligibilitycriteria.• Appraisingqualityofpapers.• Extractingdatafrompapers.• Writingtoauthorsofpapersforadditionalinformation.• Providingadditionaldataaboutpapers.• Obtainingandscreeningdataonunpublishedstudies.• Datamanagementforthereview.

23

• Analysisofdata.• Interpretationofdata.• Providingapolicyperspective.• Writingthereview(orprotocol).• Securingfundingforthereview.• Performingpreviousworkthatwasthefoundationofthecurrentreview.

PATRICESUTTON

PatriceSuttonisanAcademicCoordinatorwiththeUCSFProgramonReproductiveHealthandtheEnvironment(PRHE).ShehasbeenspearheadingPRHE'sresearchtranslationeffortssince2008andhasbeentheprojectleadontheNavigationGuidesystematicreviewmethodologysinceitsinceptionin2009.PatriceistheDirectoroftheCommunityOutreachandTranslationCoreofPRHE’sPregnancyExposurestoEnvironmentalChemicals(PEEC)Children’sCenter.PatricehasaMastersofPublicHealthfromU.C.BerkeleyinEnvironmentalHealthSciences.Patricehasover27yearsofexperienceinoccupationalandenvironmentalhealthresearch,industrialhygiene,publichealthpractice,policydevelopmentandcommunity-basedadvocacy.AsacontractortoCalifornia'sstatehealthdepartmentfrom1987to2006,shewasresponsibleforconductingallaspectsofresearchinvestigationsspanningadisparaterangeofissues,includingleadpoisoning,tuberculosis,asthma,andpesticide-illness.Shehasextensiveexperiencecollaboratingwithdirectly-impactedworkplaceandcommunity-basedpopulations,labor,andgovernmentalandnon-governmentalorganizationsinthedevelopmentofresearchstrategiesandpolicyrecommendations.Shealsohasextensiveexperienceasavolunteerinsupportofcommunitiesandworkersimpactedbythenuclearweaponsproductioncycleandhaspublishedover50peer-reviewedscientificarticlesandgovernmenttechnicalreports.


• Conceivingthereview.• Designingthereview.• Designingsearchstrategies.• Appraisingqualityofpapers.• Interpretationofdata.• Providingamethodologicalperspective.• Providingapolicyperspective.• Reviewingtheprotocol.• Providinggeneraladviceonthereview.• Securingfundingforthereview.• Performingpreviousworkthatwasthefoundationofthecurrentreview.

AMYPADULA

AmyPadulaisanInstructoratStanfordUniversityintheDepartmentofPediatricsandtheDivisionofNeonatology.ShewillbestartingasanAssistantAdjunctProfessoratUCSan

24

FranciscoinMarch2016.ShereceivedherPhDinEpidemiologyfromUCBerkeleyin2010.Sheisaperinatalandenvironmentalepidemiologistandspecializesinenvironmentalexposuresduringpregnancy.Herworkhasevaluatedtheeffectsofairpollutiononpulmonaryfunctioninchildrenandadversebirthoutcomesincludingpretermbirth,lowbirthweightandstructuralbirthdefects.


IwillcontributetothereviewofFormaldehydeandAsthmausingtheNavigationGuideinthefollowingways:Iwillassistwithdesigningthereviewandresearchstrategies.Iwillscreenbothtitle/abstractsandfulltextagainsteligibilitycriteria.Iwillappraisethequalityofthepaperandextractdatafromthem.Iwillhelpwiththeinterpretationofthedataandprovideamethodologic(bothepidemiologicandstatistical)perspective.Iwillassistwithwritingandrevisingthereview.Iwillprovidegeneraladviceontheprotocolandreview.

MICHAELCABANA

Dr.MichaelCabanaisaProfessorofPediatrics,EpidemiologyandBiostatistics,aswellasamemberofthecorefacultyatthePhilipR.LeeInstituteforHealthPolicyStudiesattheUniversityofCalifornia,SanFrancisco(UCSF).Dr.Cabana’sresearchfocusesonclinicalmanagementofasthma,epidemiology,andprevention.Dr.CabanaisPrincipalInvestigatorfortheTrialofInfantProbioticSupplementation(TIPS)toPreventEarlyMarkersofAsthma(R01HL80074),aswellasclinicaltrialsfocusedoninfantgrowthandcolic.Inaddition,Dr.CabanaisthePrincipalInvestigatorfortheUCSFPediatricAsthmaNetResearchCenter(U10HL074204)oneofninesitesintheUnitedStates.Hehasextensiveexperienceinpractice-basedresearchandhascollaboratedwithover120pediatricpracticesinseveralnationalrandomizedcontrolledtrialsfocusedonprimarycaremanagementofasthma.Dr.CabanahasservedonstudysectionsfortheNIHandadvisorygroupsfortheNationalCommitteeonQualityAssurance.Dr.Cabanahasmaintainedanactivepresenceinclinicalmedicine,servingasChiefoftheUCSFDivisionofGeneralPediatricssince2005.


• Appraisingqualityofpapers• Providingexpertiseondatatobeextractedfrompapers.• Providingexpertiseondataanalysis.• Interpretationofdata.• Providingamethodologicalperspective.• Providingaclinicalperspective.• Writingpartsofthereview(orprotocol).• Providinggeneraladviceonthereview.

25

ERICAKOUSTAS

EricaisaconsultanttotheUCSFProgramonReproductiveHealthandtheEnvironment(PRHE).ShebeganherinvolvementontheNavigationGuidesystematicreviewmethodologyin2010andwasoneoftheleadauthorsonthefirstNavigationGuidecasestudyasanOakRidgeInstituteforScienceandEducation(ORISE)Post-doctoralFellowandanAmericanAssociationfortheAdvancementofScience(AAAS)ScienceandTechnologyPolicyFellowattheU.S.EnvironmentalProtectionAgency(EPA).Overthelastfiveyears,EricahasbeeninvolvedintherefinementoftheNavigationGuidemethodologyandparticipatedintwoadditionalcasestudies.InadditiontoherextensiveexperiencewiththeNavigationGuidemethodology,Ericahasalsoworkedinotherareasofenvironmentalhealth,includingeffortstodevelopindicatorsforchildren’senvironmentalhealthandimprovechemicalriskassessmentssotheycanbetterinformbenefit-costanalysisandregulatorydecision-making.EricaholdsaPh.D.inMolecularBiologyfromtheUniversityofColoradoDenverandaB.S.inMolecularBiologyfromtheUniversityofDenver.


• Datacollectionforthereview.• Screeningsearchresults.• Screeningretrievedpapersagainsteligibilitycriteria.• Appraisingqualityofpapers.• Extractingdatafrompapers.• Obtainingandscreeningdataonunpublishedstudies.• Datamanagementforthereview.• Interpretationofdata.• Providingmethodologicalperspective.• Providingpolicyperspective.• Writingthereview(orprotocol).• Providinggeneraladviceonthereview.• Providingpreviousworkthatwasthefoundationofthecurrentreview.

HANNAVESTERINEN

HannaVesterinenisapostdoctoralresearcherworkingasaconsultantforPRHE.In2013shecompletedaPhDatTheUniversityofEdinburghwhereshefocusedondevelopingsystematicreviewandmeta-analysismethodologyinanimalstudies.Thisresearchresultedinthepublicationofa“practicalguide”forotherresearchersapplyingmeta-analysistodatafromanimalstudies.Hannaalsosuccessfullyappliedthesetechniquestoareviewofoverathousandanimalstudiesinmultiplesclerosisresearchtoassesspotentialreasonsforthefailuretotranslateefficacyfrombenchtobedside.Hanna’sresearchhasledtonumerouspublishedsystematicreviewsofindividualinterventionsordruggroupsinseveralanimalmodelsofneurologicaldiseasesincludingParkinson’sDisease,Huntingdon’sdisease,gliomaandstroke.Additionallysheusedbothhumanandanimaldatatoidentifyapotentialtreatmentforprogressivemultiplesclerosis

26

whichwasbothpublishedandtakenforwardtoaclinicaltrialwhichisnowunderway.FollowingthecompletionofherPhD,HannawasemployedasapostdoctoralresearchfellowatPRHEwheresheworkedontwoNavigationGuidecase-studies,triclosanandglomerularfiltrationrate,aswellasconductingasystematicreviewonthecumulativeeffectofpre-natalexposuretostressandchemicalsonfetaldevelopment.


Mycontributiontothereviewwillbetoscreensearchresults,appraisethequalityofstudies,extractdatafrompapers,enterdataintothedatabaseaswellasprovidinggeneraladviceonthereviewandaidingintheeditingofthemanuscript.

EVANSM.WHITAKER

Dr.WhitakerisamedicalresearchlibrarianattheUniversityofCalifornia,SanFranciscoandservesasEducationConsultantfortheSchoolsofMedicineandPharmacy.Dr.Whitakercombinesextensiveclinicalexperienceasafamilyphysicianwithformaltrainingand3yearsexperienceasaresearchlibrarianbasedatamedicalschoolrichwitheducationalscholarshipandinterprofessionalcollaborationstoadvancehealthprofessionstraining.Hedevelopsandpresentscontenttostudents,residents,fellowsandfacultyintheUCSFenvironment.Hehasassistedonquestionformulation,protocolconstruction,methodssectionwritingforover30systematicreviews.Dr.Whitakerwillassistwithallstepsofthescopingreviewprocesswithaparticularemphasisondevelopingthesearchquestion,constructingthesearchcriteria,andsynthesizingresults.


• Designingsearchstrategies.• Undertakingsearches.• Organizingretrievalofpapers.• Screeningretrievedpapersagainsteligibilitycriteria.• Appraisingqualityofpapers.• Obtainingunpublishedpapers.• Providingaclinicalperspective.• Writingtherevieworprotocol.• Providinggeneraladviceonthereview.

NATALYNDANIELS

NatalynDanielsisaResearchAssistantworkingwithPRHE.NatalynreceivedaB.A.fromUCBerkeleyin2011.Inconductingherundergraduatethesis,shebecamethefirsttodevelopanexperimentprotocolandmethodologytotesttheEcologicalValenceTheory.Herinterestin

27

reproductionandenvironmentalhealthstemsfromherworkasanambulanceemergencymedicaltechnicianandherpreviousResearchAnalystappointmentintheDivisionofAdolescentandYoungAdultMedicineatUCSF.AsaResearchAnalyst,sheevaluatedastate-widecasemanagementframeworkgearedtowardsimprovingaPositiveYouthDevelopmentinterventionforpregnantandparentingteenagersinCalifornia.Shecompletedanextensiveliteraturereviewanddatacollectionprocess,andisaco-authoronthe“Maternal,Child,andAdolescentHealthAdolescentFamilyLifeProgramPositiveYouthDevelopmentFormativeEvaluationReport.”


• Conductabstractandfull-textscreeningsofpublishedstudiesinthesearchresults• Conductabstractandfull-textscreeningsofunpublishedstudiesinthesearchresults• AssessandapplytheNavigationGuideratingcategoriestodeterminethequalityofthe

evidence• Cometoanoverallratingofthestrengthoftheevidence• Providegeneralinputregardingthereview,protocol,andratings

TRACEYJ.WOODRUFF

Dr.WoodruffisProfessorintheDepartmentofObstetrics,Gynecology,andReproductiveSciencesandPhilipRLeeInstituteforHealthPolicyStudiesattheUniversityofCalifornia,SanFranciscoandtheDirectoroftheProgramonReproductiveHealthandtheEnvironment.Shehasdoneextensiveresearchandpolicydevelopmentonenvironmentalhealthissues,withaparticularemphasisonearly-lifedevelopment.Herresearchincludesevaluatingprenatalexposurestoenvironmentalchemicalsandrelatedadversepregnancyoutcomes,andcharacterizingdevelopmentalrisks.Dr.WoodruffconceivedofandwastheleadforthecollaborativeeffortwhichdevelopedtheNavigationGuidesystematicreviewmethodology.Shehasauthorednumerousscientificpublicationsandbookchapters,andhasbeenquotedwidelyinthepress,includingUSAToday,theSanFranciscoChronicle,andWebMD.ShewaspreviouslyattheUSEPA,whereshewasaseniorscientistandpolicyadvisorintheOfficeofPolicy,andauthorofnumerousgovernmentdocuments.SheisanAssociateEditorofEnvironmentalHealthPerspectives.ShewasappointedbythegovernorofCaliforniain2012totheScienceAdvisoryBoardoftheDevelopmentalandReproductiveToxicant(DART)IdentificationCommittee.


• Conceivingthereview.• Designingthereview.• Designingsearchstrategies.• Appraisingqualityofpapers.• Providingexpertiseondatatoextractfrompapers.• Providingexpertiseondataanalysis.• Interpretationofdata.• Providingamethodologicalperspective.

28

• Providingapolicyperspective.• Providingexpertiseonwritingthereview(orprotocol).• Providinggeneraladviceonthereview.• Securingfundingforthereview.• Performingpreviousworkthatwasthefoundationofthecurrentreview.

INFORMATIONSPECIALISTCONSULTANT

LESLEYSKALLA

LesleySkallaisaresearchanalystandsciencelibrarianforMDB,Inc.inDurham,NorthCarolina.SheprovidesarangeofscientificsupportservicesfortheNationalInstituteofEnvironmentalHealthSciences(NIEHS),fromliteraturesearchestograntcodingandportfolioanalysis.Anexpertsearcher,Lesleyprovidesbothtraditionalandsystematicreviewliteraturesearchesusinganumberofbibliographicdatabases,includingPubMed,WebofScience,BIOSIS,andEmbase.AstheinformationspecialistforthefourthNavigationGuideProof-of-Conceptcasestudy(Associationbetweendevelopmentalexposurestoambientairpollutionandautism),Lesleywasresponsiblefordesigning,conducting,anddocumentingboththebibliographicandgreyliteraturesearch.LesleyreceivedherPhDinAnimalSciencewithaspecializationinReproductivePhysiologyfromtheUniversityofIllinoisinUrbanaandherMastersofInformationandLibrarySciencefromtheUniversityofNorthCarolinainChapelHill.Throughherdualtrainingasascientistandasalibrarian,Lesleyprovidesclientswithauniqueskillsetwell-suitedforliterature-basedevaluationprojects.Shehasoverfiveyearsofexperienceconductingliteraturesearchesinthefieldofenvironmentalhealthsciences.

29

AUTHORCONFLICTOFINTERESTSTATEMENTS

68

AppendixII.SearchTermsAliteraturesearchwillbedevelopedandconductedcollectivelybetweentwoinformationspecialists(LR&EW)betweenusingthedatabase-specificsearchtermsbelow.

PreliminaryScopingsearch:

Search PubMed

#1

(("Asthma"[Mesh]ORasthma[tiab]OR“airwayhyper-responsiveness”[tiab]OR“airwayhyperresponsiveness”[tiab]OR"RespiratoryHypersensitivity"[Mesh]ORbronchialhypersensitivity[tiab]ORbronchialhyper-sensitivity[tiab]OR“airwayhyper-sensitivity”[tiab]OR“airwayhypersensitivity”[tiab]OR“reactiveairway”[tiab]OR“reactiveairways”[tiab]OR“airwayinflammation”[tiab]OR“airwayobstruction”[tiab]ORwheeze[tiab]ORwheezes[tiab]ORwheezing[tiab]ORdyspnea[tiab]ORspirometry[tiab]OR“lungfunction”[tiab]OR“lungfunctions”[tiab]OR“respiratoryfunction”[tiab]OR“respiratoryfunctions”[tiab]OR“pulmonaryfunction”[tiab]OR“pulmonaryfunctions”[tiab]OR"BronchialHyperreactivity”[MH]OR“bronchialhyperreactivity”[tiab]OR“bronchialhyper-reactivity”[tiab]OR“bronchialspasm”[tiab]OR“bronchialspasms”[tiab]OR"BronchialSpasm"[Mesh]OR"AirwayResistance”[MH]OR“airwayresistance”[tiab]OR“respiratoryhealth”[tiab]OR"RespiratoryFunctionTests"[MeSHTerms]OR"RespiratorySounds"[Mesh:NoExp]OR"Bronchoconstriction"[Mesh]OR“bronchialconstriction”[tiab]OR“bronchialconstrictions”[tiab]OR“reactiveairwaydisease”[tiab]))AND(("Formaldehyde"[Mesh]ORFormaldehyde[tiab]OROxomethane[tiab]ORMethanal[tiab]ORFormol[tiab]ORFormalin[tiab]ORparaformaldehyde[tiab]OR“Medium-densityfibreboard”[tiab]OR“medium-densityfiberboard”[tiab]ORparticleboard[tiab]ORparticle-board[tiab]ORplywood[tiab]ORwood-based[tiab]))

WewilluseSwift-Review(http://www.sciome.com/swift-review/)toconductapreliminaryassessmentofthesizeandscopeoftheliteratureretrievedwithourscopingreviewsearchstrategy.Swift-reviewwillalsobeusedtofine-tuneourselectionofMeSHterms.

PubMedsearchstrategy:

Search PubMed

69

#1

((("Asthma"[Mesh]ORasthma*[tiab]ORreactiveairway[tiab]ORreactiveairways[tiab]ORairwayinflammation[tiab]ORwheeze[tiab]ORwheezes[tiab]ORwheezing[tiab]ORdyspnea[tiab]OR"RespiratoryFunctionTests"[MeSHTerms]OR"RespiratorySounds"[Mesh:noexp]ORspirometry[tiab]ORlungfunction[tiab]ORlungfunctions[tiab]ORrespiratoryfunction[tiab]ORrespiratoryfunctions[tiab]ORpulmonaryfunction[tiab]ORpulmonaryfunctions[tiab]OR"ForcedExpiratoryVolume"[Mesh]OR"PeakExpiratoryFlowRate"[Mesh]OR"ForcedExpiratoryVolume"[Mesh]ORFEV1[tiab]ORPEFR[tiab]ORPEF[tiab]OR"peakexpiratory"[tiab]OR"forcedexpiratoryvolume"[tiab]OR"forcedexpiratoryflow"[tiab]OR"BronchialHyperreactivity"[MH]ORbronchialhyperreactivity[tiab]ORbronchialhyper-reactivity[tiab]ORbronchialhypersensitivity[tiab]ORbronchialhyper-sensitivity[tiab]OR"BronchialSpasm"[Mesh]ORbronchialspasm[tiab]ORbronchialspasms[tiab]OR"AirwayResistance"[MH]OR"AirwayObstruction"[mh]ORairwayobstruction[tiab]ORairwayresistance[tiab]OR"Bronchoconstriction"[Mesh]ORbronchialconstriction[tiab]ORbronchialconstrictions[tiab]ORrespiratoryhealth[tiab]ORreactiveairwaydisease[tiab])))AND((("Formaldehyde"[Mesh]OR50-00-0[RN]ORFormaldehyde[tiab]OROxomethane[tiab]ORMethanal[tiab]ORFormol[tiab]ORFormalin[tiab]ORparaformaldehyde[tiab]ORMedium-densityfibreboard[tiab]ORmedium-densityfiberboard[tiab]ORparticleboard[tiab]ORparticle-board[tiab]ORplywood[tiab]ORwood-based[tiab]ORwood-product*[tiab]ORcomposite-wood[tiab]ORpressed-wood[tiab]ORvarnish*ORlaminate[tiab]OR"FloorsandFloorcoverings"[Mesh]ORflooring[tiab]OR"polyurethanefoam"[SupplementaryConcept]OR"polyurethanefoam"[tiab]OR"ureaformaldehydefoam"[SupplementaryConcept]OR"ureaformaldehydefoam"[tiab]OR"Adhesives"[Mesh]ORAdhesives[tiab]ORsalon[tiab]ORhair-straighten*[tiab]ORhair-smooth*[tiab]OR(trailer*[tiab]AND(housing[tiab]OR"housing"[Mesh]))ORTravel-trailer*[tiab]ORmanufactured-home*[tiab]ORmobile-home*[tiab]ORmanufactured-hous*[tiab]ORmodular-home*[tiab])OR"temporaryhousingunit"[tiab]OR"temporaryhousingunits"[tiab]OR(("ConstructionMaterials"[Mesh]OR"Housing"[Mesh:noexp])AND(airpollution[tiab]ORairquality[tiab]OR"Airpollution"[Mesh])AND"asthma"[Mesh]))ORcadaver[tiab]ORembalm*[tiab]ORgarment-workers[tiab]ORtextile-workers[tiab]ORwood-workers[tiab]ORwoodworkers[tiab])

70

WebofScienceCoreCollection:

Search WebofScience

#1

(asthmaOR“reactiveairway”OR“reactiveairways”OR“airwayinflammation”ORwheezeORwheezesORwheezingORdyspneaOR“lungfunctiontest”ORspirometryOR“lungfunction”OR“lungfunctions”OR“respiratoryfunction”ORrespiratoryfunctionsOR“pulmonaryfunction”OR“pulmonaryfunctions”OR“bronchushyperreactivity”OR“bronchialhyperreactivity”OR“bronchialhyper-reactivity”OR“bronchialhypersensitivity”OR“bronchialhyper-sensitivity”ORbronchospasmORbronchospasmOR“bronchialspasm”OR“bronchialspasms”ORairwayresistanceORairwayobstructionOR“airwayobstruction”OR“airwayresistance”ORbronchoconstrictionOR“bronchialconstriction”OR“bronchialconstrictions”OR“respiratoryhealth”OR“reactiveairwaydisease”)AND(50-00-0ORformaldehydeORoxomethaneORmethanalORformolORformalinORparaformaldehydeOR“medium-densityfibreboard”OR“medium-densityfiberboard”ORparticleboardORparticle-boardORplywoodORwood-basedORcomposite-woodORpressed-woodOR“polyurethanefoam”OR“polyurethanefoam”OR“ureaformaldehyde”ORadhesiveORadhesivesORpolyurethanefoamORpolyurethane*foamORureaformaldehydefoamORadhesivesOR((trailer*ANDhous*)OR(traveltrailer*ORmanufacturedhome*ORmobilehome*ORmanufacturedhous*ORmodularhome*OR“temporaryhousingunit”OR“temporaryhousingunits”)OR(“buildingmaterial”ORhousing)AND(“airpollution”OR“airquality”)AND(asthma)))

BiosisPreviews:

Search BiosisPreviews

#1

(asthmaOR“reactiveairway”OR“reactiveairways”OR“airwayinflammation”ORwheezeORwheezesORwheezingORdyspneaOR“lungfunctiontest”ORspirometryOR“lungfunction”OR“lungfunctions”OR“respiratoryfunction”ORrespiratoryfunctionsOR“pulmonaryfunction”OR“pulmonaryfunctions”OR“bronchushyperreactivity”OR“bronchialhyperreactivity”OR“bronchialhyper-reactivity”OR“bronchialhypersensitivity”OR

71

“bronchialhyper-sensitivity”ORbronchospasmORbronchospasmOR“bronchialspasm”OR“bronchialspasms”ORairwayresistanceORairwayobstructionOR“airwayobstruction”OR“airwayresistance”ORbronchoconstrictionOR“bronchialconstriction”OR“bronchialconstrictions”OR“respiratoryhealth”OR“reactiveairwaydisease”)AND(50-00-0ORformaldehydeORoxomethaneORmethanalORformolORformalinORparaformaldehydeOR“medium-densityfibreboard”OR“medium-densityfiberboard”ORparticleboardORparticle-boardORplywoodORwood-basedORcomposite-woodORpressed-woodOR“polyurethanefoam”OR“polyurethanefoam”OR“ureaformaldehyde”ORadhesiveORadhesivesORpolyurethanefoamORpolyurethane*foamORureaformaldehydefoamORadhesivesOR((trailer*ANDhous*)OR(traveltrailer*ORmanufacturedhome*ORmobilehome*ORmanufacturedhous*ORmodularhome*OR“temporaryhousingunit”OR“temporaryhousingunits”)OR(“buildingmaterial”ORhousing)AND(“airpollution”OR“airquality”)AND(asthma)))

Embase:

Search Embase

#1

(‘Asthma’/expORasthma:ab,tiOR‘reactiveairway*’:ab,tiOR‘airwayinflammation’:ab,tiOR'wheezing'/expORwheez*:ab,tiORdyspnea:ab,tiOR'lungfunctiontest'/expORspirometry:ab,tiOR‘lungfunction’:ab,tiOR‘lungfunctions’:ab,tiOR‘respiratoryfunction’:ab,tiOR‘respiratoryfunctions’:ab,tiOR‘pulmonaryfunction’:ab,tiOR‘pulmonaryfunctions’:ab,tiOR‘ForcedExpiratoryVolume’/expOR‘PeakExpiratoryFlow’/expORFEV1:ab,tiORPEFR:ab,tiOR

72

PEF:ab,tiOR‘peakexpiratory’:ab,tiOR‘forcedexpiratoryvolume’:ab,tiOR‘forcedexpiratoryflow’:ab,tiOR'bronchushyperreactivity'/expOR‘bronchialhyperreactivity’:ab,tiOR‘bronchialhyper-reactivity’:ab,tiOR‘bronchialhypersensitivity’:ab,tiOR‘bronchialhyper-sensitivity’:ab,tiOR'bronchospasm'/expORbronchospasm:ab,tiOR‘bronchialspasm’:ab,tiOR‘bronchialspasms’:ab,tiOR'airwayresistance'/expOR'airwayobstruction'/expOR‘airwayobstruction’:ab,tiOR‘airwayresistance’:ab,tiOR'bronchoconstriction'/expOR‘bronchialconstriction’:ab,tiOR‘bronchialconstrictions’:ab,tiOR‘respiratoryhealth’:ab,tiOR‘reactiveairwaydisease’:ab,ti)

#2 'formaldehyde'/expOR'formaldehyde'OR'50-00-0':rnORformaldehyde:ab,tiORoxomethane:ab,tiORmethanal:ab,tiORformol:ab,tiORformalin:ab,tiORparaformaldehyde:ab,tiOR'medium-densityfibreboard':ab,tiOR'medium-densityfiberboard':ab,tiORparticleboard:ab,tiOR'particle-board':ab,tiORplywood:ab,tiOR'wood-based':ab,tiOR'composite-wood':ab,tiOR'pressed-wood':ab,tiOR‘varnish*’:ab,tiOR‘laminate’:ab,tiOR‘Floorcoverings’:ab,tiORflooring:ab,tiOR

73

'polyurethanefoam'/expOR'adhesiveagent'/expOR'polyurethanefoam':ab,tiOR'polyurethanfoam'/expOR'ureaformaldehydefoam':ab,tiORadhesives:ab,tiORsalon:ab,tiORhair-straighten*:ab,tiORhair-smooth*:ab,tiOR(trailer*:ab,tiAND(housing:ab,tiOR'housing'/expORhousing))OR'traveltrailer*':ab,tiOR'manufacturedhome*':ab,tiOR'mobilehome*':ab,tiOR'manufacturedhous*':ab,tiOR'modularhome*':ab,tiOR'temporaryhousingunit*':ab,tiOR('buildingmaterial/exp'OR'housing'/expOR'housing'AND('airpollution':ab,tiOR'airquality':ab,tiOR'airpollution'/expOR'airpollution')AND('asthma'/expORasthmaORasthma:ab,ti))ORcadaver:ab,tiORembalm*:ab,tiORgarment-workers:ab,tiORtextile-workers:ab,tiORwood-workers:ab,tiORwoodworkers:ab,ti

#3 #1AND#2

ToxlineandDART:

Search Toxline&DART

#1

Formaldehydeandasthma

AppendixIII.OtherResourcesforLiteratureSearchToxicologicalwebsitestosearch

Inmostcases,thesimplifiedsearchstrategy“asthmaAND(formaldehydeOR“50-00-0”)wasusedtosearchtoxicologicalwebsites/databases.

• ATSDRInteractionProfileshttp://www.atsdr.cdc.gov/interactionprofiles/index.asp• ATSDRToxicologicalProfileshttp://www.atsdr.cdc.gov/toxprofiles/index.asp

74

• ACTOR• CalEPAOfficeofEnvironmentalHealthHazardAssessment

http://www.oehha.ca.gov/risk.html,http://oehha.ca.gov/air.html• ChemIDhttp://chem.sis.nlm.nih.gov/chemidplus/• EPAAcuteExposureGuidelineLevelshttp://www.epa.gov/oppt/aegl/chemlist.htm• EPAIRISinternetwww.epa.gov/iris• EPANEPISandNSCEPhttp://www.epa.gov/nscep/• EPAScienceInventoryhttp://cfpub.epa.gov/si/• EPASubstanceRegistrySystem

http://ofmpub.epa.gov/sor_internet/registry/substreg/searchandretrieve/substancesearch/search.do

• HealthCanadaFirstPriorityListAssessmentshttp://www.hcsc.gc.ca/hecssesc/exsd/psl1.htm

• HealthCanadaSecondPriorityListAssessmentshttp://www.hcsc.gc.ca/hecssesc/exsd/psl2.htm

• HazardousSubstancesDataBankhttp://toxnet.nlm.nih.gov/cgi-bin/sis/htmlgen?HSDB• IPCSINCHEMhttp://www.inchem.org/• NIOSHTIC2http://www2.cdc.gov/nioshtic2/Nioshtic2.htm• ToxicologyDataNetworkhttp://toxnet.nlm.nih.gov/• RTECSToxcenterhttp://www.cdc.gov/niosh/rtecs/default.html• WHOassessments–CICADS,EHChttp://www.who.int/ipcs/assessment/en/• USEPAHealthandEnvironmentalStudiesOnlinehttp://hero.epa.gov/• FIFRAdocket:http://www.regulations.gov

Greyliteraturedatabasestosearch

Inmostcases,thesimplifiedsearchstrategy“asthmaAND(formaldehydeOR“50-00-0”)wasusedtosearchgreyliteraturedatabases.

Google:http://www.google.comGoogleScholar:http://scholar.google.com/Databaseoffederally-fundedscientificresearch:Science.govScienceResearch.com(SciencefederatedsearchenginebyDeepWebTechnologies):http://scienceresearch.com/Oaisterdatabase(anopen-sourcerepositoryofdifficult-to-access,academically-orienteddigitaltresources):http://www.oclc.org/oaisterOpenGrey:http://www.opengrey.eu/ProceedingsFirst(Coverseverypublishedcongress,symposium,conference,exposition,workshopandmeetingreceivedbyTheBritishLibraryDocumentSupplyCentre)PapersFirst(Accesstoindividualpaperspresentedatconferencesworldwide)ProQuestDissertationsandTheses(Internationalrepositoryofgraduatedissertationsandtheses)

75

AppendixIV.ExclusionCriteriaScreeningFormsandAmendmentstoClarifyScreeningProcess

TitleandAbstractScreeningForm

INSTRUCTIONS:

Whenexcludingareference,pleaseselectonlyONE(1)exclusionreason.PleasereviewtheexclusionreasonsinorderandselecttheFIRSTexclusionreasonrelevanttothereferencebeingscreened.Pleaseaddinanyadditionalnotesinthecommentboxtoexplainyourselectionifnecessary.

Categories:• Exclude—Articleisareviewofformaldehydeexposureandasthma;• Exclude—Articlecontainsnooriginaldata(e.g.,editorial,reviewpapernotrelevantto

studyquestion,etc.)• Exclude—Articledidnotinvolvehumansubjects(i.e.,animalevidenceorcelllinesonly,

casereportofsinglehuman,etc.);• Exclude—Articledidnotreportformaldehydeexposure,asdefinedbyPECOstatement;• Exclude—Articledidnotreportoutcomesofasthma,asdefinedbyPECOstatement;• Exclude—Otherreason(explanationrequired).

• Include—Retrievefullarticle

Comments:Explainherereasonforexclusionifotherthanreasonsprovidedin#1above,andanyotherrelevantcomments.

AmendmentstoTitleandAbstractScreeningProcess

Addhereanyadditionaldetailsifnecessary.

Full-TextScreeningForm

INSTRUCTIONS:

Whenexcludingareference,pleaseselectonlyONE(1)exclusionreason.PleasereviewtheexclusionreasonsinorderandselecttheFIRSTexclusionreasonrelevanttothereferencebeingscreened.Pleaseaddinanyadditionalnotesinthecommentboxtoexplainyourselectionifnecessary.

Categories(selectone):

76

• Exclude—Articleisareviewofformaldehydeexposureandasthma;• Exclude—Articlecontainsnooriginaldata(e.g.,editorial,reviewpapernotrelevantto

studyquestion,etc.);• Exclude—Articledidnotinvolvehumansubjects(i.e.,animalevidenceorcelllineonly,

casereportofsinglehuman,etc.);• Exclude—Articledoesnotreportexposurestoformaldehyde,asdefinedbythePECO

statement;• Exclude—Articledoesnotreportdiagnosisofasthma,asthmasignsorsymptoms,asthma

exacerbation,orindirectmeasuresofasthma,asdefinedbythePECOstatement;• Exclude—Therewasnocomparatorgroup;• Exclude—Duplicatestudy;• Exclude—Otherreasons(explanationrequired).

• POSSIBLYInclude—otherlanguage(specifywhichlanguagebelowincomments);

• IncludeDefinition:Studymeetsinclusioncriteriaasfollows:

Populationstudiedishumans.Exposureisanyindoororoutdoorsourcesofairborneinhalationexposuretoformaldehyde(maternal/paternalpriortoconception,maternalexposureduringpregnancy,orexposuretotheindividualduringanylifestage),includingbutnotlimitedtooccupational,outdoorambient,indoorhouseholdsettings,and/orexposuretohouseholdproducts(medium-densityfiberboard,particleboard,compositewood,pressedwood,polyurethanefoam,adhesives,mobilehomes,trailerhomes,etc.)thatoccurredpriororconcurrenttodiagnosis,exacerbation,orothermeasureofasthma.Comparatorishumansexposedtolowerlevelsofformaldehydethanthemorehighlyexposedhumans.Outcomeisanyofthefollowingasthma-relatedoutcomes:diagnosisofasthma(clinicalasthmadiagnosisbyahealthcareprovider,whichmayincludeparentalreportofaclinicalasthmadiagnosisfortheirchild),asthmasigns,symptoms,orexacerbation(asthmasymptoms(cough,wheeze,chestpain,etc.),asthmasymptomsleadingtobronchodilatoruse(albuterolonly),asthma-specificemergencydepartmentvisits,outpatientvisits,orhospitaladmissions,asthma-specificICUadmissions/intubations,asthma-specificdetailedmedicationuse,AsthmaSyndromeUtilityIndex(ASUI)retrospectivequestionnaire,useofsystemiccorticosteroidsforasthma,methacholinechallengetest,and/orchangesinspirometrymeasurese.g.,FEV1,FVC,FEV1/FVC,PEForPEFR,etc.overtime)orindirectmeasuresofasthma(schoolabsences,lossofworkerdays,etc.impactingthequalityoflifeasaresultofasthmaeffects).

Comments:(explainhereifreasonforexclusionisotherthanreasonsprovidedinabove,explainwhythisispossiblyaduplicatestudy,orspeculatewhatlanguagestudyappearstobeifnotinEnglish)

77

AmendmentstoFullTextScreeningProcess

Addhereanyadditionaldetailsifnecessary.

78

AppendixV.DataCollectionFormsThesourcecriteriachecklistsforextractiontermsinclude:goldstandardpublicationchecklist(GSPC);ARRIVEguidelines(ARRIVE);CochraneHandbookforSystematicReviewsofInterventionsdatacollectionchecklist(Cochrane);GRADEcriteriaforrandomizedcontroltrials(GRADE).

DataCollectionforHumanstudies

Fieldsarefree-formexceptwherechoices(initalics)areshown

SOURCE

Refid:

Reviewer:

Publicationyear:

Authors’declaredconflictsofinterest:

• Nonedeclared

• Declared

Ifdeclared,providedetails:

Studyfundingsource:

• Governmentgrant

• Industryfunded

• Nonprofitorganizationgrant

• Other

Studyfundingsourcedetails:

Whatarethestudyobjectives?:

Site(s)ofdatacollection(city,state,country):

METHODS

Studyduration/dates:

Studydesign:

79

• Cross-sectional

• Cohort,prospective

• Cohort,retrospective

• Case-control

• Ecological

• Other(listdetailsbelow)

Studydesigndetails:

STUDYPOPULATIONCHARACTERISTICS

Cohort(givedescription,e.g.NHANES2004-2006)

Samplesizeoftotalcohort

Totalnumberofstudygroups

Descriptionofreferencegroup

Samplesize(eachstudygroup)

Targetsamplesize

Participation/follow-uprates

Inclusion/exclusioncriteria/recruitmentstrategy

Age(eachexposuregroup)

Co-morbidities

Otherrelevantdetails(listbelow)

Exposuremeasurementtiming:

• Maternal/paternalexposurepriortoconception

• Inutero

• Prenatalperiod

• Infancyperiod(upto24months)

80

• Childhoodperiod(24monthsandafter)

• Other(detailsbelow)

Exposuremeasurementtimingdetails:

Sourceofexposuredata:

• Biomonitoring(listspecificmatrix)

• Environmentalmonitoring(listspecificmatrix)

• Questionnaire(listspecificdeterminantofexposure)

• Other(specify)

Rangeofconcentrationsofformaldehydemeasured,andunits:

Frequencyofexposuremeasurementsifmorethanonce:

Numberofreplicatemeasurementstaken:

Otherchemicalinformation:

Outcomesmeasured:

Methodofasthmaoutcomemeasurement/assessment:

Sex(whereoutcomemeasured):

• Malesonly

• Femalesonly

• Malesandfemales


Numbersubjectsanalyzed(forexposureandoutcome):

Numberofmissingparticipants:

RESULTS

81

Statisticalmethods:

• Statisticaltestsemployed

• Statistic(oddsratio,adjustedoddsratio,betaestimate,etc.)

• p-valuesgiven

• Confidenceintervalsgiven

• Confoundingadjustmentsinstatisticaltests

Wereknownconfoundersaccountedforbystudydesign?

Wereknownconfoundersaccountedforbyanalysis?

Howweredatareported(mean,median,rawdata,etc.)?:

Asthmameasurement/assessmentdataforeachgroup(i.e.,outcome),ifavailable:

Howasthmameasurement/assessmentdatawerereported(table,figure,etc.),ifavailable:

Summarydataforeachgroup

Estimateofeffectwithconfidenceintervalandp-value

Howwasprecisionreported(standarderror,CI,etc.)?:

• Standarderror

• Standarddeviation

• Confidenceintervals


• Notstated

Howprecisionreporteddetails:

Precisionestimates:

Howprecisionestimateswerereported(table,figure,etc):

Miscellaneouscommentsbyreviewerregardingdataanalysis:

82

AppendixVI.InstructionsforMakingRiskofBiasDeterminations

HumanStudies

Note:Thesecriteriaforjudgingriskofbiasareforhumanstudiesonlysincewearenotevaluatinganimalstudiesinthiscasestudy.

Instructions:

• Pleaseevaluateeachindividualstudyforthefollowingnineriskofbiasdomains.Pleaseanswer“lowrisk,”“probablylowrisk,”“probablyhighrisk,”“highrisk,”or“notapplicable”andprovidedetails/justificationforyourrating.Ifthereisempiricalevidenceorotherknowledgethatinformsthedirectionofbias,pleaseincludethisinyouransweraswell;however,ifthereisnotenoughinformationtodosopleasedonotguessatthedirectionofbias.

• Additionally,pleasenotethatsomeinternalvalidityissuescouldpotentiallybeappropriatelycapturedinseveraldifferentriskofbiasconsiderations.Inthissituation,pleaseselectthesinglemostappropriatedomaintoevaluatethispotentialbias,toavoiddouble-countingthesameinternalvalidityconcern.

1.Arethestudygroupsatriskofnotrepresentingtheirsourcepopulationsinamannerthatmightintroduceselectionbias?Thesourcepopulationisviewedasthepopulationforwhichstudyinvestigatorsaretargetingtheirstudyquestionofinterest.Examplesofconsiderationsforthisriskofbiasdomaininclude:1)levelofdetailreportedforparticipantinclusion/exclusion(includingdetailsfrompreviouslypublishedpapersreferencedinthearticleforanexistingcohort);2)participationratesandwhetherthisdifferedbyexposureoroutcomegroup;3)attritionratesandreasons;and4)comparisonsofstudycharacteristicsbetweenthestudypopulationandfullcohort.CriteriaforajudgmentofLOWriskofbias(i.e.,answer:“No”):

EITHER:a)Thedescriptionsofthesourcepopulation,inclusion/exclusioncriteria,recruitmentandenrollmentprocedures,participationandfollow-upratesweresufficientlydetailed,andadequatedataweresuppliedonthedistributionofrelevantstudysampleandpopulationcharacteristicstosupporttheassertionthatriskofselectioneffectswasminimal.ORb)Althoughthedescriptionsand/ordataasindicatedin“a”abovesuggestedthepotentialforselectioneffects,adequatesupportwasgivenindicatingthatpotentialselectioneffectswerenotdifferentialacrossbothexposureandoutcome.

83

ORc)Althoughthedescriptionsand/ordataasindicatedin“a”abovesuggestedthepotentialforselectioneffectsandtherewasnosupportindicatingthatpotentialselectioneffectswerenotdifferentialacrossbothexposureandoutcome,selectionfactorsappearedtobewell-understood,weremeasuredinthedataset,andappropriateadjustmentposthoctechniqueswereusedtocontrolforselectionbias.

CriteriaforthejudgmentofPROBABLYLOWriskofbias(i.e.,answer:“ProbablyNo”):

Thereisinsufficientinformationaboutparticipantselectiontopermitajudgmentoflowriskofbias,butthereisindirectevidencewhichsuggeststhatinclusion/exclusioncriteria,recruitmentandenrollmentprocedures,andparticipationandfollow-uprateswereconsistentacrossgroupsasdescribedbythecriteriaforajudgmentoflowriskofbias.

CriteriaforthejudgmentofPROBABLYHIGHriskofbias(i.e.,answer:“ProbablyYes”):

Thereisinsufficientinformationaboutparticipantselectiontopermitajudgmentofhighriskofbias,butthereisindirectevidencewhichsuggeststhatinclusion/exclusioncriteria,recruitmentandenrollmentprocedures,andparticipationandfollow-uprateswereinconsistentacrossgroups,asdescribedbythecriteriaforajudgmentofhighriskofbias.

CriteriaforthejudgmentofHIGHriskofbias(i.e.,answer:“Yes”):

a) Therewereindicationsfromdescriptionsofthesourcepopulation,

inclusion/exclusioncriteria,recruitmentandenrollmentprocedures,participationandfollow-uprates,ordataonthedistributionofrelevantstudysampleandpopulationcharacteristicsthatriskofselectioneffectsweresubstantial;and

b) Therewasnosupporttoindicatethatpotentialselectioneffectswerenotdifferentialacrossbothexposureandoutcome;and

c) Adjustmentposthoctechniqueswerenotusedtocontrolforselectionbias.CriteriaforthejudgmentofNOTAPPLICABLE(riskofbiasdomainisnotapplicabletostudy):

Thereisevidencethatparticipantselectionisnotanelementofstudydesigncapableofintroducingriskofbiasinthestudy.

2.Wasknowledgeofthegroupassignmentsinadequatelyprevented(i.e.,blindedormasked)duringthestudy,potentiallyleadingtosubjectivemeasurementofeitherexposureoroutcome?CriteriaforajudgmentofLOWriskofbias(i.e.,answer:“No”):

Anyofthefollowing:

84

• Noblinding,butthereviewauthorsjudgethattheoutcomemeasuresaswellastheexposuremeasuresarenotlikelytobeinfluencedbylackofblinding(suchasdifferentialoutcomeassessmentwheretheoutcomeisassessedusingdifferentmeasurementorestimationmetricsacrosstheexposuregroups,ordifferentialexposureassessmentwhereexposureisassessedusingdifferentmeasurementorestimationmetricsacrossthediagnosticoroutcomegroups);or

• Blindingofkeystudypersonnelwasensured,anditisunlikelythattheblindingcouldhavebeenbroken;or

• Somekeystudypersonnelwerenotblinded,butexposureandoutcomeassessmentwasblindedandthenon-blindingofothersisunlikelytointroducebias.


Thereisinsufficientinformationaboutblindingtopermitajudgmentoflowriskofbias,butthereisindirectevidencewhichsuggeststhestudywasadequatelyblinded,asdescribedbythecriteriaforajudgmentoflowriskofbias.Forexample,investigatorswereeffectivelyblindedtotheexposureand/oroutcomegroupsiftheexposurewasmeasuredbyaseparateentityandtheoutcomewasobtainedfromahospitalrecord.


Thereisinsufficientinformationaboutblindingtopermitajudgmentofhighriskofbias,butthereisindirectevidencewhichsuggeststhestudywasnotadequatelyblinded,asdescribedbythecriteriaforajudgmentofhighriskofbias.


Anyofthefollowing:• Noblindingorincompleteblinding,andtheoutcomemeasuresorexposure

measuresislikelytobeinfluencedbylackofblinding(i.e.,differentialoutcomeorexposureassessment);or

• Blindingofkeystudypersonnelattempted,butlikelythattheblindingcouldhavebeenbrokensoastointroducebias;or

• Somekeystudypersonnelwerenotblinded,andthenon-blindingofotherswaslikelytointroducebias.

CriteriaforthejudgmentofNOTAPPLICABLE(riskofbiasdomainisnotapplicabletostudy):

Thereisevidencethatblindingisnotanelementofstudydesigncapableofintroducingriskofbiasinthestudy.

3.Wereexposureassessmentmethodslackingaccuracy?

Thefollowinglistofconsiderationsrepresentsacollectionoffactorsproposedbyexpertsinvariousfieldsthatmaypotentiallyinfluencetheinternalvalidityoftheexposureassessmentinasystematicmanner(notthose

85

thatmayrandomlyaffectoverallstudyresults).Theseshouldbeinterpretedonlyassuggestedconsiderations,andshouldnotbeviewedasscoringorachecklist.ListofConsiderations:Possibleexposureassessmentmetrics:

1) Modeling2) Monitoring3) Biomarkers4) Surrogatemeasure(i.e.,parentalsurveyofhouseholdexposures)

Foreach,overallconsiderationsinclude:

1) Whatisthequalityofthemetricbeingused?2) Hasthemetricbeenvalidatedforthescenarioforwhichitisbeingused?3) Istheexposuremeasuredinthestudyasurrogateforformaldehyde(i.e.,liveinanewhome)?4) Whatwasthetemporalcoverage(i.e.,shortorlong-termexposure,childhood,occupational)?5) Didtheanalysisaccountforpredictionuncertainty?6) Howwasmissingdataaccountedfor,andanydataimputationsincorporated?7) Weresensitivityanalysesperformed?

Inparticular,forexposureassessmentmodels:

1) Weretheinputdatainthestudysuspectedtosystematicallyunder-orover-estimateexposure?2) Whattypeofmodelwasused(geostatisticalinterpolation,land-useregression,dispersionmodels,

personalairsamplingmodels,hybridmodels,etc.)?3) Weremeteorologicalvariablesincorporatedinthemodelandjustifiedbyauthorsintheirselection?4) Weredataonlanduse,topography,traffic,monitoringdata,emissionrates,etc.incorporatedand

justifiedbyauthorsintheirselection?5) Whatwasthespatialvariation(e.g.,distancefromsource)andgeographic/spatialaccuracy(county,

censustract,individualresidence)?6) Whatwasthetemporalspecificityandvariation(accuracytoleveloftheday,pregnancytrimester,

year,etc.?)7) Whatwastheaddresscompleteness(e.g.,onlyhomeaddressatonepointintime,ormorecomplete

addresshistorythroughoutpregnancy/postnatallifeandotherlocationssuchaswork)?8) Whatwasthespace-timecoverageofthemodel?9) Weretime-activitypatternsaccountedfor?10) Wereconcentrationsofformaldehyderesultingfromaproductofsecondaryformationaccountedfor

inthemodel?CriteriaforajudgmentofLOWriskofbias(i.e.,answer:“No”):

Thereviewersjudgethatthereislowriskofexposuremisclassification,i.e.:• Thereishighconfidenceintheaccuracyoftheexposureassessmentmethods,

suchasmethodsthathavebeentestedforvalidityandreliabilityinmeasuringthetargetedexposure;or

• Less-establishedorlessdirectexposuremeasurementsarevalidatedagainstwell-establishedordirectmethods;or:

A) Biomarkers:adirectmeasureofformaldehydeexposureduringthetimeperiodthatexposureisconsideredrelevant(i.e.,asdefinedinthePECOstatement)wasused,andthereissufficientevidencethatrelevantfactorsfromtheListofConsiderationsabovewouldimplyminimalriskofbiasintheexposureassessment;or

86

B) Monitoring:directandpersonalmonitoringdevicesorotherindoorairmonitoringdevicewereusedthathavebeenvalidatedfortheformaldehydeexposureandscenarioforwhichitwasusedandthereissufficientevidencethatrelevantfactorsfromtheListofConsiderationsabovewouldimplyminimalriskofbiasintheexposureassessment;or

C) Modeling:themodelaccountedforthetime-activitypatternspecifictoeachresearchparticipant,(e.g.includesmorethanexposureattheresidentialaddress)andincludedmodelingmethodsthathavebeenvalidatedorshowntohaveahighdegreeofspatialaccuracy(e.g.pointlocation),and/ormethodsthatarethemselvesvalidatedwithgoodagreementcomparedtoperson-basedairdatacollection;andthereissufficientevidencethatrelevantfactorsfromtheListofConsiderationsabovewouldimplyminimalriskofbiasintheexposureassessment.ANDifapplicable(e.g.forlaboratorymeasurements),appropriateQA/QCformethodsaredescribedandaresatisfactory,withatleastthreeofthefollowingitemsreported,oratleasttwoofthefollowingitemsreportedplusevidenceofsatisfactoryperformanceinahighqualityinter-laboratorycomparison:

• Limitofdetectionorquantification;• standardsrecovery;• measureofrepeatability;• investigationandpreventionofblankscontamination.


Thereisinsufficientinformationabouttheexposureassessmentmethodstopermitajudgmentoflowriskofbias,butthereisindirectevidencethatsuggeststhatmethodswererobust,asdescribedbythecriteriaforajudgmentoflowriskofbias.StudiesonlyreportingthattheQA/QCitemsaboveweresatisfactorybutnotreportingalloftheactualnumbersmayreceiveajudgmentof“probablylowriskofbias.”Additionally:

A) Biomarkers:ameasurethatincludedformaldehydeexposureduringthetimeperiodthatexposureisconsideredrelevantandhasbeenvalidatedasadirectmeasureofexposure(i.e.,asdefinedinthePECOstatement)wasused,orthereissomeevidencethatrelevantfactorsfromtheListofConsiderationsabovewouldimplyminimalriskofbiasintheexposureassessment.

B) Monitoring:methodologieswhichdirectlyassessexposurewereused,suchaspersonalexposureinstrumentsorindoorairmonitors,buthadnotbeenvalidatedforthatpurpose,orifsuchinstrumentswerewornforlessthan4hoursperday,orthereissomeevidencethatrelevantfactorsfromtheListofConsiderationsabovewouldimplyminimalriskofbiasintheexposureassessment.

C) Modeling:themodelusedmethodsthatdonotmeetthecriteriaofincludingtime-activitypatternsANDspatialaccuracy,andsomaynothavethelevelofvalidationcomparedtoperson-basedairmeasurement,butincludemeasurementsthathaveevidenceofquality,suchasgood-qualitydatainputs,validationagainstarea-basedairmeasurement,orotherestablishmentsoftheaccuracyofthedatainputsandmodels,orthereissomeevidencethatrelevantfactorsfromtheListofConsiderationsabovewouldimplyminimalriskofbiasintheexposureassessment.

87


Thereisinsufficientinformationabouttheexposureassessmentmethodstopermitajudgmentofhighriskofbias,butthereisindirectevidencethatsuggeststhatmethodswerenotrobust,asdescribedbythecriteriaforajudgmentofhighriskofbias.Additionally:

A) Biomarkers:thisincludesindirectmeasuresofformaldehydeexposurebutnotspecific

tothisexposure,suchasDNAadducts,inflammationoroxidativestress,duringthetimeperiodthatexposureisconsideredrelevant(i.e.,developmentalperiodasdefinedinthePECOstatement),orthereissomeevidencethatrelevantfactorsfromtheListofConsiderationsabovewouldimplyriskofbiasintheexposureassessment.

B) Monitoring:measurementofexposuresthatmaynothavebeenvalidatedforusetostudyformaldehydeexposurewereused,orthereissomeevidencethatrelevantfactorsfromtheListofConsiderationsabovewouldimplyriskofbiasintheexposureassessment.

C) Modeling:modelswereusedthathavenotbeencomparedtoperson-basedorarea-basedairmeasurementsandhavesuspicionofproblemsestimatingtrueexposurebecause,forexample,theydonothavespatialaccuracy(e.g.county-levelmeasures),donotpertaintothecorrecttimeframe,arebasedonlimiteddata,ordifferinmethodologybetweencasesandcontrolsinastudy,orthereissomeevidencethatrelevantfactorsfromtheListofConsiderationsabovewouldimplyriskofbiasintheexposureassessment.


Thereviewersjudgethatthereishighriskofexposuremisclassificationandanyoneofthefollowing:

• Thereislowconfidenceintheaccuracyoftheexposureassessmentmethods;or• Less-establishedorlessdirectexposuremeasurementsarenotvalidatedandare

suspectedtointroducebiasthatimpactstheoutcomeassessment(example:participantsareaskedtoreportexposurestatusretrospectively,subjecttorecallbias);or

• Uncertainhowexposureinformationwasobtained;or:

A) Biomarkers:ThereissufficientevidencethatrelevantfactorsfromtheListofConsiderationsabovewouldimplyriskofbiasintheexposureassessment.

B) Monitoring:InformationfromdatabasesorotherwisewasgatheredthatindirectlyassessedexposurewithoutconsideringvariablesnotedintheListofConsiderationsabove,suchasspatialvariability,landuseregression,etc.,orthereissufficientevidencethatrelevantfactorsfromtheListofConsiderationsabovewouldimplyriskofbiasintheexposureassessment.

C) Modeling:themodelusedhasbeendemonstratednottopertaintoarea-basedorperson-basedmeasuresorhasotherwisebeenpreviouslydemonstratedtobeunabletodescribeairlevelsofexposureforassigningexposureinaresearchsituation,orthereissufficient

88

evidencethatrelevantfactorsfromtheListofConsiderationsabovewouldimplyriskofbiasintheexposureassessment.


Thereisevidencethatexposureassessmentmethodsarenotcapableofintroducingriskofbiasinthestudy.

4.Wereoutcomeassessmentmethodslackingaccuracy?CriteriaforajudgmentofLOWriskofbias(i.e.,answer:“No”):

Thereviewersjudgethatthereislowriskofoutcomemisclassification,i.e.:

• Outcomeswereassessedanddefinedconsistentlyacrossallstudyparticipants,usingvalidandreliablemeasures(notethatalloutcomeassessmentmeasurescapturedinthePECOstatement(clinicalasthmadiagnosisbyhealthpractitioner,asthma-relatedhospitaladmissionsoremergencydepartmentvisitsoroutpatientvisitsorICUadmissions/intubations,asthma-relatedmedicationuse,ASUIretrospectivequestionnaire,useofsystemiccorticosteroidsforasthma,wheezing,methacholinechallengetest,and/orspirometrymeasure)areconsideredbeforehandtobevalidandreliable,unlessotherinformationprovidedwithinthestudywarrantsaconsiderationotherwise);or

• Less-establishedorlessdirectoutcomemeasurementsarevalidatedagainstwell-establishedordirectmethods;or

• Appropriatesensitivityanalyseswereconductedthatsuggesttheinfluenceofoutcomemisclassificationwouldbeminimal

• AND,ifapplicable,appropriateQA/QCformethodsisdescribedandissatisfactory.


Thereisinsufficientinformationabouttheoutcomeassessmentmethodstopermitajudgmentoflowriskofbias,butthereisindirectevidencewhichsuggeststhatmethodswererobust,asdescribedbythecriteriaforajudgmentoflowriskofbias.AppropriateQA/QCformethodsarenotdescribedbutthereviewauthorsjudgethattheoutcomeandtheoutcomeassessmentareobjectiveanduniformacrossstudygroups.


Thereisinsufficientinformationabouttheoutcomeassessmentmethodstopermitajudgmentofhighriskofbias,butthereisindirectevidencewhichsuggeststhatmethodswerenotrobust,asdescribedbythecriteriaforajudgmentofhighriskofbias.

89


Thereviewersjudgethatthereishighriskofoutcomemisclassificationandanyoneofthefollowing:

• Thereislowconfidenceintheaccuracyoftheoutcomeassessmentmethods;or• Less-establishedorlessdirectoutcomemeasurementsarenotvalidatedandare

suspectedtointroducebiasthatimpactstheoutcomeassessment• Uncertainhowoutcomeinformationwasobtained


Thereisevidencethatoutcomeassessmentmethodsarenotcapableofintroducingriskofbiasinthestudy.

5.Waspotentialconfoundinginadequatelyincorporated?Listofimportantpotentialconfounders,collectivelygeneratedbyreviewauthorspriortotheinitiationofscreeningforstudiesbasedonexpertopinionandknowledgegatheredfromtheliterature:TierI:Importantconfounders

Occupationalstudies–(1)smokingstatusorexposuretosecondhandorenvironmentaltobaccosmoke(unlessconductedatatimeorplaceknowntohavebannedsmokingintheworkplaceinwhichcasewewouldonlyconsideractivesmoking);and(2)socioeconomicstatus.

Non-occupationalstudies–(1)age,instudieswhereparticipantsare6yearsoldorless;(2)smokingstatusorexposuretosecondhandorenvironmentaltobaccosmoke;and(3)socioeconomicstatus/parentaleducationinstudiesofchildren.

Age,smokingstatusincludingexposuretosecondhandorenvironmentaltobaccosmoke,andsocioeconomicstatus/parentaleducationareconsideredimportantconfoundersforasthereisevidencetheyarerelatedtoboththeexposure(formaldehyde)andhealthoutcome(asthma).

Age(instudiesofchildren6yearsoldorless)isconsideredanimportantconfounderbecauseitcanimpactthediagnosisofasthma,particularlyatyoungerageranges(<6years)(Morganetal.2005).Agemayalsobeassociatedwiththelevelsofformaldehydeexposures,inhalationrates,timespentindoorsathomeversusatschool,mayvarybyage(Moyaetal.2004).

Smokingstatusandexposuretosecondhand/environmentaltobaccosmokeareconsideredimportantconfoundersbecausecigarettesmokecontainsformaldehyde(WordHealthOrganization2004,Bolteetal.2008)andexposuretocigarettesmokeisalsogenerallyarecognizedriskfactorforasthmainchildhood(Ferranteetal.2014).

90

Socioeconomicstatus(andparentaleducationasasurrogatemeasure)isconsideredanimportantconfounderbecauseSESisrelatedtoindoorexposurestoformaldehyde(Hunetal.2009)andalsorelatedtotheoutcomeofasthma(Bhanetal.2015).

TierII:Otherpotentiallyimportantconfounders:

Race/ethnicity,sex,height,weight,BMI,obesitystatus,parentalorfamilyhistoryofasthma,allergy,additionalenvironmental(non-formaldehyde)exposures.

Theseconfounderswereselectedfromevidenceintheliterature(BjornsonandMitchell1999,Langleyetal.2003,Brüskeetal.2014)thatsuggestedthesemightbepotentialconfounders,stronglyassociatedwithoneofeitherexposureoroutcomebutnotasstronglyassociatedwiththeother,orthatmaybepotentialsurrogatesforoneoftheimportantconfounderslistedabove.Toavoidover-adjustmentforconfounders,theseTierIIconfoundersarecharacteristicsthatcouldpotentiallybeadjustedfor,butwouldnotbeaprimaryconsiderationinevaluatingastudy’sriskofbias.

Seasonisconsideredanpotentialconfounderbecauseitcanimpactasthmasymptomsinchildren,whicharemorelikelytooccurinthefall(SearsandJohnston2007).Likewise,formaldehydeexposuresbothindoorandoutdoormayalsovarybyseason(CaliforniaEnvironmentalProtectionAgency1992,U.S.ConsumerProductSafetyCommission2015).Ageinstudies(involvingindividuals>6yearsold)wasconsideredapotentialconfounderitcanbeassociatedwithdifferenttypesofoutdoorexposuresandthelevelofexposure(sincerespiratoryratesandlevelsofcontactwithfloorandwoodproductsdifferbyagegroup)(Beameretal.2012,Hospitaletal.2015);however,age>6yearsisnotknowntobeassociatedwithasthmadiagnosis,signsorsymptomsandsoaslongasageisaconsiderationinassessingpulmonaryfunctiontests,itwouldnotbeconsideredapotentiallyimportantconfounder.

CriteriaforajudgmentofLOWriskofbias(i.e.,answer:“No”):

Thestudyappropriatelyassessedandaccountedfor(i.e.,matched,stratified,excludedcertainpopulations(i.e.,smokers)orstatisticallycontrolledfor)allimportantconfounders(TierI)usingappropriatestatisticaltechniques,orreportedthatimportantconfounderswereevaluatedandomittedbecauseinclusiondidnotsubstantiallyaffecttheresults.Thedeterminationofspecificconfoundersmayalsobeinformedby,butnotlimitedto,thestudiesincludedintheoverallreview,ANDthestudyappropriatelyassessedandaccountedfor(i.e.,matched,stratified,orstatisticallycontrolledfor)otherpotentiallyimportantconfoundersrelevant(TierII)usingappropriatestatisticaltechniques,orreportedthattheseconfounderswereevaluatedandomittedbecauseinclusiondidnotsubstantiallyaffecttheresults,

91

ANDtheimportantpotentialconfoundersweremeasuredconsistentlyacrossstudygroupsusingvalidandreliablemethods,ortheinfluenceofcovariatemeasurementerrorwasdetermined,throughsensitivityanalysis,tobeminimal.


Thestudyappropriatelyaccountedformostbutnotalloftheimportantconfounders(TierI)orusedappropriatestatisticaltechniques;ANDsomeoftheotherpotentiallyimportantconfoundersrelevant(TierII)usingappropriatestatisticaltechniques,ORreportedthattheseconfounderswereevaluatedandomittedbecauseinclusiondidnotsubstantiallyaffecttheresults;ANDthisisnotexpectedtointroducesubstantialbias.


Thestudyevaluatedsomebutnotalloftheimportantconfounders(TierI),ANDsomebutnotalloftheotherpotentiallyimportantconfoundersrelevant(TierII),ORusedquestionablestatisticaltechniquesforconfounderadjustment;ANDthisisexpectedtointroducesubstantialbias.


Thestudydidnotaccountfororevaluatemultipleimportantconfounders(TierI),ANDdidnotaccountfororevaluatemultipleotherpotentiallyimportantconfoundersrelevant(TierII),ORtheimportantpotentialconfounderswereinappropriatelymeasuredand/orinappropriatelyanalyzedacrossstudygroups.


Thereisevidencethatoutcomeassessmentmethodsarenotcapableofintroducingriskofbiasinthestudy.

6.Wereincompleteoutcomedatainadequatelyaddressed?CriteriaforajudgmentofLOWriskofbias(i.e.,answer:“No”):

Participantswerefollowedlongenoughtoobtainoutcomemeasurements

92

ORanyoneofthefollowing:

• Nomissingoutcomedata;or• Reasonsformissingoutcomedataunlikelytoberelatedtotrueoutcome(for

survivaldata,censoringunlikelytointroducebias);or• Attritionormissingoutcomedatabalancedinnumbersacrossexposuregroups,

withsimilarreasonsformissingdataacrossgroups;or• Fordichotomousoutcomedata,theproportionofmissingoutcomescompared

withobservedeventrisknotenoughtohavearelevantimpactontheexposureeffectestimate;or

• Forcontinuousoutcomedata,plausibleeffectsize(differenceinmeansorstandardizeddifferenceinmeans)amongmissingoutcomesnotenoughtohavearelevantimpactontheobservedeffectsize;or

• MissingdatahavebeenimputedusingappropriatemethodsCriteriaforthejudgmentofPROBABLYLOWriskofbias(i.e.,answer:“ProbablyNo”):

Thereisinsufficientinformationaboutincompleteoutcomedatatopermitajudgmentoflowriskofbias,butthereisindirectevidencewhichsuggestsincompleteoutcomedatawasadequatelyaddressed,asdescribedbythecriteriaforajudgmentoflowriskofbias.


Thereisinsufficientinformationaboutincompleteoutcomedatatopermitajudgmentofhighriskofbias,butthereisindirectevidencewhichsuggestsincompleteoutcomedatawasnotadequatelyaddressed,asdescribedbythecriteriaforajudgmentofhighriskofbias.


ParticipantswerenotfollowedlongenoughtoobtainoutcomemeasurementsORanyoneofthefollowing:

• Reasonformissingoutcomedatalikelytoberelatedtotrueoutcome,witheitherimbalanceinnumbersorreasonsformissingdataacrossexposuregroups;or

• Fordichotomousoutcomedata,theproportionofmissingoutcomescomparedwithobservedeventriskenoughtoinducebiologicallyrelevantbiasininterventioneffectestimate;or

• Forcontinuousoutcomedata,plausibleeffectsize(differenceinmeansorstandardizeddifferenceinmeans)amongmissingoutcomesenoughtoinducebiologicallyrelevantbiasinobservedeffectsize;or

• Potentiallyinappropriateapplicationofimputation.CriteriaforthejudgmentofNOTAPPLICABLE(riskofbiasdomainisnotapplicabletostudy):

Thereisevidencethatincompleteoutcomedataisnotcapableofintroducingriskofbiasinthestudy.

93

7.Doesthestudyreportappeartohaveselectiveoutcomereporting?


Allofthestudy’spre-specified(primaryandsecondary)outcomesoutlinedinthepre-publishedprotocolorthepublishedmanuscript’smethods,abstract,and/orintroductionsectionthatareofinterestinthereviewhavebeenreportedinthepre-specifiedway.


Thereisinsufficientinformationaboutselectiveoutcomereportingtopermitajudgmentoflowriskofbias,butthereisindirectevidencewhichsuggeststhestudywasfreeofselectivereporting,asdescribedbythecriteriaforajudgmentoflowriskofbias.Thisincludesifapre-publishedprotocolisnotavailablebutthestudy’spre-specified(primaryandsecondary)outcomesoutlinedinthepublishedmanuscript’smethods,abstract,and/orintroductionsectionthatareofinterestinthereviewhavebeenreportedinthepre-specifiedway.


Thereisinsufficientinformationaboutselectiveoutcomereportingtopermitajudgmentofhighriskofbias,butthereisindirectevidencewhichsuggeststhestudywasnotfreeofselectivereporting,asdescribedbythecriteriaforajudgmentofhighriskofbias.Thisincludesifapre-publishedprotocolisnotavailableandthestudy’spre-specified(primaryandsecondary)outcomesoutlinedinthepublishedmanuscript’smethods,abstract,and/orintroductionsectionthatareofinterestinthereviewhavenotbeenreportedinthepre-specifiedway.


Anyoneofthefollowing:• Notallofthestudy’spre-specifiedprimaryoutcomes(asoutlinedinthepre-

publishedprotocolorpublishedmanuscript’smethods,abstract,and/orintroduction)havebeenreported;or

• Oneormoreprimaryoutcomesisreportedusingmeasurements,analysismethodsorsubsetsofthedata(e.g.subscales)thatwerenotpre-specified;or

• Oneormorereportedprimaryoutcomeswerenotpre-specified(unlessclearjustificationfortheirreportingisprovided,suchasanunexpectedeffect);or

• OneormoreoutcomesofinterestarereportedincompletelyCriteriaforthejudgmentofNOTAPPLICABLE(riskofbiasdomainisnotapplicabletostudy):

Thereisevidencethatselectiveoutcomereportingisnotcapableofintroducingriskofbiasinthestudy.

94

8.Didthestudyreceiveanysupportfromacompany,studyauthor,orotherentityhavingafinancialinterestinanyoftheexposuresstudied?CriteriaforajudgmentofLOWriskofbias(i.e.,answer:“No”):

Thestudydidnotreceivesupportfromacompany,studyauthor,orotherentityhavingafinancialinterestintheoutcomeofthestudy.Examplesincludethefollowing:

• Fundingsourceislimitedtogovernment,non-profitorganizations,oracademicgrantsfundedbygovernment,foundationsand/ornon-profitorganizations;

• Chemicalsorothertreatmentusedinstudywerepurchasedfromasupplier;• Companyaffiliatedstaffarenotmentionedintheacknowledgementssection;• Authorswerenotemployeesofacompanywithafinancialinterestintheoutcome

ofthestudy;• Companywithafinancialinterestintheoutcomeofthestudywasnotinvolvedin

thedesign,conduct,analysis,orreportingofthestudyandauthorshadcompleteaccesstothedata;

• Studyauthorsmakeaclaimdenyingconflictsofinterest;• Studyauthorsareunaffiliatedwithcompanieswithfinancialinterest,andthereis

noreasontobelieveaconflictofinterestexists;• Allstudyauthorsareaffiliatedwithagovernmentagency(areprohibitedfrom

involvementinprojectsforwhichthereisaconflictofinterestoranappearanceofconflictofinterest).


Thereisinsufficientinformationtopermitajudgmentoflowriskofbias,butthereisindirectevidencewhichsuggeststhestudywasfreeofsupportfromacompany,studyauthor,orotherentityhavingafinancialinterestintheoutcomeofthestudy,asdescribedbythecriteriaforajudgmentoflowriskofbias.


Thereisinsufficientinformationtopermitajudgmentofhighriskofbias,butthereisindirectevidencewhichsuggeststhestudywasnotfreeofsupportfromacompany,studyauthor,orotherentityhavingafinancialinterestintheoutcomeofthestudy,asdescribedbythecriteriaforajudgmentofhighriskofbias.


Thestudyreceivedsupportfromacompany,studyauthor,orotherentityhavingafinancialinterestintheoutcomeofthestudy.Examplesofsupportinclude:

• Researchfunds;• Chemicals,equipmentortestingprovidedatnocost;• Writingservices;

95

• Author/stafffromstudywasemployeeorotherwiseaffiliatedwithcompanywithfinancialinterest;

• Companylimitedauthoraccesstothedata;• Companywasinvolvedinthedesign,conduct,analysis,orreportingofthestudy;• Studyauthorsclaimaconflictofinterest


Thereisevidencethatconflictsofinterestarenotcapableofintroducingriskofbiasinthestudy.

9.Didthestudyappeartohaveotherproblemsthatcouldputitatariskofbias?


Thestudyappearstobefreeofothersourcesofbias.


Thereisinsufficientinformationtopermitajudgmentoflowriskofbias,butthereisindirectevidencewhichsuggeststhestudywasfreeofotherthreatstovalidity.


Thereisinsufficientinformationtopermitajudgmentofhighriskofbias,butthereisindirectevidencewhichsuggeststhestudywasnotfreeofotherthreatstovalidity,asdescribedbythecriteriaforajudgmentofhighriskofbias.

CriteriaforthejudgmentofHIGHriskofbias(i.e.,answer:“Yes”):Thereisatleastoneimportantriskofbias.Forexample,thestudy:

• Hadapotentialsourceofbiasrelatedtothespecificstudydesignused;or• Stoppedearlyduetosomedata-dependentprocess(includingaformal-stopping

rule);or• Theconductofthestudyisaffectedbyinterimresults(e.g.recruitingadditional

participantsfromasubgroupshowinggreaterorlessereffect);or• Hasbeenclaimedtohavebeenfraudulent;or• Hadsomeotherproblem

96

Appendix VII. Instructions for Grading the Quality and Strength of Evidence

A. GradingQuality

Eachofthecategoriestoconsiderindowngradingorupgradingtheevidenceisdescribedindetailbelow.Pleaserecordyourresultsonthechartattheendofeachcategory,includingabriefexplanationforyourratings.

DowngradeCategories

Category1.QualityofStudyLimitations(RiskofBias)(Guyattetal.2011)

Possibleratings:0=nochange;-1or-2downgrade1or2levels

Theevidencefromstudiescanberateddownifmostoftherelevantevidencecomesfromstudiesthatsufferfromahighriskofbias.Riskofbiasisratedbyoutcomeacrossstudies.Studylimitationsforeachoutcomeforindividualstudiesandacrossstudiesaresummarizedintheheatmaps.

GRADEoutlinesthefollowingprinciplesformovingfromriskofbiasinindividualstudiestoratingqualityofevidenceacrossstudies.

1.Indecidingontheoverallqualityofevidence,onedoesnotaverageacrossstudies(forinstanceifsomestudieshavenoseriouslimitations,someseriouslimitations,andsomeveryseriouslimitations,onedoesnotautomaticallyratequalitydownbyonelevelbecauseofanaverageratingofseriouslimitations).Rather,judiciousconsiderationofthecontributionofeachstudy,withageneralguidetofocusonthehigh-qualitystudiesiswarranted.a

2.Thisjudiciousconsiderationrequiresevaluatingtheextenttowhicheachstudycontributestowardtheestimateofmagnitudeofeffect.ThecontributionthateachstudymakeswillusuallyaNote: Limitations to GRADE’s risk of bias assessments as stated by GRADE: “First, empirical evidence supporting the criteria is limited. Attempts to show systematic difference between studies that meet and do not meet specific criteria have shown inconsistent results. Second, the relative weight one should put on the criteria remains uncertain. The GRADE approach is less comprehensive than many systems, emphasizing simplicity and parsimony over completeness. GRADE’s approach does not provide a quantitative rating of risk of bias. Although such a rating has advantages, we share with the Cochrane Collaboration methodologists a reluctance to provide a risk of bias score that, by its nature, must make questionable assumptions about the relative extent of bias associated with individual items and fails to consider the context of the individual items.”

97

reflectstudysamplesizeandnumberofoutcomeevents.Largerstudieswithmanyeventswillcontributemore,muchlargerstudieswithmanymoreeventswillcontributemuchmore.

3.Oneshouldbeconservativeinthejudgmentofratingdown.Thatis,oneshouldbeconfidentthatthereissubstantialriskofbiasacrossmostofthebodyofavailableevidencebeforeoneratesdownforriskofbias.

4.Theriskofbiasshouldbeconsideredinthecontextofotherlimitations.If,forinstance,reviewersfindthemselvesinaclose-callsituationwithrespecttotwoqualityissues(riskofbiasand,say,precision),GRADEsuggestsratingdownforatleastoneofthetwo.

5.Notwithstandingthefirstfourprinciples,reviewerswillfaceclose-callsituations.Youshouldacknowledgethatyouareinsuchasituation,makeitexplicitwhyyouthinkthisisthecase,andmakethereasonsforyourultimatejudgmentapparent.

RatingforRiskofBias(StudyLimitations)

0nochange

-1decreasequality1level

-2decreasequality2levels

Rationaleforyourjudgment

Human

Category2.IndirectnessofEvidence


Qualityofevidence(yourconfidenceinestimatesofeffect)maydecreasewhensubstantialdifferencesexistbetweenthepopulation,exposure,oroutcomesmeasuredintheresearchstudiesunderconsiderationinthereview.

Evidenceisdirectwhenitdirectlycomparestheexposuresinwhichweareinterestedinthepopulationsinwhichweareinterestedandmeasuresoutcomesimportanttothestudyquestion(inGRADEtheoutcomesmustbeimportanttopatients).

98

BasedonGRADE(Guyattetal.2011),evidencecanbeindirectinoneofthreeways.b

1. Thepopulationstudieddiffersfromthepopulationofinterest(thetermapplicabilityisoftenusedforthisformofindirectness).GRADEstatesthatingeneral,oneshouldnotratedownforpopulationdifferencesunlessonehascompellingreasontothinkthatthebiologyinthepopulationofinterestissodifferentthanthepopulationtestedthatthemagnitudeofeffectwilldiffersubstantially.AccordingtoGRADE,mostoften,thiswillnotbethecase.

2. Theintervention(exposure)testedmaydifferfromtheexposureofinterest,i.e.,adifferenceinthechemical,routeand/ordose.Decisionsregardingindirectnessofpopulationsandexposuredependonanunderstandingofwhetherbiologicalorsocialfactorsaresufficientlydifferentthatonemightexpectsubstantialdifferencesinthemagnitudeofeffect.GRADEalsostates,“Aswithallotheraspectsofratingqualityofevidence,thereisacontinuumofsimilarityoftheinterventionthatwillrequirejudgment.Itisrare,andusuallyunnecessary,fortheintendedpopulationsandinterventionstobeidenticaltothoseinthestudies,andweshouldonlyratedownifthedifferencesareconsideredsufficienttomakeadifferenceinoutcomelikely.”

3. Outcomesmaydifferfromthoseofprimaryinterest;forinstance,surrogateoutcomesthatarenotthemselvesimportant,butmeasuredinthepresumptionthatchangesinthesurrogatereflectchangesinanimportantoutcome.Thedifferencebetweendesiredandmeasuredoutcomesmayrelatetotimeframe.Whenthereisadiscrepancybetweenthetimeframeofmeasurementandthatofinterest,whethertoratedownbyoneortwolevelswilldependonthemagnitudeofthediscrepancy.Anothersourceofindirectnessrelatedtomeasurementofoutcomesistheuseofsubstituteorsurrogateendpointsinplaceoftheexposedpopulation’simportantoutcomeofinterest.Ingeneral,theuseofasurrogateoutcomerequiresratingdownthequalityofevidencebyone,oreventwo,levels.Considerationofthebiology,mechanism,andnaturalhistoryofthediseasecanbehelpfulinmakingadecisionaboutindirectness.Surrogatesthatarecloserintheputativecausalpathwaytotheadverseoutcomeswarrantratingdownbyonlyonelevelforindirectness.GRADEstatesthatrarely,surrogatesaresufficientlywellestablishedthatoneshouldchoosenottoratedownqualityofevidenceforindirectness.Ingeneral,evidencebasedonsurrogateoutcomesshouldusuallytriggerratingdown,whereastheothertypesofindirectnesswillrequireamoreconsideredjudgment.

RatingforIndirectness

0nochange



b GRADE includes a fourth type of indirectness that occurs when there are no direct (i.e., head-to-head) comparisons between two or more interventions of interest. This criterion is not relevant to our study question; it could be relevant to future case studies.

99


Human

Category3.InconsistencyofEvidence


AccordingtoCochrane,“whenstudiesyieldwidelydifferingestimatesofeffect(heterogeneityorvariabilityinresults)investigatorsshouldlookforrobustexplanationsforthatheterogeneity.…Whenheterogeneityexistsandaffectstheinterpretationofresults,butauthorsfailtoidentifyaplausibleexplanation,thequalityoftheevidencedecreases.”

BasedonGRADE(Guyattetal.2011),abodyofevidenceisnotratedupinqualityifstudiesyieldconsistentresults,butmayberateddowninqualityifinconsistent.Theirstatedreasonisthataconsistentbiaswillleadtoconsistent,spuriousfindings.

GRADEsuggestsratingdownthequalityofevidenceiflargeinconsistency(heterogeneity)instudyresultsremainsafterexplorationofapriorihypothesesthatmightexplainheterogeneity.Judgmentoftheextentofheterogeneityisbasedonsimilarityofpointestimates,extentofoverlapofconfidenceintervals,andstatisticalcriteria.GRADE’srecommendationsrefertoinconsistenciesineffectsize,specificallytorelativemeasures(riskratiosandhazardratiosoroddsratios),notabsolutemeasures.

BasedonGRADE,reviewersshouldconsiderratingdownforinconsistencywhen:

1.Pointestimatesvarywidelyacrossstudies;

2.Confidenceintervals(CIs)showminimalornooverlap;

3.Thestatisticaltestforheterogeneity-whichteststhenullhypothesisthatallstudiesinameta-analysishavethesameunderlyingmagnitudeofeffect-showsalowP-value;

4.TheI2-whichquantifiestheproportionofthevariationinpointestimatesduetoamong-studydifferences-islarge.(I.e.,theI2indexquantifiesthedegreeofheterogeneityinameta-analysis).

GRADEstatesthatinconsistencyisimportantonlywhenitreducesconfidenceinresultsin

100

relationtoaparticulardecision.Evenwheninconsistencyislarge,itmaynotreduceconfidenceinresultsregardingaparticulardecision.Forexample,studiesthatareinconsistentrelatedtothemagnitudeofabeneficialorharmfuleffect(butareinthesamedirection)wouldnotberateddown;ininstanceswhenresultsareinconsistentastowhetherthereisabenefitorharmoftreatment,GRADEwouldratedownthequalityofevidenceasaresultofvariabilityinresults,becausethemeaningoftheinconsistencyissorelevanttothedecisiontotreatornottotreat.

RatingforInconsistency

0nochange




Human

Category4.ImprecisionofEvidence


Cochranestatesthatwhenstudieshavefewparticipantsandfewevents,andthushavewideconfidenceintervals(CIs),authorscanlowertheirratingofthequalityofevidence.Theseratingsofprecisionaremadeasjudgmentsbyreviewauthors.Theratingsaremadebylookingacrossstudies,or,ifavailable,ontheresultsofameta-analysis.

GRADEdefinesevidencequalitydifferentlyforsystematicreviewsandguidelines.Forsystematicreviews,qualityreferstoconfidenceintheestimatesofeffect.Forguidelines,qualityreferstotheextenttowhichconfidenceintheeffectestimateisadequatetosupportaparticulardecision(Guyattetal.2011).Forthepurposeofstep3ofNavigationGuide,wewillusethesystematicreviewdefinition,becausethedecisionphasedoesnotoccuruntilstep4whenrecommendationsforpreventionaremade.Thus,whenreviewingthedataforimprecision,evaluateyourconfidenceintheestimateoftheeffect.

AccordingtoGRADE,toalargeextent,CIsinformtheimpactofrandomerroronevidencequality.Thus,whenconsideringimprecision,theissueiswhethertheCIaroundtheestimateof

101

exposureeffectissufficientlynarrow.Ifitisnot,GRADEratesdowntheevidencequalitybyonelevel(forinstance,fromhightomoderate).IftheCIisverywide,GRADEmightratedownbytwolevels.

RatingforImprecision

0nochange




Human

Category5.PublicationBias

Possibleratings:0=nochange;-1or-2downgrade1or2levelsGRADE(Guyattetal.2011)andCochrane(HigginsandGreen2011)assesspublicationbiasinasimilarmanner.Whereas“selectiveoutcomereporting”isassessedforeachstudyincludedinthereviewaspartoftheriskofbiasassessment,“publicationbias”isassessedonthebodyofevidence.GRADEstatesthat“whenanentirestudyremainsunreportedandtheresultsrelatetothesizeoftheeffect-publicationbias-onecanassessthelikelihoodofpublicationbiasonlybylookingatagroupofstudies.”

Cochrane’sdefinitionofpublicationbiasis“thepublicationornon-publicationofresearchfindingsdependingonthenatureanddirectionoftheresults.”CochraneandGRADEareprimarilyconcernedwithoverestimatesoftrueeffectsoftreatmentsorpharmaceuticals,especiallyrelatedto“smallstudieseffects”,i.e.,thetendencyforestimatesofaninterventiontobemorebeneficialinsmallerstudies.Thereisempiricalevidenceintheclinicalsciencesthatpublicationandotherreportingbiasesresultinoverestimatingtheeffectsofinterventions(HigginsandGreen2011).

102

Incontrast,inenvironmentalhealth,weareprimarilyconcernedwithunderestimatingthetrueeffectsofachemicalexposure,sinceinmanycasespopulationwideexposurehasalreadyoccurred.Wearealsoconcernedthatstudiesfindingnoassociationarelesslikelytobepublishedbecausejournalsarelesslikelytopublish“negative”findings.

ApplyingthisinvertedconcerntoGRADE’sassessmentforpublicationbias,leadstotheseconsiderationswhenratingpublicationbias:

• Earlynegativestudies,particularlyifsmallinsize,aresuspect.(GRADEisconcernedwithearlypositivestudies).

• Authorsofsystematicreviewsshouldsuspectpublicationbiaswhenstudiesareuniformlysmall,particularlywhensponsoredbytheindustry.(SameasGRADE)

• Empiricalexaminationofpatternsofresults(e.g.,funnelplots)maysuggestpublicationbiasbutshouldbeinterpretedwithcaution.(SameasGRADE)

• Morecompellingthananyofthesetheoreticalexercisesisauthors’successinobtainingtheresultsofsomeunpublishedstudiesanddemonstratingthatthepublishedandunpublisheddatashowdifferentresults.(SameasGRADE)

• Comprehensivesearchesoftheliteratureincludingunpublishedstudies,i.e.,thegreyliterature,andasearchforresearchinotherlanguagesareimportanttoaddressingpublicationbias.NotethatCochranealsostates“comprehensivesearchingisnotsufficienttopreventsomesubstantialpotentialbiases.”

RatingforPublicationBias

0nochange




Human

UpgradeCategories

GRADEstatesthatthecircumstancesforupgradinglikelyoccurinfrequentlyandareprimarilyrelevanttoobservationalandothernon-randomizedstudies.Althoughitispossibletorateupresultsfromrandomizedcontrolledtrials,GRADEhasyettofindacompellingcircumstancefordoingso(Guyattetal.2011).

103

GRADEspecifies3categoriesforincreasingthequalityofevidence(Guyattetal.2011)

Category6.LargeMagnitudeofEffect

Possibleratings:0=nochange;+1or+2upgrade1or2levels

Modelingstudiessuggeststhatconfounding(fromnon-randomallocation)aloneisunlikelytoexplainassociationswitharelativerisk(RR)greaterthan2(orlessthan0.5),andveryunlikelytoexplainassociationswithanRRgreaterthan5(orlessthan0.2).Thus,thesearethedefinitionsof“largemagnitudeofeffect”usedbyGRADEtoupgrade1or2levels,respectively.Also,GRADEismorelikelytorateupiftheeffectisrapidandoutofkeepingwithpriortrajectory;usuallysupportedbyindirectevidence.GRADEpresentsempiricalevidencetosupporttheseconclusions,andstatesthat“althoughfurtherresearchiswarranted,bothmodelingandempiricalworksuggestthesizeofbiasfromconfoundingisunpredictableindirectionbutboundedinsize.Hence,theGRADEgrouphaspreviouslysuggestedguidelinesforratingqualityofevidenceupbyonecategory(typicallyfromlowtomoderate)forassociationsgreaterthan2,andupbytwocategoriesforassociationsgreaterthan5.”ApplyingtheGRADEdefinitionsoflargemagnitudeofeffecti.e.,RRgreaterthan2or5isproblematicinenvironmentalhealthbecausefordichotomousoutcomesRRisafunctionoftheexposurecomparator;thesedefinitionsalsoarenotapplicabletoresultsfromcontinuousvariables.Atpresent,wedonothaveanempiricallydefined“largemagnitudeofeffect.”Therefore,forthepurposeofthiscasestudy,reviewauthorsshouldassesswhethertheresultsindicatealargemagnitudeofeffectusingtheirexpertjudgmentof“largeeffects”inenvironmentalhealthandstatetheirdefinitionfordiscussionbythegroup.RatingforLargeMagnitudeofEffect

0nochange

+1increasequality1level

+2increasequality2levels


Human

104

Category7.Dose-response


Possibleconsiderationsincludeconsistentdoseresponsegradientsinoneormultiplestudies,and/ordoseresponseacrossstudies,dependingontheoverallrelevancetothebodyofevidence.RatingforDose-Response

0nochange




Human

Category8.ResidualConfoundingIncreasesConfidence(NationalToxicologyProgram2015)


Upgradeifconsiderationofallplausibleresidualconfounders,biases,oreffectmodificationwouldunderestimatetheeffectorsuggestaspuriouseffectwhenresultsshownoeffect.Ifastudyreportsanassociationdespitethepresenceofresidualconfounding,biasesoreffectmodificationthatwoulddiminishtheassociation,confidenceintheassociationisincreased.GRADEprovidesanillustrativeexamplerelatedtobias:ratingupobservationalevidencefindinglackofassociationbetweenvaccinationandautism,whichoccurreddespiteempiricallyconfirmedbiasthatparentsofautisticchildrenmaybemorelikelytoremembertheirvaccineexperience.Thenegativefindingsdespitethisformofrecallbiassuggestratingupthequalityofevidence(Guyattetal.2011).

105

RatingforResidualConfoundingIncreasesConfidence

0nochange




Human

Theresultsofthereviewers’ratingsbypopulationwillbecompiledanddiscussedleadingtoafinaldecisiononoverallqualityofhumanevidence.Therationaleforthedecisionwillbefullydocumented.

106

1. Finaldecisiononoverallqualityofhumanevidence:

(Example:Moderatequalityisupgraded1steptohighforXYZreason(s))

----High

----Moderate

----Low

B.RatetheStrengthofEvidence

Theevidencequalityratingswillbetranslatedintostrengthofevidenceforeachpopulationbasedonacombinationoffourcriteria:(1)Qualityofbodyofevidence;(2)Directionofeffect;(3)Confidenceineffect;and(4)Othercompellingattributesofthedatathatmayinfluencecertainty.ThestrengthofevidenceratingsaresummarizedinTable1below,wheretheirmeaningisfurtherdefined.

applying the navigation guide systematic review ... · 2 protocol information review authors juleen...

Documents