applying the navigation guide systematic review ... · 2 protocol information review authors juleen...
Post on 15-Feb-2019
220 Views
Preview:
TRANSCRIPT
1
ApplyingtheNavigationGuideSystematicReviewMethodologyCaseStudy#6
AssociationbetweenFormaldehydeExposureand
Asthma
ASystematicReviewoftheEvidenceProtocolApril2016
2
PROTOCOLINFORMATIONReviewAuthorsJuleenLam,PhD,MHS,MSAssociateResearchScientistDepartmentofObstetrics,Gynecology&ReproductiveSciencesProgramonReproductiveHealthandtheEnvironmentUniversityofCalifornia,SanFranciscoPatriceSutton,MPHAcademicCoordinatorDepartmentofObstetrics,Gynecology&ReproductiveSciencesProgramonReproductiveHealthandtheEnvironmentUniversityofCalifornia,SanFranciscoAmyM.Padula,PhD,MScNovember12015–February29,2016: Instructor Dept.ofPediatrics,DivisionofNeonatology StanfordUniversity March1-October31,2016:AssistantProfessorDepartmentofObstetrics,Gynecology&ReproductiveSciencesProgramonReproductiveHealthandtheEnvironmentUniversityofCalifornia,SanFranciscoMichaelD.Cabana,MD,MPHProfessorofPediatrics,Epidemiology&BiostatisticsPhilipR.LeeInstituteforHealthPolicyStudiesUniversityofCalifornia,SanFranciscoEricaKoustas,PhDScientificconsultanttoUCSFHannaVesterinen,PhDScientificconsultanttoUCSFEvansWhitaker,MD,MLISEducationConsultantfortheSchoolsofMedicineandPharmacyUniversityofCalifornia,SanFranciscoLesleySkalla,PhD,MSLSInformationSpecialistMDB,Inc.NatalynDanielsResearchAssistantDepartmentofObstetrics,Gynecology&ReproductiveSciencesProgramonReproductiveHealthandtheEnvironmentUniversityofCalifornia,SanFrancisco
3
TraceyJ.Woodruff,PhD,MPHProfessorandDirectorDepartmentofObstetrics,Gynecology&ReproductiveSciencesProgramonReproductiveHealthandtheEnvironmentUniversityofCalifornia,SanFranciscoContact:JuleenLamUniversityofCaliforniaatSanFranciscoDepartmentofOB/GYN&RSMailStop0132,55016thSt,7thFloor,SanFrancisco,CA94143Juleen.lam@ucsf.edu;(415)476-3219
4
TABLEOFCONTENTS
PROTOCOLINFORMATION...............................................................................................................................2
BACKGROUND.......................................................................................................................................................5NavigationGuideSystematicReviewMethodology............................................................................................5RationaleforReview:FormaldehydeandAsthma................................................................................................6
AIM...........................................................................................................................................................................8
METHODS...............................................................................................................................................................8ReviewTeam...........................................................................................................................................................................8CriteriaforSelectingStudies..........................................................................................................................................9PECOStatement...................................................................................................................................................................9SearchMethods...................................................................................................................................................................10StudySelectionCriteria...................................................................................................................................................14DataCollection....................................................................................................................................................................17RiskofBiasDetermination............................................................................................................................................17DataAnalysis.......................................................................................................................................................................18QualityandStrengthofEvidenceRatings.............................................................................................................20
SUPPLEMENTARYINFORMATION................................................................................................................22AppendixI.Coauthor/LibrarianBiosketches,ProjectContributionStatements,andConflictofInterestStatements...........................................................................................................................................................22AppendixII.SearchTerms.............................................................................................................................................68AppendixIII.OtherResourcesforLiteratureSearch........................................................................................73
APPENDIXIV.EXCLUSIONCRITERIASCREENINGFORMSANDAMENDMENTSTOCLARIFYSCREENING
PROCESS...................................................................................................................................................................................75AppendixV.DataCollectionForms..........................................................................................................................78AppendixVI.InstructionsforMakingRiskofBiasDeterminations.........................................................82AppendixVII.InstructionsforGradingtheQualityandStrengthofEvidence.........................................96
ABOUTTHEARTICLE.....................................................................................................................................109
ACKNOWLEDGEMENTS.................................................................................................................................109
REFERENCES.....................................................................................................................................................110
5
BACKGROUND
NavigationGuideSystematicReviewMethodology
Robustmethodstoevaluateavailablescientificevidencetoreachconclusionsregardingthestrengthofevidencearefundamentaltospeedingthetranslationofthescienceintopoliciesanddecisionstoimprovehealthoutcomes.Intheclinicalsciences,methodsofevidenceintegrationhaveplayedatransformativeroleinthetimelyincorporationofscienceintotherapeutic,preventiveandcosteffectiveactionattheindividualandsocietallevel(Fox2010).Beginningin2009,researchersbegantoexploretheapplicationofsystematicandrobustmethodsofevidenceintegrationinenvironmentalhealthsciences(Woodruffetal.2011,Rooneyetal.2014).In2014,tworeportsbytheNationalAcademyofSciences(NAS)stronglyendorsedtheuptakeofsuchimprovedmethodsofevidenceintegrationinenvironmentalhealthsciences,andspecificallyencouragedtheirusebytheU.S.EnvironmentalProtectionAgency(EPA)indeterminationsofwhetherenvironmentalchemicalsareharmfultohumanhealth(NationalResearchCouncil2014,2014).TheUSEPAisworkingtoinitiatestepstoincorporateprinciplesofsystematicreviewintoitsIRISprocess(NationalResearchCouncil2014,USEnvironmentalProtectionAgency2014),whiletheNationalInstituteforEnvironmentalHealthSciences’(NIEHS)NationalToxicologyProgram(NTP)OfficeofHealthAssessmentandTranslation(OHAT)hasbeendevelopingthetools,expertise,casestudies,andotherinfrastructurethatwillfacilitateincreasedutilizationofsystematicreviewmethodologies(Rooneyetal.2014,NationalToxicologyProgram2015).
TheNavigationGuidesystematicreviewmethodology(NavigationGuide)wasdevelopedin2011aspartofaninterdisciplinarycollaborationbetweenclinicians,academicians,andpractitionersinanattempttoharmonizetheapproachesforassessingevidenceintheclinicalscienceswithenvironmentalhealthsciences(Woodruffetal.2011).TheNavigationGuideisasystematicandtransparentmethodologythatproceedsfrombestpracticesintheclinicalarena,i.e.,Cochrane(HigginsandGreen2011)andtheGradingofRecommendationsAssessmentDevelopmentandEvaluation(GRADEWorkingGroup2014),butaccountsforthedifferencesinevidenceanddecisioncontextinvolvedinenvironmentalhealthriskassessments,suchastherelianceonanimaltoxicologyandhumanobservationalstudiesintheabsenceofrandomizedcontrolledtrials(RCTs)(Mandrioli2016).Todate,theNavigationGuidehasbeenappliedinfiveproof-of-conceptstudies:1.Toevaluatethehumanandnon-humanevidenceofperfluorooctanoicacid(PFOA)onfetalgrowth(Johnsonetal.2014,Koustasetal.2014,Lametal.2014).FromthisapplicationoftheNavigationGuide,reviewauthorsconcludedthatPFOAwas“knowntobetoxic”tohumanreproductionanddevelopment,basedonafindingof“moderate”qualityand“sufficient”strengthofboththehumanandnon-humanmammalianevidence.
6
2.Toevaluatethehumanandnon-humanevidenceoftheassociationbetweenfetalgrowthandglomerularfiltrationrate(GFR)inordertoassessthestrengthoftheevidenceofa‘reversecausality’hypothesis:thatthesizeofadevelopingfetusmayaffectmaternalGFRsuchthatasmallfetusleadstoreducedplasmavolumeexpansion,reducedGFR,andsubsequentlyhigherconcentrationsofbiomarkersinmaternalserum.Thishadbeenproposedasapotentialalternateexplanationforobservationalstudiesdocumentinganinverseassociationbetweenprenatalexposuretochemicalsclearedrenallyandfetalgrowth(Savitz2007,Whitworthetal.2012,Loccisanoetal.2013).Theauthorsofthisreviewfoundinsufficientevidencetosupporttheplausibilityofthereversecausalityhypothesisandrecommendedfurtherhighqualityresearch(Vesterinenetal.2014).
3.Toevaluatethehumanandnon-humanevidenceoftriclosanonreproductiveand/ordevelopmentaltoxicity.Thisreviewhasbeencompletedandthemanuscriptiscurrentlyinpress(Johnsonetal.2015).
4.Toevaluatethehumanevidenceoftherelationshipbetweenairpollutionandautismspectrumdisorder.Thiscasestudyiscompletedandthemanuscriptiscurrentlybeingdraftedforpublication(Lametal.2015).
5.ToevaluatethehumanevidenceoftherelationshipbetweenPBDEexposureandintelligenceorattention-relatedbehaviorconditions.Thiscasestudyiscompletedandthemanuscriptiscurrentlybeingdraftedforpublication(Lametal.2015).
TheresultsofthesecasestudiestodatedemonstratethatthemethodsunderdevelopmentbytheUSEPAandtheNTParefullyachievable(Johnsonetal.2014,Koustasetal.2014,Lametal.2014,WoodruffandSutton2014).
This6thcasestudyoftheNavigationGuidesystematicreviewmethodinenvironmentalhealthwillassessthehumanevidenceforeffectsofexposuretoformaldehydeandasthma-relatedoutcomes.ThehumanhealthrationaleforthisreviewrelatestotherecentattentiontopopulationexposuretoformaldehydebytheUSEPAthroughitsattemptstoregulateexposuretoformaldehydewithitsauthorityundertheToxicSubstancesControlAct(TSCA).
RationaleforReview:FormaldehydeandAsthmaFormaldehydeisthesimplestofallaldehydesandexistsatroomtemperatureasanearlycolorlessgaswithapungent,suffocatingodor(AgencyforToxicSubstancesandDiseaseRegistry1999).Ithasnumerousindustrialandcommercialusesasasolutionortoproduceindustrialresinsinturnusedtomanufactureadhesivesandbindersinnumerouscommercialproducts(NationalToxicologyProgram2010).FormaldehydewasrecognizedasaknownhumancarcinogenbytheInternationalAgencyforResearchonCancer(IARC)in2004andbytheNationalToxicologyProgram(NTP)withintheNationalInstitutesforEnvironmentalHealth(NIEHS)in2011(InternationalAgencyforResearchonCancer2006,NationalToxicologyProgram2010).Formaldehydeisacommonindoorairpollutantfoundinvirtuallyallhomesandbuildings.Itis
7
presentinmanyhouseholdproducts,suchasfoaminsulationandcleaningandpersonalcareproductsand,mostcommonly,intheformofpressedwoodproductssuchasparticleboardandplywood(NationalToxicologyProgram2010).Low-incomepopulationsaredisproportionatelyatriskofexposuretoformaldehydefromcompositewoodproductsinhomesbuiltusingcheaperbuildingmaterials.However,newerhomesgenerallyhavebeenfoundtoraiseconcernsregardinglevelsofformaldehydeoffgassedfromnewhousingmaterials,withavailabilityandratesofventilationhavingminimalimpactonexposurelevels(ParkandIkeda2006).Furthermore,formaldehydeexposurecanalsobeofconcerninbuildingsotherthanresidentialhomes—forinstance,formaldehydehasbeenmeasuredatlevelsexceedingexposurelimitsinchildcaresettingsinCalifornia(Bradmanetal.2016),thusindicatingthatwidespreadexposureinmanysettingsispotentiallyofconcern.Currentlythereisnoregulationofthelevelofformaldehydeallowedincompositewoodproducts.Evaluatingthescienceforhealthimpactsofformaldehydewillprovidetheevidencebaseformakingdecisionsaboutitsuseinbuildingproductsandassuchwillhelptoprotectthehealthofvulnerablepopulations.
Therelationshipbetweenasthmaandexposuretoformaldehydehasbeenunderevaluationforatleast25years.In1990,USEPAreleaseditsIntegratedRiskInformationSystem(IRIS)non-cancerassessmentofformaldehyde;in1998theAgencybeganareassessmentofthetopic;andinJune2010USEPAreleasedadraftIRISassessment.(NationalResearchCouncil2011)In2011theNationalAcademyofSciencespublishedanindependentreviewoftheUSEPA’sIRISassessmentofformaldehyde,to“providecandidandcriticalcommentsthatwillassisttheinstitution…[in]makingits[USEPA’s]publishedreportassoundaspossibleandtoensurethatthereportmeetsinstitutionalstandardsofobjectivity,evidence,andresponsivenesstothestudycharge”(NationalResearchCouncil2011).
Ingeneral,theNAS’reviewoftheIRISformaldehydeassessmentwashighlycriticalofUSEPA’snon-systematic,descriptive,andunspecifiedformatforstudyevaluation;intermsofasthma,theNASwasadditionallycriticalofthefactthatUSEPAdidnotanalyzetheevidenceaccordingtospecificasthmaoutcomes,i.e.,incidentasthma(theoccurrenceofnewcases),prevalentasthma(thepresenceofasthmaatthetimeofstudy),orexacerbationofestablishedasthma(NationalResearchCouncil2011).YettheUSEPAdidnotdirectlyaddressthesecriticismsinitsformaldehydeandcompositewoodproductsrulemaking.Rather,theAgencyjudgeda2010meta-analysisofformaldehyde-associatedchildhoodasthmaconductedbyMcGwinetal.(McGwinetal.2011)tobethemostrelevantstudytouseinitsbenefitsassessmentforasthma.Itappearsthatbasedonthedocumentedchangesintherule’scostbenefitanalysisbetween2012and2013,theOfficeofManagementandBudget(OMB)lackedconfidenceinUSEPA’suseoftheMcGwinmeta-analysisofasthmaandformaldehyde.
ThissystematicreviewoftheevidencelinkingexposuretoformaldehydeandasthmawillbeconductedusingtheNavigationGuide,amethodologythatspecificallyaccountsfortheweaknessesidentifiedbytheNASintheIRISformaldehydeassessment.
8
Aim
StudyQuestion
Ouraimistoanswerthequestion:“Isexposuretoformaldehydeassociatedwithdiagnosis,signs,symptoms,exacerbation,orothermeasuresofasthmainhumans?”
Objectives:
• Identifystudiesorexperimentsconductedinhumansconcerningtheassociationofexposuretoformaldehydewithasthmaoutcomes;
• Evaluatetheevidenceforaneffectacrossstudiesandifappropriate,conductameta-analysisoftheeffectsofexposuretoformaldehydeandasthma,andassessforpotentialsourcesofheterogeneity;
• Assesstheriskofbiasofindividualstudiesand,whereappropriate,assesstheirimpact(includingdirection)onmeasuresofestimatedeffectsize;
• Ratethequalityoftheoverallbodyofhumanevidence;and• Ratethestrengthofthehumanevidenceontheeffectofexposuretoformaldehydeon
asthmaoutcomesaccordingtooneofthefollowingfourstatements:1.Sufficientevidenceoftoxicity;2.Limitedevidenceoftoxicity;3.Inadequateevidenceoftoxicity;or4.Evidenceoflackoftoxicity.
Methods
ReviewTeamAtthebeginningofthecasestudy,UCSFwillassembleareviewteamconsistingofexpertsfromavarietyofresearchfieldsrelevanttothestudyquestionathand(i.e.,epidemiology,clinicalsciences,specificallyasthmaoutcomeassessment,exposureassessment,biostatistics,librarysciences,andsystematicreviewmethodology).Thefirstauthor(JL),ProjectDirector(PS)andseniorauthor(TW)willcollectivelyidentifypotentialreviewteammembersbasedontheirresearchinterests,expertise,availability,capacitytomeetprojectdeadlines,andtheabsenceofanyrealorpotentialconflictofinterestandinvitereviewteammembers.
Thelistofcoauthors,theirbiographicalsketches,proposedcontributionstotheproject,andacompletedconflictofinterestformaredocumentedinAppendixI.Specificrolesandresponsibilitiesforreviewauthorswillbedocumentedthroughouttheprotocol,i.e.,thedesignandconductofthesearch,applyinginclusion/exclusioncriteria,assessingriskofbiasforincludedstudies,dataextractiondataanalysis,andratingthequalityandstrengthoftheevidence.Theconductofthecasestudy,itsconclusionsandpublicationsarethesoleresponsibilityofthereviewteammembers.
9
Allreviewteammemberswillactivelyparticipateindevelopingtheprotocolandreviewingandapprovingthefinalmanuscript.Intheeventthatamemberofthereviewteamwasacoauthorofastudyunderreview,thatmembermustrecusethemselvesfromtheevaluatingthequalityandstrengthofthatstudy.
Throughoutthecourseofthereviewwewillalsoengagetopicexpertswithabroadersetofinterestsandexpertise.Topicexpertswillprovideconsultationasneeded.Wewilldocumentandacknowledgethecontributionofallindividualswhoparticipatedastopicexperts.Thecontributionoftopicexpertsislimitedtoadvisingthereviewteamanddoesnotconstituteauthorshiporagreementordisagreementwiththereviewteam’sfindings.
CriteriaforSelectingStudiesStudiesthatareeligibleforreviewwilladdressthestudyquestionaboveandthefollowingcharacteristicsasoutlinedinthefollowing“PECO”aid.
PECOStatement
“PECO”isanaidusedtoformulateananswerablequestioninasystematicreviewofhealthstudies.Theacronymstandsfor“Population/Participants”,“Exposure,”“Comparator”and“Outcomes.”
Population:Humans
Exposure:Anyindoororoutdoorsourcesofairborneinhalationexposuretoformaldehyde,includingbutnotlimitedtooccupational,outdoorambient,indoorhouseholdsettings,and/orexposuretohouseholdproductsthatoccurredpriororconcurrenttodiagnosis,exacerbation,orothermeasureofasthma.Exposurepriortodiagnosis,exacerbation,orothermeasureofasthmaisdefinedasmaternal/paternalpriortoconception,maternalexposureduringpregnancy,orexposuretotheindividualduringanylifestageEXCLUDINGactiveandpassivesmoking.Householdproductsthatmayserveassourcesofairborneinhalationexposuretoformaldehydeincludemedium-densityfiberboard,particleboard,compositewood,pressedwood,polyurethanefoam,adhesives,mobilehomes,trailerhomes,customkitchencabinetry,etc.Comparator:Humansexposedtolowerlevelsofformaldehydethanthemorehighlyexposedhumans.
Thisdefinitionisintendedtocaptureonlystudieswithsomecomparatorgroupasabaselineforcomparisonwhenexaminingeffectsofformaldehydeexposure.Thisisintendedtoincludegroups
10
definedbycase-controlstudydesigns;forinstancecomparingtheformaldehydeexposurelevelsforpeoplewithasthmaversusthosewithout.
Outcomes:Anyofthefollowingasthma-relatedoutcomes:diagnosisofasthma,asthmasignsorsymptoms,asthmaexacerbation(requiringsystemictreatment),orindirectmeasuresofasthma.Diagnosisofasthmaisdefinedasaclinicalasthmadiagnosisbyahealthcareprovider,whichmayincludeparentalreportofaclinicalasthmadiagnosisfortheirchild.
Directmeasuresofasthmasignsorsymptomsorexacerbationaredefinedas:asthmasymptoms(cough,wheeze,chestpain,etc.),asthmasymptomsleadingtobronchodilatoruse(albuterolonly),asthma-specificemergencydepartmentvisits,outpatientvisits,orhospitaladmissions,asthma-specificICUadmissions/intubations,asthma-specificdetailedmedicationuse,AsthmaSyndromeUtilityIndex(ASUI)retrospectivequestionnaire,useofsystemiccorticosteroidsforasthma,methacholinechallengetest,and/orchangesinspirometrymeasurese.g.,FEV1,FVC,FEV1/FVC,PEForPEFR,etc.overtime.
Thetermasthma“exacerbation’(e.g.,asthmaflare-up,asthmaattack,etc.)hasmanydifferentdefinitionsintheclinicalliterature.Forthisanalysis,wewilldefineanasthmaexacerbationasanyincreaseinsymptomsthatrequiretheuseofsystemiccorticosteroid(Fuhlbrigge2012).
Indirectmeasuresofasthmaimpactsaredefinedasbeinginindividualswithnorestrictiononage,measuredas:schoolabsences,lossofworkerdays,etc.impactingthequalityoflifeasaresultofasthmaeffects.
SearchMethods Oursearchtermsandsearchstrategyweredevelopedbyalibrariantrainedinsystematicreviewmethodology(LS)(seeAppendixIforLS’sbiosketch),incollaborationwithasecondInformationSpecialist(EW),whohastrainingandexpertisewithsearchingtheclinicalliteraturebutwillbelearningtheNavigationGuidesystematicreviewmethodologybyworkingwithLSduringthiscasestudy(seeAppendixIforEW’sbiosketch).Collectivelyourlibrarianteamwilldesignandimplementthesearchforrelevantstudies.
Wewillperformelectronicsearchesofonlinedatabases(PubMed,WebofScienceCoreCollection,BiosisPreviews,Embase,GoogleScholar,andToxline/DART)usingthesearchtermsoutlinedinAppendixII.Forpracticalityreasons,GoogleScholarwillbesearchedwithasimplifiedsearchstringdictatedbythelimitationsofthesearchengine(AppendixII),andthefirst100relevance-rankedsearchresultswillbeevaluated.Oursearchwillnotbelimitedbylanguageorpublicationdate.
Tofacilitatedevelopmentofthesearchstrategy,wewillbeginbyperformingapreliminaryscopingassessmentinPubMedofthenatureandextentoftheavailableliteraturerelevanttoourPECOstatement.Wewillaccomplishthreethingswiththisreview:wewillidentifycontrolledvocabulary(MedicalSubjectHeadingsorMeSH)andsynonymsforourtwosearchconceptsof
11
formaldehydeandasthma,wewillidentifyproxytermsforbothconcepts,andwewillestimatethetypeandamountofinformationextantaboutourtopic.Inaddition,weplantoestimatethenumberofhumanstudiesavailableandidentifydifferenttypesofasthmaoutcomesavailableintheliterature.Thescopingsearchtermsandstrategywillbedocumentedandusedtoinformouroverallsearchstrategy.ThescopingstrategyandtermsarelistinAppendixII.
Toassistinthedevelopmentofalistoftermsrelevanttooursearchstrategy,wewillusetheMedicalSubjectHeadings(MeSH)databasetocompilesynonymsforformaldehydeandoutcomesrelatedtoasthma.WewillusethefollowingMeSHtermstoguideoursearch:
Asthma
RespiratoryFunctionTests
Respiratorysounds
ForcedExpiratoryVolume
PeakExpiratoryFlowRate
Bronchoconstriction
BronchialSpasm
Formaldehyde
Floorsorfloorcoverings
Adhesives
Airpollution
Constructionmaterials
Housing
Thefollowingarticleswereidentifiedinthescopingsearchorwereknowntotheauthorsasrelevanttoourstudy.Eachwasexaminedforterminologyandreferences.DuringdevelopmentofafinalsearchstrategyinPubMedthissetofarticleswasusedtotestforrecall.Inadditionwewillidentifyfurthersynonymsfromthefollowingknownresearcharticlesonformaldehydeandasthma:
1) RumchevKB,SpickettJT,BulsaraMK,etal.2002.Domesticexposuretoformaldehydesignificantlyincreasesriskofasthmainyoungchildren.EurRepirJ.20(2):403-408.
2) SchachterEN,WitekTJJr.,BrodyDJ,etal.1987.Astudyofrespiratoryeffectsfromexposureto2.0ppmformaldehydeinoccupationallyexposedworkers.EnvironRes.44(2):188-205.
3) G.Smedje,D.Norbäck.Incidenceofasthmadiagnosisandself-reportedallergyinrelationtotheschoolenvironment—afouryearfollow-upinschoolchildren.IntJTubercLungDis.5(2001),pps.1059–1066.
12
4) GarrettMH,HooperBM,RaymentPR,AbramsonMJ.2001.Increasedriskofallergyinchildrenduetoformaldehydeexposureinhomes.Allergy.54:330-337.
5) ZhaoZZhangZ,WangZ,FermM,LiangY,NorbäckD.2008.AsthmaticsymptomsamongpupilsinrelationtowinterindoorandoutdoorairpollutioninschoolsinTaiyuan,China.EnvironHealthPerspect.116(1):90-7.
6) KrzyzanowskiM,QuackenbossJJ,LebowitzMD.1990.Chronicrespiratoryeffectsofindoorformaldehydeexposure.EnvironRes.52(2):117-25.
7) HulinM,CaillaudD,Annesi-MaesanoI.2010.Indoorairpollutionandchildhoodasthma:variationsbetweenurbanandruralareas.IndoorAir.20(6):502-14.
8) DriscollTR,CareyRN,PetersS,GlassDC,BenkeG,ReidA,FritschiL.2016.TheAustralianWorkExposuresStudy:Prevalenceofoccupationalexposuretoformaldehyde.AnnOccupHyg.60(1):132-8.
Theseeightselectedpaperswererelevanttothestudyquestion,werepublishedindifferentyearsandjournals,authoredbyavarietyofresearchers,andtheyexaminedavarietyoftopicsrelevanttothestudyquestion.
Additionally,weusedthefollowinglistofbroadsurveysandsynthesestocompileoutcomesearchterms.Furthermore,weselectedbroadsurveysrelatedtoasthmatoreviewforcompilingoutcomesearchterms.Theseincluded:
1. McGwinG,LienertJ,KennedyJI.2010.Formaldehydeexposureandasthmainchildren:asystematicreview.EnvironHealthPerspect.118(3):313-7.doi:10.1289/ehp.0901143.
2. YaoY,LiangW,ZhuL,DuanY,JinY,HeL.2015.Relationshipbetweentheconcentrationofformaldehydeintheairandasthmainchildren:ameta-analysis.IntJClinExpMed.8(6):8358-62.
3. ReddelHK,TaylorR,BatemanED,etal.2009.AnofficialAmericanThoracicSociety/EuropeanRespiratorySocietyStatement:Asthmacontrolandexacerbations.AmJRespirCritCareMed180:59-99.DOI:10.1164/rccm.200801-060ST
4. FuhlbriggeA,PedenD,ApterAJ,etal.2012.Asthmaoutcomes:Exacerbations.JAllergyClinImmunol.129:S34-S48.DOI:10.1016/j.jaci.2011.12.983
5. KrishnanJA,LemanskeRF,CaninoGJ,etal.2012.Asthmaoutcomes:Symptoms.JAllergyClinImmunol.129:S124-S135.DOI:10.1016/j.jaci.2011.12.981
PubMedFortheexposurecomponentofthePECOstatement,wewillcombinetermsrepresentingformaldehyde,itssynonymsandnotableenvironmental/householdsourcesusingtheBoolean“OR”operator.Wewillnotincludeexposuretoformaldehydeasaproductofcombustion;homeheating,cigarettesmoking,exhaustexposureswereexcluded.Fortheoutcomes,wewillsearchusingtermsrepresentingasthma,itssynonyms,anditsproxiescombinedwiththeBoolean“OR”operator.Proxytermswillincludemeasuresoflungfunctionandsignsorsymptomsofasthma.WewillincluderelevantMeSHheadings(usingthe[mesh]fieldtag)andtitleand/orabstract
13
keywordsidentifiedinthescopingphaseofthesearch(usingthe[ti]or[tiab]fieldtags).Wewilladdtheformaldehyderegistrynumber[rn]totheformaldehydesearchstring.
WewillcombinetheexposureandoutcomesearchesusingtheBoolean“AND”operatortoidentifyarticleswithbothconcepts.
PubMedwillserveasourprimaryonlinedatabase.OnceourPubMedsearchstrategyisfinalized,thesearchwillbetranslatedtoperformoptimallyineachadditionaldatabase.
WebofScienceandBiosisPreviewsWewillmodifythePubMedsearchtoworkinWebofScienceCoreCollectionandBiosisPreviewsasbothdatabasesarehousedinThomsonReutersWebofScienceplatform.Todoso,wewillremovethePubMed-specific[mesh],[tiab],and[ti]fieldtagsandwhileretainingthekeywordsandphrases.WewillperformatopicsearchinWebofSciencewiththistranslatedsearchstring;atopicsearchexaminesthetitle,abstract,author-definedkeywords,and“KeywordsPlus”fieldsofWebofScience.
Embase
WewilldevelopourEmbasesearchfilterusingthesamemethodasdescribedaboveforWebofScienceCoreCollectionandBiosisPreviews.WewilltranslateMeSHtermstothenearestEmtreeThesaurusequivalentandusethe“:ab,ti.”functiontolimitthesearchtoabstractsandtitles.Wewillusethe:rnfunctiontosearchregistrynumbersforformaldehyde.
Toxnetdatabases(ToxlineandDART)Duetothesearchinterfacelimitationsweusedasimplifiedsearchstring(“formaldehydeandasthma”)forToxlineandDart,thetwotoxicologicliteraturedatabasesofToxnet.SeeAppendixIIfordetails.
SearchresultsfromeachdatabasewillbestoredinitsownEndNotecollectionforaccountingpurposes.WewillthencreateacombinedEndNotecollectionforalldatabases.The“findduplicates”functioninEndNotewillbeusedtofindandremoveduplicatesinthecombinedfolder.PubMedwillbeconsideredourprimaryonlinedatabasesothatduplicatesfromnon-PubMeddatabaseswillberemoved.
SearchingOtherResources
Wewilluseavarietyofmethodstofindadditionalinformationnotidentifiedthroughsearchesofonlinebibliographicdatabases.Theso-calledgreyliteratureincludestechnicalreportsfromgovernmentagenciesorscientificresearchgroups,workingpapersfromresearchgroupsorcommittees,whitepapers,preprints,conferenceproceedings,personalcommunications,dissertationsandtheses,etc.
Thesemethodsinclude:• SearchingthewebsitesanddatabaseslistedinAppendixIII.• LocatingconferenceabstractsfromWebofScienceCoreCollection,BIOSISPreviews,
Embase,ProceedingsFirst,andPapersFirst.
14
• Handsearchingthereferencelistsofallstudiesincludedafterfulltextreview(priortostudyauthorcontact,ifapplicable)usingWebofSciencetosearchforarticlescitedbyandthosethatcitedtheincludedstudies.
• Handsearchingthereferencelistsofallexcludedstudiesidentifiedduringourliteraturesearchasreviewsorsynthesesrelevanttoourstudyquestion.
• AGoogleandGoogleScholarsearchusingdomainoperatorsof.eduand.gov.Thefirst100resultsfromeachsearchwillbeevaluated.
• Personalcommunicationwithauthorstorequestunpublisheddataoriftheyhaveknowledgeofadditionaldatafromotherauthors.
• Havingexpertsinthefieldofformaldehydeandasthmareviewourlistofincludedstudiesforcompleteness.
StudySelectionCriteria
AllsearchresultswillbeimportedormanuallyenteredintoEndNote(Versionx7)referencemanagementsoftware.WewilluseEndNotetoeliminateanyduplicatereferencesbeforewebeginevaluatingtheeligibilityofthestudiesidentified.
TitleandabstractscreeningEachreferencewillbescreenedinduplicate.Fourreviewers(EK,ND,AP,HV)willindependentlyconductatitleandabstractreviewofeachreferencefromtheliteraturesearchresultstodeterminewhetheritmeetstheselectioncriteriaforinclusion.Eachauthorwillbeassignedanon-randomsubsetofreferencestoscreen,toensurethatallreferencesarescreenedinduplicateandtoensurethatthesametwoauthorsdonotalwaysscreenthesamereferences(i.e.HVwillbeassignedthefirstthreequarters(75%)ofthereferences;EKthelastthreequarters(75%);NDthe1stquarter(25%);APthelastquarter(25%)).
Referenceswhichareincludedatthetitle/abstractscreeninglevelwillbesubjecttoafulltextreviewbythesamefourauthors(moredetailfollowsinthenextsection).
Intheeventthatthereisadiscrepancybetweenreviewers,thedefaultwillbetopushthereferenceforwardtothenextstepintheprocess(i.e.,ifthetworeviewersdisagreeonwhetherthestudyisrelevantatthetitleandabstractscreeninglevel,thereferencewillbeincludedbydefaultforfull-textscreening).
Toensurequalitycontrol,oneauthor(JL)willperformtitleandabstractscreeningofarandomselection(usingarandomnumbergeneratorassignment)offivepercentofthesearchresultsor5papers,whicheverisgreater.Thesedeterminationswillbecomparedtotheotherreviewers’determinationsforthesestudies.
Thereviewofarticlesagainstthepre-specifiedinclusionandexclusioncriteriawillbeperformedusingastructuredforminDistillerSR(EvidencePartners;availableat:http://www.systematic-review.net),anonlineapplicationdesignedspecificallyforthescreeninganddataextractionphasesofasystematicreview(seeAppendixIVfortitleandabstractinclusion/exclusionform).
15
Reportsinanylanguage,fromanyyear,willbeeligibleforinclusion.Allreportsthatcomparehumansexposedtoformaldehydetoappropriatecomparatorsandevaluatethemfortheasthma-relatedhealthoutcomesasdescribedinthePECOstatementsabovewillbeeligibleforinclusion.
Thetitle/abstractscreeningformwillbeusedtoscreenandEXCLUDEreferencesifoneormoreofthefollowingcriteriaaremet:
1. Articleisareviewofformaldehydeexposureandasthma;2. Articlecontainsnooriginaldata(e.g.,editorial,reviewpapernotrelevanttostudy
question,etc.)3. Articledidnotinvolvehumansubjects(i.e.,animalevidenceorcelllinesonly,case
reportofsinglehuman,etc.);4. Articledidnotreportformaldehydeexposure,asdefinedbyPECOstatement;5. Articledidnotreportoutcomesofasthma,asdefinedbyPECOstatement;6. Otherreason(explanationrequired).
Thecriteriaforanarticlebeingareviewarticleisseparatelycategorizedfromothertypesofnon-originaldatasothatreviewarticlesmayberetainedandsearchedincaseanyofitsreferencesmaybeidentifiedforinclusionandtoultimatelycomparetheresultsofourreviewtothemethodsandconclusionsofpreviousreviews.Thefollowinginstructionswillbeprovidedtoreviewauthorsconductingthetitleandabstractscreening:
“Whenexcludingareference,pleaseselectonlyONE(1)exclusionreason.PleasereviewtheexclusionreasonsinorderandselecttheFIRSTexclusionreasonrelevanttothereferencebeingscreened.Pleaseaddinanyadditionalnotesinthecommentboxtoexplainyourselectionifnecessary.”
ThefollowingtypesofrecordswillbeINCLUDEDatthetitle/abstractlevel:
• Studiesofanydesignconductedinhumansinvestigatingassociationsbetweenformaldehydeexposureandasthma,asdefinedbythePECOstatement.
Forcitationswherethedatabasecontainsnoabstract,authorswillattempttoobtaintheabstractsfromanInternetsearch.ArticlesforwhichtheabstractremainsunavailablewillbescreenedbasedontitlesandPubMedMeSHheadings.Anystudynotexcludedbasedonthestatedcriteriaabovewillbeincludedforfull-textreview.Updateddetailstoinstructionsandinterpretationsfortitleandabstractscreening(additionaltowhatisprovidedhereintheprotocol)maybeaddedtoAppendixIVtothoroughlydocumenttheprocessofthereviewteamduringthescreeningprocess,ifnecessary.
Full-TextScreeningReferenceswhichareincludedatthetitle/abstractscreeninglevelwillbesubjecttoafulltextreviewbythesamefourauthorsinvolvedintitleandabstractscreening(EK,ND,AP,HV).Eachreferencewillbescreenedindependentlyandinduplicate.Eachauthorwillbeassignedanon-randomsubsetofreferencestoscreen,toensurethatallreferencesarescreenedinduplicateandtoensurethatthesametwoauthorsdonotalwaysscreenthesamereferences(i.e.HVwillbe
16
assignedthefirstthreequarters(75%)ofthereferences;EKthelastthreequarters(75%);NDthe1stquarter(25%);APthelastquarter(25%)).
Oneauthor(JL)willbebroughtintosettleanydiscrepanciesbetweenthereviewersresultingfromeachstepofthereviewprocessifnecessary.Intheeventthatthediscrepancyultimatelycannotberesolved,thedefaultwillbetoincludethestudy.
Toensurequalitycontrol,oneauthor(JL)willperformfulltextscreeningofarandomselection(usingarandomnumbergeneratorassignment)offivepercentorfivepapers,whicheverisgreater,ofsearchresultseligibleforfulltextreview.Thesedeterminationswillbecomparedtotheotherreviewers’determinationsforthesestudies.
Thereviewofarticlesagainstthepre-specifiedinclusionandexclusioncriteriawillbeperformedusingastructuredforminDistillerSR(EvidencePartners;availableat:http://www.systematic-review.net),anonlineapplicationdesignedspecificallyforthescreeninganddataextractionphasesofasystematicreview(seeAppendixIVforfulltextinclusion/exclusionform).
CitationseligibleforfulltextreviewwillbescreenedandEXCLUDEDifoneormoreofthefollowingcriteriaaremet:
1. Articleisareviewofformaldehydeexposureandasthma;2. Articlecontainsnooriginaldata(e.g.,editorial,reviewpapernotrelevanttostudy
question,etc.);3. Articledidnotinvolvehumansubjects(i.e.,animalevidenceorcelllineonly,casereport
ofsinglehuman,etc.);4. Articledoesnotreportexposuretoformaldehyde,asdefinedbythePECOstatement;5. Articledoesnotreportdiagnosisofasthma,asthmasignsorsymptoms,asthma
exacerbation,orindirectmeasuresofasthma,asdefinedbythePECOstatement;6. Therewasnocomparatorgroup;7. Duplicatestudy;8. Otherreasons(explanationrequired).
9. POSSIBLYInclude—otherlanguage(specifywhichlanguagebelowincomments);
Thefollowinginstructionswillbeprovidedtoreviewauthorsconductingfulltextscreening:
“Whenexcludingareference,pleaseselectonlyONE(1)exclusionreason.PleasereviewtheexclusionreasonsinorderandselecttheFIRSTexclusionreasonrelevanttothereferencebeingscreened.Pleaseaddinanyadditionalnotesinthecommentboxtoexplainyourselectionifnecessary.”
CitationswillbeINCLUDEDiftheymeetthePECOstatementcriteria,thatis,theirsubjectsrepresenthumans,theyincludeexposurecomparisonsforrelevantperiods,andtheyreporttheoutcomeofasthmaasdefinedinthePECOstatement.
Forarticles(includingnon-Englisharticles)thatarenotavailableinthedatabase,wewillattempttoobtainarticlesfromabroadInternetsearch.Potentiallyrelevantnon-EnglisharticleswillbetranslatedintoEnglishtodetermineeligibility.Alistofallexcludedstudiesandabriefreasonfor
17
exclusionwillbedocumentedfortransparencyandreportedinthestudyfindings(HigginsandGreen2011).
DataCollectionDatawillbeextractedfromeachincludedstudytoevaluatereportingquality,riskofbias,and/ortoconductstatisticalanalyses.Datawillbeextractedfromeachstudyinduplicate(byHVandEK)andcomparedunderthedirectionofathirdco-author(JL)forqualityassurance/qualitycontrolandtoresolveanydiscrepancies.DatawillbeextractedusinganAccessdatabaseformthathasbeenmodifiedfrompreviousapplications(seeAppendixVforthestudycharacteristicsdatacollectionform).Thedataextractionformwillbepilot-testedonasmallnumberofstudiestoidentifyanyissuesandmaximizeconsistencyacrossdataextractorsbeforeextractingthefullsetofincludedstudies.
Foreverystudythatdoesnotreportallthedataneededfordataanalysisorriskofbiasassessment,wewillrequestthesedatafromthestudycontactauthorbyemail.Ifstudyauthorsdonotrespondtorequestsafterbeingcontactedthroughatleast2emailmessagesoverthecourseof1month,reviewauthorswillnotethatattemptstocontactstudyresearcherswereunsuccessful.
RiskofBiasDeterminationRiskofbiaswillbeassessedforhumanstudiesusingdomainsfromtheCochraneCollaboration’s“RiskofBias”toolandtheAgencyforHealthcareResearchandQuality’s(AHRQ)criteria(HigginsandDeeks2011,Viswanathanetal.2012).Thesetoolshavebeenmodifiedtomakethemappropriateforhumanobservationalstudies,andincludedomainsthataddressrecruitmentstrategy,blinding,confounding,exposureassessment,outcomeassessment,incompleteoutcomedata,selectiveoutcomereporting,andconflictofinterest(AppendixVI).WehavemodifiedthesetoolsandappliedthemtoevaluateriskofbiasinfivepreviouscasestudiesapplyingtheNavigationGuidesystematicreviewmethodology(Johnsonetal.2014,Vesterinenetal.2014,Johnsonetal.2015,Lametal.2015,Lametal.2015).
AlthoughCochraneandAHRQdonotcurrentlyincorporateaconflictofinterestdomainasapotentialsourceofbias,theNavigationGuidetooldoesincludethisdomain,representedasfundingsourceordeclaredconflictsofinterest.Thisdecisionwasinformedbyempiricaldatafrommeta-analysesconductedonpharmacologicaltreatmentsandstudiesofriskofbiasandsponsorship(Rosemanetal.2011,Lundhetal.2012,Krauthetal.2013).WewillalsosearchforeachstudyinPubMedandnoteiftherehasbeenaretractionofthepublishedarticleinordertodetermineifthestudymaybefraudulentorifanycorrectionshavebeenpublished.
Reviewauthorswithrelevanttopicmatterexpertise(AP,TW,PS,HV,EK,MC)willassessriskofbias.Eachincludedstudywillberatedinduplicate.Reviewauthorswillbeassignedasmallsampleofstudies(e.g.,2)topilottheriskofbiasassessmenttool.Allreviewauthors(ratersaswellasothercoauthors)willthenmeettodiscusstheseratings,identifypotentialissuesorconfusion,andthenestablishconsistentrulestoensureconsistencyinratings.Anydecisionsorrulesestablishedduringthismeetingwillbedocumentedandappendedtotheprotocolfortransparency.Reviewauthorswillthenratetheremainingstudies,applyinganyrelevantdecisionsorrulesdiscussedsubsequenttothepilotrating.
AnydiscrepanciesbetweenROBraterswillbeinitiallyreviewedbyJLandthetworaterstoattempttoreachconsensusontheratings.Anyremainingdiscrepancieswillthenbereviewedby
18
JLandallotherriskofbiasraters.If,uponfurtherdiscussiontheraterscannotreachagreementonanappropriateriskofbiasdeterminationforaparticulardomain,theratingjudgmentwillbeselectedasfollows:ifonereviewermakesajudgmentof‘high’riskofbiasandtheothermakesajudgmentof‘probablyhigh’riskofbias,the‘high’riskofbiasjudgmentwillbeused,etc.Ifadditionaldataorinformationisacquiredfromstudyauthors,riskofbiasjudgmentswillbemodifiedtoreflecttheupdatedstudyinformation.Thediscussionamongraterswillbeusedtoinformratingsacrosssimilarstudies.Allfinalratingswillbereviewedtoensureconsistencyacrossratingsofstudieswithsimilarexposureoroutcomeassessmentmethods,studydesigns,etc.
Toensurequalitycontrol,JLwillalsomakeriskofbiasdeterminationsforarandomselection(usingarandomnumbergeneratorassignment)offivepercentofor5includedstudies,whicheverisgreaterandthesewillbecomparedtootherreviewers’determinationsforthesestudies.
Wewillattempttominimizetheimpactofpublicationbiasby:(1)implementingacomprehensivesearchoftheliteratureusingmultiplesourcesandmethodsinordertoidentifypublishedaswellasunpublishedstudiesthatmeettheeligibilitycriteria;and(2)ifpossible,usingfunnelplotanalysisand/orotherstatisticalanalyses(e.g.,Eggerregression(LightandPillemer1984)and“trimandfill”(DuvalandTweedie2000)ofthestudiesincludedinthesystematicreview,asappropriate.Thesestatisticalapproacheshavebeenrecommendedonlywhenthenumberofstudiesincludedinthemeta-analysisissufficientlylarge(Sterneetal.2011);andsotheseanalyseswillonlybeperformedwhen>10studiesareincludedinthemeta-analysis.Intheeventthatthenumberofstudiesincludedinthemeta-analysisistoosmalltheseanalyticalapproacheswillnotbepursued.Furthermore,intheeventofsubstantialbetween-studyheterogeneity,thesemethodsareknowntoperformpoorlyandsowewilltestforbetween-studyheterogeneityaswelltomakethedeterminationofwhetherthismethodwouldbeappropriateforthecollectionofincludedstudies(Higgins2011).
DataAnalysisWeintendtoperformameta-analysistosummarizetheeffectsofexposuretoformaldehydeonasthma,andtoassesstheimpactofstudydesigncharacteristicsonfindings.Todothis,firstwewillcompileandreviewimportantcharacteristicsfromeachstudytoestablishcomparabilitybetweenstudiesortoidentifydatatransformationsnecessarytoensuresuchcomparability.Keycharacteristicsinclude:
• Studydesign(cross-sectional,case-control,etc.)• Populationstudied(includinggeographicregion,sexand/orageofindividualswhen
assessed)• Exposurelevels,methodofmeasurement,andtimingofmeasurement• Healthoutcomeassessed,thetest/assessmenttoolused,andtimingofassessmentrelative
toexposure• Typeofdata/summarystatisticavailable
Thefollowingareexamplesofcharacteristicsthatwouldpotentiallybeconsiderednon-combinable:
19
• Non-quantitativeexposure(i.e.,whereexposureisnotquantifiedbutclassifiedashigh/loworothercategoriesofexposure)inanoccupationalsetting(i.e.,byjobclassification)wouldnotbecombinablewithnon-quantitativeexposureincommunitysettingsintheabsenceofdatatovalidatethatthesequalitativemeasureswerecomparable.
• Exposuretoambientairwouldpossiblynotbecombinablewithindoorairwithoutknowledgethattheairpollutantlevelsand/orcompositionofeachwassimilar.
• Moredirectmeasuresofasthmahealthoutcomes(i.e.,diagnosisofasthma,asthmasignsandsymptoms,asthmaexacerbation)wouldnotbecombinablewithindirectmeasuresofschoolabsences,lossofworkerdays,etc.impactingqualityoflifeasaresultofasthmaeffects.
• Effectsonchildrenandadultsmaybeaddressedseparately.
Summariesofthesecharacteristicsforeachincludedstudywillbeassessedbytwoormorereviewerauthors(JL,EK,HV,AP,TW)todeterminecomparabilitybetweenstudiesandtoidentifyanyheterogeneityconcerns.JLwillthenidentifystudieswithsufficientmethodologicalhomogeneitywithrespecttopopulation,studydesign,studyduration,exposurelevelandhealthoutcomeamongotherconsiderationsthatcanbecombinedinameta-analysis.Iftransformationstoreportedeffectestimatesarenecessarytoacommonscaleacrossdifferenttests/assessmentsofasthmaormeasuresofformaldehydeexposure,thesewillbedocumented.Thestatistician(TBD)andseniorauthor(TW)willreviewstudycharacteristicsandrecommendationsofJLregardingmeta-analysis.
Ifameta-analysisisdeemedappropriate,JLincollaborationwiththestatisticianwillidentifyappropriatestatisticalmethodstoanalyzethedata,andtodeterminewhetherfurthermodificationsarerequiredpriortoperformingthemeta-analysis.Ourinitialproposedapproachistoextractfromeachstudydeemedcombinabletheadjustedeffectestimates(oddsratios,relativerisks,linearregressionbetaestimates,etc.)aswellastherelevantscalereportedforeach(log-transformed,percontinuousunitincreaseofexposure,etc.).Wewillthenestablishacommonscaleandeffectestimateandeitherperformthenecessarycalculationstotransformeacheffectestimatetoacommonscaleifpossible,andcontactstudyauthorstoobtainthenecessarytransformedeffectestimatesifnecessary.Wewillalsotesttheseestimates(calculatingandinterpretingtheI2estimateaswellasachi-squaredtestforheterogeneity)toinvestigatewhetherstatisticalheterogeneityispresent.Furthermore,wewillattempttodeterminethecausesofpotentialheterogeneityamongresultsforstudiestodetermineifafixedeffectorrandomeffectsmodelisappropriate.Theseestimateswillthenbecombinedacrosscomparablestudies,usingeitherthefixedorrandomeffectsmodeltoaccountforpotentialheterogeneityacrossstudies.Thefinalquantitativeresultwillbethecombinedadjustedeffectestimatesonthesamescalewithanassociatedconfidenceinterval.OuranalysisplanwillberefinedbyJLincollaborationwiththestatisticianasneededbasedonthedatathatenterthereview.
Intheeventthattheseproposedmethodsfordataanalysisarealteredtotailortotheevidencebasefromincludedstudies,theprotocolwillbeamendedaccordinglyandthereasonsforchangewillbejustifiedinthedocumentation.
Toteststatisticalheterogeneityacrossthestudyestimates,wewillestimatethevariancecomponentcorrespondingtobetween-studyvariability(“Cochran’sQ”),andusealikelihoodratiotestforthenullhypothesisthatbetween-studyvariabilityisabsent.Ap-valueof0.05orlesswillbeconsideredstatisticallysignificant.Furthermore,toassesstheimpactofbetween-study
20
heterogeneityonthemeta-analysis,theI2teststatisticwillbecalculatedandevaluatedbyconsideringthemagnitude/directionofeffects,strengthofevidenceforheterogeneity(e.g.,p-valuefromachisquaredtestoraconfidenceintervalforI2),andtheCochrane’sguidetointerpretationasfollows:
• 0%-40%:mightnotbeimportant;• 30%-60%:mayrepresentmoderateheterogeneity;• 50%-90%:mayrepresentsubstantialheterogeneity;• 75%-100%:considerableheterogeneity.
Wewillalsoperformsensitivityanalysesbyexaminingtheeffectsofexcludingstudieswithparticularlyheterogeneousresultsaswellasperformingsubgroupanalysesbasedonexcludingsubsetsofstudieswithsharedcharacteristicsthatmightbeinfluential.Ifasufficientnumberofstudiesareavailable,analternativeapproachwouldbetoperformasubgroupanalysis,clusteringstudiesbyspecifiedcharacteristicstodeterminetheimpactonstatisticalheterogeneity.
Asdiscussedaboveinthe“RiskofBiasDetermination”section,ifpossible,i.e.thereareenoughstudies,wewillassessforthepresenceofpublicationbiasbyfunnelplottingandEggerregressionontheestimatesofeffectsize(LightandPillemer1984)andpredicttheimpactofhypothetical“missing”studiesusing“TrimandFill”(DuvalandTweedie2000).
QualityandStrengthofEvidenceRatingsUponcompletionofthedatacollection,riskofbiasdeterminations,anddataanalysis,eachofthereviewauthorswillindependentlycomparetheresultsofthesystematicreviewtothecriteriaoutlinedintheNavigationGuidesystematicreviewmethodologyforratingthequalityandstrengthoftheevidence.Allreviewauthorswillbegivenexplicitdirectionsbeforerating(seeAppendixVII,“InstructionsforRatingtheQualityandStrengthofEvidence”).
Theinitialqualitylevelofhumanobservationaldatawillstartatmoderate,ashasbeenassignedinpriorcasestudiesofapplyingtheNavigationGuidemethodology(WoodruffandSutton2014).
Factorsthatmaydecreasethequalitylevelofthebodyofevidenceinclude:1. RiskofBiasAcrossStudies:Studylimitations–asubstantialriskofbiasacrossbodyof
evidence;2. Indirectness:Evidencewasnotdirectlycomparabletothequestionofinterest(i.e.,
population,exposure,comparator,andoutcome).3. Inconsistency:Widelydifferentestimatesofeffect(heterogeneityorvariabilityinresults);4. Imprecision:Studieshadfewparticipantsandfewevents(wideconfidenceintervals);and5. PublicationBias:Studiesmissingfrombodyofevidence,resultinginanoverestimateor
underestimateoftrueeffectsfromexposure.
Factorsthatmayincreasethequalitylevelofthebodyofevidenceinclude:
1. Largemagnitudeofeffect:Upgradedifmodelingsuggestedconfoundingaloneunlikelytoexplainassociationswithlargemagnitudeofeffect.
2. Dose-response:Upgradedifconsistentdoseresponsegradientinoneormultiplestudies,and/ordoseresponseacrossstudies.
3. Residualconfoundingincreasesconfidence(NationalToxicologyProgram2015):Upgradedifconsiderationofallplausibleresidualconfounders,biases,oreffectmodificationwould
21
underestimatetheeffectorsuggestaspuriouseffectwhenresultsshownoeffect.Ifastudyreportsanassociationdespitethepresenceofresidualconfounding,biasesoreffectmodificationthatwoulddiminishtheassociation,confidenceintheassociationisincreased.GRADEprovidesanillustrativeexamplerelatedtobias:ratingupobservationalevidencefindinglackofassociationbetweenvaccinationandautism,whichoccurreddespiteempiricallyconfirmedbiasthatparentsofautisticchildrenmaybemorelikelytoremembertheirvaccineexperience.Thenegativefindingsdespitethisformofrecallbiassuggestratingupthequalityofevidence(Guyattetal.2011).
Possibleratingsforqualityofevidenceare“high,”“moderate,”or“low.”Possibledowngradesorupgradesare:0(nochange),-1(1leveldowngrade),–2(2leveldowngrade),+1(1levelupgrade)or+2(2levelupgrade).Theratingsoftheseparatefactorsarenotaddedtogetherintoascore,e.g.a-1downgradeforinconsistencyanda-1downgradeforimprecisiondoesnotautomaticallydictateanoverall-2downgradeforthebodyofevidence.Judgmentisexercisedtodetermineiftherationalebehindeachdowngradewarrantsanoveralldowngradeof1or2levels.Thesameappliestoupgradingtheoverallbodyofevidence.Likewise,a-1downgradeforonefactoranda+1upgradeforanotherfactordonotautomaticallycanceloutanddeterminen0downgradesorupgradesfortheoverallbodyofevidence.
Authorswhodecidetoratequalitydownorupneedtospecifythe1or2criteriamostresponsiblefortheirdecisionwhiledocumentingallfactorsthatcontributedtothefinaldecision.Afterindependentlyevaluatingthequalityoftheevidence,reviewauthorswillcomparetheirevaluationsandanydiscrepanciesbetweenthereviewers’decisionswillberesolvedthroughdiscussionuntilconsensusisreached,ifpossible.Therationaleforeachdecisiononeachofthefivefactorswillberecorded.Alackofconsensusonanyspecificfactordoesnotprecludeconsensusontheoverallqualityoftheevidence.
Subsequenttoratingthequalityoftheevidence,thereviewauthorswillratethestrengthofevidence.Theoverallstrengthofthebodyofhumanevidenceisbasedonacombinationoffourcriteria:(1)Qualityofbodyofevidence(i.e.,theratingfromthepreviousstep);(2)Directionofeffect;(3)Confidenceineffect;and(4)Othercompellingattributesofthedatathatmayinfluencecertainty.TheresultsofratingofthestrengthofthehumanevidencewillthenbecomparedtothecriteriaspecifiedintheNavigationGuidesystematicreviewmethodologyanddescribedaccordingtooneofthefollowingfourconcludingstatements:1.Sufficient;2.Limited;3.Inadequate;or4.Evidenceoflackoftoxicity(Table1)(Woodruffetal.2011,Johnsonetal.2015).Anydiscrepanciesbetweenthereviewers’decisionswillberesolvedthroughdiscussion.Theseniorauthor(TW)willbetheultimatearbiterofthediscrepanciesthatcannotberesolvedthroughconsensusamongthereviewauthors.Theresultsofthereview,includingimplicationsforpublichealth,willbecompiledinamanuscriptforsubmissiontothepeer-reviewliterature.
22
SUPPLEMENTARYINFORMATION
AppendixI.Coauthor/LibrarianBiosketches,ProjectContributionStatements,andConflictofInterestStatements
JULEENLAM
JuleenLamisanAssociateResearchScientistattheUniversityofCaliforniaatSanFrancisco(UCSF).ShehasbeeninvolvedwiththeNavigationGuidesince2011whileemployedattheEnvironmentalProtectionAgency’sOfficeofPolicyattheNationalCenterforEnvironmentalEconomicsasanOakRidgeInstituteforScienceandEducation(ORISE)postdoctoralfellowandasaresearcherattheJohnsHopkinsUniversityBloombergSchoolofPublicHealthintheDepartmentofHealth,PolicyandManagement.ShehasbeeninvolvedinfourcasestudiestodateapplyingtheNavigationGuidetoaddressproblemsinthefieldofenvironmentalhealth.JuleenreceivedherPhDfromJohnsHopkinsUniversityinEnvironmentalHealthPolicy,MHSfromJohnsHopkinsUniversityinBiostatistics,MSfromGeorgeWashingtonUniversityinEnvironmentalEngineeringManagement,andtwoBSdegreesfromtheUniversityofCaliforniaatDavisinMathandEnvironmentalToxicology.Shehasoveradecadeofexperienceinenvironmentalhealthresearchandpolicy,holdingpositionsatstateandfederalgovernmentagencies,academicinstitutions,andintheconsultingandnonprofitsectors.Shespecializesinanalysisofenvironmentalhealthdataandfocusesherresearchonthetranslationofscientificfindingsintomakinginformeddecisionsandpolicies.
Contributionstothereview:
• Conceivingthereview.• Designingthereview.• Coordinatingthereview.• Datacollectionforthereview.• Designingsearchstrategies.• Screeningsearchresults.• Organizingretrievalofpapers.• Screeningretrievedpapersagainsteligibilitycriteria.• Appraisingqualityofpapers.• Extractingdatafrompapers.• Writingtoauthorsofpapersforadditionalinformation.• Providingadditionaldataaboutpapers.• Obtainingandscreeningdataonunpublishedstudies.• Datamanagementforthereview.
23
• Analysisofdata.• Interpretationofdata.• Providingapolicyperspective.• Writingthereview(orprotocol).• Securingfundingforthereview.• Performingpreviousworkthatwasthefoundationofthecurrentreview.
PATRICESUTTON
PatriceSuttonisanAcademicCoordinatorwiththeUCSFProgramonReproductiveHealthandtheEnvironment(PRHE).ShehasbeenspearheadingPRHE'sresearchtranslationeffortssince2008andhasbeentheprojectleadontheNavigationGuidesystematicreviewmethodologysinceitsinceptionin2009.PatriceistheDirectoroftheCommunityOutreachandTranslationCoreofPRHE’sPregnancyExposurestoEnvironmentalChemicals(PEEC)Children’sCenter.PatricehasaMastersofPublicHealthfromU.C.BerkeleyinEnvironmentalHealthSciences.Patricehasover27yearsofexperienceinoccupationalandenvironmentalhealthresearch,industrialhygiene,publichealthpractice,policydevelopmentandcommunity-basedadvocacy.AsacontractortoCalifornia'sstatehealthdepartmentfrom1987to2006,shewasresponsibleforconductingallaspectsofresearchinvestigationsspanningadisparaterangeofissues,includingleadpoisoning,tuberculosis,asthma,andpesticide-illness.Shehasextensiveexperiencecollaboratingwithdirectly-impactedworkplaceandcommunity-basedpopulations,labor,andgovernmentalandnon-governmentalorganizationsinthedevelopmentofresearchstrategiesandpolicyrecommendations.Shealsohasextensiveexperienceasavolunteerinsupportofcommunitiesandworkersimpactedbythenuclearweaponsproductioncycleandhaspublishedover50peer-reviewedscientificarticlesandgovernmenttechnicalreports.
Contributionstothereview:
• Conceivingthereview.• Designingthereview.• Designingsearchstrategies.• Appraisingqualityofpapers.• Interpretationofdata.• Providingamethodologicalperspective.• Providingapolicyperspective.• Reviewingtheprotocol.• Providinggeneraladviceonthereview.• Securingfundingforthereview.• Performingpreviousworkthatwasthefoundationofthecurrentreview.
AMYPADULA
AmyPadulaisanInstructoratStanfordUniversityintheDepartmentofPediatricsandtheDivisionofNeonatology.ShewillbestartingasanAssistantAdjunctProfessoratUCSan
24
FranciscoinMarch2016.ShereceivedherPhDinEpidemiologyfromUCBerkeleyin2010.Sheisaperinatalandenvironmentalepidemiologistandspecializesinenvironmentalexposuresduringpregnancy.Herworkhasevaluatedtheeffectsofairpollutiononpulmonaryfunctioninchildrenandadversebirthoutcomesincludingpretermbirth,lowbirthweightandstructuralbirthdefects.
Contributionstothereview:
IwillcontributetothereviewofFormaldehydeandAsthmausingtheNavigationGuideinthefollowingways:Iwillassistwithdesigningthereviewandresearchstrategies.Iwillscreenbothtitle/abstractsandfulltextagainsteligibilitycriteria.Iwillappraisethequalityofthepaperandextractdatafromthem.Iwillhelpwiththeinterpretationofthedataandprovideamethodologic(bothepidemiologicandstatistical)perspective.Iwillassistwithwritingandrevisingthereview.Iwillprovidegeneraladviceontheprotocolandreview.
MICHAELCABANA
Dr.MichaelCabanaisaProfessorofPediatrics,EpidemiologyandBiostatistics,aswellasamemberofthecorefacultyatthePhilipR.LeeInstituteforHealthPolicyStudiesattheUniversityofCalifornia,SanFrancisco(UCSF).Dr.Cabana’sresearchfocusesonclinicalmanagementofasthma,epidemiology,andprevention.Dr.CabanaisPrincipalInvestigatorfortheTrialofInfantProbioticSupplementation(TIPS)toPreventEarlyMarkersofAsthma(R01HL80074),aswellasclinicaltrialsfocusedoninfantgrowthandcolic.Inaddition,Dr.CabanaisthePrincipalInvestigatorfortheUCSFPediatricAsthmaNetResearchCenter(U10HL074204)oneofninesitesintheUnitedStates.Hehasextensiveexperienceinpractice-basedresearchandhascollaboratedwithover120pediatricpracticesinseveralnationalrandomizedcontrolledtrialsfocusedonprimarycaremanagementofasthma.Dr.CabanahasservedonstudysectionsfortheNIHandadvisorygroupsfortheNationalCommitteeonQualityAssurance.Dr.Cabanahasmaintainedanactivepresenceinclinicalmedicine,servingasChiefoftheUCSFDivisionofGeneralPediatricssince2005.
Contributionstothereview:
• Appraisingqualityofpapers• Providingexpertiseondatatobeextractedfrompapers.• Providingexpertiseondataanalysis.• Interpretationofdata.• Providingamethodologicalperspective.• Providingaclinicalperspective.• Writingpartsofthereview(orprotocol).• Providinggeneraladviceonthereview.
25
ERICAKOUSTAS
EricaisaconsultanttotheUCSFProgramonReproductiveHealthandtheEnvironment(PRHE).ShebeganherinvolvementontheNavigationGuidesystematicreviewmethodologyin2010andwasoneoftheleadauthorsonthefirstNavigationGuidecasestudyasanOakRidgeInstituteforScienceandEducation(ORISE)Post-doctoralFellowandanAmericanAssociationfortheAdvancementofScience(AAAS)ScienceandTechnologyPolicyFellowattheU.S.EnvironmentalProtectionAgency(EPA).Overthelastfiveyears,EricahasbeeninvolvedintherefinementoftheNavigationGuidemethodologyandparticipatedintwoadditionalcasestudies.InadditiontoherextensiveexperiencewiththeNavigationGuidemethodology,Ericahasalsoworkedinotherareasofenvironmentalhealth,includingeffortstodevelopindicatorsforchildren’senvironmentalhealthandimprovechemicalriskassessmentssotheycanbetterinformbenefit-costanalysisandregulatorydecision-making.EricaholdsaPh.D.inMolecularBiologyfromtheUniversityofColoradoDenverandaB.S.inMolecularBiologyfromtheUniversityofDenver.
Contributionstothereview:
• Datacollectionforthereview.• Screeningsearchresults.• Screeningretrievedpapersagainsteligibilitycriteria.• Appraisingqualityofpapers.• Extractingdatafrompapers.• Obtainingandscreeningdataonunpublishedstudies.• Datamanagementforthereview.• Interpretationofdata.• Providingmethodologicalperspective.• Providingpolicyperspective.• Writingthereview(orprotocol).• Providinggeneraladviceonthereview.• Providingpreviousworkthatwasthefoundationofthecurrentreview.
HANNAVESTERINEN
HannaVesterinenisapostdoctoralresearcherworkingasaconsultantforPRHE.In2013shecompletedaPhDatTheUniversityofEdinburghwhereshefocusedondevelopingsystematicreviewandmeta-analysismethodologyinanimalstudies.Thisresearchresultedinthepublicationofa“practicalguide”forotherresearchersapplyingmeta-analysistodatafromanimalstudies.Hannaalsosuccessfullyappliedthesetechniquestoareviewofoverathousandanimalstudiesinmultiplesclerosisresearchtoassesspotentialreasonsforthefailuretotranslateefficacyfrombenchtobedside.Hanna’sresearchhasledtonumerouspublishedsystematicreviewsofindividualinterventionsordruggroupsinseveralanimalmodelsofneurologicaldiseasesincludingParkinson’sDisease,Huntingdon’sdisease,gliomaandstroke.Additionallysheusedbothhumanandanimaldatatoidentifyapotentialtreatmentforprogressivemultiplesclerosis
26
whichwasbothpublishedandtakenforwardtoaclinicaltrialwhichisnowunderway.FollowingthecompletionofherPhD,HannawasemployedasapostdoctoralresearchfellowatPRHEwheresheworkedontwoNavigationGuidecase-studies,triclosanandglomerularfiltrationrate,aswellasconductingasystematicreviewonthecumulativeeffectofpre-natalexposuretostressandchemicalsonfetaldevelopment.
Contributionstothereview:
Mycontributiontothereviewwillbetoscreensearchresults,appraisethequalityofstudies,extractdatafrompapers,enterdataintothedatabaseaswellasprovidinggeneraladviceonthereviewandaidingintheeditingofthemanuscript.
EVANSM.WHITAKER
Dr.WhitakerisamedicalresearchlibrarianattheUniversityofCalifornia,SanFranciscoandservesasEducationConsultantfortheSchoolsofMedicineandPharmacy.Dr.Whitakercombinesextensiveclinicalexperienceasafamilyphysicianwithformaltrainingand3yearsexperienceasaresearchlibrarianbasedatamedicalschoolrichwitheducationalscholarshipandinterprofessionalcollaborationstoadvancehealthprofessionstraining.Hedevelopsandpresentscontenttostudents,residents,fellowsandfacultyintheUCSFenvironment.Hehasassistedonquestionformulation,protocolconstruction,methodssectionwritingforover30systematicreviews.Dr.Whitakerwillassistwithallstepsofthescopingreviewprocesswithaparticularemphasisondevelopingthesearchquestion,constructingthesearchcriteria,andsynthesizingresults.
Contributionstothereview:
• Designingsearchstrategies.• Undertakingsearches.• Organizingretrievalofpapers.• Screeningretrievedpapersagainsteligibilitycriteria.• Appraisingqualityofpapers.• Obtainingunpublishedpapers.• Providingaclinicalperspective.• Writingtherevieworprotocol.• Providinggeneraladviceonthereview.
NATALYNDANIELS
NatalynDanielsisaResearchAssistantworkingwithPRHE.NatalynreceivedaB.A.fromUCBerkeleyin2011.Inconductingherundergraduatethesis,shebecamethefirsttodevelopanexperimentprotocolandmethodologytotesttheEcologicalValenceTheory.Herinterestin
27
reproductionandenvironmentalhealthstemsfromherworkasanambulanceemergencymedicaltechnicianandherpreviousResearchAnalystappointmentintheDivisionofAdolescentandYoungAdultMedicineatUCSF.AsaResearchAnalyst,sheevaluatedastate-widecasemanagementframeworkgearedtowardsimprovingaPositiveYouthDevelopmentinterventionforpregnantandparentingteenagersinCalifornia.Shecompletedanextensiveliteraturereviewanddatacollectionprocess,andisaco-authoronthe“Maternal,Child,andAdolescentHealthAdolescentFamilyLifeProgramPositiveYouthDevelopmentFormativeEvaluationReport.”
Contributionstothereview:
• Conductabstractandfull-textscreeningsofpublishedstudiesinthesearchresults• Conductabstractandfull-textscreeningsofunpublishedstudiesinthesearchresults• AssessandapplytheNavigationGuideratingcategoriestodeterminethequalityofthe
evidence• Cometoanoverallratingofthestrengthoftheevidence• Providegeneralinputregardingthereview,protocol,andratings
TRACEYJ.WOODRUFF
Dr.WoodruffisProfessorintheDepartmentofObstetrics,Gynecology,andReproductiveSciencesandPhilipRLeeInstituteforHealthPolicyStudiesattheUniversityofCalifornia,SanFranciscoandtheDirectoroftheProgramonReproductiveHealthandtheEnvironment.Shehasdoneextensiveresearchandpolicydevelopmentonenvironmentalhealthissues,withaparticularemphasisonearly-lifedevelopment.Herresearchincludesevaluatingprenatalexposurestoenvironmentalchemicalsandrelatedadversepregnancyoutcomes,andcharacterizingdevelopmentalrisks.Dr.WoodruffconceivedofandwastheleadforthecollaborativeeffortwhichdevelopedtheNavigationGuidesystematicreviewmethodology.Shehasauthorednumerousscientificpublicationsandbookchapters,andhasbeenquotedwidelyinthepress,includingUSAToday,theSanFranciscoChronicle,andWebMD.ShewaspreviouslyattheUSEPA,whereshewasaseniorscientistandpolicyadvisorintheOfficeofPolicy,andauthorofnumerousgovernmentdocuments.SheisanAssociateEditorofEnvironmentalHealthPerspectives.ShewasappointedbythegovernorofCaliforniain2012totheScienceAdvisoryBoardoftheDevelopmentalandReproductiveToxicant(DART)IdentificationCommittee.
Contributionstothereview:
• Conceivingthereview.• Designingthereview.• Designingsearchstrategies.• Appraisingqualityofpapers.• Providingexpertiseondatatoextractfrompapers.• Providingexpertiseondataanalysis.• Interpretationofdata.• Providingamethodologicalperspective.
28
• Providingapolicyperspective.• Providingexpertiseonwritingthereview(orprotocol).• Providinggeneraladviceonthereview.• Securingfundingforthereview.• Performingpreviousworkthatwasthefoundationofthecurrentreview.
INFORMATIONSPECIALISTCONSULTANT
LESLEYSKALLA
LesleySkallaisaresearchanalystandsciencelibrarianforMDB,Inc.inDurham,NorthCarolina.SheprovidesarangeofscientificsupportservicesfortheNationalInstituteofEnvironmentalHealthSciences(NIEHS),fromliteraturesearchestograntcodingandportfolioanalysis.Anexpertsearcher,Lesleyprovidesbothtraditionalandsystematicreviewliteraturesearchesusinganumberofbibliographicdatabases,includingPubMed,WebofScience,BIOSIS,andEmbase.AstheinformationspecialistforthefourthNavigationGuideProof-of-Conceptcasestudy(Associationbetweendevelopmentalexposurestoambientairpollutionandautism),Lesleywasresponsiblefordesigning,conducting,anddocumentingboththebibliographicandgreyliteraturesearch.LesleyreceivedherPhDinAnimalSciencewithaspecializationinReproductivePhysiologyfromtheUniversityofIllinoisinUrbanaandherMastersofInformationandLibrarySciencefromtheUniversityofNorthCarolinainChapelHill.Throughherdualtrainingasascientistandasalibrarian,Lesleyprovidesclientswithauniqueskillsetwell-suitedforliterature-basedevaluationprojects.Shehasoverfiveyearsofexperienceconductingliteraturesearchesinthefieldofenvironmentalhealthsciences.
29
AUTHORCONFLICTOFINTERESTSTATEMENTS
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
AppendixII.SearchTermsAliteraturesearchwillbedevelopedandconductedcollectivelybetweentwoinformationspecialists(LR&EW)betweenusingthedatabase-specificsearchtermsbelow.
PreliminaryScopingsearch:
Search PubMed
#1
(("Asthma"[Mesh]ORasthma[tiab]OR“airwayhyper-responsiveness”[tiab]OR“airwayhyperresponsiveness”[tiab]OR"RespiratoryHypersensitivity"[Mesh]ORbronchialhypersensitivity[tiab]ORbronchialhyper-sensitivity[tiab]OR“airwayhyper-sensitivity”[tiab]OR“airwayhypersensitivity”[tiab]OR“reactiveairway”[tiab]OR“reactiveairways”[tiab]OR“airwayinflammation”[tiab]OR“airwayobstruction”[tiab]ORwheeze[tiab]ORwheezes[tiab]ORwheezing[tiab]ORdyspnea[tiab]ORspirometry[tiab]OR“lungfunction”[tiab]OR“lungfunctions”[tiab]OR“respiratoryfunction”[tiab]OR“respiratoryfunctions”[tiab]OR“pulmonaryfunction”[tiab]OR“pulmonaryfunctions”[tiab]OR"BronchialHyperreactivity”[MH]OR“bronchialhyperreactivity”[tiab]OR“bronchialhyper-reactivity”[tiab]OR“bronchialspasm”[tiab]OR“bronchialspasms”[tiab]OR"BronchialSpasm"[Mesh]OR"AirwayResistance”[MH]OR“airwayresistance”[tiab]OR“respiratoryhealth”[tiab]OR"RespiratoryFunctionTests"[MeSHTerms]OR"RespiratorySounds"[Mesh:NoExp]OR"Bronchoconstriction"[Mesh]OR“bronchialconstriction”[tiab]OR“bronchialconstrictions”[tiab]OR“reactiveairwaydisease”[tiab]))AND(("Formaldehyde"[Mesh]ORFormaldehyde[tiab]OROxomethane[tiab]ORMethanal[tiab]ORFormol[tiab]ORFormalin[tiab]ORparaformaldehyde[tiab]OR“Medium-densityfibreboard”[tiab]OR“medium-densityfiberboard”[tiab]ORparticleboard[tiab]ORparticle-board[tiab]ORplywood[tiab]ORwood-based[tiab]))
WewilluseSwift-Review(http://www.sciome.com/swift-review/)toconductapreliminaryassessmentofthesizeandscopeoftheliteratureretrievedwithourscopingreviewsearchstrategy.Swift-reviewwillalsobeusedtofine-tuneourselectionofMeSHterms.
PubMedsearchstrategy:
Search PubMed
69
#1
((("Asthma"[Mesh]ORasthma*[tiab]ORreactiveairway[tiab]ORreactiveairways[tiab]ORairwayinflammation[tiab]ORwheeze[tiab]ORwheezes[tiab]ORwheezing[tiab]ORdyspnea[tiab]OR"RespiratoryFunctionTests"[MeSHTerms]OR"RespiratorySounds"[Mesh:noexp]ORspirometry[tiab]ORlungfunction[tiab]ORlungfunctions[tiab]ORrespiratoryfunction[tiab]ORrespiratoryfunctions[tiab]ORpulmonaryfunction[tiab]ORpulmonaryfunctions[tiab]OR"ForcedExpiratoryVolume"[Mesh]OR"PeakExpiratoryFlowRate"[Mesh]OR"ForcedExpiratoryVolume"[Mesh]ORFEV1[tiab]ORPEFR[tiab]ORPEF[tiab]OR"peakexpiratory"[tiab]OR"forcedexpiratoryvolume"[tiab]OR"forcedexpiratoryflow"[tiab]OR"BronchialHyperreactivity"[MH]ORbronchialhyperreactivity[tiab]ORbronchialhyper-reactivity[tiab]ORbronchialhypersensitivity[tiab]ORbronchialhyper-sensitivity[tiab]OR"BronchialSpasm"[Mesh]ORbronchialspasm[tiab]ORbronchialspasms[tiab]OR"AirwayResistance"[MH]OR"AirwayObstruction"[mh]ORairwayobstruction[tiab]ORairwayresistance[tiab]OR"Bronchoconstriction"[Mesh]ORbronchialconstriction[tiab]ORbronchialconstrictions[tiab]ORrespiratoryhealth[tiab]ORreactiveairwaydisease[tiab])))AND((("Formaldehyde"[Mesh]OR50-00-0[RN]ORFormaldehyde[tiab]OROxomethane[tiab]ORMethanal[tiab]ORFormol[tiab]ORFormalin[tiab]ORparaformaldehyde[tiab]ORMedium-densityfibreboard[tiab]ORmedium-densityfiberboard[tiab]ORparticleboard[tiab]ORparticle-board[tiab]ORplywood[tiab]ORwood-based[tiab]ORwood-product*[tiab]ORcomposite-wood[tiab]ORpressed-wood[tiab]ORvarnish*ORlaminate[tiab]OR"FloorsandFloorcoverings"[Mesh]ORflooring[tiab]OR"polyurethanefoam"[SupplementaryConcept]OR"polyurethanefoam"[tiab]OR"ureaformaldehydefoam"[SupplementaryConcept]OR"ureaformaldehydefoam"[tiab]OR"Adhesives"[Mesh]ORAdhesives[tiab]ORsalon[tiab]ORhair-straighten*[tiab]ORhair-smooth*[tiab]OR(trailer*[tiab]AND(housing[tiab]OR"housing"[Mesh]))ORTravel-trailer*[tiab]ORmanufactured-home*[tiab]ORmobile-home*[tiab]ORmanufactured-hous*[tiab]ORmodular-home*[tiab])OR"temporaryhousingunit"[tiab]OR"temporaryhousingunits"[tiab]OR(("ConstructionMaterials"[Mesh]OR"Housing"[Mesh:noexp])AND(airpollution[tiab]ORairquality[tiab]OR"Airpollution"[Mesh])AND"asthma"[Mesh]))ORcadaver[tiab]ORembalm*[tiab]ORgarment-workers[tiab]ORtextile-workers[tiab]ORwood-workers[tiab]ORwoodworkers[tiab])
70
WebofScienceCoreCollection:
Search WebofScience
#1
(asthmaOR“reactiveairway”OR“reactiveairways”OR“airwayinflammation”ORwheezeORwheezesORwheezingORdyspneaOR“lungfunctiontest”ORspirometryOR“lungfunction”OR“lungfunctions”OR“respiratoryfunction”ORrespiratoryfunctionsOR“pulmonaryfunction”OR“pulmonaryfunctions”OR“bronchushyperreactivity”OR“bronchialhyperreactivity”OR“bronchialhyper-reactivity”OR“bronchialhypersensitivity”OR“bronchialhyper-sensitivity”ORbronchospasmORbronchospasmOR“bronchialspasm”OR“bronchialspasms”ORairwayresistanceORairwayobstructionOR“airwayobstruction”OR“airwayresistance”ORbronchoconstrictionOR“bronchialconstriction”OR“bronchialconstrictions”OR“respiratoryhealth”OR“reactiveairwaydisease”)AND(50-00-0ORformaldehydeORoxomethaneORmethanalORformolORformalinORparaformaldehydeOR“medium-densityfibreboard”OR“medium-densityfiberboard”ORparticleboardORparticle-boardORplywoodORwood-basedORcomposite-woodORpressed-woodOR“polyurethanefoam”OR“polyurethanefoam”OR“ureaformaldehyde”ORadhesiveORadhesivesORpolyurethanefoamORpolyurethane*foamORureaformaldehydefoamORadhesivesOR((trailer*ANDhous*)OR(traveltrailer*ORmanufacturedhome*ORmobilehome*ORmanufacturedhous*ORmodularhome*OR“temporaryhousingunit”OR“temporaryhousingunits”)OR(“buildingmaterial”ORhousing)AND(“airpollution”OR“airquality”)AND(asthma)))
BiosisPreviews:
Search BiosisPreviews
#1
(asthmaOR“reactiveairway”OR“reactiveairways”OR“airwayinflammation”ORwheezeORwheezesORwheezingORdyspneaOR“lungfunctiontest”ORspirometryOR“lungfunction”OR“lungfunctions”OR“respiratoryfunction”ORrespiratoryfunctionsOR“pulmonaryfunction”OR“pulmonaryfunctions”OR“bronchushyperreactivity”OR“bronchialhyperreactivity”OR“bronchialhyper-reactivity”OR“bronchialhypersensitivity”OR
71
“bronchialhyper-sensitivity”ORbronchospasmORbronchospasmOR“bronchialspasm”OR“bronchialspasms”ORairwayresistanceORairwayobstructionOR“airwayobstruction”OR“airwayresistance”ORbronchoconstrictionOR“bronchialconstriction”OR“bronchialconstrictions”OR“respiratoryhealth”OR“reactiveairwaydisease”)AND(50-00-0ORformaldehydeORoxomethaneORmethanalORformolORformalinORparaformaldehydeOR“medium-densityfibreboard”OR“medium-densityfiberboard”ORparticleboardORparticle-boardORplywoodORwood-basedORcomposite-woodORpressed-woodOR“polyurethanefoam”OR“polyurethanefoam”OR“ureaformaldehyde”ORadhesiveORadhesivesORpolyurethanefoamORpolyurethane*foamORureaformaldehydefoamORadhesivesOR((trailer*ANDhous*)OR(traveltrailer*ORmanufacturedhome*ORmobilehome*ORmanufacturedhous*ORmodularhome*OR“temporaryhousingunit”OR“temporaryhousingunits”)OR(“buildingmaterial”ORhousing)AND(“airpollution”OR“airquality”)AND(asthma)))
Embase:
Search Embase
#1
(‘Asthma’/expORasthma:ab,tiOR‘reactiveairway*’:ab,tiOR‘airwayinflammation’:ab,tiOR'wheezing'/expORwheez*:ab,tiORdyspnea:ab,tiOR'lungfunctiontest'/expORspirometry:ab,tiOR‘lungfunction’:ab,tiOR‘lungfunctions’:ab,tiOR‘respiratoryfunction’:ab,tiOR‘respiratoryfunctions’:ab,tiOR‘pulmonaryfunction’:ab,tiOR‘pulmonaryfunctions’:ab,tiOR‘ForcedExpiratoryVolume’/expOR‘PeakExpiratoryFlow’/expORFEV1:ab,tiORPEFR:ab,tiOR
72
PEF:ab,tiOR‘peakexpiratory’:ab,tiOR‘forcedexpiratoryvolume’:ab,tiOR‘forcedexpiratoryflow’:ab,tiOR'bronchushyperreactivity'/expOR‘bronchialhyperreactivity’:ab,tiOR‘bronchialhyper-reactivity’:ab,tiOR‘bronchialhypersensitivity’:ab,tiOR‘bronchialhyper-sensitivity’:ab,tiOR'bronchospasm'/expORbronchospasm:ab,tiOR‘bronchialspasm’:ab,tiOR‘bronchialspasms’:ab,tiOR'airwayresistance'/expOR'airwayobstruction'/expOR‘airwayobstruction’:ab,tiOR‘airwayresistance’:ab,tiOR'bronchoconstriction'/expOR‘bronchialconstriction’:ab,tiOR‘bronchialconstrictions’:ab,tiOR‘respiratoryhealth’:ab,tiOR‘reactiveairwaydisease’:ab,ti)
#2 'formaldehyde'/expOR'formaldehyde'OR'50-00-0':rnORformaldehyde:ab,tiORoxomethane:ab,tiORmethanal:ab,tiORformol:ab,tiORformalin:ab,tiORparaformaldehyde:ab,tiOR'medium-densityfibreboard':ab,tiOR'medium-densityfiberboard':ab,tiORparticleboard:ab,tiOR'particle-board':ab,tiORplywood:ab,tiOR'wood-based':ab,tiOR'composite-wood':ab,tiOR'pressed-wood':ab,tiOR‘varnish*’:ab,tiOR‘laminate’:ab,tiOR‘Floorcoverings’:ab,tiORflooring:ab,tiOR
73
'polyurethanefoam'/expOR'adhesiveagent'/expOR'polyurethanefoam':ab,tiOR'polyurethanfoam'/expOR'ureaformaldehydefoam':ab,tiORadhesives:ab,tiORsalon:ab,tiORhair-straighten*:ab,tiORhair-smooth*:ab,tiOR(trailer*:ab,tiAND(housing:ab,tiOR'housing'/expORhousing))OR'traveltrailer*':ab,tiOR'manufacturedhome*':ab,tiOR'mobilehome*':ab,tiOR'manufacturedhous*':ab,tiOR'modularhome*':ab,tiOR'temporaryhousingunit*':ab,tiOR('buildingmaterial/exp'OR'housing'/expOR'housing'AND('airpollution':ab,tiOR'airquality':ab,tiOR'airpollution'/expOR'airpollution')AND('asthma'/expORasthmaORasthma:ab,ti))ORcadaver:ab,tiORembalm*:ab,tiORgarment-workers:ab,tiORtextile-workers:ab,tiORwood-workers:ab,tiORwoodworkers:ab,ti
#3 #1AND#2
ToxlineandDART:
Search Toxline&DART
#1
Formaldehydeandasthma
AppendixIII.OtherResourcesforLiteratureSearchToxicologicalwebsitestosearch
Inmostcases,thesimplifiedsearchstrategy“asthmaAND(formaldehydeOR“50-00-0”)wasusedtosearchtoxicologicalwebsites/databases.
• ATSDRInteractionProfileshttp://www.atsdr.cdc.gov/interactionprofiles/index.asp• ATSDRToxicologicalProfileshttp://www.atsdr.cdc.gov/toxprofiles/index.asp
74
• ACTOR• CalEPAOfficeofEnvironmentalHealthHazardAssessment
http://www.oehha.ca.gov/risk.html,http://oehha.ca.gov/air.html• ChemIDhttp://chem.sis.nlm.nih.gov/chemidplus/• EPAAcuteExposureGuidelineLevelshttp://www.epa.gov/oppt/aegl/chemlist.htm• EPAIRISinternetwww.epa.gov/iris• EPANEPISandNSCEPhttp://www.epa.gov/nscep/• EPAScienceInventoryhttp://cfpub.epa.gov/si/• EPASubstanceRegistrySystem
http://ofmpub.epa.gov/sor_internet/registry/substreg/searchandretrieve/substancesearch/search.do
• HealthCanadaFirstPriorityListAssessmentshttp://www.hcsc.gc.ca/hecssesc/exsd/psl1.htm
• HealthCanadaSecondPriorityListAssessmentshttp://www.hcsc.gc.ca/hecssesc/exsd/psl2.htm
• HazardousSubstancesDataBankhttp://toxnet.nlm.nih.gov/cgi-bin/sis/htmlgen?HSDB• IPCSINCHEMhttp://www.inchem.org/• NIOSHTIC2http://www2.cdc.gov/nioshtic2/Nioshtic2.htm• ToxicologyDataNetworkhttp://toxnet.nlm.nih.gov/• RTECSToxcenterhttp://www.cdc.gov/niosh/rtecs/default.html• WHOassessments–CICADS,EHChttp://www.who.int/ipcs/assessment/en/• USEPAHealthandEnvironmentalStudiesOnlinehttp://hero.epa.gov/• FIFRAdocket:http://www.regulations.gov
Greyliteraturedatabasestosearch
Inmostcases,thesimplifiedsearchstrategy“asthmaAND(formaldehydeOR“50-00-0”)wasusedtosearchgreyliteraturedatabases.
Google:http://www.google.comGoogleScholar:http://scholar.google.com/Databaseoffederally-fundedscientificresearch:Science.govScienceResearch.com(SciencefederatedsearchenginebyDeepWebTechnologies):http://scienceresearch.com/Oaisterdatabase(anopen-sourcerepositoryofdifficult-to-access,academically-orienteddigitaltresources):http://www.oclc.org/oaisterOpenGrey:http://www.opengrey.eu/ProceedingsFirst(Coverseverypublishedcongress,symposium,conference,exposition,workshopandmeetingreceivedbyTheBritishLibraryDocumentSupplyCentre)PapersFirst(Accesstoindividualpaperspresentedatconferencesworldwide)ProQuestDissertationsandTheses(Internationalrepositoryofgraduatedissertationsandtheses)
75
AppendixIV.ExclusionCriteriaScreeningFormsandAmendmentstoClarifyScreeningProcess
TitleandAbstractScreeningForm
INSTRUCTIONS:
Whenexcludingareference,pleaseselectonlyONE(1)exclusionreason.PleasereviewtheexclusionreasonsinorderandselecttheFIRSTexclusionreasonrelevanttothereferencebeingscreened.Pleaseaddinanyadditionalnotesinthecommentboxtoexplainyourselectionifnecessary.
Categories:• Exclude—Articleisareviewofformaldehydeexposureandasthma;• Exclude—Articlecontainsnooriginaldata(e.g.,editorial,reviewpapernotrelevantto
studyquestion,etc.)• Exclude—Articledidnotinvolvehumansubjects(i.e.,animalevidenceorcelllinesonly,
casereportofsinglehuman,etc.);• Exclude—Articledidnotreportformaldehydeexposure,asdefinedbyPECOstatement;• Exclude—Articledidnotreportoutcomesofasthma,asdefinedbyPECOstatement;• Exclude—Otherreason(explanationrequired).
• Include—Retrievefullarticle
Comments:Explainherereasonforexclusionifotherthanreasonsprovidedin#1above,andanyotherrelevantcomments.
AmendmentstoTitleandAbstractScreeningProcess
Addhereanyadditionaldetailsifnecessary.
Full-TextScreeningForm
INSTRUCTIONS:
Whenexcludingareference,pleaseselectonlyONE(1)exclusionreason.PleasereviewtheexclusionreasonsinorderandselecttheFIRSTexclusionreasonrelevanttothereferencebeingscreened.Pleaseaddinanyadditionalnotesinthecommentboxtoexplainyourselectionifnecessary.
Categories(selectone):
76
• Exclude—Articleisareviewofformaldehydeexposureandasthma;• Exclude—Articlecontainsnooriginaldata(e.g.,editorial,reviewpapernotrelevantto
studyquestion,etc.);• Exclude—Articledidnotinvolvehumansubjects(i.e.,animalevidenceorcelllineonly,
casereportofsinglehuman,etc.);• Exclude—Articledoesnotreportexposurestoformaldehyde,asdefinedbythePECO
statement;• Exclude—Articledoesnotreportdiagnosisofasthma,asthmasignsorsymptoms,asthma
exacerbation,orindirectmeasuresofasthma,asdefinedbythePECOstatement;• Exclude—Therewasnocomparatorgroup;• Exclude—Duplicatestudy;• Exclude—Otherreasons(explanationrequired).
• POSSIBLYInclude—otherlanguage(specifywhichlanguagebelowincomments);
• IncludeDefinition:Studymeetsinclusioncriteriaasfollows:
Populationstudiedishumans.Exposureisanyindoororoutdoorsourcesofairborneinhalationexposuretoformaldehyde(maternal/paternalpriortoconception,maternalexposureduringpregnancy,orexposuretotheindividualduringanylifestage),includingbutnotlimitedtooccupational,outdoorambient,indoorhouseholdsettings,and/orexposuretohouseholdproducts(medium-densityfiberboard,particleboard,compositewood,pressedwood,polyurethanefoam,adhesives,mobilehomes,trailerhomes,etc.)thatoccurredpriororconcurrenttodiagnosis,exacerbation,orothermeasureofasthma.Comparatorishumansexposedtolowerlevelsofformaldehydethanthemorehighlyexposedhumans.Outcomeisanyofthefollowingasthma-relatedoutcomes:diagnosisofasthma(clinicalasthmadiagnosisbyahealthcareprovider,whichmayincludeparentalreportofaclinicalasthmadiagnosisfortheirchild),asthmasigns,symptoms,orexacerbation(asthmasymptoms(cough,wheeze,chestpain,etc.),asthmasymptomsleadingtobronchodilatoruse(albuterolonly),asthma-specificemergencydepartmentvisits,outpatientvisits,orhospitaladmissions,asthma-specificICUadmissions/intubations,asthma-specificdetailedmedicationuse,AsthmaSyndromeUtilityIndex(ASUI)retrospectivequestionnaire,useofsystemiccorticosteroidsforasthma,methacholinechallengetest,and/orchangesinspirometrymeasurese.g.,FEV1,FVC,FEV1/FVC,PEForPEFR,etc.overtime)orindirectmeasuresofasthma(schoolabsences,lossofworkerdays,etc.impactingthequalityoflifeasaresultofasthmaeffects).
Comments:(explainhereifreasonforexclusionisotherthanreasonsprovidedinabove,explainwhythisispossiblyaduplicatestudy,orspeculatewhatlanguagestudyappearstobeifnotinEnglish)
77
AmendmentstoFullTextScreeningProcess
Addhereanyadditionaldetailsifnecessary.
78
AppendixV.DataCollectionFormsThesourcecriteriachecklistsforextractiontermsinclude:goldstandardpublicationchecklist(GSPC);ARRIVEguidelines(ARRIVE);CochraneHandbookforSystematicReviewsofInterventionsdatacollectionchecklist(Cochrane);GRADEcriteriaforrandomizedcontroltrials(GRADE).
DataCollectionforHumanstudies
Fieldsarefree-formexceptwherechoices(initalics)areshown
SOURCE
Refid:
Reviewer:
Publicationyear:
Authors’declaredconflictsofinterest:
• Nonedeclared
• Declared
Ifdeclared,providedetails:
Studyfundingsource:
• Governmentgrant
• Industryfunded
• Nonprofitorganizationgrant
• Other
Studyfundingsourcedetails:
Whatarethestudyobjectives?:
Site(s)ofdatacollection(city,state,country):
METHODS
Studyduration/dates:
Studydesign:
79
• Cross-sectional
• Cohort,prospective
• Cohort,retrospective
• Case-control
• Ecological
• Other(listdetailsbelow)
Studydesigndetails:
STUDYPOPULATIONCHARACTERISTICS
Cohort(givedescription,e.g.NHANES2004-2006)
Samplesizeoftotalcohort
Totalnumberofstudygroups
Descriptionofreferencegroup
Samplesize(eachstudygroup)
Targetsamplesize
Participation/follow-uprates
Inclusion/exclusioncriteria/recruitmentstrategy
Age(eachexposuregroup)
Co-morbidities
Otherrelevantdetails(listbelow)
Exposuremeasurementtiming:
• Maternal/paternalexposurepriortoconception
• Inutero
• Prenatalperiod
• Infancyperiod(upto24months)
80
• Childhoodperiod(24monthsandafter)
• Other(detailsbelow)
Exposuremeasurementtimingdetails:
Sourceofexposuredata:
• Biomonitoring(listspecificmatrix)
• Environmentalmonitoring(listspecificmatrix)
• Questionnaire(listspecificdeterminantofexposure)
• Other(specify)
Rangeofconcentrationsofformaldehydemeasured,andunits:
Frequencyofexposuremeasurementsifmorethanonce:
Numberofreplicatemeasurementstaken:
Otherchemicalinformation:
Outcomesmeasured:
Methodofasthmaoutcomemeasurement/assessment:
Sex(whereoutcomemeasured):
• Malesonly
• Femalesonly
• Malesandfemales
• Other(detailsbelow)
Numbersubjectsanalyzed(forexposureandoutcome):
Numberofmissingparticipants:
RESULTS
81
Statisticalmethods:
• Statisticaltestsemployed
• Statistic(oddsratio,adjustedoddsratio,betaestimate,etc.)
• p-valuesgiven
• Confidenceintervalsgiven
• Confoundingadjustmentsinstatisticaltests
Wereknownconfoundersaccountedforbystudydesign?
Wereknownconfoundersaccountedforbyanalysis?
Howweredatareported(mean,median,rawdata,etc.)?:
Asthmameasurement/assessmentdataforeachgroup(i.e.,outcome),ifavailable:
Howasthmameasurement/assessmentdatawerereported(table,figure,etc.),ifavailable:
Summarydataforeachgroup
Estimateofeffectwithconfidenceintervalandp-value
Howwasprecisionreported(standarderror,CI,etc.)?:
• Standarderror
• Standarddeviation
• Confidenceintervals
• Other(detailsbelow)
• Notstated
Howprecisionreporteddetails:
Precisionestimates:
Howprecisionestimateswerereported(table,figure,etc):
Miscellaneouscommentsbyreviewerregardingdataanalysis:
82
AppendixVI.InstructionsforMakingRiskofBiasDeterminations
HumanStudies
Note:Thesecriteriaforjudgingriskofbiasareforhumanstudiesonlysincewearenotevaluatinganimalstudiesinthiscasestudy.
Instructions:
• Pleaseevaluateeachindividualstudyforthefollowingnineriskofbiasdomains.Pleaseanswer“lowrisk,”“probablylowrisk,”“probablyhighrisk,”“highrisk,”or“notapplicable”andprovidedetails/justificationforyourrating.Ifthereisempiricalevidenceorotherknowledgethatinformsthedirectionofbias,pleaseincludethisinyouransweraswell;however,ifthereisnotenoughinformationtodosopleasedonotguessatthedirectionofbias.
• Additionally,pleasenotethatsomeinternalvalidityissuescouldpotentiallybeappropriatelycapturedinseveraldifferentriskofbiasconsiderations.Inthissituation,pleaseselectthesinglemostappropriatedomaintoevaluatethispotentialbias,toavoiddouble-countingthesameinternalvalidityconcern.
1.Arethestudygroupsatriskofnotrepresentingtheirsourcepopulationsinamannerthatmightintroduceselectionbias?Thesourcepopulationisviewedasthepopulationforwhichstudyinvestigatorsaretargetingtheirstudyquestionofinterest.Examplesofconsiderationsforthisriskofbiasdomaininclude:1)levelofdetailreportedforparticipantinclusion/exclusion(includingdetailsfrompreviouslypublishedpapersreferencedinthearticleforanexistingcohort);2)participationratesandwhetherthisdifferedbyexposureoroutcomegroup;3)attritionratesandreasons;and4)comparisonsofstudycharacteristicsbetweenthestudypopulationandfullcohort.CriteriaforajudgmentofLOWriskofbias(i.e.,answer:“No”):
EITHER:a)Thedescriptionsofthesourcepopulation,inclusion/exclusioncriteria,recruitmentandenrollmentprocedures,participationandfollow-upratesweresufficientlydetailed,andadequatedataweresuppliedonthedistributionofrelevantstudysampleandpopulationcharacteristicstosupporttheassertionthatriskofselectioneffectswasminimal.ORb)Althoughthedescriptionsand/ordataasindicatedin“a”abovesuggestedthepotentialforselectioneffects,adequatesupportwasgivenindicatingthatpotentialselectioneffectswerenotdifferentialacrossbothexposureandoutcome.
83
ORc)Althoughthedescriptionsand/ordataasindicatedin“a”abovesuggestedthepotentialforselectioneffectsandtherewasnosupportindicatingthatpotentialselectioneffectswerenotdifferentialacrossbothexposureandoutcome,selectionfactorsappearedtobewell-understood,weremeasuredinthedataset,andappropriateadjustmentposthoctechniqueswereusedtocontrolforselectionbias.
CriteriaforthejudgmentofPROBABLYLOWriskofbias(i.e.,answer:“ProbablyNo”):
Thereisinsufficientinformationaboutparticipantselectiontopermitajudgmentoflowriskofbias,butthereisindirectevidencewhichsuggeststhatinclusion/exclusioncriteria,recruitmentandenrollmentprocedures,andparticipationandfollow-uprateswereconsistentacrossgroupsasdescribedbythecriteriaforajudgmentoflowriskofbias.
CriteriaforthejudgmentofPROBABLYHIGHriskofbias(i.e.,answer:“ProbablyYes”):
Thereisinsufficientinformationaboutparticipantselectiontopermitajudgmentofhighriskofbias,butthereisindirectevidencewhichsuggeststhatinclusion/exclusioncriteria,recruitmentandenrollmentprocedures,andparticipationandfollow-uprateswereinconsistentacrossgroups,asdescribedbythecriteriaforajudgmentofhighriskofbias.
CriteriaforthejudgmentofHIGHriskofbias(i.e.,answer:“Yes”):
a) Therewereindicationsfromdescriptionsofthesourcepopulation,
inclusion/exclusioncriteria,recruitmentandenrollmentprocedures,participationandfollow-uprates,ordataonthedistributionofrelevantstudysampleandpopulationcharacteristicsthatriskofselectioneffectsweresubstantial;and
b) Therewasnosupporttoindicatethatpotentialselectioneffectswerenotdifferentialacrossbothexposureandoutcome;and
c) Adjustmentposthoctechniqueswerenotusedtocontrolforselectionbias.CriteriaforthejudgmentofNOTAPPLICABLE(riskofbiasdomainisnotapplicabletostudy):
Thereisevidencethatparticipantselectionisnotanelementofstudydesigncapableofintroducingriskofbiasinthestudy.
2.Wasknowledgeofthegroupassignmentsinadequatelyprevented(i.e.,blindedormasked)duringthestudy,potentiallyleadingtosubjectivemeasurementofeitherexposureoroutcome?CriteriaforajudgmentofLOWriskofbias(i.e.,answer:“No”):
Anyofthefollowing:
84
• Noblinding,butthereviewauthorsjudgethattheoutcomemeasuresaswellastheexposuremeasuresarenotlikelytobeinfluencedbylackofblinding(suchasdifferentialoutcomeassessmentwheretheoutcomeisassessedusingdifferentmeasurementorestimationmetricsacrosstheexposuregroups,ordifferentialexposureassessmentwhereexposureisassessedusingdifferentmeasurementorestimationmetricsacrossthediagnosticoroutcomegroups);or
• Blindingofkeystudypersonnelwasensured,anditisunlikelythattheblindingcouldhavebeenbroken;or
• Somekeystudypersonnelwerenotblinded,butexposureandoutcomeassessmentwasblindedandthenon-blindingofothersisunlikelytointroducebias.
CriteriaforthejudgmentofPROBABLYLOWriskofbias(i.e.,answer:“ProbablyNo”):
Thereisinsufficientinformationaboutblindingtopermitajudgmentoflowriskofbias,butthereisindirectevidencewhichsuggeststhestudywasadequatelyblinded,asdescribedbythecriteriaforajudgmentoflowriskofbias.Forexample,investigatorswereeffectivelyblindedtotheexposureand/oroutcomegroupsiftheexposurewasmeasuredbyaseparateentityandtheoutcomewasobtainedfromahospitalrecord.
CriteriaforthejudgmentofPROBABLYHIGHriskofbias(i.e.,answer:“ProbablyYes”):
Thereisinsufficientinformationaboutblindingtopermitajudgmentofhighriskofbias,butthereisindirectevidencewhichsuggeststhestudywasnotadequatelyblinded,asdescribedbythecriteriaforajudgmentofhighriskofbias.
CriteriaforthejudgmentofHIGHriskofbias(i.e.,answer:“Yes”):
Anyofthefollowing:• Noblindingorincompleteblinding,andtheoutcomemeasuresorexposure
measuresislikelytobeinfluencedbylackofblinding(i.e.,differentialoutcomeorexposureassessment);or
• Blindingofkeystudypersonnelattempted,butlikelythattheblindingcouldhavebeenbrokensoastointroducebias;or
• Somekeystudypersonnelwerenotblinded,andthenon-blindingofotherswaslikelytointroducebias.
CriteriaforthejudgmentofNOTAPPLICABLE(riskofbiasdomainisnotapplicabletostudy):
Thereisevidencethatblindingisnotanelementofstudydesigncapableofintroducingriskofbiasinthestudy.
3.Wereexposureassessmentmethodslackingaccuracy?
Thefollowinglistofconsiderationsrepresentsacollectionoffactorsproposedbyexpertsinvariousfieldsthatmaypotentiallyinfluencetheinternalvalidityoftheexposureassessmentinasystematicmanner(notthose
85
thatmayrandomlyaffectoverallstudyresults).Theseshouldbeinterpretedonlyassuggestedconsiderations,andshouldnotbeviewedasscoringorachecklist.ListofConsiderations:Possibleexposureassessmentmetrics:
1) Modeling2) Monitoring3) Biomarkers4) Surrogatemeasure(i.e.,parentalsurveyofhouseholdexposures)
Foreach,overallconsiderationsinclude:
1) Whatisthequalityofthemetricbeingused?2) Hasthemetricbeenvalidatedforthescenarioforwhichitisbeingused?3) Istheexposuremeasuredinthestudyasurrogateforformaldehyde(i.e.,liveinanewhome)?4) Whatwasthetemporalcoverage(i.e.,shortorlong-termexposure,childhood,occupational)?5) Didtheanalysisaccountforpredictionuncertainty?6) Howwasmissingdataaccountedfor,andanydataimputationsincorporated?7) Weresensitivityanalysesperformed?
Inparticular,forexposureassessmentmodels:
1) Weretheinputdatainthestudysuspectedtosystematicallyunder-orover-estimateexposure?2) Whattypeofmodelwasused(geostatisticalinterpolation,land-useregression,dispersionmodels,
personalairsamplingmodels,hybridmodels,etc.)?3) Weremeteorologicalvariablesincorporatedinthemodelandjustifiedbyauthorsintheirselection?4) Weredataonlanduse,topography,traffic,monitoringdata,emissionrates,etc.incorporatedand
justifiedbyauthorsintheirselection?5) Whatwasthespatialvariation(e.g.,distancefromsource)andgeographic/spatialaccuracy(county,
censustract,individualresidence)?6) Whatwasthetemporalspecificityandvariation(accuracytoleveloftheday,pregnancytrimester,
year,etc.?)7) Whatwastheaddresscompleteness(e.g.,onlyhomeaddressatonepointintime,ormorecomplete
addresshistorythroughoutpregnancy/postnatallifeandotherlocationssuchaswork)?8) Whatwasthespace-timecoverageofthemodel?9) Weretime-activitypatternsaccountedfor?10) Wereconcentrationsofformaldehyderesultingfromaproductofsecondaryformationaccountedfor
inthemodel?CriteriaforajudgmentofLOWriskofbias(i.e.,answer:“No”):
Thereviewersjudgethatthereislowriskofexposuremisclassification,i.e.:• Thereishighconfidenceintheaccuracyoftheexposureassessmentmethods,
suchasmethodsthathavebeentestedforvalidityandreliabilityinmeasuringthetargetedexposure;or
• Less-establishedorlessdirectexposuremeasurementsarevalidatedagainstwell-establishedordirectmethods;or:
A) Biomarkers:adirectmeasureofformaldehydeexposureduringthetimeperiodthatexposureisconsideredrelevant(i.e.,asdefinedinthePECOstatement)wasused,andthereissufficientevidencethatrelevantfactorsfromtheListofConsiderationsabovewouldimplyminimalriskofbiasintheexposureassessment;or
86
B) Monitoring:directandpersonalmonitoringdevicesorotherindoorairmonitoringdevicewereusedthathavebeenvalidatedfortheformaldehydeexposureandscenarioforwhichitwasusedandthereissufficientevidencethatrelevantfactorsfromtheListofConsiderationsabovewouldimplyminimalriskofbiasintheexposureassessment;or
C) Modeling:themodelaccountedforthetime-activitypatternspecifictoeachresearchparticipant,(e.g.includesmorethanexposureattheresidentialaddress)andincludedmodelingmethodsthathavebeenvalidatedorshowntohaveahighdegreeofspatialaccuracy(e.g.pointlocation),and/ormethodsthatarethemselvesvalidatedwithgoodagreementcomparedtoperson-basedairdatacollection;andthereissufficientevidencethatrelevantfactorsfromtheListofConsiderationsabovewouldimplyminimalriskofbiasintheexposureassessment.ANDifapplicable(e.g.forlaboratorymeasurements),appropriateQA/QCformethodsaredescribedandaresatisfactory,withatleastthreeofthefollowingitemsreported,oratleasttwoofthefollowingitemsreportedplusevidenceofsatisfactoryperformanceinahighqualityinter-laboratorycomparison:
• Limitofdetectionorquantification;• standardsrecovery;• measureofrepeatability;• investigationandpreventionofblankscontamination.
CriteriaforthejudgmentofPROBABLYLOWriskofbias(i.e.,answer:“ProbablyNo”):
Thereisinsufficientinformationabouttheexposureassessmentmethodstopermitajudgmentoflowriskofbias,butthereisindirectevidencethatsuggeststhatmethodswererobust,asdescribedbythecriteriaforajudgmentoflowriskofbias.StudiesonlyreportingthattheQA/QCitemsaboveweresatisfactorybutnotreportingalloftheactualnumbersmayreceiveajudgmentof“probablylowriskofbias.”Additionally:
A) Biomarkers:ameasurethatincludedformaldehydeexposureduringthetimeperiodthatexposureisconsideredrelevantandhasbeenvalidatedasadirectmeasureofexposure(i.e.,asdefinedinthePECOstatement)wasused,orthereissomeevidencethatrelevantfactorsfromtheListofConsiderationsabovewouldimplyminimalriskofbiasintheexposureassessment.
B) Monitoring:methodologieswhichdirectlyassessexposurewereused,suchaspersonalexposureinstrumentsorindoorairmonitors,buthadnotbeenvalidatedforthatpurpose,orifsuchinstrumentswerewornforlessthan4hoursperday,orthereissomeevidencethatrelevantfactorsfromtheListofConsiderationsabovewouldimplyminimalriskofbiasintheexposureassessment.
C) Modeling:themodelusedmethodsthatdonotmeetthecriteriaofincludingtime-activitypatternsANDspatialaccuracy,andsomaynothavethelevelofvalidationcomparedtoperson-basedairmeasurement,butincludemeasurementsthathaveevidenceofquality,suchasgood-qualitydatainputs,validationagainstarea-basedairmeasurement,orotherestablishmentsoftheaccuracyofthedatainputsandmodels,orthereissomeevidencethatrelevantfactorsfromtheListofConsiderationsabovewouldimplyminimalriskofbiasintheexposureassessment.
87
CriteriaforthejudgmentofPROBABLYHIGHriskofbias(i.e.,answer:“ProbablyYes”):
Thereisinsufficientinformationabouttheexposureassessmentmethodstopermitajudgmentofhighriskofbias,butthereisindirectevidencethatsuggeststhatmethodswerenotrobust,asdescribedbythecriteriaforajudgmentofhighriskofbias.Additionally:
A) Biomarkers:thisincludesindirectmeasuresofformaldehydeexposurebutnotspecific
tothisexposure,suchasDNAadducts,inflammationoroxidativestress,duringthetimeperiodthatexposureisconsideredrelevant(i.e.,developmentalperiodasdefinedinthePECOstatement),orthereissomeevidencethatrelevantfactorsfromtheListofConsiderationsabovewouldimplyriskofbiasintheexposureassessment.
B) Monitoring:measurementofexposuresthatmaynothavebeenvalidatedforusetostudyformaldehydeexposurewereused,orthereissomeevidencethatrelevantfactorsfromtheListofConsiderationsabovewouldimplyriskofbiasintheexposureassessment.
C) Modeling:modelswereusedthathavenotbeencomparedtoperson-basedorarea-basedairmeasurementsandhavesuspicionofproblemsestimatingtrueexposurebecause,forexample,theydonothavespatialaccuracy(e.g.county-levelmeasures),donotpertaintothecorrecttimeframe,arebasedonlimiteddata,ordifferinmethodologybetweencasesandcontrolsinastudy,orthereissomeevidencethatrelevantfactorsfromtheListofConsiderationsabovewouldimplyriskofbiasintheexposureassessment.
CriteriaforthejudgmentofHIGHriskofbias(i.e.,answer:“Yes”):
Thereviewersjudgethatthereishighriskofexposuremisclassificationandanyoneofthefollowing:
• Thereislowconfidenceintheaccuracyoftheexposureassessmentmethods;or• Less-establishedorlessdirectexposuremeasurementsarenotvalidatedandare
suspectedtointroducebiasthatimpactstheoutcomeassessment(example:participantsareaskedtoreportexposurestatusretrospectively,subjecttorecallbias);or
• Uncertainhowexposureinformationwasobtained;or:
A) Biomarkers:ThereissufficientevidencethatrelevantfactorsfromtheListofConsiderationsabovewouldimplyriskofbiasintheexposureassessment.
B) Monitoring:InformationfromdatabasesorotherwisewasgatheredthatindirectlyassessedexposurewithoutconsideringvariablesnotedintheListofConsiderationsabove,suchasspatialvariability,landuseregression,etc.,orthereissufficientevidencethatrelevantfactorsfromtheListofConsiderationsabovewouldimplyriskofbiasintheexposureassessment.
C) Modeling:themodelusedhasbeendemonstratednottopertaintoarea-basedorperson-basedmeasuresorhasotherwisebeenpreviouslydemonstratedtobeunabletodescribeairlevelsofexposureforassigningexposureinaresearchsituation,orthereissufficient
88
evidencethatrelevantfactorsfromtheListofConsiderationsabovewouldimplyriskofbiasintheexposureassessment.
CriteriaforthejudgmentofNOTAPPLICABLE(riskofbiasdomainisnotapplicabletostudy):
Thereisevidencethatexposureassessmentmethodsarenotcapableofintroducingriskofbiasinthestudy.
4.Wereoutcomeassessmentmethodslackingaccuracy?CriteriaforajudgmentofLOWriskofbias(i.e.,answer:“No”):
Thereviewersjudgethatthereislowriskofoutcomemisclassification,i.e.:
• Outcomeswereassessedanddefinedconsistentlyacrossallstudyparticipants,usingvalidandreliablemeasures(notethatalloutcomeassessmentmeasurescapturedinthePECOstatement(clinicalasthmadiagnosisbyhealthpractitioner,asthma-relatedhospitaladmissionsoremergencydepartmentvisitsoroutpatientvisitsorICUadmissions/intubations,asthma-relatedmedicationuse,ASUIretrospectivequestionnaire,useofsystemiccorticosteroidsforasthma,wheezing,methacholinechallengetest,and/orspirometrymeasure)areconsideredbeforehandtobevalidandreliable,unlessotherinformationprovidedwithinthestudywarrantsaconsiderationotherwise);or
• Less-establishedorlessdirectoutcomemeasurementsarevalidatedagainstwell-establishedordirectmethods;or
• Appropriatesensitivityanalyseswereconductedthatsuggesttheinfluenceofoutcomemisclassificationwouldbeminimal
• AND,ifapplicable,appropriateQA/QCformethodsisdescribedandissatisfactory.
CriteriaforthejudgmentofPROBABLYLOWriskofbias(i.e.,answer:“ProbablyNo”):
Thereisinsufficientinformationabouttheoutcomeassessmentmethodstopermitajudgmentoflowriskofbias,butthereisindirectevidencewhichsuggeststhatmethodswererobust,asdescribedbythecriteriaforajudgmentoflowriskofbias.AppropriateQA/QCformethodsarenotdescribedbutthereviewauthorsjudgethattheoutcomeandtheoutcomeassessmentareobjectiveanduniformacrossstudygroups.
CriteriaforthejudgmentofPROBABLYHIGHriskofbias(i.e.,answer:“ProbablyYes”):
Thereisinsufficientinformationabouttheoutcomeassessmentmethodstopermitajudgmentofhighriskofbias,butthereisindirectevidencewhichsuggeststhatmethodswerenotrobust,asdescribedbythecriteriaforajudgmentofhighriskofbias.
89
CriteriaforthejudgmentofHIGHriskofbias(i.e.,answer:“Yes”):
Thereviewersjudgethatthereishighriskofoutcomemisclassificationandanyoneofthefollowing:
• Thereislowconfidenceintheaccuracyoftheoutcomeassessmentmethods;or• Less-establishedorlessdirectoutcomemeasurementsarenotvalidatedandare
suspectedtointroducebiasthatimpactstheoutcomeassessment• Uncertainhowoutcomeinformationwasobtained
CriteriaforthejudgmentofNOTAPPLICABLE(riskofbiasdomainisnotapplicabletostudy):
Thereisevidencethatoutcomeassessmentmethodsarenotcapableofintroducingriskofbiasinthestudy.
5.Waspotentialconfoundinginadequatelyincorporated?Listofimportantpotentialconfounders,collectivelygeneratedbyreviewauthorspriortotheinitiationofscreeningforstudiesbasedonexpertopinionandknowledgegatheredfromtheliterature:TierI:Importantconfounders
Occupationalstudies–(1)smokingstatusorexposuretosecondhandorenvironmentaltobaccosmoke(unlessconductedatatimeorplaceknowntohavebannedsmokingintheworkplaceinwhichcasewewouldonlyconsideractivesmoking);and(2)socioeconomicstatus.
Non-occupationalstudies–(1)age,instudieswhereparticipantsare6yearsoldorless;(2)smokingstatusorexposuretosecondhandorenvironmentaltobaccosmoke;and(3)socioeconomicstatus/parentaleducationinstudiesofchildren.
Age,smokingstatusincludingexposuretosecondhandorenvironmentaltobaccosmoke,andsocioeconomicstatus/parentaleducationareconsideredimportantconfoundersforasthereisevidencetheyarerelatedtoboththeexposure(formaldehyde)andhealthoutcome(asthma).
Age(instudiesofchildren6yearsoldorless)isconsideredanimportantconfounderbecauseitcanimpactthediagnosisofasthma,particularlyatyoungerageranges(<6years)(Morganetal.2005).Agemayalsobeassociatedwiththelevelsofformaldehydeexposures,inhalationrates,timespentindoorsathomeversusatschool,mayvarybyage(Moyaetal.2004).
Smokingstatusandexposuretosecondhand/environmentaltobaccosmokeareconsideredimportantconfoundersbecausecigarettesmokecontainsformaldehyde(WordHealthOrganization2004,Bolteetal.2008)andexposuretocigarettesmokeisalsogenerallyarecognizedriskfactorforasthmainchildhood(Ferranteetal.2014).
90
Socioeconomicstatus(andparentaleducationasasurrogatemeasure)isconsideredanimportantconfounderbecauseSESisrelatedtoindoorexposurestoformaldehyde(Hunetal.2009)andalsorelatedtotheoutcomeofasthma(Bhanetal.2015).
TierII:Otherpotentiallyimportantconfounders:
Race/ethnicity,sex,height,weight,BMI,obesitystatus,parentalorfamilyhistoryofasthma,allergy,additionalenvironmental(non-formaldehyde)exposures.
Theseconfounderswereselectedfromevidenceintheliterature(BjornsonandMitchell1999,Langleyetal.2003,Brüskeetal.2014)thatsuggestedthesemightbepotentialconfounders,stronglyassociatedwithoneofeitherexposureoroutcomebutnotasstronglyassociatedwiththeother,orthatmaybepotentialsurrogatesforoneoftheimportantconfounderslistedabove.Toavoidover-adjustmentforconfounders,theseTierIIconfoundersarecharacteristicsthatcouldpotentiallybeadjustedfor,butwouldnotbeaprimaryconsiderationinevaluatingastudy’sriskofbias.
Seasonisconsideredanpotentialconfounderbecauseitcanimpactasthmasymptomsinchildren,whicharemorelikelytooccurinthefall(SearsandJohnston2007).Likewise,formaldehydeexposuresbothindoorandoutdoormayalsovarybyseason(CaliforniaEnvironmentalProtectionAgency1992,U.S.ConsumerProductSafetyCommission2015).Ageinstudies(involvingindividuals>6yearsold)wasconsideredapotentialconfounderitcanbeassociatedwithdifferenttypesofoutdoorexposuresandthelevelofexposure(sincerespiratoryratesandlevelsofcontactwithfloorandwoodproductsdifferbyagegroup)(Beameretal.2012,Hospitaletal.2015);however,age>6yearsisnotknowntobeassociatedwithasthmadiagnosis,signsorsymptomsandsoaslongasageisaconsiderationinassessingpulmonaryfunctiontests,itwouldnotbeconsideredapotentiallyimportantconfounder.
CriteriaforajudgmentofLOWriskofbias(i.e.,answer:“No”):
Thestudyappropriatelyassessedandaccountedfor(i.e.,matched,stratified,excludedcertainpopulations(i.e.,smokers)orstatisticallycontrolledfor)allimportantconfounders(TierI)usingappropriatestatisticaltechniques,orreportedthatimportantconfounderswereevaluatedandomittedbecauseinclusiondidnotsubstantiallyaffecttheresults.Thedeterminationofspecificconfoundersmayalsobeinformedby,butnotlimitedto,thestudiesincludedintheoverallreview,ANDthestudyappropriatelyassessedandaccountedfor(i.e.,matched,stratified,orstatisticallycontrolledfor)otherpotentiallyimportantconfoundersrelevant(TierII)usingappropriatestatisticaltechniques,orreportedthattheseconfounderswereevaluatedandomittedbecauseinclusiondidnotsubstantiallyaffecttheresults,
91
ANDtheimportantpotentialconfoundersweremeasuredconsistentlyacrossstudygroupsusingvalidandreliablemethods,ortheinfluenceofcovariatemeasurementerrorwasdetermined,throughsensitivityanalysis,tobeminimal.
CriteriaforthejudgmentofPROBABLYLOWriskofbias(i.e.,answer:“ProbablyNo”):
Thestudyappropriatelyaccountedformostbutnotalloftheimportantconfounders(TierI)orusedappropriatestatisticaltechniques;ANDsomeoftheotherpotentiallyimportantconfoundersrelevant(TierII)usingappropriatestatisticaltechniques,ORreportedthattheseconfounderswereevaluatedandomittedbecauseinclusiondidnotsubstantiallyaffecttheresults;ANDthisisnotexpectedtointroducesubstantialbias.
CriteriaforthejudgmentofPROBABLYHIGHriskofbias(i.e.,answer:“ProbablyYes”):
Thestudyevaluatedsomebutnotalloftheimportantconfounders(TierI),ANDsomebutnotalloftheotherpotentiallyimportantconfoundersrelevant(TierII),ORusedquestionablestatisticaltechniquesforconfounderadjustment;ANDthisisexpectedtointroducesubstantialbias.
CriteriaforthejudgmentofHIGHriskofbias(i.e.,answer:“Yes”):
Thestudydidnotaccountfororevaluatemultipleimportantconfounders(TierI),ANDdidnotaccountfororevaluatemultipleotherpotentiallyimportantconfoundersrelevant(TierII),ORtheimportantpotentialconfounderswereinappropriatelymeasuredand/orinappropriatelyanalyzedacrossstudygroups.
CriteriaforthejudgmentofNOTAPPLICABLE(riskofbiasdomainisnotapplicabletostudy):
Thereisevidencethatoutcomeassessmentmethodsarenotcapableofintroducingriskofbiasinthestudy.
6.Wereincompleteoutcomedatainadequatelyaddressed?CriteriaforajudgmentofLOWriskofbias(i.e.,answer:“No”):
Participantswerefollowedlongenoughtoobtainoutcomemeasurements
92
ORanyoneofthefollowing:
• Nomissingoutcomedata;or• Reasonsformissingoutcomedataunlikelytoberelatedtotrueoutcome(for
survivaldata,censoringunlikelytointroducebias);or• Attritionormissingoutcomedatabalancedinnumbersacrossexposuregroups,
withsimilarreasonsformissingdataacrossgroups;or• Fordichotomousoutcomedata,theproportionofmissingoutcomescompared
withobservedeventrisknotenoughtohavearelevantimpactontheexposureeffectestimate;or
• Forcontinuousoutcomedata,plausibleeffectsize(differenceinmeansorstandardizeddifferenceinmeans)amongmissingoutcomesnotenoughtohavearelevantimpactontheobservedeffectsize;or
• MissingdatahavebeenimputedusingappropriatemethodsCriteriaforthejudgmentofPROBABLYLOWriskofbias(i.e.,answer:“ProbablyNo”):
Thereisinsufficientinformationaboutincompleteoutcomedatatopermitajudgmentoflowriskofbias,butthereisindirectevidencewhichsuggestsincompleteoutcomedatawasadequatelyaddressed,asdescribedbythecriteriaforajudgmentoflowriskofbias.
CriteriaforthejudgmentofPROBABLYHIGHriskofbias(i.e.,answer:“ProbablyYes”):
Thereisinsufficientinformationaboutincompleteoutcomedatatopermitajudgmentofhighriskofbias,butthereisindirectevidencewhichsuggestsincompleteoutcomedatawasnotadequatelyaddressed,asdescribedbythecriteriaforajudgmentofhighriskofbias.
CriteriaforthejudgmentofHIGHriskofbias(i.e.,answer:“Yes”):
ParticipantswerenotfollowedlongenoughtoobtainoutcomemeasurementsORanyoneofthefollowing:
• Reasonformissingoutcomedatalikelytoberelatedtotrueoutcome,witheitherimbalanceinnumbersorreasonsformissingdataacrossexposuregroups;or
• Fordichotomousoutcomedata,theproportionofmissingoutcomescomparedwithobservedeventriskenoughtoinducebiologicallyrelevantbiasininterventioneffectestimate;or
• Forcontinuousoutcomedata,plausibleeffectsize(differenceinmeansorstandardizeddifferenceinmeans)amongmissingoutcomesenoughtoinducebiologicallyrelevantbiasinobservedeffectsize;or
• Potentiallyinappropriateapplicationofimputation.CriteriaforthejudgmentofNOTAPPLICABLE(riskofbiasdomainisnotapplicabletostudy):
Thereisevidencethatincompleteoutcomedataisnotcapableofintroducingriskofbiasinthestudy.
93
7.Doesthestudyreportappeartohaveselectiveoutcomereporting?
CriteriaforajudgmentofLOWriskofbias(i.e.,answer:“No”):
Allofthestudy’spre-specified(primaryandsecondary)outcomesoutlinedinthepre-publishedprotocolorthepublishedmanuscript’smethods,abstract,and/orintroductionsectionthatareofinterestinthereviewhavebeenreportedinthepre-specifiedway.
CriteriaforthejudgmentofPROBABLYLOWriskofbias(i.e.,answer:“ProbablyNo”):
Thereisinsufficientinformationaboutselectiveoutcomereportingtopermitajudgmentoflowriskofbias,butthereisindirectevidencewhichsuggeststhestudywasfreeofselectivereporting,asdescribedbythecriteriaforajudgmentoflowriskofbias.Thisincludesifapre-publishedprotocolisnotavailablebutthestudy’spre-specified(primaryandsecondary)outcomesoutlinedinthepublishedmanuscript’smethods,abstract,and/orintroductionsectionthatareofinterestinthereviewhavebeenreportedinthepre-specifiedway.
CriteriaforthejudgmentofPROBABLYHIGHriskofbias(i.e.,answer:“ProbablyYes”):
Thereisinsufficientinformationaboutselectiveoutcomereportingtopermitajudgmentofhighriskofbias,butthereisindirectevidencewhichsuggeststhestudywasnotfreeofselectivereporting,asdescribedbythecriteriaforajudgmentofhighriskofbias.Thisincludesifapre-publishedprotocolisnotavailableandthestudy’spre-specified(primaryandsecondary)outcomesoutlinedinthepublishedmanuscript’smethods,abstract,and/orintroductionsectionthatareofinterestinthereviewhavenotbeenreportedinthepre-specifiedway.
CriteriaforthejudgmentofHIGHriskofbias(i.e.,answer:“Yes”):
Anyoneofthefollowing:• Notallofthestudy’spre-specifiedprimaryoutcomes(asoutlinedinthepre-
publishedprotocolorpublishedmanuscript’smethods,abstract,and/orintroduction)havebeenreported;or
• Oneormoreprimaryoutcomesisreportedusingmeasurements,analysismethodsorsubsetsofthedata(e.g.subscales)thatwerenotpre-specified;or
• Oneormorereportedprimaryoutcomeswerenotpre-specified(unlessclearjustificationfortheirreportingisprovided,suchasanunexpectedeffect);or
• OneormoreoutcomesofinterestarereportedincompletelyCriteriaforthejudgmentofNOTAPPLICABLE(riskofbiasdomainisnotapplicabletostudy):
Thereisevidencethatselectiveoutcomereportingisnotcapableofintroducingriskofbiasinthestudy.
94
8.Didthestudyreceiveanysupportfromacompany,studyauthor,orotherentityhavingafinancialinterestinanyoftheexposuresstudied?CriteriaforajudgmentofLOWriskofbias(i.e.,answer:“No”):
Thestudydidnotreceivesupportfromacompany,studyauthor,orotherentityhavingafinancialinterestintheoutcomeofthestudy.Examplesincludethefollowing:
• Fundingsourceislimitedtogovernment,non-profitorganizations,oracademicgrantsfundedbygovernment,foundationsand/ornon-profitorganizations;
• Chemicalsorothertreatmentusedinstudywerepurchasedfromasupplier;• Companyaffiliatedstaffarenotmentionedintheacknowledgementssection;• Authorswerenotemployeesofacompanywithafinancialinterestintheoutcome
ofthestudy;• Companywithafinancialinterestintheoutcomeofthestudywasnotinvolvedin
thedesign,conduct,analysis,orreportingofthestudyandauthorshadcompleteaccesstothedata;
• Studyauthorsmakeaclaimdenyingconflictsofinterest;• Studyauthorsareunaffiliatedwithcompanieswithfinancialinterest,andthereis
noreasontobelieveaconflictofinterestexists;• Allstudyauthorsareaffiliatedwithagovernmentagency(areprohibitedfrom
involvementinprojectsforwhichthereisaconflictofinterestoranappearanceofconflictofinterest).
CriteriaforthejudgmentofPROBABLYLOWriskofbias(i.e.,answer:“ProbablyNo”):
Thereisinsufficientinformationtopermitajudgmentoflowriskofbias,butthereisindirectevidencewhichsuggeststhestudywasfreeofsupportfromacompany,studyauthor,orotherentityhavingafinancialinterestintheoutcomeofthestudy,asdescribedbythecriteriaforajudgmentoflowriskofbias.
CriteriaforthejudgmentofPROBABLYHIGHriskofbias(i.e.,answer:“ProbablyYes”):
Thereisinsufficientinformationtopermitajudgmentofhighriskofbias,butthereisindirectevidencewhichsuggeststhestudywasnotfreeofsupportfromacompany,studyauthor,orotherentityhavingafinancialinterestintheoutcomeofthestudy,asdescribedbythecriteriaforajudgmentofhighriskofbias.
CriteriaforthejudgmentofHIGHriskofbias(i.e.,answer:“Yes”):
Thestudyreceivedsupportfromacompany,studyauthor,orotherentityhavingafinancialinterestintheoutcomeofthestudy.Examplesofsupportinclude:
• Researchfunds;• Chemicals,equipmentortestingprovidedatnocost;• Writingservices;
95
• Author/stafffromstudywasemployeeorotherwiseaffiliatedwithcompanywithfinancialinterest;
• Companylimitedauthoraccesstothedata;• Companywasinvolvedinthedesign,conduct,analysis,orreportingofthestudy;• Studyauthorsclaimaconflictofinterest
CriteriaforthejudgmentofNOTAPPLICABLE(riskofbiasdomainisnotapplicabletostudy):
Thereisevidencethatconflictsofinterestarenotcapableofintroducingriskofbiasinthestudy.
9.Didthestudyappeartohaveotherproblemsthatcouldputitatariskofbias?
CriteriaforajudgmentofLOWriskofbias(i.e.,answer:“No”):
Thestudyappearstobefreeofothersourcesofbias.
CriteriaforthejudgmentofPROBABLYLOWriskofbias(i.e.,answer:“ProbablyNo”):
Thereisinsufficientinformationtopermitajudgmentoflowriskofbias,butthereisindirectevidencewhichsuggeststhestudywasfreeofotherthreatstovalidity.
CriteriaforthejudgmentofPROBABLYHIGHriskofbias(i.e.,answer:“ProbablyYes”):
Thereisinsufficientinformationtopermitajudgmentofhighriskofbias,butthereisindirectevidencewhichsuggeststhestudywasnotfreeofotherthreatstovalidity,asdescribedbythecriteriaforajudgmentofhighriskofbias.
CriteriaforthejudgmentofHIGHriskofbias(i.e.,answer:“Yes”):Thereisatleastoneimportantriskofbias.Forexample,thestudy:
• Hadapotentialsourceofbiasrelatedtothespecificstudydesignused;or• Stoppedearlyduetosomedata-dependentprocess(includingaformal-stopping
rule);or• Theconductofthestudyisaffectedbyinterimresults(e.g.recruitingadditional
participantsfromasubgroupshowinggreaterorlessereffect);or• Hasbeenclaimedtohavebeenfraudulent;or• Hadsomeotherproblem
96
Appendix VII. Instructions for Grading the Quality and Strength of Evidence
A. GradingQuality
Eachofthecategoriestoconsiderindowngradingorupgradingtheevidenceisdescribedindetailbelow.Pleaserecordyourresultsonthechartattheendofeachcategory,includingabriefexplanationforyourratings.
DowngradeCategories
Category1.QualityofStudyLimitations(RiskofBias)(Guyattetal.2011)
Possibleratings:0=nochange;-1or-2downgrade1or2levels
Theevidencefromstudiescanberateddownifmostoftherelevantevidencecomesfromstudiesthatsufferfromahighriskofbias.Riskofbiasisratedbyoutcomeacrossstudies.Studylimitationsforeachoutcomeforindividualstudiesandacrossstudiesaresummarizedintheheatmaps.
GRADEoutlinesthefollowingprinciplesformovingfromriskofbiasinindividualstudiestoratingqualityofevidenceacrossstudies.
1.Indecidingontheoverallqualityofevidence,onedoesnotaverageacrossstudies(forinstanceifsomestudieshavenoseriouslimitations,someseriouslimitations,andsomeveryseriouslimitations,onedoesnotautomaticallyratequalitydownbyonelevelbecauseofanaverageratingofseriouslimitations).Rather,judiciousconsiderationofthecontributionofeachstudy,withageneralguidetofocusonthehigh-qualitystudiesiswarranted.a
2.Thisjudiciousconsiderationrequiresevaluatingtheextenttowhicheachstudycontributestowardtheestimateofmagnitudeofeffect.ThecontributionthateachstudymakeswillusuallyaNote: Limitations to GRADE’s risk of bias assessments as stated by GRADE: “First, empirical evidence supporting the criteria is limited. Attempts to show systematic difference between studies that meet and do not meet specific criteria have shown inconsistent results. Second, the relative weight one should put on the criteria remains uncertain. The GRADE approach is less comprehensive than many systems, emphasizing simplicity and parsimony over completeness. GRADE’s approach does not provide a quantitative rating of risk of bias. Although such a rating has advantages, we share with the Cochrane Collaboration methodologists a reluctance to provide a risk of bias score that, by its nature, must make questionable assumptions about the relative extent of bias associated with individual items and fails to consider the context of the individual items.”
97
reflectstudysamplesizeandnumberofoutcomeevents.Largerstudieswithmanyeventswillcontributemore,muchlargerstudieswithmanymoreeventswillcontributemuchmore.
3.Oneshouldbeconservativeinthejudgmentofratingdown.Thatis,oneshouldbeconfidentthatthereissubstantialriskofbiasacrossmostofthebodyofavailableevidencebeforeoneratesdownforriskofbias.
4.Theriskofbiasshouldbeconsideredinthecontextofotherlimitations.If,forinstance,reviewersfindthemselvesinaclose-callsituationwithrespecttotwoqualityissues(riskofbiasand,say,precision),GRADEsuggestsratingdownforatleastoneofthetwo.
5.Notwithstandingthefirstfourprinciples,reviewerswillfaceclose-callsituations.Youshouldacknowledgethatyouareinsuchasituation,makeitexplicitwhyyouthinkthisisthecase,andmakethereasonsforyourultimatejudgmentapparent.
RatingforRiskofBias(StudyLimitations)
0nochange
-1decreasequality1level
-2decreasequality2levels
Rationaleforyourjudgment
Human
Category2.IndirectnessofEvidence
Possibleratings:0=nochange;-1or-2downgrade1or2levels
Qualityofevidence(yourconfidenceinestimatesofeffect)maydecreasewhensubstantialdifferencesexistbetweenthepopulation,exposure,oroutcomesmeasuredintheresearchstudiesunderconsiderationinthereview.
Evidenceisdirectwhenitdirectlycomparestheexposuresinwhichweareinterestedinthepopulationsinwhichweareinterestedandmeasuresoutcomesimportanttothestudyquestion(inGRADEtheoutcomesmustbeimportanttopatients).
98
BasedonGRADE(Guyattetal.2011),evidencecanbeindirectinoneofthreeways.b
1. Thepopulationstudieddiffersfromthepopulationofinterest(thetermapplicabilityisoftenusedforthisformofindirectness).GRADEstatesthatingeneral,oneshouldnotratedownforpopulationdifferencesunlessonehascompellingreasontothinkthatthebiologyinthepopulationofinterestissodifferentthanthepopulationtestedthatthemagnitudeofeffectwilldiffersubstantially.AccordingtoGRADE,mostoften,thiswillnotbethecase.
2. Theintervention(exposure)testedmaydifferfromtheexposureofinterest,i.e.,adifferenceinthechemical,routeand/ordose.Decisionsregardingindirectnessofpopulationsandexposuredependonanunderstandingofwhetherbiologicalorsocialfactorsaresufficientlydifferentthatonemightexpectsubstantialdifferencesinthemagnitudeofeffect.GRADEalsostates,“Aswithallotheraspectsofratingqualityofevidence,thereisacontinuumofsimilarityoftheinterventionthatwillrequirejudgment.Itisrare,andusuallyunnecessary,fortheintendedpopulationsandinterventionstobeidenticaltothoseinthestudies,andweshouldonlyratedownifthedifferencesareconsideredsufficienttomakeadifferenceinoutcomelikely.”
3. Outcomesmaydifferfromthoseofprimaryinterest;forinstance,surrogateoutcomesthatarenotthemselvesimportant,butmeasuredinthepresumptionthatchangesinthesurrogatereflectchangesinanimportantoutcome.Thedifferencebetweendesiredandmeasuredoutcomesmayrelatetotimeframe.Whenthereisadiscrepancybetweenthetimeframeofmeasurementandthatofinterest,whethertoratedownbyoneortwolevelswilldependonthemagnitudeofthediscrepancy.Anothersourceofindirectnessrelatedtomeasurementofoutcomesistheuseofsubstituteorsurrogateendpointsinplaceoftheexposedpopulation’simportantoutcomeofinterest.Ingeneral,theuseofasurrogateoutcomerequiresratingdownthequalityofevidencebyone,oreventwo,levels.Considerationofthebiology,mechanism,andnaturalhistoryofthediseasecanbehelpfulinmakingadecisionaboutindirectness.Surrogatesthatarecloserintheputativecausalpathwaytotheadverseoutcomeswarrantratingdownbyonlyonelevelforindirectness.GRADEstatesthatrarely,surrogatesaresufficientlywellestablishedthatoneshouldchoosenottoratedownqualityofevidenceforindirectness.Ingeneral,evidencebasedonsurrogateoutcomesshouldusuallytriggerratingdown,whereastheothertypesofindirectnesswillrequireamoreconsideredjudgment.
RatingforIndirectness
0nochange
-1decreasequality1level
Rationaleforyourjudgment
b GRADE includes a fourth type of indirectness that occurs when there are no direct (i.e., head-to-head) comparisons between two or more interventions of interest. This criterion is not relevant to our study question; it could be relevant to future case studies.
99
-2decreasequality2levels
Human
Category3.InconsistencyofEvidence
Possibleratings:0=nochange;-1or-2downgrade1or2levels
AccordingtoCochrane,“whenstudiesyieldwidelydifferingestimatesofeffect(heterogeneityorvariabilityinresults)investigatorsshouldlookforrobustexplanationsforthatheterogeneity.…Whenheterogeneityexistsandaffectstheinterpretationofresults,butauthorsfailtoidentifyaplausibleexplanation,thequalityoftheevidencedecreases.”
BasedonGRADE(Guyattetal.2011),abodyofevidenceisnotratedupinqualityifstudiesyieldconsistentresults,butmayberateddowninqualityifinconsistent.Theirstatedreasonisthataconsistentbiaswillleadtoconsistent,spuriousfindings.
GRADEsuggestsratingdownthequalityofevidenceiflargeinconsistency(heterogeneity)instudyresultsremainsafterexplorationofapriorihypothesesthatmightexplainheterogeneity.Judgmentoftheextentofheterogeneityisbasedonsimilarityofpointestimates,extentofoverlapofconfidenceintervals,andstatisticalcriteria.GRADE’srecommendationsrefertoinconsistenciesineffectsize,specificallytorelativemeasures(riskratiosandhazardratiosoroddsratios),notabsolutemeasures.
BasedonGRADE,reviewersshouldconsiderratingdownforinconsistencywhen:
1.Pointestimatesvarywidelyacrossstudies;
2.Confidenceintervals(CIs)showminimalornooverlap;
3.Thestatisticaltestforheterogeneity-whichteststhenullhypothesisthatallstudiesinameta-analysishavethesameunderlyingmagnitudeofeffect-showsalowP-value;
4.TheI2-whichquantifiestheproportionofthevariationinpointestimatesduetoamong-studydifferences-islarge.(I.e.,theI2indexquantifiesthedegreeofheterogeneityinameta-analysis).
GRADEstatesthatinconsistencyisimportantonlywhenitreducesconfidenceinresultsin
100
relationtoaparticulardecision.Evenwheninconsistencyislarge,itmaynotreduceconfidenceinresultsregardingaparticulardecision.Forexample,studiesthatareinconsistentrelatedtothemagnitudeofabeneficialorharmfuleffect(butareinthesamedirection)wouldnotberateddown;ininstanceswhenresultsareinconsistentastowhetherthereisabenefitorharmoftreatment,GRADEwouldratedownthequalityofevidenceasaresultofvariabilityinresults,becausethemeaningoftheinconsistencyissorelevanttothedecisiontotreatornottotreat.
RatingforInconsistency
0nochange
-1decreasequality1level
-2decreasequality2levels
Rationaleforyourjudgment
Human
Category4.ImprecisionofEvidence
Possibleratings:0=nochange;-1or-2downgrade1or2levels
Cochranestatesthatwhenstudieshavefewparticipantsandfewevents,andthushavewideconfidenceintervals(CIs),authorscanlowertheirratingofthequalityofevidence.Theseratingsofprecisionaremadeasjudgmentsbyreviewauthors.Theratingsaremadebylookingacrossstudies,or,ifavailable,ontheresultsofameta-analysis.
GRADEdefinesevidencequalitydifferentlyforsystematicreviewsandguidelines.Forsystematicreviews,qualityreferstoconfidenceintheestimatesofeffect.Forguidelines,qualityreferstotheextenttowhichconfidenceintheeffectestimateisadequatetosupportaparticulardecision(Guyattetal.2011).Forthepurposeofstep3ofNavigationGuide,wewillusethesystematicreviewdefinition,becausethedecisionphasedoesnotoccuruntilstep4whenrecommendationsforpreventionaremade.Thus,whenreviewingthedataforimprecision,evaluateyourconfidenceintheestimateoftheeffect.
AccordingtoGRADE,toalargeextent,CIsinformtheimpactofrandomerroronevidencequality.Thus,whenconsideringimprecision,theissueiswhethertheCIaroundtheestimateof
101
exposureeffectissufficientlynarrow.Ifitisnot,GRADEratesdowntheevidencequalitybyonelevel(forinstance,fromhightomoderate).IftheCIisverywide,GRADEmightratedownbytwolevels.
RatingforImprecision
0nochange
-1decreasequality1level
-2decreasequality2levels
Rationaleforyourjudgment
Human
Category5.PublicationBias
Possibleratings:0=nochange;-1or-2downgrade1or2levelsGRADE(Guyattetal.2011)andCochrane(HigginsandGreen2011)assesspublicationbiasinasimilarmanner.Whereas“selectiveoutcomereporting”isassessedforeachstudyincludedinthereviewaspartoftheriskofbiasassessment,“publicationbias”isassessedonthebodyofevidence.GRADEstatesthat“whenanentirestudyremainsunreportedandtheresultsrelatetothesizeoftheeffect-publicationbias-onecanassessthelikelihoodofpublicationbiasonlybylookingatagroupofstudies.”
Cochrane’sdefinitionofpublicationbiasis“thepublicationornon-publicationofresearchfindingsdependingonthenatureanddirectionoftheresults.”CochraneandGRADEareprimarilyconcernedwithoverestimatesoftrueeffectsoftreatmentsorpharmaceuticals,especiallyrelatedto“smallstudieseffects”,i.e.,thetendencyforestimatesofaninterventiontobemorebeneficialinsmallerstudies.Thereisempiricalevidenceintheclinicalsciencesthatpublicationandotherreportingbiasesresultinoverestimatingtheeffectsofinterventions(HigginsandGreen2011).
102
Incontrast,inenvironmentalhealth,weareprimarilyconcernedwithunderestimatingthetrueeffectsofachemicalexposure,sinceinmanycasespopulationwideexposurehasalreadyoccurred.Wearealsoconcernedthatstudiesfindingnoassociationarelesslikelytobepublishedbecausejournalsarelesslikelytopublish“negative”findings.
ApplyingthisinvertedconcerntoGRADE’sassessmentforpublicationbias,leadstotheseconsiderationswhenratingpublicationbias:
• Earlynegativestudies,particularlyifsmallinsize,aresuspect.(GRADEisconcernedwithearlypositivestudies).
• Authorsofsystematicreviewsshouldsuspectpublicationbiaswhenstudiesareuniformlysmall,particularlywhensponsoredbytheindustry.(SameasGRADE)
• Empiricalexaminationofpatternsofresults(e.g.,funnelplots)maysuggestpublicationbiasbutshouldbeinterpretedwithcaution.(SameasGRADE)
• Morecompellingthananyofthesetheoreticalexercisesisauthors’successinobtainingtheresultsofsomeunpublishedstudiesanddemonstratingthatthepublishedandunpublisheddatashowdifferentresults.(SameasGRADE)
• Comprehensivesearchesoftheliteratureincludingunpublishedstudies,i.e.,thegreyliterature,andasearchforresearchinotherlanguagesareimportanttoaddressingpublicationbias.NotethatCochranealsostates“comprehensivesearchingisnotsufficienttopreventsomesubstantialpotentialbiases.”
RatingforPublicationBias
0nochange
-1decreasequality1level
-2decreasequality2levels
Rationaleforyourjudgment
Human
UpgradeCategories
GRADEstatesthatthecircumstancesforupgradinglikelyoccurinfrequentlyandareprimarilyrelevanttoobservationalandothernon-randomizedstudies.Althoughitispossibletorateupresultsfromrandomizedcontrolledtrials,GRADEhasyettofindacompellingcircumstancefordoingso(Guyattetal.2011).
103
GRADEspecifies3categoriesforincreasingthequalityofevidence(Guyattetal.2011)
Category6.LargeMagnitudeofEffect
Possibleratings:0=nochange;+1or+2upgrade1or2levels
Modelingstudiessuggeststhatconfounding(fromnon-randomallocation)aloneisunlikelytoexplainassociationswitharelativerisk(RR)greaterthan2(orlessthan0.5),andveryunlikelytoexplainassociationswithanRRgreaterthan5(orlessthan0.2).Thus,thesearethedefinitionsof“largemagnitudeofeffect”usedbyGRADEtoupgrade1or2levels,respectively.Also,GRADEismorelikelytorateupiftheeffectisrapidandoutofkeepingwithpriortrajectory;usuallysupportedbyindirectevidence.GRADEpresentsempiricalevidencetosupporttheseconclusions,andstatesthat“althoughfurtherresearchiswarranted,bothmodelingandempiricalworksuggestthesizeofbiasfromconfoundingisunpredictableindirectionbutboundedinsize.Hence,theGRADEgrouphaspreviouslysuggestedguidelinesforratingqualityofevidenceupbyonecategory(typicallyfromlowtomoderate)forassociationsgreaterthan2,andupbytwocategoriesforassociationsgreaterthan5.”ApplyingtheGRADEdefinitionsoflargemagnitudeofeffecti.e.,RRgreaterthan2or5isproblematicinenvironmentalhealthbecausefordichotomousoutcomesRRisafunctionoftheexposurecomparator;thesedefinitionsalsoarenotapplicabletoresultsfromcontinuousvariables.Atpresent,wedonothaveanempiricallydefined“largemagnitudeofeffect.”Therefore,forthepurposeofthiscasestudy,reviewauthorsshouldassesswhethertheresultsindicatealargemagnitudeofeffectusingtheirexpertjudgmentof“largeeffects”inenvironmentalhealthandstatetheirdefinitionfordiscussionbythegroup.RatingforLargeMagnitudeofEffect
0nochange
+1increasequality1level
+2increasequality2levels
Rationaleforyourjudgment
Human
104
Category7.Dose-response
Possibleratings:0=nochange;+1or+2upgrade1or2levels
Possibleconsiderationsincludeconsistentdoseresponsegradientsinoneormultiplestudies,and/ordoseresponseacrossstudies,dependingontheoverallrelevancetothebodyofevidence.RatingforDose-Response
0nochange
+1increasequality1level
+2increasequality2levels
Rationaleforyourjudgment
Human
Category8.ResidualConfoundingIncreasesConfidence(NationalToxicologyProgram2015)
Possibleratings:0=nochange;+1or+2upgrade1or2levels
Upgradeifconsiderationofallplausibleresidualconfounders,biases,oreffectmodificationwouldunderestimatetheeffectorsuggestaspuriouseffectwhenresultsshownoeffect.Ifastudyreportsanassociationdespitethepresenceofresidualconfounding,biasesoreffectmodificationthatwoulddiminishtheassociation,confidenceintheassociationisincreased.GRADEprovidesanillustrativeexamplerelatedtobias:ratingupobservationalevidencefindinglackofassociationbetweenvaccinationandautism,whichoccurreddespiteempiricallyconfirmedbiasthatparentsofautisticchildrenmaybemorelikelytoremembertheirvaccineexperience.Thenegativefindingsdespitethisformofrecallbiassuggestratingupthequalityofevidence(Guyattetal.2011).
105
RatingforResidualConfoundingIncreasesConfidence
0nochange
+1increasequality1level
+2increasequality2levels
Rationaleforyourjudgment
Human
Theresultsofthereviewers’ratingsbypopulationwillbecompiledanddiscussedleadingtoafinaldecisiononoverallqualityofhumanevidence.Therationaleforthedecisionwillbefullydocumented.
106
1. Finaldecisiononoverallqualityofhumanevidence:
(Example:Moderatequalityisupgraded1steptohighforXYZreason(s))
----High
----Moderate
----Low
B.RatetheStrengthofEvidence
Theevidencequalityratingswillbetranslatedintostrengthofevidenceforeachpopulationbasedonacombinationoffourcriteria:(1)Qualityofbodyofevidence;(2)Directionofeffect;(3)Confidenceineffect;and(4)Othercompellingattributesofthedatathatmayinfluencecertainty.ThestrengthofevidenceratingsaresummarizedinTable1below,wheretheirmeaningisfurtherdefined.
top related