11

Bandit Thinkhamrop, PhD.(Statistics)Bandit Thinkhamrop, PhD.(Statistics)Dept. of Biostatistics & DemographyDept. of Biostatistics & Demography

Faculty of Public HealthFaculty of Public HealthKhon Kaen University Khon Kaen University

BiostatisticsBiostatisticsforfor

Critical AppraisalCritical Appraisal

BiostatisticsBiostatisticsforfor

Critical AppraisalCritical Appraisal

22

Core QuestionsCore Questions

1.1. What is the research What is the research questionquestion??2.2. What is the What is the answeranswer or the conclusion? or the conclusion?3.3. What is the What is the magnitudemagnitude of effect being used as of effect being used as

the basis of the conclusion?the basis of the conclusion?4.4. How large is the effect – How large is the effect – conclusive or notconclusive or not??5.5. How likely is the How likely is the validityvalidity of the magnitude of of the magnitude of

effect - accurate, over or under estimate, effect - accurate, over or under estimate, questionable, unassessable?questionable, unassessable?

6.6. Can the conclusion be Can the conclusion be acceptableacceptable? Why?? Why?7.7. How this paper could beHow this paper could be improved improved??

33

1. What is the research 1. What is the research questionquestion??

State the research questionState the research question

Focus on the primary research question or Focus on the primary research question or the one that lead to the main conclusionthe one that lead to the main conclusion

Example:Example:– What are predictors of low birth weight?What are predictors of low birth weight?– What is the efficacy of the HIV vaccine?What is the efficacy of the HIV vaccine?

44

2. What is the 2. What is the answeranswer or the conclusion? or the conclusion?

Extract the answer to the primary research Extract the answer to the primary research question question

Usually be obtained from the main Usually be obtained from the main conclusion of the study conclusion of the study

Example:Example:– Low BMI of mother and not received ANC Low BMI of mother and not received ANC

increase risk of low birth weightincrease risk of low birth weight– HIV vaccine can prevent HIV infectionHIV vaccine can prevent HIV infection

55

3. What is the 3. What is the magnitudemagnitude of effect being of effect being used as the basis of the conclusion?used as the basis of the conclusion?

If reported, transcribe it. If not, calculate it based If reported, transcribe it. If not, calculate it based on the reported results.on the reported results.Its the 95%CI is essentialIts the 95%CI is essentialSummarize the magnitude of effect as a forest Summarize the magnitude of effect as a forest plot-like diagram against the meaningful level is plot-like diagram against the meaningful level is recommended recommended Example:Example:– OR (95%CI) of each factor on low birth weight : Low BMI of OR (95%CI) of each factor on low birth weight : Low BMI of

mother (OR: 3.2; 2.50 to 4.50), and not received ANC increase mother (OR: 3.2; 2.50 to 4.50), and not received ANC increase risk of low birth weight (OR: 1.6; 1.02 to 2.18)risk of low birth weight (OR: 1.6; 1.02 to 2.18)

– HIV vaccine efficacy is 30% (95%CI: 1.1% to 52.2%)HIV vaccine efficacy is 30% (95%CI: 1.1% to 52.2%)

66

Forest plot for the DifferenceForest plot for the Difference

0 1 2 3-1-2-3

Male (0.9; 0.1 to 1.7)

Female (1.4; -0.1 to 2.9)

Flavor Intervention B Flavor Intervention A

Overall (1.8; 0.6 to 3.0)

Minimum meaningful level

77

Forest plot for RR, IRR, OR, HRForest plot for RR, IRR, OR, HR

0 1 2 3 40.500.330.25

Minimum meaningful level

Low BMI of mother (3.20; 2.50 to 4.50)

Protective effect Risk effect

Received ANC (1.60; 1.02 to 2.18)

88

Notice the wrong scaleNotice the wrong scale

99

1010

4. How large is the effect – 4. How large is the effect – conclusive or notconclusive or not??

Compare the magnitude of both lower and Compare the magnitude of both lower and upper boundary of the 95%CI with the upper boundary of the 95%CI with the meaningful level of effect meaningful level of effect

To be conclusive, the CI needs NOT To be conclusive, the CI needs NOT include the meaningful cut pointinclude the meaningful cut point

Conclusive findings imply sufficient sample Conclusive findings imply sufficient sample size size

1111

5. How likely is the 5. How likely is the validityvalidity of the of the magnitude of effect-magnitude of effect- accurate, over or accurate, over or under estimate, questionable, unassessableunder estimate, questionable, unassessable??

Describe briefly how the magnitude of effect Describe briefly how the magnitude of effect was obtainedwas obtainedDescribe how it could be wrong by considering Describe how it could be wrong by considering these three issues:these three issues:1.1. Wrong data?Wrong data?

Selection biasSelection biasInformation biasInformation biasConfounding biasConfounding bias

2.2. Wrong analysis?Wrong analysis?Applied inappropriate statistical methodApplied inappropriate statistical method

3.3. Poor or unclear reportingPoor or unclear reportingImportant information was not providedImportant information was not provided

1212

6. Can the conclusion be 6. Can the conclusion be acceptableacceptable??

Answer : Yes/No and Why?Answer : Yes/No and Why?

Explain the mechanisms as to how the Explain the mechanisms as to how the magnitude of effect could be wrong, if not magnitude of effect could be wrong, if not acceptedaccepted

Elaborate more by describing what Elaborate more by describing what position of this evidence in your evidence-position of this evidence in your evidence-base practicebase practice

1313

7. How this paper could be7. How this paper could be improved improved??

No research is perfect – there are rooms No research is perfect – there are rooms to improve.to improve.

Go beyond what had been reported in the Go beyond what had been reported in the paper to what should have been done to paper to what should have been done to be better under unlimited resourcesbe better under unlimited resources