guidelines in evaluating the validity and applicability of a research study

7/27/2019 Guidelines in Evaluating the Validity and Applicability of a Research Study

1/3

Lecture: Guidelines in Evaluating the Validity and Applicability of a

Research study

1. What is the research design?2. What is the level of quality of the Research study according to the U.S. Preventive Services Task

Force?

3. Are the results of the study valid?a. Randomizationb. Follow-up (80% or better)c. Blinding of the clinicians, patients, research personnels (the more blinding the

better

d. Baseline similarities (established at the start of the trial)4. What are the results of the research study?5. Will the results help in caring for my patient?6. What is the Level of Recommendation according to the U.S. Preventive Services Task Force

DESIGN OF THE RESEARCH STUDY

Meta Analysis

Examines a number of valid studies on a topic and combine the results using accepted statistical

methodology as if they were from one large study.

Double blind study

A study which neither the patient nor the physician knows whether the patient is receiving the treatment

of interest or the control treatment.

Randomized controlled clinical trialsPlanned projects that study the effect of a therapy on real patients. They include methodologies that

reduce the potential for bias (randomization and blinding) and that allow for comparison between

intervention groups and control groups (no intervention).

Cohort StudiesThis study involves a large population and follow patients who have a specific condition or receive a

particular treatment over time and compare them with another group that has not been affected by the

condition or treatment being studied. Cohort studies are observational and not as reliable as randomized

controlled studies, since the two groups may differ in ways other than in the variable under study.

Case Control StudiesStudies in which patients who already have a specific condition are compared with people who do not.

They often rely on medical records and patient recall for data collection. These types of studies are often

less reliable than randomized controlled trials and cohort studies because showing a statistical

relationship does not mean than one factor necessarily caused the other.

Case series and Case reports

Collections of reports on the treatment of individual patients or a report on a single patient.


2/3

Ideas, Editorials, Opinions

Exchange of ideas-Subjective

Animal Research

Trial that involves animals as samples

Test Tube research

trial that tests a substance in a test tube

LEVEL of EVIDENCE

Level I

Evidence obtained from at least one properly designed randomized controlled trial

Level II-1

Evidence obtained from well-designed controlled trials without randomization

Level II-2

Evidence obtained from well-designed cohort or case-control analytic studies, preferably from more than

one center or research group.

Level II-3

Evidence obtained from multiple time series with or without the intervention. Dramatic results in

uncontrolled trials might also be regarded as this type of evidence.

Level III

Opinions of respected authorities, based on clinical experience, descriptive studies, or reports of expertcommittees.

LEVEL OF RECOMMENDATION

The U.S. Preventive Services Task Force (USPSTF) assigns one of five letter grades to each of its

recommendations (A, B, C, D, or I).

What the Grades Mean and Suggestions for Practice

A The USPSTF recommends the service. There is high certainty that the net

benefit is substantial. Offer or provide this service.

B The USPSTF recommends the service. There is high certainty that the net

benefit is moderate or there is moderate certainty that the net benefit is

moderate to substantial. Offer or provide this service.

C Clinicians may provide this service to selected patients depending on individual

circumstances. However, for most individuals without signs or symptoms there

is likely to be only a small benefit from this service. Offer or provide this


3/3

service only if other considerations support the offering or providing the service

in an individual patient.

D The USPSTF recommends against the service. There is moderate or high

certainty that the service has no net benefit or that the harms outweigh the

benefits. Discourage the use of this service.

I The USPSTF concludes that the current evidence is insufficient to assess the

balance of benefits and harms of the service. Evidence is lacking, of poor

quality, or conflicting, and the balance of benefits and harms cannot be

determined. Read the clinical considerations section of USPSTF

Recommendation Statement. If the service is offered, patients should

understand the uncertainty about the balance of benefits and harms.

LEVELS OF CERTAINTY REGARDING NET BENEFIT

DescriptionHIGH: The available evidence usually includes consistent results from well-designed, well-conducted

studies in representative primary care populations. These studies assess the effects of the preventive

service on health outcomes. This conclusion is therefore unlikely to be strongly affected by the results of

future studies.

MODERATE: The available evidence is sufficient to determine the effects of the preventive service on

health outcomes, but confidence in the estimate is constrained by such factors as:

The number, size, or quality of individual studies.

Inconsistency of findings across individual studies.

Limited generalizability of findings to routine primary care practice.

Lack of coherence in the chain of evidence.As more information becomes available, the magnitude or direction of the observed effect could change,

and this change may be large enough to alter the conclusion.

LOW: The available evidence is insufficient to assess effects on health outcomes. Evidence is insufficient

because of:

The limited number or size of studies.

Important flaws in study design or methods.

Inconsistency of findings across individual studies.

Gaps in the chain of evidence.

Findings not generalizable to routine primary care practice.

Lack of information on important health outcomes.More information may allow estimation of effects on health outcomes.

The USPSTF defines certainty as "likelihood that the USPSTF assessment of the net benefit of a

preventive service is correct." The net benefit is defined as benefit minus harm of the preventive service

as implemented in a general, primary care population. The USPSTF assigns a certainty level based on the

nature of the overall evidence available to assess the net benefit of a preventive service.

guidelines in evaluating the validity and applicability of a research study

Documents