the good, the bad & the ugly of ems research. definitions research: careful, systematic, patient...

The Good, The Bad & The Ugly of EMS Research

The Good, The Bad & The Ugly of EMS Research

DefinitionsDefinitions

Research: careful, systematic, patient study and investigation in some field of knowledge, undertaken to discover or establish facts or principles.

Research: careful, systematic, patient study and investigation in some field of knowledge, undertaken to discover or establish facts or principles.

DefinitionsDefinitions

Science:The state or fact of knowing; knowledge.

Systematized knowledge derived from observation, study, and experimentation carried on in order to determine the nature or principles of what is being studied.

Science:The state or fact of knowing; knowledge.

Systematized knowledge derived from observation, study, and experimentation carried on in order to determine the nature or principles of what is being studied.

Is EMS Art or Science?Is EMS Art or Science?

The knowledge of EMS is science.

The way it is applied is art.

Excellent EMTs know the science of EMS and use the art of EMS to apply the science.

The knowledge of EMS is science.

The way it is applied is art.

Excellent EMTs know the science of EMS and use the art of EMS to apply the science.

Art or Science?Art or Science?

Some health practices are more about art than science.

Chiropractic, for example, has little science and a lot of art.

Some health practices are more about art than science.

Chiropractic, for example, has little science and a lot of art.


Neurology today has a great deal of science and little art.

Neurology today has a great deal of science and little art.


When you have a life threatening illness or injury, would you seek out a health care provider whose practice was more art or more science?

When you have a life threatening illness or injury, would you seek out a health care provider whose practice was more art or more science?


EMS is the safety net of society.

We are often the first to provide care to the injured and the infirm.

Our practices must be based on science.

EMS is the safety net of society.

We are often the first to provide care to the injured and the infirm.

Our practices must be based on science.

But we must apply them with art!But we must apply them with art!

ResearchResearch

Most medical research is based on the application of the scientific method.

Most medical research is based on the application of the scientific method.

The Scientific MethodThe Scientific Method

The scientific method is the process by which scientists, collectively and over time, endeavor to construct an accurate (that is, reliable, consistent and non-arbitrary) representation of the world.

The scientific method is the process by which scientists, collectively and over time, endeavor to construct an accurate (that is, reliable, consistent and non-arbitrary) representation of the world.


Steps:1. Observe some aspect of the universe2. Invent a tentative description, called a hypothesis,

that is consistent with what you observed.3. Use the hypothesis to make predictions.4. Test those predictions by experiments or further

observation and modify the hypothesis in the light of your results.

5. Repeat steps 3 and 4 until there are no discrepancies between theory and experiment and/or observation.

Steps:1. Observe some aspect of the universe2. Invent a tentative description, called a hypothesis,

that is consistent with what you observed.3. Use the hypothesis to make predictions.4. Test those predictions by experiments or further

observation and modify the hypothesis in the light of your results.

5. Repeat steps 3 and 4 until there are no discrepancies between theory and experiment and/or observation.


Hypothesis: groundwork, foundation supposition; an unproven theory.

Hypothesis: groundwork, foundation supposition; an unproven theory.


The great advantage of the scientific method is that it is unprejudiced:

One does not have to believe a given researcher, one can redo the experiment and determine whether his or her results are true or false. The conclusions will hold irrespective of the state of mind, or the religious persuasion, or the state of consciousness of the investigator and/or the subject of the investigation. Faith, defined as a belief that is not based on logical proof or material evidence, does not determine whether a scientific theory is adopted or discarded.

The great advantage of the scientific method is that it is unprejudiced:

One does not have to believe a given researcher, one can redo the experiment and determine whether his or her results are true or false. The conclusions will hold irrespective of the state of mind, or the religious persuasion, or the state of consciousness of the investigator and/or the subject of the investigation. Faith, defined as a belief that is not based on logical proof or material evidence, does not determine whether a scientific theory is adopted or discarded.


A theory is accepted not based on the prestige or convincing powers of the proponent, but on the results obtained through observations and/or experiments which anyone can reproduce.

The results obtained using the scientific method are repeatable.

A theory is accepted not based on the prestige or convincing powers of the proponent, but on the results obtained through observations and/or experiments which anyone can reproduce.

The results obtained using the scientific method are repeatable.


There are many types of “pseudoscientific” theories which seem based on a mantle of apparent experimental evidence but that, when examined closely, are nothing but statements of faith.

There are many types of “pseudoscientific” theories which seem based on a mantle of apparent experimental evidence but that, when examined closely, are nothing but statements of faith.


“Faith is believing what you know ain’t so.”

Mark Twain, 1894

“Following the Equator”

“Faith is believing what you know ain’t so.”

Mark Twain, 1894

“Following the Equator”

Science versus PseudoscienceScience versus Pseudoscience

How to tell the difference:Anecdotes do not make a science.

Scientific language does not make a science.

Bold statements do not make claims true.

Heresy does not equal correctness.

Reversal of the burden of proof.

Rumors do not equal reality.

How to tell the difference:Anecdotes do not make a science.

Scientific language does not make a science.

Bold statements do not make claims true.

Heresy does not equal correctness.

Reversal of the burden of proof.

Rumors do not equal reality.

Science versus PseudoscienceScience versus Pseudoscience

How to tell the difference:Failures are rationalized.Use of emotive words and false analogies.Ad ignorantiam reasoning (if you can’t disprove a claim, it must be true).Ad hominem or tu quoque statements.Overreliance on authorities.Circular reasoning.Reductio ad absurdum reasoning.

How to tell the difference:Failures are rationalized.Use of emotive words and false analogies.Ad ignorantiam reasoning (if you can’t disprove a claim, it must be true).Ad hominem or tu quoque statements.Overreliance on authorities.Circular reasoning.Reductio ad absurdum reasoning.


Enzyte has to work….after all, Bob is smiling!

Enzyte has to work….after all, Bob is smiling!


Pseudoscience is still a major part of our culture. Many practitioners resort to pseudoscience and other anecdotal practices.

Pseudoscience is still a major part of our culture. Many practitioners resort to pseudoscience and other anecdotal practices.


Typically uses terms and images to appear “scientific”.

Typically uses terms and images to appear “scientific”.


Reliance on “names” and anecdotes.

Dr. Greg Cynaumon?Physician?

Psychologist?

Nutritionist?

Chiropractor?

Master’s and Doctorate in psychology from Sierra University (known diploma mill).

Reliance on “names” and anecdotes.

Dr. Greg Cynaumon?Physician?

Psychologist?

Nutritionist?

Chiropractor?

Master’s and Doctorate in psychology from Sierra University (known diploma mill).


No clinical evidence CortiSlim works.

No information about contents (natural or herbal).

Dr. Talcott fringe nutritionist with loose affiliations with University of Utah.

One of the largest scams in recent history.

FDA issued a cease and desist order and fined Talbot and Cynamoun.

No clinical evidence CortiSlim works.

No information about contents (natural or herbal).

Dr. Talcott fringe nutritionist with loose affiliations with University of Utah.

One of the largest scams in recent history.

FDA issued a cease and desist order and fined Talbot and Cynamoun.

Use of the Scientific Method in EMSUse of the Scientific Method in EMS

Observation: During a discussion you and your coworkers bring up the idea that there are more psychiatric emergencies when the moon is full.

Hypothesis: Psychiatric emergencies are more common when the lunar cycle is in the full moon phase.

Observation: During a discussion you and your coworkers bring up the idea that there are more psychiatric emergencies when the moon is full.

Hypothesis: Psychiatric emergencies are more common when the lunar cycle is in the full moon phase.


Prediction: You predict that there will be more EMS calls for psychiatric emergencies when the moon is full as compared to other times of the month.

Prediction: You predict that there will be more EMS calls for psychiatric emergencies when the moon is full as compared to other times of the month.


Testing: You set up an experimental protocol to test your hypothesis.

For this you:Determine you will conduct the study for a year.

Determine when a full moon is scheduled to appear.

Determine that you will declare two days before and two days after the full moon a five-day “full moon period.”

Testing: You set up an experimental protocol to test your hypothesis.

For this you:Determine you will conduct the study for a year.

Determine when a full moon is scheduled to appear.

Determine that you will declare two days before and two days after the full moon a five-day “full moon period.”


For this you:Define what a psychiatric emergency will be.

Gather information over the course of the year that includes the date of all psychiatric emergency patients.

When the data has been gathered for the period previously determined, you determine:

1. How many psychiatric patients were there?

2. How many were treated during the full moon periods?

For this you:Define what a psychiatric emergency will be.

Gather information over the course of the year that includes the date of all psychiatric emergency patients.

When the data has been gathered for the period previously determined, you determine:

1. How many psychiatric patients were there?

2. How many were treated during the full moon periods?


For this you:Analyze the data based on the number of emergencies that arose during full moon periods and compare to the number of emergencies during the other periods.

Perform any statistical tests necessary to understand the data.

For this you:Analyze the data based on the number of emergencies that arose during full moon periods and compare to the number of emergencies during the other periods.

Perform any statistical tests necessary to understand the data.


Results:Results:

Full Moon PeriodDays=65

Other PeriodDays = 300

Total Days = 365

Ψ patients = 10 Ψ patients = 74 Total Ψ patients = 84

Total Patients = 2,105




Results: Your study finds:During full moon period 0.5% of patients were Ψ patients.

During other period 0.8% of patients were Ψ patients.

During the entire study period 0.7% of patients were Ψ patients.

Results: Your study finds:During full moon period 0.5% of patients were Ψ patients.

During other period 0.8% of patients were Ψ patients.

During the entire study period 0.7% of patients were Ψ patients.


Results: Your study finds:The incidence of Ψ patients during the full moon period was 0.15 patients per day.

The incidence of Ψ patients during the other moon phases was 0.25 patients per day.

The incidence of Ψ patients overall was 0.23 patients per day.

Results: Your study finds:The incidence of Ψ patients during the full moon period was 0.15 patients per day.

The incidence of Ψ patients during the other moon phases was 0.25 patients per day.

The incidence of Ψ patients overall was 0.23 patients per day.


Psych patients are less common during the full moon.

HYPOTHESIS DISSPROVED!

Psych patients are less common during the full moon.

HYPOTHESIS DISSPROVED!


Revise hypothesis:New hypothesis: Psychiatric emergencies are no more frequent during full moon lunar phases than during other lunar phases.

HYPOTHESIS ACCURATE.Report findings. Paper should be detailed enough that anybody can follow it and repeat your experiment.

Repeat study to determine whether findings can be repeated.

Revise hypothesis:New hypothesis: Psychiatric emergencies are no more frequent during full moon lunar phases than during other lunar phases.

HYPOTHESIS ACCURATE.Report findings. Paper should be detailed enough that anybody can follow it and repeat your experiment.

Repeat study to determine whether findings can be repeated.

EMS ResearchEMS Research

The more valid a study, the closer it is to the truth!

The more valid a study, the closer it is to the truth!


Bias: A mental leaning or inclination; partiality; prejudice; bent.

The more a study design adheres to the scientific method, the less chance for bias to affect the outcomes.

Bias: A mental leaning or inclination; partiality; prejudice; bent.

The more a study design adheres to the scientific method, the less chance for bias to affect the outcomes.


Types of biases:Selection bias

Volunteer bias (tends to favor the treatment group)

Nonrespondent bias

Group not homogeneous

Measurement biasInstrument bias

Insensitive measure bias

Expectation bias

Recall bias (people remember positive events rather than negative events)

Attention bias (aware of participation)

Types of biases:Selection bias

Volunteer bias (tends to favor the treatment group)

Nonrespondent bias

Group not homogeneous

Measurement biasInstrument bias

Insensitive measure bias

Expectation bias

Recall bias (people remember positive events rather than negative events)

Attention bias (aware of participation)


Types of biases:Intervention bias

Contamination bias

Compliance bias

Withdrawal bias

Proficiency bias

Publication bias

Cherry picking (mainly retrospective studies)

Trashcan bias

Types of biases:Intervention bias

Contamination bias

Compliance bias

Withdrawal bias

Proficiency bias

Publication bias

Cherry picking (mainly retrospective studies)

Trashcan bias


Avoiding bias:Randomize subjects

Prospective design

Blind subjects and researchers

Contract to publish irregardless of findings

Admit conflicts of interest

Use null hypothesis

Avoiding bias:Randomize subjects

Prospective design

Blind subjects and researchers

Contract to publish irregardless of findings

Admit conflicts of interest

Use null hypothesis


Bias (sometimes flat deception) has been a common practice in medicine and EMS.

Quackery and bias still permeates many aspects of modern healthcare.

Bias (sometimes flat deception) has been a common practice in medicine and EMS.

Quackery and bias still permeates many aspects of modern healthcare.


Validity: Whether the study measures what it was supposed to measure.

Validity refers to the appropriateness of the interpretation of the results of a study.

External Validity: The extent and appropriateness of the generalizability of results.

Internal Validity: The basic minimum control, measure, analysis and procedures necessary to make results interpretable.

Validity: Whether the study measures what it was supposed to measure.

Validity refers to the appropriateness of the interpretation of the results of a study.

External Validity: The extent and appropriateness of the generalizability of results.

Internal Validity: The basic minimum control, measure, analysis and procedures necessary to make results interpretable.


Constant: A characteristic or condition that is the same for all individuals in a study.

Variable:A characteristic that takes on different values or conditions for different individuals.

Dependent Variable: the variable being affected or assumed to be affected my the independent variable.Independent Variables: the variables that affects (or is assumed to affect) the dependent variable under study.Experimental Variable: at least one independent variable being manipulated by the researcher.

Constant: A characteristic or condition that is the same for all individuals in a study.

Variable:A characteristic that takes on different values or conditions for different individuals.

Dependent Variable: the variable being affected or assumed to be affected my the independent variable.Independent Variables: the variables that affects (or is assumed to affect) the dependent variable under study.Experimental Variable: at least one independent variable being manipulated by the researcher.

Levels of EvidenceLevels of Evidence


Center for Evidence-Based Medicine (Oxford)

Ia. Meta-analysis of RCTsIb. One RCT.IIa. Controlled trial without randomisation.IIb. One other type of quasi-experimental study.III. Descriptive studies, such as comparative studies, correlation studies, and case-control studies.IV. Expert committee reports or opinions, or clinical experience of respected authorities or both.

Center for Evidence-Based Medicine (Oxford)

Ia. Meta-analysis of RCTsIb. One RCT.IIa. Controlled trial without randomisation.IIb. One other type of quasi-experimental study.III. Descriptive studies, such as comparative studies, correlation studies, and case-control studies.IV. Expert committee reports or opinions, or clinical experience of respected authorities or both.


American Heart Association1. Positive randomized controlled trials.

2. Neutral randomized controlled trials.

3. Prospective, non-randomized controlled trials.

4. Retrospective, non-randomized controlled trials

5. Case series (no control group)

6. Animal studies

7. Extrapolations

8. Rational conjecture (common sense)

American Heart Association1. Positive randomized controlled trials.

2. Neutral randomized controlled trials.

3. Prospective, non-randomized controlled trials.

4. Retrospective, non-randomized controlled trials

5. Case series (no control group)

6. Animal studies

7. Extrapolations

8. Rational conjecture (common sense)

Study DesignsStudy Designs

Randomized Controlled Trial (RCT):A group of patients are randomized into an experimental group and a control group. These groups are followed up for the variables/outcomes of interest.

RCTs most closely approximate the scientific method and are the most valid of studies.

Randomized Controlled Trial (RCT):A group of patients are randomized into an experimental group and a control group. These groups are followed up for the variables/outcomes of interest.

RCTs most closely approximate the scientific method and are the most valid of studies.

Randomized Controlled TrialRandomized Controlled Trial


Houston MAST Study:Constant: All patients with abdominal trauma with hypotension transported by the HFD to Ben Taub Hospital.

Dependent Variable: survival from trauma.

Independent Variables: age, sex, location of injury, transport time, etc.

Experimental Variable: Application or non-application of the MAST.

Houston MAST Study:Constant: All patients with abdominal trauma with hypotension transported by the HFD to Ben Taub Hospital.

Dependent Variable: survival from trauma.

Independent Variables: age, sex, location of injury, transport time, etc.

Experimental Variable: Application or non-application of the MAST.

ExampleExample

Mattox KL, Bickell B, Pepe PE, Burch J, Feliciano D. Prospective MAST study in 911 patients. J. Trauma. 1989;29:1104-12

911 trauma patients > 15 years of age with systolic BP < 90 mmHg were randomized by alternate day assignment. All transported by HFD to Ben Taub hospital trauma center.

Experimental variable: MAST or no-MAST

Dependent variable: survival from trauma

Mattox KL, Bickell B, Pepe PE, Burch J, Feliciano D. Prospective MAST study in 911 patients. J. Trauma. 1989;29:1104-12

911 trauma patients > 15 years of age with systolic BP < 90 mmHg were randomized by alternate day assignment. All transported by HFD to Ben Taub hospital trauma center.

Experimental variable: MAST or no-MAST

Dependent variable: survival from trauma

ExampleExample

Independent variables: etiology, age, race, sex, location of injury, trauma scores, injury severity scores. Scores revealed the two groups to be statistically identically matched.

Findings: Mortality rate 31% in the MAST group and 25% in the non-MAST group.

Difference statistically significant.

Independent variables: etiology, age, race, sex, location of injury, trauma scores, injury severity scores. Scores revealed the two groups to be statistically identically matched.

Findings: Mortality rate 31% in the MAST group and 25% in the non-MAST group.

Difference statistically significant.

Double Blind StudyDouble Blind Study

Two orthopedic surgeons looking at an ECG.


A double blind study is one in which neither the patient nor the physician knows whether the patient is receiving the treatment of interest or the control treatment.

It is a type of RCT.

A double blind study is one in which neither the patient nor the physician knows whether the patient is receiving the treatment of interest or the control treatment.

It is a type of RCT.


Bracken MB, Shephard MJ, Collins WF, et al. A randomized, controlled trial of methylprednisolone or naloxone in the treatment of acute spinal-cord injury. NEJM 1990;322(20):1405-11 Patients with spinal cord injury randomized by computer to receive methyprednisolone, naloxone, or placebo.Researchers and patients did not know which drug was which (prepared in pharmacy)

Bracken MB, Shephard MJ, Collins WF, et al. A randomized, controlled trial of methylprednisolone or naloxone in the treatment of acute spinal-cord injury. NEJM 1990;322(20):1405-11 Patients with spinal cord injury randomized by computer to receive methyprednisolone, naloxone, or placebo.Researchers and patients did not know which drug was which (prepared in pharmacy)

Quasi-Experimental StudiesQuasi-Experimental Studies

Quasi-experimental studies use intact groups of subjects rather than assigning subjects to groups at random.

Quasi-experimental studies use intact groups of subjects rather than assigning subjects to groups at random.

Quasi-Experimental StudiesQuasi-Experimental Studies

Non-randomized controlled trial:1 group receives intervention

1 group receives no-intervention (control)

Subjects assigned to groups by methods other than randomization.

Non-randomized controlled trial:1 group receives intervention

1 group receives no-intervention (control)

Subjects assigned to groups by methods other than randomization.

Cohort StudiesCohort Studies

A Cohort Study is a study in which patients who presently have a certain condition and/or receive a particular treatment are followed over time and compared with another group who are not affected by the condition under investigation.

A Cohort Study is a study in which patients who presently have a certain condition and/or receive a particular treatment are followed over time and compared with another group who are not affected by the condition under investigation.

Cohort StudyCohort Study


Framingham Heart Study:People in Framingham, MA have been followed for over 50 years.Conducted by National Heart, Lung, and Blood Institute (NHLBI) and Boston University.5,209 residents between 30 and 60 years of age initially enrolled.In 1971, 5,124 children (and their spouses) of the original cohort added.500 minority members have been added.

Framingham Heart Study:People in Framingham, MA have been followed for over 50 years.Conducted by National Heart, Lung, and Blood Institute (NHLBI) and Boston University.5,209 residents between 30 and 60 years of age initially enrolled.In 1971, 5,124 children (and their spouses) of the original cohort added.500 minority members have been added.


Framingham Heart Study:Every two to four years, study participants are given extensive medical examinations including a medical history, blood tests and other sophisticated tests such as bone scans, eye exams and echocardiograms assessing multiple aspects of their current health status.

Framingham Heart Study:Every two to four years, study participants are given extensive medical examinations including a medical history, blood tests and other sophisticated tests such as bone scans, eye exams and echocardiograms assessing multiple aspects of their current health status.


Framingham Heart Study (results):Risk factors for heart disease identified.

Better understanding of the effects of lifestyle.

More than 1,000 papers published significantly adding to the knowledge base of heart disease.

Study continues….

Framingham Heart Study (results):Risk factors for heart disease identified.

Better understanding of the effects of lifestyle.

More than 1,000 papers published significantly adding to the knowledge base of heart disease.

Study continues….

Case-Control StudiesCase-Control Studies

Case Control Studies: Case control studies are studies in which patients who already have a certain condition are compared with people who do not.

Case Control Studies: Case control studies are studies in which patients who already have a certain condition are compared with people who do not.

Case-Control StudiesCase-Control Studies

Case SeriesCase Series

A case series is a report on a series of patients with an outcome of interest. No control group is involved.

A case series is a report on a series of patients with an outcome of interest. No control group is involved.

Case ReportCase Report

A case report is a narrative report of an interesting case.A case report is a narrative report of an interesting case.

Historical StudiesHistorical Studies

Historical studies are systematized searches for the facts and then using the information to describe, analyze, and interpret the past.

Historical studies are systematized searches for the facts and then using the information to describe, analyze, and interpret the past.


Bledsoe BE, Smith MG. Medical Helicopter Accidents in the United States: A 10-Year Review. J Trauma. 2004;56:1325-1329

Bledsoe BE, Smith MG. Medical Helicopter Accidents in the United States: A 10-Year Review. J Trauma. 2004;56:1325-1329


Methods: Review of all air medical helicopter accidents in the US from 1997-2002 from the NTSB database.

Methods: Review of all air medical helicopter accidents in the US from 1997-2002 from the NTSB database.

Systematic ReviewsSystematic Reviews

A summary of the medical literature that uses explicit methods to systematically search, critically appraise, and synthesize the world literature on a specific issue.

A summary of the medical literature that uses explicit methods to systematically search, critically appraise, and synthesize the world literature on a specific issue.

Systematic ReviewsSystematic Reviews

Bledsoe BE. Critical Incident Stress Management (CISM): Benefit or Risk for Emergency Services? Prehosp Emerg Care. 2003;7:272-279

Bledsoe BE. Critical Incident Stress Management (CISM): Benefit or Risk for Emergency Services? Prehosp Emerg Care. 2003;7:272-279

Meta-AnalysisMeta-Analysis

A meta-analysis is a systematic review that uses quantitative methods to summarize the results.

A meta-analysis is a systematic review that uses quantitative methods to summarize the results.


Meta-analysis of RCTs represents the highest level of scientific evidence.

Allows for more objective appraisal of the evidence.

Reduces the possibility of false negative results.

Meta-analysis of RCTs represents the highest level of scientific evidence.

Allows for more objective appraisal of the evidence.

Reduces the possibility of false negative results.


van Emmerik AAP, Kamphuis JH, Hulsbosch AM, Emmelkamp PMG. Single-session debriefing after psychological trauma: a meta-analysis. Lancet. 2002;360:766-771

van Emmerik AAP, Kamphuis JH, Hulsbosch AM, Emmelkamp PMG. Single-session debriefing after psychological trauma: a meta-analysis. Lancet. 2002;360:766-771

Meta-Analysis (Observational Studies)Meta-Analysis (Observational Studies)

Less valid than RCTs.

Still valid in that it gives better information of the population as a whole.

Many EMS interventions cannot be tested with an RCT as it might be unethical to withhold care from the control group.

Less valid than RCTs.

Still valid in that it gives better information of the population as a whole.

Many EMS interventions cannot be tested with an RCT as it might be unethical to withhold care from the control group.


Bledsoe BE, Wesley AK, Eckstein M, Dunn TM, O’Keefe MF. Helicopter transport of trauma patients: a meta-analysis. J Trauma. 2006;60:1257-1266

Observational meta-analysis examining validated trauma scoring systems in trauma patients transported from the scene to a trauma center.

Bledsoe BE, Wesley AK, Eckstein M, Dunn TM, O’Keefe MF. Helicopter transport of trauma patients: a meta-analysis. J Trauma. 2006;60:1257-1266

Observational meta-analysis examining validated trauma scoring systems in trauma patients transported from the scene to a trauma center.


Conclusions:2 out of 3 trauma patients transported from the scene to a trauma center have minor injuries based on validated trauma scoring criteria.

1 out of 4 patients transported are discharged from the emergency department.

Conclusions:2 out of 3 trauma patients transported from the scene to a trauma center have minor injuries based on validated trauma scoring criteria.

1 out of 4 patients transported are discharged from the emergency department.

Animal StudiesAnimal Studies

Animal studies provide insight into biology.

For ethical reasons, some studies cannot be carried out on humans.

Computer modeling are replacing many animal studies.

Animal studies provide insight into biology.

For ethical reasons, some studies cannot be carried out on humans.

Computer modeling are replacing many animal studies.

Expert OpinionExpert Opinion

Expert opinions can take various forms:Systematic reviews

Narrative reviews

Pure opinion pieces

Expert opinions can take various forms:Systematic reviews

Narrative reviews

Pure opinion pieces

Rational ConjectureRational Conjecture

Lowest level of scientific validity.

But, overall very important.

“You don’t have to run a Chi-Square test on common sense”

Lowest level of scientific validity.

But, overall very important.

“You don’t have to run a Chi-Square test on common sense”

JournalsJournals

Not all journals are the same.

Impact rating:Cites

Articles

Readership

Not all journals are the same.

Impact rating:Cites

Articles

Readership

JournalsJournals

Journal Impact Rating (2005)

NEJM 44.016

Lancet 23.407

JAMA 23.332

Circulation 11.632

BMJ 9.052

Crit Care Med 5.077

Ann Emerg Med 2.782

Acad Emerg Med 1.789

J Trauma 1.722

StatisticsStatistics

Statistics are essential to any scientific analysis to eliminate bias and chance.

“Torture numbers and they’ll confess to anything.” (Gregg Easterbrook)

“Satan delights equally in statistics and in quoting scripture.... ” (H.G. Wells)

“The average human has one breast and one testicle.” (Des McHale)

Statistics are essential to any scientific analysis to eliminate bias and chance.

“Torture numbers and they’ll confess to anything.” (Gregg Easterbrook)

“Satan delights equally in statistics and in quoting scripture.... ” (H.G. Wells)

“The average human has one breast and one testicle.” (Des McHale)

SummarySummary

EMS must be driven by science.

Science is based upon quality research.

EMS providers of the future must be able to understand, and in certain cases, conduct valid research.

EMS must be driven by science.

Science is based upon quality research.

EMS providers of the future must be able to understand, and in certain cases, conduct valid research.

SummarySummary

Break TimeBreak Time

GoodGood

Gaucshe M, Lewis RJ, StrattonSJ, et al. Effect of Out-of-Hospital Pediatric Intubation on Survival and Neurologic Outcome: A Clinical Trial. JAMA. 2000;283:783-790

Gaucshe M, Lewis RJ, StrattonSJ, et al. Effect of Out-of-Hospital Pediatric Intubation on Survival and Neurologic Outcome: A Clinical Trial. JAMA. 2000;283:783-790

GoodGood

Methodology: RCT

Randomization: By calendar day.

Journal: JAMA

Participants: 830 consecutive patients 12 years and weight < 40 kg

Interventions: BVM only (odd days)

N= 410

BVM then ETI (even days)N= 420

Methodology: RCT

Randomization: By calendar day.

Journal: JAMA

Participants: 830 consecutive patients 12 years and weight < 40 kg

Interventions: BVM only (odd days)

N= 410

BVM then ETI (even days)N= 420

Outcome Measures:Survival to discharge

Neurologic status at discharge

Results:No improvement in survival or neurological outcome in either group.

Outcome Measures:Survival to discharge

Neurologic status at discharge

Results:No improvement in survival or neurological outcome in either group.

BadBad

Timmerman S, Cardoso LF, Ramires JAF, Halperin H. Improved Hemodynamic Performance with a Novel Chest Compression Device During Treatment of in-hospital Cardiac Arrest. Resusc. 2004;61:273-280

Timmerman S, Cardoso LF, Ramires JAF, Halperin H. Improved Hemodynamic Performance with a Novel Chest Compression Device During Treatment of in-hospital Cardiac Arrest. Resusc. 2004;61:273-280

BadBad

Methodology: Case series

Randomization: None.

Journal: Resuscitation

Participants: 31 terminally-ill, fresh dead Brazilians who failed 10 minutes of ACLS. Age 68 10 years

Interventions: Swan-Ganz catheters placed

Methodology: Case series


Journal: Resuscitation

Participants: 31 terminally-ill, fresh dead Brazilians who failed 10 minutes of ACLS. Age 68 10 years

Interventions: Swan-Ganz catheters placed

Interventions:90 seconds A-CPR (60 compressions / minute) followed by manual CPR (100 compressions / minute).

No ventilations

Subjects then intubated. Ventilated (12 / minute)

Epinephrine given

Interventions:90 seconds A-CPR (60 compressions / minute) followed by manual CPR (100 compressions / minute).

No ventilations

Subjects then intubated. Ventilated (12 / minute)

Epinephrine given

BadBad

Outcome Measures:CPP

MAP

Results:Usable blood pressures obtained in 16/31 patients (52%)

Rosary or wake interrupted (25%)

CPP:A-CPR: 20 12

M-CPR: 15 11

Outcome Measures:CPP

MAP

Results:Usable blood pressures obtained in 16/31 patients (52%)

Rosary or wake interrupted (25%)

CPP:A-CPR: 20 12

M-CPR: 15 11

PAP:A-CPR: 115 42

M-CPR: 153 28

Results:Increased CPP is related to increased chances of resuscitation. A-CPR improved CPP.

PAP:A-CPR: 115 42

M-CPR: 153 28

Results:Increased CPP is related to increased chances of resuscitation. A-CPR improved CPP.

UglyUgly

Everly GS, Flannery RB, Eyler V A. Critical Incident Stress Management (CISM): A Statistical Review of the Literature. Psych Quart. 2002;73:171-182

Everly GS, Flannery RB, Eyler V A. Critical Incident Stress Management (CISM): A Statistical Review of the Literature. Psych Quart. 2002;73:171-182

UglyUgly

Methodology: Meta-analysis?


Journal:

Participants: Meta-analysis of 8 CISM investigations.

Evaluation: Results pooled and Cohen’s d calculated

Methodology: Meta-analysis?


Journal:

Participants: Meta-analysis of 8 CISM investigations.

Evaluation: Results pooled and Cohen’s d calculated

Results: Significant effect size (Cohen’s d) of 3.11 showing a beneficial outcome from CISM.

Results: Significant effect size (Cohen’s d) of 3.11 showing a beneficial outcome from CISM.

d =v 2

1 +v 22 /2

mean1- mean2

You can’t obtain a Cohen’s d without a

control group!

UglyUglySTUDY CISM MEASURES PROBLEM

Busuttil (1995) Used CISM to treat PTSD in British Air Force personnel.

PTSD Scale CISM is not a treatment for PTSD

Flannery (1995) ASAP AssaultsQuestionnaire

No mention of CISM in paper





Richards (1999) Debriefing following bank robbery in NSW

IES CISM provided at 10 0r 24 hours

Mitchell (2000) CISM provided for volunteer firefighters with PTSD 3 years after a tornado

Diagnostic interviews CISM is not a treatment for PTSD

Flannery (2000) ASAP Assaults No mention of CISM in paper

Flannery (2000) ASAP Assaults CISM mentioned

GoodGood

Wang HE, Kupas DF, Hostler D, Cooney R, Yealey DM, Lave JR. Procedural Experience with out-of-hospital endotrachela intubation. Crit Care Med. 2005;33:

Wang HE, Kupas DF, Hostler D, Cooney R, Yealey DM, Lave JR. Procedural Experience with out-of-hospital endotrachela intubation. Crit Care Med. 2005;33:

GoodGood

Methodology: Analysis of administrative database

Randomization: None

Journal: Critical Care Medicine

Participants:Records: 1,544,791

11,484 ETIs by 5,245 rescuers.

Methodology: Analysis of administrative database

Randomization: None

Journal: Critical Care Medicine

Participants:Records: 1,544,791

11,484 ETIs by 5,245 rescuers.

Outcome Measures:Of 5,245 rescuers:

67% performed 2 ETIs.

39% performed 0 ETIs/

Mean ETIs = 1

Results:Out-of-hospital intubation is an uncommon event.

Outcome Measures:Of 5,245 rescuers:

67% performed 2 ETIs.

39% performed 0 ETIs/

Mean ETIs = 1

Results:Out-of-hospital intubation is an uncommon event.

BadBad

Amatangelo M, Thomas SH, Harrison T, Wedel SK. Analysis of Patients Discharged from Receiving Hospitals within 24 Hours of Air Medical Transport. Air Med J. 1997;16:44-14

Amatangelo M, Thomas SH, Harrison T, Wedel SK. Analysis of Patients Discharged from Receiving Hospitals within 24 Hours of Air Medical Transport. Air Med J. 1997;16:44-14

BadBad

Methodology: Retrospective review of flight records

Randomization: None

Journal: Air Medical Journal

Participants:945 flights

42 (4.4%) discharged less than 24 hours of transport.

Methodology: Retrospective review of flight records

Randomization: None

Journal: Air Medical Journal

Participants:945 flights

42 (4.4%) discharged less than 24 hours of transport.

Results: Patients met air transport criteria.

A priori

UglyUgly

Ohshige K, Kawakami C, Kubota K, Tochikubo O. A contingent valuation study of the appropriate user price for ambulance service. Acad Emerg Med. 2005;12:932-940

Ohshige K, Kawakami C, Kubota K, Tochikubo O. A contingent valuation study of the appropriate user price for ambulance service. Acad Emerg Med. 2005;12:932-940

UglyUgly

Methodology: Questionnaire survey

Randomization: None

Journal: Academic Emergency Medicine

Participants:

3,363 ambulance service customers sent survey.

2,029 (60.3%) returned survey.

Methodology: Questionnaire survey

Randomization: None

Journal: Academic Emergency Medicine

Participants:

3,363 ambulance service customers sent survey.

2,029 (60.3%) returned survey.

Results:Public inelastic (not concerned) about cost of ambulance service when case is an emergency.

Public fairly elastic about price in nonserious situations.

Thus, a ¥ 20,000 (US$ 190.00) user fee will reduce the number of nonemergency calls.

Results:Public inelastic (not concerned) about cost of ambulance service when case is an emergency.

Public fairly elastic about price in nonserious situations.

Thus, a ¥ 20,000 (US$ 190.00) user fee will reduce the number of nonemergency calls.

You don’t have to run a Chi

Square test on common sense.

the good, the bad & the ugly of ems research. definitions research: careful, systematic, patient...

Documents

science of ems

little science

definitions science

ems art

art of ems

scientific method hypothesis

great deal of science

ugly of ems research