experimental design part i

8/3/2019 Experimental Design Part I

1/19

1

Quantitative ResearchPart I. Basic Terms and Concepts

Ann de Peyster, PhD


2/19

2

Quantitative Methods

A definition

A survey or experiment that provides asoutput a quantitative or numeric description ofsome fraction of the population, called thesample.


3/19

3

Importance-of-Statistics Disclaimer

A working knowledge of statistics isabsolutely essential to being a successfulquantitative methods researcher.

Unfortunately, time does not permit muchcoverage of statistics within thisintroductory overview research class.


4/19

4

Terms and Concepts

Study purpose, objectives/aims, researchquestions, and hypotheses (review these andother terms in Dr. dePs first Ppt)

Quantitative/qualitative research and data

Causal relationships vs. associations Groups: study group or experimental group,

control group, group assignment Variables: dependent, independent Independence/independent samples

Validity Confounders Bias Hypothesis testing statistics, variability, statistical

significance


5/19

5

Causal relationships vs. associations

In research involving human subjects,when the condition you are studying iscorrelated with a risk factor or other

variable, saying A is associated withB isusually preferable to saying A causes B.Why?

Hint:


6/19

6

Groups

Study or experimental group: receivingthe medication or experiencing the otherfactor(s) under study

Controlgroup: a group of subjectsclosely resembling the treatment group inmany demographic variables but notreceiving the medication or experiencingthe other factor(s) under study andthereby serving as a comparison group.

Group assignment: Random or based onpre-determined selection criteria (more onthis later in Part II)


7/19

7

Variablesdependent and independent

In an experiment, the independentvariable is the variable that is varied ormanipulated by the researcher, and thedependentvariable is the response thatis measured.

Maybe not intuitive?

Independent = Remember:I/you (theexperimenter) controls

Dependent = depends on the independentvariable


8/19

8

Examples of independent and

dependent variables( 2 and 3 adapted from: http://www.uncp.edu/home/collierw/ivdv.htm

The following are hypotheses for two different studies:

1. Hispanic women living in close proximity to agriculturalland (


9/19

9

Examples, continuedThe following is a description of a study:3. A director of residential living on a large university campus is

concerned about the large turnover rate in resident assistants(RAs). In recent years many RAs have left their positions beforecompleting even 1 year in their assignments. The director wantsto identify the environmental health factors that predict longevity

of an RA in their position, defined as continuing in the position aminimum of 2 years.The director decides to assess the following variables as possible

contributors to longevity in the position: safety, ease and generalcomfort of the built environment of the residence (e.g., darkstair wells or elevators?); any evidence of sick buildingsyndrome (e.g., number of student residents and RAsexperiencing headaches or flu-like symptoms during the year ineach residence); tobacco smoke permeating residence hallscaused by students smoking illegally in their rooms.

IVs: All environmental variables listed above that thedirector/experimenter has decided to include in this study.

DV: RA commitment to position or not (i.e., continuing in positionfor 2 years or not continuing).


10/19

10

Independence, independent samples

If you survey a person (e.g, by interview)about exposure to a chemical ten times ondifferent days in one month are these

samples independent? If you measure biological samples (e.g,

blood or other body fluids) from a singleperson ten times in one month are these

samples independent?


11/19

11

Independence, independent samples

If you survey ten people (e.g, by interview) onceeach in one month are these samplesindependent?

If you take biological samples (e.g, blood orother body fluids) from ten different people onceeach in one month are these samplesindependent?

Suppose two very different types of datavariables are measured once in the same person(e.g., survey response and blood values) arethose samples independent of each other?Could these be correlated?


12/19

12

Validity (of a study)

Conclusions drawn from analyzingresearch data are only acceptable to thedegree to which they are determined to be

valid (=relevant, meaningful,justifiable, believable)


13/19

13

Confounders (confound~confuse)

A confounderis a factor that is linked tothe variable or outcome of interest and isunevenly distributed between the studygroups.

Examples of confounding: 1. Researchers completely forget to ask about

possible job-related exposures to mercury in astudy investigating correlation of mercury

levels in hair with consumption of largepredator fish. (Occupational exposure tomercury should be acknowledged as apossible confounder.)


14/19

14

Examples of confounding, continued

2. In a childhood asthma intervention study conductedat a pre-school, the researchers know parental smokingmay be confounder so they also document smokingpractices of the parents. They find the amount ofcigarettes and cigars smoked at home by the

intervention group parents is similar to parents of thecontrol group not receiving the intervention.

These researchers are lucky! They made an effort torule out smoking as a potential confounder in thisexperiment. In this case they were successful.But if the distribution of smoking was different inthe two groups, then the research report should

state that different tobacco smoking habits by theparents of the two groups could confoundinterpretation of results.)


15/19

15

Confounding and spurious

correlations (=false or inappropriate)

A study finds that the numbers of drinks menconsume in sleazy bars is positively correlatedwith their incidence of lung cancer doesdrinking alcohol cause lung cancer?

A study finds that freshman women are morebeautiful than seniors is educationdebeautifying?


16/19

16

Bias: systematic error in an estimate or inference(Shadish, Cook and Cambell, 2002, Experimental and Quasi-Experimenal Designs)

Selection biases, which may result in thesubjects in the sample being unrepresentative ofthe population of interest

Measurement biases, which include issuesrelated to how the outcome of interest wasmeasured

Intervention (exposure) biases, which involvedifferences in how the treatment or interventionwas carried out, or how subjects were exposed

to the factor of interest

Publication bias: tendency for researchers andreviewers to not publish nonsignificant findings


17/19

17

Bias, continued

Obvious examples of potential for bias inresearch that can easily be avoided: Volunteers willing to take your 100-page survey

probably have much more leisure time than youraverage person in the general population of interestbeing studied (=selection bias)

Jargon or other rarely-used words used in your surveymay not have been completely understood by the lesseducated or inexperienced study subjects, and theresearchers only discover this after the study ends

(=measurement bias) Researcher providing an educational intervention by

conversing with men and women in a study isinstinctively more relaxed and friendly toward womeninterviewees and may not be aware of this(=intervention bias)


18/19

18

Importance of Statistics Disclaimer

A working knowledge of statistics isabsolutely essential to being a successfulresearcher.

Unfortunately, time does not permit morecoverage of statistics now within thisintroductory overview research class.


19/19

19

Hypothesis testing statistics are often used todetermine differences between experimental groups

Rock bottom basics!

When the means of two groups (e.g.,experimental vs. control) are significantly

different

experimental design part i

Documents