Agenda

• Review of Last week

• Learn about types of Research designs– How are they different from each other?

From other things?

• Applying what you learned with in-class exercises

• Using what you learned toward your projects

• List all the ways used to measure gun ownership in the article– For each way, identify whether used as

predictor or criterion variable

• Describe (with a diagram) the hypotheses– For each, identify the kind of hypothesis

• Explain (with a diagram) the logic behind the hypotheses

Review: Students answer

Types of Research Designs

Typical Examples of Research Designs

• Field Studies

• Laboratory Experiments

Two criteria differentiating research designs

1. The obtrusiveness of the operations they use

2. Specificity of the Systems they study

Particular Behavior Systems

Universal Behavior Systems

ObtrusiveOperations

UnobtrusiveOperations Field Studies

Lab Experiments

Runkel & McGrath, 1972

Example of a Laboratory Experiment

Example of a field Study

• Conference Board study– What is it particular to?– In what way is it unobtrusive?

Adding a third criterion to differentiate research designs

1. Degree to which settings are natural to the behavior being studied

2. The obtrusiveness of the operations they use

3. Specificity of the Systems they study

Natural Settings

ContrivedSetting

ObtrusiveOperations

UnobtrusiveOperations

Field Studies

Lab Experiments

Runkel & McGrath, 1972

Field Experiments

Particular Behavior Systems

Universal Behavior Systems

Adding a fourth criterion to differentiate research designs

1. Precision of measurement

2. Degree to which settings are natural to the behavior being studied

3. The obtrusiveness of the operations they use

4. Specificity of the Systems they study

Natural Settings

ContrivedSetting

ObtrusiveOperations

UnobtrusiveOperations

Field Studies

Lab Experiments

Runkel & McGrath, 1972

Field Experiments

MaxContext

Maximum Precision

ExperimentalSimulations

• Field Study– Handgun magazine sales & gun related

deaths

Examples from Guns & Crime Article

• Experimental Simulation– Radhakrishnan (1996) Dissertation study

• Field Experiment– Radhakrishnan Writing Study (2006)

Examples from IRE2002Y Instructor’s Research

• Field Experiment– Gun magazine sales in Year 1 related to

homicide rate in Year 2– Homicide rate in Year 1 not related to gun

magazine sales in Year 2

Example from Guns & Crime Article

• Which design establishes the causal relationship with more precision? Why/how?

Practice concepts learned so far with Guns & Crime Article

Adding a fifth criterion differentiating research designs

1. Degree to which to the behavior being studied is independent of the setting

2. Precision of measurement

3. Degree to which settings are natural to the behavior being studied

4. The obtrusiveness of the operations they use

5. Specificity of the Systems they study

Setting Dependent

SettingIndependent

ObtrusiveOperations

UnobtrusiveOperations

Field Studies

Lab Experiments

Runkel & McGrath, 1972

Field ExperimentsSample

Surveys

Examples

• Sample Survey– Gallup Polls– Spector, Radhakrishan et al. (2001)– Schimmack, Radhakrishnan et al. (2002)

• Field Study– Conference Board Studies

A sixth criterion differentiating research designs

1. Degree to which to they maximize generality of behaviors studied by

– Large samples– Multiple variables

2. Degree to which to the behavior being studied is independent of the setting

3. Precision of measurement

4. Degree to which settings are natural to the behavior being studied

5. The obtrusiveness of the operations they use

6. Specificity of the Systems they study

Particular Behavior SystemsUniversal Behavior Systems

ObtrusiveOperations

UnobtrusiveOperations

FieldExperiments

LabExperiments

MaximumPrecision

MaximumGenerality

Formal Theory

Sample Surveys

Runkel& McGrath, 1972

MinimumGeneralityExperimental

Simulations

Examining all criteria differentiating research designs

simultaneously

Particular Behavior SystemsUniversal Behavior Systems

ObtrusiveOperations

UnobtrusiveOperations

NaturalSettings

ContrivedSettings

FieldStudies

FieldExperiments

LabExperiments

MaximumContext

MaximumPrecision

MaximumGenerality

Formal Theory

Sample Surveys

SettingIndependent

Behavior notmeasured

ComputerSimulations

Runkel& McGrath, 1972

ExperimentalSimulations

Review & Next Steps…

• Types of research designs

• How research designs are different from– Types of data analyses– Types of data

• Research design= way of collecting data

• Statistical Analysis =way of analyzing data

Research Designs vs. Statistical Analyses

Examples of Statistical Analyses• Means

• Frequencies

• Correlations

• Regression

• ANOVA

• Data collected in field studies or sample surveys lend themselves to correlational analyses

• Sometimes, the term “correlational method” is (confusingly) used to imply field study or a sample survey in contrast to the experimental method that is used to imply laboratory or field experiments– There is no such thing as experimental

analyses

Research Designs vs. Statistical Analyses

Meta-analysis: An example of a statistical analysis

• A way of analysing data collected across multiple settings & samples to estimate overall size of relation between variables• Helps in

• Developing theory, identifying research needs, • Establishing validity/effectiveness of tools

(e.g., teaching writing)

• Replace sample surveys• Better than qualitative literature reviews

Meta-analyses vs. sample surveys

• Sample Survey– A way of collecting data

• Meta analyses – A way of analysing data

• Can involve sample surveys, field studies, lab experiments

• Examples of Qualitative Data:– Themes from Interview transcripts– Themes from focus group transcripts – Some kinds of observational/archival data

• Examples of Quantitative Data: – What are examples of quantitative data in

the guns & crime article

Research Designs vs. Type of data

• Quantitative & qualitative data can be collected in all research designs

Research Designs vs. Type of data

Review & Next steps

• Types of research designs

• How research designs are different from– Types of data analyses– Types of data

• Applying what you learned

• What is the type of research design used in each study description in handouts from last week

Applying what you learned:Student do exercises & answer

orally

Review & Next steps

• Research designs

• Applying what you learned

• Using what you learned in your projects

Choose a Research Design for your project that

1. Adequately test the hypothesis

2. Is efficient in using available resources

3. Rejects/retains the hypothesis via statistical means

4. Can statistically control for some extraneous factors

5. Ensures that results are generalizable

Choose a Research Designthat adequately tests the

hypothesis

• Hypotheses determine participants, variables measured & data analysis methods– Students answer orally: What are some

(tentative) hypotheses tested in your projects?

Choose a design that is efficient in using available

resources

• Resources: time, material resources, expertise– Students answer orally: What are

possible designs you can use in student projects?

Choose a Research Design that can reject/retain the hypothesis via statistical

analyses

• Need reliable measures — Future readings & lectures

• Need large enough sample to detect true effect & avoid errors-- Future readings & lectures

• Control=Eliminate extraneous variables that can affect dependent variables increases confidence in cause-effect conclusion – Depends on type of design

Choose a Research Design that can control for some

extraneous variables

Controlling Extraneous Variables in Experimental Designs

• Random Assignment• Distributing extraneous variable across

groups randomly

• Matching Groups• Spreading the extraneous variable across

groups via systematic assignment• Less effective than random assignment

(p.148 Sekaran)

• Control Group • Group that receives no manipulation

Controlling Extraneous Variables

• Extraneous variables are reliably measured and effects are statistically controlled• Future lectures on Power & Statistical Analyses

• Typically done in – Field studies– Field Experiments – Sample surveys

• Students give examples from Guns & Crime articles

Choose a design that gives generalizable results

• Generalizable=Replicate to other samples and other contexts – Random selection of participants– Large number of participants – Large number of contexts

• E.g., meta-analysis

Ensure your project design

1. Adequately test the hypothesis

2. Is efficient in using available resources

3. Rejects/retains the hypothesis via statistical means

4. Can statistically control for some extraneous factors

5. Ensures that results are generalizable

What you learned today

• Types of research designs

• How research designs are different from– Types of data analyses– Types of data