Introduction

Theory

Hypotheses

Data

Verification

Theory buildingHypothesis generation

Measurement issuesResearch designSampling issues

Statistical analysisInterpretation

Presentation of results

GeneralizationCumulationModification

Common Sense vs. ScienceProcedures

CS: Quick acceptance of explanationsS: Slow and deliberate theory building and testing

HypothesesCS: “Selective” testing and post hoc explanationsS: Systematic testing, a priori hypotheses, and empirical basis

PhenomenaCS: May include untestable phenomenaS: Only concerned with observable and testable phenomena

ModificationCS: Modification through selective recollectionS: Replication and systematic testing - self-correcting

Sources of discovery(McCall & Bobko)

(1) Serendipity - chance discovery(2) Sagacity - ability to recognize what

unexpected results mean(3) N=1 - qualitative studies(4) Socializing - interacting with others can

create insight(5) Metaphors - applying metaphors to the

unknown(6) Error variance - pay attention to the

abnormal

TYPE I AND TYPE II ERRORS

Ho True Ho FalseTRUE STATE OF WORLD

DECISION Reject Ho

Fail toReject Ho

Type I Errorp =

Type II Errorp =

Correct decisionp = 1 -

Correct decisionp = 1 - power

Important Concepts in Research

Forms of ControlInternal ValidityExternal Validity

ControlManipulation

Elimination or inclusion

Statistical

Randomization

Types of Validity (C&C, 1979)

Statistical conclusion

Construct

Internal

External

Authorship DecisionsWhat is authorship?

A professional contribution that is creative and intellectual in natureIncludes

Developing research designs/ideasWriting portions of a manuscriptIntegrating theoryIntegrating conceptual modelsMaking decisions about data analysisInterpreting results

Authorship Decisions

What authorship is notBased on time and effortDoes not include

Advice on what data analysis to doResearch idea onlyLiterature searchesData collectionDesign of equipment

Authorship DecisionsSome guidelines include:

Discuss up frontRenegotiate as things changeConsider where students fall on the line of competence, greater competence typically yields greater contributionShould not be affected by paid participation (controversial)Get advice if you are unsure what constitutes authorship

How do you do research that is:

• Significant?• Likely to be published in good

journals?• Likely to be read by peers?• Likely to be cited by peers?

Persistence

1. Doing good research is persistence, persistence, persistence. –rethink, rethink—re-examine the issue over and over again more fine-grained each time. You need to think deeply about what you want to do.

Discussion and Collaboration

Research benefits by discussing ideas with others. It helps to focus your thoughts and you benefit by the thoughts of others when you air your ideas.

Multi-authored articles are OK. Try to have teams with complementary skills—some limitations.Go to conventions and network.

Doing

You will not learn research from a research course; you learn research by doing research. The more research you do the better future research will be—you develop a template for doing research"Good" research only raises more questions. You realize how little you know about the phenomena and what you need to know.

TheoryResearch without a strong theoretical basis less likely to make a contribution and be published.Research frequently ignores the mediating process when documenting relationships—We miss the underlying reason why the event took place or depend on theory to explain the event. Process measures always help.The hardest part of the research is deciding on the problem and hypotheses (theoretical part).

MethodsTriangulation – combine an experimental (manipulation) study with a surveyMulti-source data (multi people, methods)Pilot what you do on a small sample firstUse reliable and valid measurementSeparate the independent from the dependent variables using different measurement strategies. Try to measure behavior not attitudes as the primary dependent variable.

Build on past research

Articles in JAP, OBHDP, Personnel Psychology 84% were coupled, building on existing research by:

Use different subject populationUse different operationalization of variablesUse different levels of variablesExamine variables together previously examined separatelyAdding mediators or moderators

Reasons For RejectionInadequate Justification For Doing Study

Unclear research questionUnclear theoretical basis for questionDoes not articulate the importance of the research

“Why is this question important?”Does not articulate the contribution

“What is new and different about the study that adds to the literature?”Sometimes the study really does not have a contribution

Methods Are Inadequate to Answer Question

Confounded variables Appropriate control variables are lacking Make statements of causality while

methods (correlation) only allow one to make statements of association.

Reasons For Rejection

Measurement IssuesIncorrect operationalization of the variables.

Common with archival dataConstruct validity - Absence of justification that variables being measured actually assess the construct. Construct validityPoor reliability of scales

Reasons For Rejection

Key Terms

Independent Variable—Variable that is manipulated in a experiment, the cause—the antecedent or predictor variable when measured.

Dependent Variable—The variable that is used to measure the effects of the independent variable in an experiment, the effect. The consequence or criterion variable when measured.

Mediation vs. ModerationResearchers often confuse mediation and moderationTwo completely different processes and analytical approachesMediation implies the effect of an independent variable on a DV occurs through another variableModeration implies the effect of an independent variable on a DV depends on the level of another variable

In terms of research terminology:

The existence of an interaction indicates the effect (relationship) of the IV with the DV is different at different values of the moderator variable.

Moderator

Moderation

IV

Moderator

DV

Example of Moderator

Males

Low

High

Y

X HighLow

Females

When X = amount of coffee consumed and Y = minutes spent on treadmill

Example of Moderator

Low

High

Y

X HighLow

Z Easy

Z Medium

Z Difficult

X = hours studied, Y = test performance,Z = test difficulty

Example of Moderation

Experts

Low

High

Y

X HighLow

Novices

Where X = blood alcohol levelY = video game score

Mediator Variables

The mechanism or process through which a variable has its effect on other variables

Obtaining PhD

Lack of Personal

Time

Marital Conflict

Assignment 1

Find an empirical article in your field of interest that tests the relationship between an independent variable dependent variable and tests the impact of a moderator variable on this relationship.

1) Describe the independent variable(s) and the dependent variable(s).

2) State the hypothesis that describes the relationship between the independent and the dependent variable.

3) State the hypothesis that describes how the moderating variable is expected to change the relationship between the independent and dependent variable.

4) Illustrate the hypothesized moderated relationship in a figure.

5) Did the results find evidence for a moderated relationship?