christian dormann 1 substantive and methodological problems in the study of stress at work european...

ChristianDormann1

Substantive and Methodological Problems in the Study of Stress at Work

European Erasmus Mundus Program

Master on Work, Organizational & Personnel Psychology

Valencia, 15.10.-17.10.2008

Substantive & Methodological Problems

in the Study of Stress at Work!

ChristianDormann2




Valencia, 15.10.-17.10.2008

Overview

1. Introduction

2. The Big 5 Substantive & Methodological

Challenges in Organizational Stress Research

2.1 Overlap (Conceptual & Measurement)

2.2 Confoundedness (Congeneric & Measurement)

2.4 Change (Intraindividual & Interindividual)

2.3 Directedness (Reversed & Reciprocal)

2.5 Time (Longitudinal & Shortitudinal)

3. Take Home Messages

ChristianDormann3




Valencia, 15.10.-17.10.2008

Why this topic

Demonstrating and understanding causal relations = Central aim of work

and organizational psychology research on stress

Experimental designs are frequently impossible to implement.

Circumventing problems associated with non-experimental designs along

two pathways:

1) Substantive considerations tailored to W&O stress research

2) Methodological considerations useful for various content domains

Introduction

ChristianDormann4




Valencia, 15.10.-17.10.2008

Why this topic

Three commonly accepted conditions must hold for a scientist to claim

that X causes Y:

1. Time precedence

2. Relationship

3. Nonspuriousness

Experiment

1. The DV is measured after the IV is established

2. Systematic variations in the DV conditional upon the IV

3. Participants are assigned to the IV by random

(Ken

ny, 1979)

Introduction

ChristianDormann5




Valencia, 15.10.-17.10.2008

Why this topic

(Ken

ny, 1979)Getting Rid of the Relationship Condition

StressorStress

Reaction+ .25

ThirdVariable

+ .50 - .50 r = +.25 + (+.50 * -.50) = +.25 - .25 = 0.00

Introduction

Suppres-sor

Mediator

ChristianDormann6




Valencia, 15.10.-17.10.2008

The Content of the Subsequent Presentation

What

- substantive (i.e. related to the content) and

- methodological (i.e. related to design and analysis)

considerations are worthwhile in order to establish

- time precedence (i.e. the ordering of IV and DV) and

- nonspuriousness (i.e., rejection of third variable explanations)

among

- variables in work and organizational stress research allowing for

- causal explanations?

=> let’s go ahead

Introduction

ChristianDormann7




Valencia, 15.10.-17.10.2008

Conceptual& Measurement

Overlap

Overlap

ChristianDormann8




Valencia, 15.10.-17.10.2008

Two Questions Addressed in this Section

1) How should stress and related concepts be defined?

2) How should stress and related concepts be measured?

Overlap

ChristianDormann9




Valencia, 15.10.-17.10.2008

Defining „Stress“Defining Stress

- stimulus/situation based

- reaction/person based

- transactional

Examples:

- Kobasa (1979): Stressful life events cause changes in, and

demand readjustment of, an average person’s normal routine.

- Selye (1936): Stress is the body’s nonspecific response to a

demand placed on it.

- Lazarus (1966): Stress occurs when an individual perceives that

the demands of an external situation are beyond his or her perceived

ability to cope with them.

Overlap

ChristianDormann10




Valencia, 15.10.-17.10.2008

Defining „Stress“Problems with Stimulus-based Definitions of Stress

- no possibility to describes qualitatively different types of stress on

the same scale (e.g., relocating and divorcing)

- no model available to allow combinations of different types of stress

- no consideration of interindividual differences in stress appraisal

- no consideration of interindividual differences in responses to stress

- everything might be regarded as stress response (even a non-

response)

Problems with Response-based Definitions of Stress

- no universal stress response pattern identified yet

- if universal stress responses existed, their meaning were probably

not unique (e.g., blood pressure due to fear vs. joy vs. exercise)

Overlap

ChristianDormann11




Valencia, 15.10.-17.10.2008

Defining „Stress“Solutions:

- Probabilistic model: “Average” person; (cf. Holland’s 1986

fundamental problem of causal inference and his concept of average

causal effects):

Stressors represent stimuli that cause stress reactions, on average

- Different conceptual perspectives:

+ Defining a processes versus

+ Defining variables

- Different terms:

+ stressor (or demand – take care!)

+ resources

+ stress reaction (strain)

Overlap

ChristianDormann12




Valencia, 15.10.-17.10.2008

Defining „Stress“

Example: Work-Family Conflict

(Stressor, Stress, Stress Reaction, Resource, Process)?

Work-family conflict is a form of interrole conflict in which the role pressures

from the work and family domains are mutually incompatible in some respect.

That is, participation in the work (family) role is made more difficult by virtue of

participation in the family (work) role.

Overlap (Gre

enhaus & B

eutel, 1985)

ChristianDormann13




Valencia, 15.10.-17.10.2008

Measurement of „Stress“The exact definition of a variable represents a prerequisite for developing

adequate measures.

Exercise: Develop 1-2 items in order to measure the following:

- family role-pressure

- work role-pressure

- mutual incompatibility

- difficulty

- participation in the work role

- participation in the family role

Definition: Work–family conflict is a form of interrole conflict in which the role

pressures from the work and family domains are mutually incompatible in some

respect. That is, participation in the work (family) role is made more difficult by virtue

of participation in the family (work) role

Overlap

ChristianDormann14




Valencia, 15.10.-17.10.2008

Measurement of „Stress“Examples:

Carlson, Kacmar, & Williams (2000): Due to all the pressures at work, sometimes when I come home I am too stressed to do the things I enjoy.

Netemeyer Boles, & McMurrian (1996): I have to put off doing things at work because of demands on my time at home.

Carlson, Kacmar, & Williams (2000): I am often so emotionally drained when I get home from work that it prevents me from contributing to my family.

Haun & Dormann (2008): I feel I have not enough time for both my work and my private life.

Haun & Dormann (2008): To what extent is your work life (nonwork life) to blame, when you encounter problems reconciling your work and your nonwork life?

What do these items actually measure?

What are potential problems?

Overlap

ChristianDormann15




Valencia, 15.10.-17.10.2008

Avoiding Overlap

Conclusion

We have to define non-overlapping concepts and measure them without

overlap in order to avoid obtaining trivial (i.e. spurious) relations!

Solutions

- Measuring stressors, resources, stress reactions objectively (take

care!)

=> The End of this Section

- Measuring (adequately!!) confonding variables and controlling (i.e.,

partialling) them:

=> The Transition to the Next Section

Overlap

ChristianDormann16




Valencia, 15.10.-17.10.2008

Objective Measurement of „Stress“

Conceptions of Objective Stressors and Resources:

1) Measurement without psychological processes involved

=> problem then: psychological stressors and resources can never be

objective (only useful for material objects and processes)

2) not illusory or unreal

=> problem then: the concept of objectivity is too broad (everything not

illusionary) to be useful. Applies only when psychopathological phenomena

are proposed

3) not being related to a specific individual's cognitive and emotional

processes (perceptions, appraisals etc).

=> problem then: concepts such as “average person” is required to

conceptually leave out “individual” cognitive processes.

Overlap

ChristianDormann17




Valencia, 15.10.-17.10.2008

Ways to Measure „Stress“ Objectively

Measuring stress-variables that are unrelated to one specific individual's cognitive and emotional processes (perceptions, appraisals etc):

1) independent observers (i.e., randomly assigned) => several problems enumerated on the next slides

2) descriptive instead of evaluative items=> more or less easy to develop, amount of objectivity unclear

3) dependent observers (e.g., colleagues, supervisors, spouses)=> difficult to obtain, and amount of objectivity unclear. For example

+ colleagues use their own (related) working conditions to infer the level of stress reactions of the target’s stress reactions

+ halo errors of supervisors (e.g., an attractive subordinate feels excellent even though he was given the most demanding

tasks)+ similarity in personality characteristics (e.g., N or NA) of spouses

cause similar biases in measurement (e.g., my wife’s time pressure is very high; my wife feels poor)

Overlap

ChristianDormann18




Valencia, 15.10.-17.10.2008

Objective Measurement of „Stress“

Problems with observer ratings:

1) Limited time of observation (work shift, work cycles, any kind of cycle)

2) Unobservability of mental processes (e.g., high concentration)

3) Effects of observation on work behavior

4) Representativeness of workplaces (sampling frequently restricted to

good/typical workplaces/hours etc)

=> Conservative estimates:

All of these four reasons lead to a decrease of the correlation between

observed stressors and stress reactions (compared with the true

correlation). Reason: The true variance of stressors is underestimated.

Overlap (Frese &

Zapf, 1

988)

ChristianDormann19




Valencia, 15.10.-17.10.2008

Objective Measurement of „Stress“ in Questionnaires

Note: Any kind of 'subjective' questionnaire report can be placed somewhere on a dimension from 'low in dependency on cognitive and emotional processing' to 'high in dependency on cognitive and emotional processing.

Examples:

Time Pressure: - How frequently do you have to skip regular breaks because of high time pressure?

+ The time pressure at work makes me feel fatigued.

Social Support: - When I have too meet tough deadlines I can count on my colleagues.

+ My colleagues are very responsive when I need help

Burnout: - Working all day is really a strain for me.+ I have headaches.

Overlap (Frese &

Zapf, 1

988)

ChristianDormann20




Valencia, 15.10.-17.10.2008

Objective Measurement of „Stress“ in Questionnaires

Overlap

ObjectiveStressor

PerceivedStessor

WellBeing

Appraisal Coping

obj./perc.Resource

Stressor R

eduction

Appra

isal

Opt

ions C

oping Options

Well Being Enhancement

ChristianDormann21




Valencia, 15.10.-17.10.2008

Measurement of „Stress“There ain’t such thing as method variance without variance in the method.

Since we usually do not vary our method of data gathering between individuals (e.g., some using questionnaires and others being observed), we better should not speak of method variance.

The things we are really talking about are psychological or psycho-physiological variables, which vary between individuals and impact on the variables as they are measured. These psychological or psycho-physiological variables have been labeled measurement-confounding variables.

When these psychological or psycho-physiological variables would impact on the variables irrespective of how they were measured, they represent congerneric confounding variables (what we usually call third variables).

Sometimes it is difficult to decide whether a confounding variable represents a measurement confounder or a congenenic confounder (e.g. negative affectivity, NA) => let’s go for the next section

Overlap

ChristianDormann22




Valencia, 15.10.-17.10.2008

Congeneric& Measurement

Confoundedness

ChristianDormann23




Valencia, 15.10.-17.10.2008

Three Questions Addressed in this Section

1) What does confounding and related terms (spuriousness, third variable-effects) mean?

2) By which mechanisms are relations among IVs and DVs confounded?

3) How to control for confounding?

Overlap

ChristianDormann24




Valencia, 15.10.-17.10.2008

Confounding (Third) Variables

Several so-called ‘third variables’ can be distinguished.

A confounding variable (CV) is a variable that meets three conditions:- Cond. 1: The CV must cause the independent variable (IV) - Cond. 2: The CV must cause the dependent variable (DV) - Cond. 3: The product of the two effects must be identical to the sign of the causal effect the IV has on the DV.

A suppressor variable (SV) is similar to a confounding variable. Condition 3 is different, however:- Cond. 3: The product of the two effects must be opposite to the sign of the causal effect the IV has on the DV.

A third variable in general terms is a variable, for which Condition 3 does not have to apply, and for which one out of the two first conditions can be relaxed:- Cond. 1/2: The CV is either correlated with the IV or with the DV.

Confoundedness

ChristianDormann25




Valencia, 15.10.-17.10.2008

Types of Third Variables

StressorStress

Reaction+

Introduction

+ +

Con-founder

-+

+-

Suppres-sor

+-+-

-++-Me-

diator

≠ 0 ≠ 0

ThirdVariable

+ +

Con-founder

Mo-derator

ChristianDormann26




Valencia, 15.10.-17.10.2008

Confounding (Third) Variabls

“One of the factors that complicates efforts to understand method effect variables is that they may have both

methodological and substantive roles"

Confoundedness (William

s, Gavin, &

William

s , 1996, p

. 88)

ChristianDormann27




Valencia, 15.10.-17.10.2008


Measurement Confounding Mechanisms:

- affect priming-model: An individual’s emotional state elicits affect-

congruent memories, which in turn influence his or her questionnaire ratings.

- affect as information-model: Individuals become aware of their current

affect as a reaction elicited by questionnaire items. They regard their affective

state as an indication about how they feel regarding the item content.

Confoundedness (cf. Brie

f, Butch

er and Ro

bertson, 19

95)

ChristianDormann28




Valencia, 15.10.-17.10.2008


Congeneric Confounding Mechanisms:

-instrumental/stressor-creation mechanism: Individuals high in NA exhibit

behavior, which leads to situations with more stressors.

-selection-mechanism: Individuals with high NA may select themselves (or are

selected) for jobs with more stressors.

Confoundedness (e.g., Sp

ector, Za

pf, Chen, &

Frese, 2

000)

ChristianDormann29




Valencia, 15.10.-17.10.2008


Congeneric Confounding Mechanisms:

- perceptual/hyper-responsivity mechanism: Individuals with high NA

perceive more stressors and more strain. NA enhances stressor perception and

strain perception.

(Strictly speaking, this mechanism applies to neither the measurement-

confounding nor the congeneric model because NA is supposed to affect the

relation between stressors and strains, but neither the constructs themselves

nor their measurements.

Instead, this model implies an interaction between “objective” stressors and

strains on the one hand and NA on the other).

Confoundedness

(Schaubroeck, G

anster, & F

ox, 1992; Spector, Z

apf, Chen, &

Frese,

2000; Watson &

Clark; 1984; W

atson & P

ennebaker, 1989)

ChristianDormann30




Valencia, 15.10.-17.10.2008

Confounding Congeneric Model

Overlap

StressorStress-reaction

+ .25

Congen. Conf.

Item

1

Item

2

Item

3

Item

1

Item

2

Item

3

+ .50 + .50

r = + .25 + .50 * .50

= + .50 *)

*) when analyzed without controlling for congeneric confounding variable.

(William

s et al., 1996)

ChristianDormann31




Valencia, 15.10.-17.10.2008

Confounding Measurement Model

Overlap


Item

1

Item

2

Item

3

Item

1

Item

2

Item

3

Meas. Conf.

+ .25

(William

s et al., 1996)

+ .50

r = + .625 *)

*) when analyzed without controlling for measurement confounding variable.

ChristianDormann32




Valencia, 15.10.-17.10.2008

Pseudo-Confoundedness

Take Care:

Beware of a logical fallacy: Even if partialling NA would reduce the stressor-

strain relations, this does not prove that NA actually confounded the

relationship.

Explanation: A common confounding variable exists (meta confounder), which

does not only affect stressors and strains, but NA (or any other supposed third

variable), too.

Mechanims:

- mood mechanism

- causality mechanism

Confoundedness

ChristianDormann33




Valencia, 15.10.-17.10.2008

Pseudo-Confoundedness

Mood Mechanism

A measurement-confounding model of mood: Mood does not only affect

measures of stressors and strains, but also measures of NA.

Therefore, partialling NA will reduce stressor-strain correlations, even though

NA is not the confounding variable.

Causality Mechanism

NA does not affect stressors; it is just the other way round.

Confoundedness

ChristianDormann34




Valencia, 15.10.-17.10.2008

Meta Confounding Measurement Model

Overlap


Item

1

Item

2

Item

3

Item

1

Item

2

Item

3

Meta Conf.

Pseudo Conf.

Item

1

Item

2

Item

3

ChristianDormann35




Valencia, 15.10.-17.10.2008

Controlling for Confounding (Third) Variables

1) Usually, explicit measurement and control of confounding variables is

required to avoid biased estimates of causal relationships.

2) Controlling for congeneric confounding variables is straightforward. Multiple

(hierarchical) regression analysis, ANCOVA, structural equation modeling

etc.

3) Controlling for measurement confounding variables is more demanding:

- avoid measurement overlap

- structural equation modeling

4) Some confounding variables can be explicitly (or are implicitly) controlled

when change is analyzed

=> let’s go for the (quite complex) next section

Overlap

ChristianDormann36




Valencia, 15.10.-17.10.2008

Interindividual& Intraindividual

Change

Change

ChristianDormann37




Valencia, 15.10.-17.10.2008

Three Questions Addressed in this Section

1) Should Interindividual Differences be used to substitute for Intraindividual Change?

2) Should difference-scores be used as measures of change?

3) Could change be used to control for confounding variables?

Change

ChristianDormann38




Valencia, 15.10.-17.10.2008

Cross-sectional analysis is based on some doubtful assumptions:

All men were created equal and any difference is due to change.

All men once had identical job conditions, and any difference …

All men once suffered from identical complaints, and any difference …

From this it follows:

1) Any current value of a variable can be interpreted as a change-score.

2) The relationship between two variables informs about how change in one variable caused a change in the other variable.

When the assumptions do not apply, then:

Potential confounders are not controlled appropriately.

Appropriately addressing change may help controlling confounders.

Change

Why is it worth to look at change?

ChristianDormann39




Valencia, 15.10.-17.10.2008

The Seven Commandments of Animal Farm

1) Whatever goes upon two legs is an enemy.

2) Whatever goes upon four legs, or has wings, is a fried

3) No animal shall wear clothes.

Change

ChristianDormann40




Valencia, 15.10.-17.10.2008


4) No animal shall sleep in a bed.

5) No animal shall drink alcohol.

6) No animal shall kill another animal.

Change

ChristianDormann41




Valencia, 15.10.-17.10.2008


7) All animals are created equal.

Change

ChristianDormann42




Valencia, 15.10.-17.10.2008

The Seven Commandments of Panel Farm

1) Whatever goes upon two legs is an enemy. 1) Whatever goes not beyond one measurement occasion is an enemy.

2) Whatever goes upon four legs, or has wings, is a fried2) Whatever goes beyond one measurement occasion, or is an experiment, is a friend.

3) No animal shall wear clothes. 3) No student shall get weary of checking results closely.

Change

ChristianDormann43




Valencia, 15.10.-17.10.2008


4) No animal shall sleep in a bed. 4) No student shall sleep at all

5) No animal shall drink alcohol. 5) No student shall drink alcohol

6) No animal shall kill another animal.6) No student shall kill time

- except he/she has to cope with data from more than one measurement occasion.

- except has to wait for Time 2 to come (will be soon).

- except he/she is sleeping over his/her decision to stop doing panel analysis.

Change

ChristianDormann44




Valencia, 15.10.-17.10.2008


7) All animals are created equal

7) All students are created equal

but some are more equal than others.

but some students are more equal than others

and only those, who are more equal than others are allowed to use difference scores.

Change

ChristianDormann45




Valencia, 15.10.-17.10.2008

95

97

99

101

103

105

94 96 98 100 102 104 106

1

2

3

Intraindividual Change vs. Interindividual Differences

Year 0

Change

ChristianDormann46




Valencia, 15.10.-17.10.2008


95

97

99

101

103

105

94 96 98 100 102 104 106

1

2

3

Year 1

Change

ChristianDormann47




Valencia, 15.10.-17.10.2008


95

97

99

101

103

105

94 96 98 100 102 104 106

1

2

3

Year 2

Change

ChristianDormann48




Valencia, 15.10.-17.10.2008


Year 3

95

97

99

101

103

105

94 96 98 100 102 104 106

1

2

3

Change

ChristianDormann49




Valencia, 15.10.-17.10.2008


Year 4

95

97

99

101

103

105

94 96 98 100 102 104 106

1

2

3

Change

ChristianDormann50




Valencia, 15.10.-17.10.2008


Year 5

95

97

99

101

103

105

94 96 98 100 102 104 106

1

2

3

Change

ChristianDormann51




Valencia, 15.10.-17.10.2008


Year 6

95

97

99

101

103

105

94 96 98 100 102 104 106

1

2

3

Change

ChristianDormann52




Valencia, 15.10.-17.10.2008


Year 7

95

97

99

101

103

105

94 96 98 100 102 104 106

1

2

3

Change

ChristianDormann53




Valencia, 15.10.-17.10.2008


Year 8

95

97

99

101

103

105

94 96 98 100 102 104 106

1

2

3

Change

ChristianDormann54




Valencia, 15.10.-17.10.2008


Year 9

95

97

99

101

103

105

94 96 98 100 102 104 106

1

2

3

Change

ChristianDormann55




Valencia, 15.10.-17.10.2008


Year 10

r =-.94

r =-.92

r =-.95

r = .17

95

97

99

101

103

105

94 96 98 100 102 104 106

1

2

3

Change

ChristianDormann56




Valencia, 15.10.-17.10.2008


95

97

99

101

103

105

94 96 98 100 102 104 106

1

2

3

Change

ChristianDormann57




Valencia, 15.10.-17.10.2008


95

97

99

101

103

105

94 96 98 100 102 104 106

1

2

3

r = 1.00

Change

ChristianDormann58




Valencia, 15.10.-17.10.2008


Year 10

r =-.94

r = .95

r =-.95

r = .38

95

97

99

101

103

105

94 96 98 100 102 104 106

1

2

3

Change

ChristianDormann59




Valencia, 15.10.-17.10.2008


95

97

99

101

103

105

94 96 98 100 102 104 106

1

2

3

Change

ChristianDormann60




Valencia, 15.10.-17.10.2008


95

97

99

101

103

105

94 96 98 100 102 104 106

1

2

3

r = 1.00

Change

ChristianDormann61




Valencia, 15.10.-17.10.2008


(A) Differences (mean levels) between subjects: cross-sectional study (T1) + allows controlling for :

- nothing=> see previous slide after data points were collapsed

(B) (Causal) effects among variables: panel study (T1-T2) + allows controlling for :

- differences (mean levels) in T1-variables between subjects => see next slide

(C) Functional differences between subjects: multiple time series (T1-Tn) + allows controlling for :

- differences in variables (mean levels) between subjects - differences in effects among variables between subjects

-> see previous slide before data points were collapsed

Change

ChristianDormann62




Valencia, 15.10.-17.10.2008


An idea previously proposed by many writers:

When we control for prior differences (T1) in a variable that we also measured at T2, then we control for those variables that caused these differences, too

For example, when we control for pre-job differences in current health complaints, then we also control for personality factors such as complaining tendencies that may cause complaining.Any relation between health complaints and stressors cannot be confounded by such personality factors if pre-job differences in current health complaints are controlled.

=> Open question: How to control for prior differences in a variable measured at T2 when the same variable was measured at T1?

=> What about the difference between T1 and T2? Let‘s have a look on the next slides …

Change

ChristianDormann63




Valencia, 15.10.-17.10.2008


9596979899

100101102103104105

94 96 98 100 102 104 106

1

2

3

r = -.96

rT1 = .40rT2 = .69rT1 = .40rT2 = .69

Change

0 1 2 3 654

-3

-2

-1

0

1

2

ChristianDormann64




Valencia, 15.10.-17.10.2008

Why not change scores?

The previous slide demonstrated that the relation (correlation) between change scores may give a much better impression of real causal processes than the relation of cross-sectionally obtained variables does give.

However, change scores usually are a biased (i.e., not true) representation of the actual change that occurred.

The next (complex) slides aim at:1. Setting up a simple regression model that has certain properties,

which we would expect most psychological systems to have.2. then re-arranging the equations to show that change should not be

expressed as, e.g. Health2 – Health1. Instead, Health2 – .70*Health1 (or any value >0 and <1) is better.

The final slides of this section then aim at demonstrating, how (and how not), repeated measures could be used to control for confounding variables.

Change

ChristianDormann65




Valencia, 15.10.-17.10.2008

Change: Should we use difference scores?

Control T1 Control T21.00

-> perfect test-retest correlation: rtt = 1.0

-> invariant variances: Var(Con T1) = Var(Con T2)

=>> no change-model

(unlikely for many biological and social systems: constant change)

Change

ChristianDormann66




Valencia, 15.10.-17.10.2008


-> lower test-retest correlation: rtt < 1.0

-> increasing variances: Var(Con T1) < Var(Con T2)

=>> esaclation model (unlikely for many biological and social systems: stationary systems, regression-to-the-mean)

Control T1 Control T2

1.00

Change

ChristianDormann67




Valencia, 15.10.-17.10.2008


-> lower test-retest correlation: rtt < 1.0

-> invariant variances: Var(Con T1) = Var(Con T2)

=>> stationarity model

(likely for many biological and social systems: However, how to give meaning to the coefficients?)


1.00

+

-

Change

ChristianDormann68




Valencia, 15.10.-17.10.2008


Rules of Path Analysis:

- There are several rules. We use the tracing rule.

- Tracing rule: to find rxy, take the sum of the product of all the paths obtained from each possible tracing between x and y (ignore paths along variables that are entered and left thorough arrowheads)

-> r = -.30 * 1.00 + 1.00 = .70 (does this help to assign meaning?)


1.00

Change

ChristianDormann69




Valencia, 15.10.-17.10.2008


Lagged Regression (1/3):

- Control T2 = .70 * Control T1 + (Standard type of equation)


.70

Change

ChristianDormann70




Valencia, 15.10.-17.10.2008


Lagged Regression (2/3):Non-standard way of writing the equation. Since:

Control T2 = .70 * Control T1 + | remove (- .70 * Control T1) on both sides

Control T2 - .70 * Control T1 = (Difference/Change in Control = )

-> reflects the actual change. It is always there in regression models.


.70

Change

ChristianDormann71




Valencia, 15.10.-17.10.2008


Lagged Regression (3/3):

Non-standard way of writing the equation. Since:

Control T2 = .70 * Control T1 +

Subtracting (1.0 * Control T1) on both sides of the equation leads to:

Control T2 - 1.0 * Control T1 = .70 * Control T1 – 1.0 * Control T1 +

Control T2 - Control T1 = -.30 * Control T1 +

Thus, when it is aimed at analyzing (causes or consequences of) the non-weighted difference score Control T2 – Control T1, Control T1 has to be used as a predictor because the equation would be not correctly specified otherwise.

ChristianDormann72




Valencia, 15.10.-17.10.2008


Control T2 - Control T1 = -.30 * Control T1 +

Okay, perhaps this is unnecessarily complex, but why shouldn‘t we use difference scores?

You could, but you should knowhow (see above)

and when (see next slides)

Change

ChristianDormann73




Valencia, 15.10.-17.10.2008

Intermediate Summary

- Interindividual differences may or may not substitute for intraindividual changes. It may lead to correct estimates, but it sometimes does not.

- When X(T1) is available, it should be included as a predictor of X(T2), and the regression weight should be empirically estimated.

- Omission of a regression path between X(T2) and X(T1) assumes the regression weight to be .00, which is almost always wrong.

- Using difference scores X(T2) – X(T1) assumes the regression weight to be 1.00, which is almost always wrong.

Change

ChristianDormann74




Valencia, 15.10.-17.10.2008

Using change to control for unknown confounding variables

Remember: The primary aim to analyze change was to overcome the assumption of cross-sectional analysis: All individuals/job conditions/health problems were equal once. All differences are due to change.

The aims of the next three modeling examples are:1. Demonstrating how (and how not) an unmeasured, stable confounding variable (e.g., gender, SES) can be controlled when follow-up measures are available.2. Demonstrating, how unmeasured third variables that may change somehow over time can be ruled out as confounders3. Demonstrating, how unmeasured third variables that may change completely over time can be controlled as confounders

Change

ChristianDormann75




Valencia, 15.10.-17.10.2008


H2-H1 = c (C2-C1)

+ s2 H1

+ 2 - 1

1)Good: The difference H2-H1 does

not depend on “something unknown”

2)Bad: H1 (independent Variable) and

1 (part of error term) are correlated.

This violates standard assumption of

regression analysis.

=> Do not use difference

scores!

Change

Health T1 Health T2

Control T1

Control T2

somethingunknown

s

c c

21

b b

Can be modeled

directly using

Structural Equation

Programs

ChristianDormann76




Valencia, 15.10.-17.10.2008

What does the CLPC-Technique really tells us?

Proportional Stationarity1)Assumption: The effects of the TV

(something unknown) remains

“relatively” (= proportional) stable:

Change

Health T1 Health T2

Control T1

Control T2

somethingunknown

+.30

+.10

somethingunknown

b e

a d

c

3) The path from Health T1 to

Control T2 implies:

rH1 C2 = b * c * d

a d_ _=b e

2) Solving for d yields: d = a * e / b

d = a * e / b

= c * a * e= b * c * a * e / b

4) The path from Control T1 to

Health T2 implies:

rC1 H2 = a * c * eec *

=a *

Conculsion: When the 2 cross-lagged correlations are equal, control and health might be spuriously related (i.e. confounded by a proportional statonary TV).

When the 2 cross-lagged correlations are different, there might be a causal relations.There is, however, no indication of causal predominance or something alike!

ChristianDormann77




Valencia, 15.10.-17.10.2008

How to control for unmeasured confounding variables?

Change

IV1

DV1 DV2

IV1 IV2

DV1 DV2

Allprevious

TVoccasion

factor

ChristianDormann78




Valencia, 15.10.-17.10.2008


Change

IV1

DV1 DV2

IV1

DV1

IV2

DV2

ChristianDormann79




Valencia, 15.10.-17.10.2008

Intermediate Summary

- Interindividual differences may or may not substitute intraindividual changes. It may lead to correct estimates, but it sometimes does not.

- Hierarchical Linear Models (HLM) are useful for analyzing intraindividual change, but they have shortcomings, too (all points below)

- Using difference scores X(T2) – X(T1) to analyze change assumes the regression weight to be 1.00, which is almost always wrong.

- Omission of a regression path between X(T2) and X(T1) assumes the regression weight to be .00, which is almost always wrong.

- Using Difference scores X(T2) – X(T1) as DV and IV eliminates unmeasured (stationary) background factors, but usually it leads to biased estimates because regression assumptions are violated.

Change

ChristianDormann80




Valencia, 15.10.-17.10.2008


1) 1st Rule: A panel design is required with at least 2 variables measured at least twice (2v2w-Design)

2) 2nd Rule: Do a CLPC-Check. If it fails, stop.

3) 3rd Rule: Use Structural Equation Modeling (SEM). It allows to comply with all other rules.

4) 4th Rule: Within Time 1, do not model causal relations among variables. SEM tempt us doing so, regression analysis does not).=> Prior (unknown) confounding variables and the history of (unknown) mutual influences among variables are controlled!!!!

5) 5th Rule: Within Time 2, do model occasion factors. Occasion factors are transient (i.e., highly fluctuating) confounding variables such as mood. They are the Achilles’ heel of cross-sectional designs, where they have to be explicitly measured in order to control them. This is not necessary in longitudinal designs.

Change

ChristianDormann81




Valencia, 15.10.-17.10.2008

Reversed& ReciprocalDirectedness

ChristianDormann82




Valencia, 15.10.-17.10.2008

Two Questions Addressed in this Section

1) What is reversed and reciprocal causation?

2) How should we control for it?

Change

ChristianDormann83




Valencia, 15.10.-17.10.2008

How What is reversed and reciprocal causation?

Change

IV1

DV1 DV2

IV1 IV2

DV1 DV2

ChristianDormann84




Valencia, 15.10.-17.10.2008

Substantive reasons for reversed causaltion

Drift hypothesis: Workers with poor health drift to better jobs.

Refugee hypothesis: Workers under strain seek better job conditions.

Zapf et al. (1996) identified only 16 studies on organizational stress.

which tested for reversed causation.

In six studies, evidence for reversed causation emerged.

Overall, evidence suggests that reversed causation is as likely as effects

of stressors/resources on strain. Hence, reciprocal causation is

probably the most plausible model.

Time

(Frese &

Zapf, 1988; Jam

es & T

etrick, 1986; Kohn &

Schooler, 1983; Leventhal

& T

omarken, 1987; O

’Brien, 1986; S

pector, 1992; Zapf et al., 1996

ChristianDormann85




Valencia, 15.10.-17.10.2008

How What is reversed and reciprocal causation?

Change

IV1

DV1 DV2

IV1 IV2

DV1 DV2

IV3

DV3

a ab b

ChristianDormann86




Valencia, 15.10.-17.10.2008

Summary of reversed and reciprocal causation issues

Reversed Causation is probably more common than usually expected.

Individuals under strain may subsequently perceive more stressors and

less resources (measurement reversed causation) or may actually

become exposed to more stressors and less resources (congeneric

reversed causation).

Individuals with good health subsequently may perceive conditions better

than they are or they may get better conditions.

Time

ChristianDormann87




Valencia, 15.10.-17.10.2008

How to control for reversed and reciprocal causation?

With regression analysis: Test an opposite model. When evidence for

hypothesized as well as reversed causation emerges, discuss both

option in the Discussion Section.

With SEM: Include both hypothesized as well as reversed causation in

some models (in those models where it is possible). Interpret your

findings.

Do NOT worry when evidence for reversed but not for hypothesized

causation emerges. Perhaps this is what is actually happening.

Time

ChristianDormann88




Valencia, 15.10.-17.10.2008

TimeIt’s

Time for a

Change

ChristianDormann89




Valencia, 15.10.-17.10.2008

Two Question Addressed in this Section

1) How long is the optimal time lag in between two measurement occasions?

2) Why is an optimal time lag optimal?

Change

ChristianDormann90




Valencia, 15.10.-17.10.2008

Why longitudinal studies?

A1. Demonstrate cause-effects relations (causality)

A2. Random assignment to IV not possible

A3. Manipulation of IV not possible

A4. Control for unmeasured confounding variables

B1. Demonstrating nonlinear effects (thresholds, phases, disruptions )

B2. Demonstrating age-related effects (growth curves, phases)

C1. Demonstrating sustainability (long term effects)

Time

ChristianDormann91




Valencia, 15.10.-17.10.2008

What is Sustainability?

One of researchers’ great desire …

… that we’d better put aside.

(Dormann, today)

Time

Development sustainability: The continuation of benefits after major assistance from the donor has been completed.

(Australian Agency for International Development 2000).

ChristianDormann92




Valencia, 15.10.-17.10.2008

Example: Effectiveness of Consultancy Sustainability = Long Term Effect?

0

1

2

3

4

5

6

7

8

9

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 520

1

2

3

4

5

6

7

8

9

10

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52

Time

ChristianDormann93




Valencia, 15.10.-17.10.2008

With impoverished theory about issues such as - when events occur, - when they change, - or how quickly they change, the empirical researcher is in a quandary. Decisions about - when to measure and - how frequently to measure critical variables are left to - intuition, - chance, - convenience, - or tradition.

None of these are particularly reliable guides ….

The ProblemWith impoverished theory about issues such as - when events occur, - when they change, - or how quickly they change, the empirical researcher is in a quandary. Decisions about - when to measure and - how frequently to measure critical variables are left to - intuition, - chance, - convenience, - or tradition. - impressions of a good ‘Gestalt’.

None of these are particularly reliable guides ….

Time (Mitchell &

James, 2001, p. 533)

ChristianDormann94




Valencia, 15.10.-17.10.2008

not men

tioned

Frequency of Different Time Lags in (almost) all Panel Studies on

Occupational Stress

7-17

3 m

onth

s

6 m

onth

s

1 ye

ar

2 ye

ars

3 ye

ars

4 ye

ars

5 ye

ars

7 ye

ars

10 y

ears

1 da

y

Note: 247 studies; time lags as reported in the abstracts.

Time

ChristianDormann95




Valencia, 15.10.-17.10.2008

Reasons for Choosing a Particular Time Lag

1. Construct: - change (e.g. change of occupation, mortality)- relative time (e.g. yearly incidence rate)- absolute time (e.g. daily work hours, blue monday)

2. Mechanisms:- strength of effects (e.g., weaker effects require longer time to unfold)

3. Operationalization:- relative time (e.g. counting the frequency of events)- absolute time (e.g. interviews following certain events)

4. Method: - control of seasonal fluctuations (e.g., in workload)- survey feedback research (e.g., feedback phase necessary)- replication study

- demonstration of sustainability (e.g., work re-design on long-term satisfaction)

5. Sample:- long lags increase panel mortality (e.g., participants move)- short lags are too demanding (e.g., 10 diaries within 2 weeks)- participants only accessible at particular points in time (e.g., rotating shifts)

Time

ChristianDormann96




Valencia, 15.10.-17.10.2008

Astonishing IssuesTime lags seem to be relatively unimportant

Time lags seem to be unbelievably flexible- e.g. job satisfaction: - laboratory experiments using 30min time lags; - passive field studies using 30year time lags; ~ 500.000 times longer)

In panel analyses, synchronous time-2-effects emerge frequently - common explanation: Evidence for short-term „true“ effects, that is, effects

taking less time than 50% of the time lag tested

Effect sizes smaller compared to experimental studies - common explanation: Lack of experimentally-induced maximization of variance

IV1

DV1

IV2

DV2

IV1

DV1 DV2

27

- Dormann’s explanation: Panel-induced minimization of variance

Time

ChristianDormann97




Valencia, 15.10.-17.10.2008

Types of Effects

(1) Linear Reaction Model

(2) Accumulation Model

(3) Dynamic Accumulation Model

(4) Adjustment Model

(5) Sleeper Effect Model

27

(Frese &

Zapf, 1

988)

(6) Constant Reaction Model

(7) Constant Accumulation Model

=> When is a strong effect of IV Time 1 on DV Time 2 observed?

Time

ChristianDormann98




Valencia, 15.10.-17.10.2008

Types of Variables and Types of Effects

IV: changing

Effect: linear

IV

DV

Linear Reaction Model(IV onset = -1 * IV offset)

Maximum lagged effect with an optimal time lag!

Time

ChristianDormann99




Valencia, 15.10.-17.10.2008


IV: changing

Effect:

cumulative

IV

DV

Accumulation Model(IV onset <> IV offset = 0)

Maximum lagged effect with a long time lag

?No! Remember consultancy example.

Time

ChristianDormann100




Valencia, 15.10.-17.10.2008


IV: changing

Effect:

cumulative

IV

DV

Dynamic Accumulation Model(IV onset <> IV offset > 0)

Maximum lagged effect with a long time lag

Time

ChristianDormann101




Valencia, 15.10.-17.10.2008


IV: changing

Effect: linear

IV

DV

Adjustment Model(IV onset 1 <> IV onset 2)

Maximum lagged effect with an optimal time lagand optimal time point

Time

ChristianDormann102




Valencia, 15.10.-17.10.2008


IV: changing

Effect: linear

IV

DV

Sleeper Effect Model(= Linear Reaction Model

+ lagged effect)

Maximum lagged effect with an optimal time lag!

Time

ChristianDormann103




Valencia, 15.10.-17.10.2008


IV: stable

Effect: linear

IV

DV

Constant Reaction Model(= Linear Reaction Model

+ stable IV)

Maximum lagged effect with a long time lag?

No! Time lags don‘t matter. A time lag is not even required

Time

ChristianDormann104




Valencia, 15.10.-17.10.2008


IV: stable

Effect:

cumulative

IV

DV

Constant Accumulation Model(= Accumulation Model

+ stable IV)

Again, a time lag is not required. Measure IV & DV late!

Time

ChristianDormann105




Valencia, 15.10.-17.10.2008


Choosing optimal time lags is important in a variety of causal models:

- linear reaction models: YES

- accumulation models: YES

- dynamic accumulation models: no

- adjustment models: YES + optimal time point

- sleeper effects models: YES

- constant reaction models: no – a time lag is not required

- constant accumulation models: no – a time lag is not required

What is an optimal time lag?

Time

ChristianDormann106




Valencia, 15.10.-17.10.2008

A ‘True’ Causal Process

IV0

DV0

IV1

DV1

IV2

DV2

IV3

DV3

Time

ChristianDormann107




Valencia, 15.10.-17.10.2008

Panel Analysis of a Causal Process

IV0

DV0

IV2

DV2

IV3IV1

DV1 DV3

i i

d d

e e e

b

b d w– i w

d – i

= e * d w+ e * i *

IV1

DV1 DV3

What is the size of b?

The optimal Time Lag (Number of ) is defined as the Time Lag, that yields the maximum effect b of the IV on the DV when the data are analyzed.

Time

ChristianDormann108




Valencia, 15.10.-17.10.2008

When does b reach ist maximum? When the Time Lag is optimally chosen!

=> = – ( )ln

d ln(d)

i ln(i)

di( )ln

The size of the true causal effect e

is irrelevant!

… some math wizard solves the problem …

Time

ChristianDormann109




Valencia, 15.10.-17.10.2008

Example 1: i = .990, d =.995; across 1 Day

= 137.01 Days

= 19.58 Weeks

= 4.57 Months

= – ( )ln

d ln(d)

i ln(i)

di( )ln

27

Time

ChristianDormann110




Valencia, 15.10.-17.10.2008

Example 2: i =.98, d = .97; across 1 Day

= 39.03 Days

= 5.58 Weeks

= 1.30 Months

= – ( )ln

d ln(d)

i ln(i)

d i( )ln

Time

ChristianDormann111




Valencia, 15.10.-17.10.2008

Effect Sizes as a Function of the Time Lag

IV1

DV1 DV2

IV1

DV1

IV2

DV2

i = .990, d = .995, e = .005i = .980, d = .970, e = .050i = .900, d = .900, e = .100

Time

ChristianDormann112




Valencia, 15.10.-17.10.2008

What is Stabiliy?

1. Stability = Effect of a Variable on itself across time

2. Stability > 1: Rarely

3. Stability < 1: Usually (mainly reflecting a regression-to-the-mean)

4. Test-Retest-Correlation

= Stability

+ Effects of Confounding Third Variables

– Measurement Error

.40 .40

.50

.19 .19

5. Stability = .50

6. Retest = .54

7. dissat. Retest = .66

iV0 IV1

TV

IV0 IV0

Time

ChristianDormann113




Valencia, 15.10.-17.10.2008

Summary

Choosing optimal time lags is of utmost importance for effect sizes.

Do not mix up

- sustainable effects in real life (sustainable WOP is possible) vs.

- cross-lagged long-term effects in panel studies (almost impossible)

For sustainable (i.e., strong) effects we need

- optimal (i.e., usually short) time lags

- or extremely stable variables

Time

ChristianDormann114




Valencia, 15.10.-17.10.2008

Implications

Stabilities of IV & DV determine the optimal lag. - We need meta-analyses of stabilities of important WOP variables!

Use quasi-trait measures less frequently(„How do you think in general about …?“)

Use state measures more frequently(„ How do you think right now about …?“)

Do more ‘shortitudinal’ research- Diary studies- Event sampling studies- Combining observational & questionnaire studies

Time

ChristianDormann115




Valencia, 15.10.-17.10.2008

Overall Summary – Christian‘s Personal Opinion

Follow-up studies are much better in controlling for all kinds of

unmeasured confounding variables.

The less confounding is built-in the concepts and measures, the more

good follow-up research approximates experiments.

Shortiduinal designs covering not more than two weeks are most

appropriate.

Dormann‘s Dear Desire: The Dublet Design:

- 2 week gap using measures focusing on the last 2 weeks

- in between: diary design using measures focusing on the immediate

moment (here and now)

Time

ChristianDormann116




Valencia, 15.10.-17.10.2008

ChristianDormann117




Valencia, 15.10.-17.10.2008

Additional Slides

ChristianDormann118




Valencia, 15.10.-17.10.2008

Further Implications

Most Studies Apply Designs with Time Lags that are much too long

Example Job Satisfaction:

- rtt = .96 (1 month; short-lagged panel studies considered in the

in the meta-analysis by Dormann & Zapf, 2001)

- minus 25% personality dispositions

- Estimated Stability ~ .71

- Assumptions: + IV (e.g. job characteristics) and DV similarly stable+ Job satisfaction may change from day to day

=> Optimal Lag: 12.6 weeks = 2.95 months

Time

ChristianDormann119




Valencia, 15.10.-17.10.2008

True Cross-lagged Effects

IV0

DV0

IV1

DV1

IV2

DV2

IV3

DV3

IV4

DV4

IV5

DV5

IV6

DV6

IV7

DV7

Time

ChristianDormann120




Valencia, 15.10.-17.10.2008

How Theory and Statistics Combine

= – ( )ln

d ln(d)

i ln(i)

d i( )ln

Statistics

Theory

Minimal Lag: Time Lag to be derived from theoretical Considerations, which is required for a change in the IV to trigger a change in the DV.

– Optimizing Lag: Time Lag to be derived from the stabilities of the IV and the DV in order for maximal effects to emerge from analysis.

Time

ChristianDormann121




Valencia, 15.10.-17.10.2008

Sexual Intercourse & Birth of a Baby

Assumption:

1) 1-week Stability of the IV: .90 (Christian Dormann‘s estimate)

2) 1-week stability of the DV: .80 (Britannica, 2004).

3) = 9 months (40 weeks, 280 days)

The appropriate equation is:

40 + ln( (.90 * ln(.90) ) / ( .80 * ln(.80) ) )/ ln(.90/.80) = 45.37.

Therefore, the optimal occasion (for the 2nd measurement) is

45.37 weeks (~ 318 Days; ~ 10.59 months) after the first.

Time

ChristianDormann122




Valencia, 15.10.-17.10.2008

Computer

Delta (d) 0,000

Stability of the IV: 0,900Stability of the DV: 0,950

if stabilities are identical: #ZAHL! if stabilities are different: 12,314

Time

christian dormann 1 substantive and methodological problems in the study of stress at work european...

Documents

study of stress

organizational stress

christian dormann

substantive considerations

central aim of work

wo stress research

stress experimental

circumventing problems