Experimental Design

Tali Sharot & Christian Kaul

With slides taken from presentations by: Tor Wager

Christian Ruff

Part I

• Taxonomy of experimental design (Friston ’97)• Aim of design• Block Design• Event Related Design• Baseline / Control• Timing

General Taxonomy of Experimental Design(Friston, 1997)

• Categorical – activation in one task as compared to that in another task– categorical designs assume that the cognitive processes can be

dissected into sub-cognitive processes & that one can add or remove cognitive processes by “Pure insertion”.

• Factorial– Factorial designs involve combining two or more factors within a

task and looking at the effect of one factor on the response to other factor

• Parametric – systematic changes in the brain responses according to some

performance attributes of task can be investigated in parametric designs

Categorical experimental design

– Subtraction • Pure insertion: assumption that one can add or remove

cognitive processes without influencing others. • activation in one task as compared to that in another task

considering the fact that the neural structures supporting cognitive and behavioural processes combine in a simple additive manner

– Conjunction • Testing multiple hypotheses• several hypotheses are tested, asking whether all the

activations in a series of task pairs, are jointly significant

Factorial design - example

• A – Low attentional load, motion• B – Low attentional load, no motion• C – High attentional load, motion• D – High attentional load, no motion

A B

C D

LOW

LOAD

HIGH

MOTION NO MOTIONLoad task Rees, Frith & Lavie (1997)

Terminology

• Simple main effects

• Main effects

• Interaction terms

A B

C D

LOW

LOAD

HIGH

MOTION NO MOTION

SIMPLE MAIN EFFECTS

• A – B: Simple main effect of motion (vs. no motion) in the context of low load

• B – D: Simple main effect of low load (vs. high load) in the context of no motion

• D – C: ?• Simple main effect of no

motion (vs. motion) in the context of high load

A B

C D

LOW

LOAD

HIGH

MOTION NO MOTION

OR

The inverse simple main effect of motion (vs. no motion) in the Context of high load

MAIN EFFECTS

• (A + B) – (C + D): • the main effect of low load

(vs. high load) irrelevant of motion

Main effect of load

• (A + C) – (B + D): ?• The main effect of motion (vs.

no motion) irrelevant of load Main effect of motion

A B

C D

LOW

LOAD

HIGH

MOTION NO MOTION

INTERACTION TERMS

• (A - B) – (C - D): • the interaction effect of

motion (vs. no motion) greater under low (vs. high) load

• (B - A) – (D - C): ?• the interaction effect of no

motion (vs. motion) greater under low (vs. high) load

A B

C D

LOW

LOAD

HIGH

MOTION NO MOTION

Factorial design in SPM5

A B

C D

LOW

LOAD

HIGH

MOTION NO MOTION

• How do we enter these effects in SPM5?

• Simple main effect of motion in the context of low load:

• A vs. B or (A – B)A B C D

[1 -1 0 0]

Factorial design in SPM5

• Main effect of low load: • (A + B) – (C + D)

• Interaction term of motion greater under low load:

• (A – B) – (C – D)

A B C D

A B C D

[1 -1 -1 1]

[1 1 -1 -1]

Factorial design in SPM5

• Interaction term of motion greater under low load:

• (A – B) – (C – D) A B C D

Parametric experimental design

• What do we want to measure? systematic changes in the brain responses

according to some performance attributes of task can be investigated in parametric designs:

Part I

• Aim of design • Block Design• Event Related Design• Baseline / Control• Timing

What we want from a design

• Interpretability: Can I relate brain data to specific psychological events?– Memory retrieval and comparison processes associated with recognition

• Power: Can I detect results?

Experimental (A) - Control (B) Experimental (A) - Control (B)

Block Design• Similar events are grouped

….

pros • Avoid rapid task-switching (patients)• Fast and easy to run • Good signal to noise

Block design - some pros & cons

cons

• Expectation (cognitive set, attention, fatigue)

• Habituation (olfactory, emotional)

• Different trials according to subjects’ responses.

Event Related Design

• Events are mixed

• Encode:

Event Related Design

Recognition Test:

……..

Response: new old old new old

Category: CR HIT HIT MISS FA

Baseline?

Known: Queen! Unknown? Aunt Jenny?

• Different stimuli & Task

Queen! Female!?

• Same stimuli, different task

• Different stimuli similar Task

+

Queen! i - pod!Queen! Mmm..Whats for dinner?...

• Similar stimuli, same task

“Baseline” here corresponds to session mean

“Cognitive” interpretation hardly possible, but useful to define regions generally involved in the task

Baseline?

Timing : the long and the short of it

3 s picture viewing3 s picture viewing

Recognition: 250 msRecognition: 250 ms “What was he in?” ““I used to wear Batman I used to wear Batman PJs…”PJs…”

Timing : the long and the short of it

Autobiographical Memory RetrievalAutobiographical Memory Retrieval

Word RecognitionWord Recognition

“Friend”

Search Episodic Retrieval &ElaborationSearch Episodic Retrieval &Elaboration

~5sc ~14sc

Response

Part II

• ROIs in early visual cortex• Multivariate decoding

• Natural viewing• Individual differences

ROI – Regions of interest

• A) anatomically defined

• B) functionally defined

What are ROIs in early visual cortex?

Stimuli 3D representation Flatmesh

How do we get ROIs?

+

SPMmip

[6.4

2772

, -62

.907

, -24

.589

]

<

< <

SPM{T435

}

on>off

SPMresults: .\model smoothedHeight threshold T = 4.818653 {p<1e-006 (unc.)}Extent threshold k = 0 voxels

Design matrix5 10 15 20 25 30

50

100

150

200

250

300

350

400

450

contrast(s)

1

0

1

2

3

4

5

6

Meridian map

50 100 150 200 250

50

100

150

200

250

V1v

V2v

V1d

V2d

=

Meridian map

50 100 150 200 250

50

100

150

200

250

V1v

V2v

V1d

V2d

Meridian map

50 100 150 200 250

50

100

150

200

250

V1v

V2v

V1d

V2d

right hemisphere dot localiser retinotopic location of dot

exemplar ROI Result

• Extracting activity-values from ROIs for all conditions.

• Then compute interaction term for activity in V5/MT greater under motion (vs. no motion) under high versus low load

• (replication: Rees et al. ’97)

cont

rast

bra

in a

ctiv

ity

ROI

Load task

Multivariate pattern analysis?

• What is MVPA?

• Methodology in which an algorithm is trained to tell two or more conditions from each other.

• The algorithm is then presented with a new set of data and categorises/classifies it into the conditions previously learned.

What questions can (& cannot) be answered with multivariate pattern analysis?

• When conventional analysis is not feasible, multivariate analysis might be an option

but what are we actually measuring?

• Assumption:

• Feature sensitive information is present in BOLD signal

Haynes & Rees (2006)

Haynes & Rees (2005)

Mean

signal

LDA

What questions can (& cannot) be answered with multivariate pattern analysis?

• Feature sensitive information is present in BOLD signal (biased competition)

• Multivariate decoding extracts this info

• Thus feature selective processing promises new insights (i.e. towards a better understanding of neuronal population coding contained in the BOLD signal)

Haynes & Rees (2006)

Haynes & Rees (2005)

Mean

signal

LDA

Multivariate pattern analysis – how to design an experiment

• Other then with conventional analysis we are asking a different question:

Does the pattern of activity contain meaningful information we can extract?

Not the level of brain activity is addressed, but the pattern of information within the activity.

Experiment 1 Question:

• Does feature selective information (left vs. right tilted orientation as measured by decoding from BOLD signal) for the irrelevant annulus change between the two central load conditions?

• Prediction (by load theory): – Feature selective information will be

reduced in high load condition

Multivariate Decodingexample Result:

Number of voxels

% c

orre

ct d

ecod

ed

50 1001

Low

High

N voxels

Acc

urac

y

Result: Feature selective info present and decoded

expected actual

The reverse-correlation method

Hasson et al., (2004)

Individual Differences

Individual Differences

Post – Scanning Questionnaires/ Tests …

Select subjects that vary on a specific dimension

Thank you…

Between- Subject Correlation

Hasson et al., (2004)

Kahn et al., (2004)

Example: Subsequent Memory

Kahn et al., (2004)