Lecture 2What is a Theoretical Contribution?

RESEARCH METHODSDOCTORAL PROGRAM, NATIONAL RESEARCH

UNIVERSITYHIGHER SCHOOL OF ECONOMICS

DR C S LEONARD

JUNE 2011

RESEARCH METHODS

2 OUTLINE

What is a theoretical contribution?

What is a concept? A variable is a concept

Validity External, Internal

8/6/2011

RESEARCH METHODS 3

THEORY

Publishing

(Evaluative)

Criteria

8/6/2011

RESEARCH METHODS

4PRACTICAL

GUIDELINES

Choose a hypothesis important in literature but for which no systematic study exists—decide in favour or against the hypothesis

Choose a theory you suspect is false and investigate if it is, indeed, and what alternate might replace it

Design research to illuminate unquestioned assumptions in the literature

Argue that a topic of importance has been overlooked

Show relation of theory from one literature to separate problem from another

8/6/2011

RESEARCH METHODS

5REVISION OF FORMAL

THEORIES?

More than just adding a variable

Showing how it changes perception of the problem

Producing data that are inconsistent with a theory

8/6/2011

RESEARCH METHODS

6BORROW FROM OTHER

DISCIPLINES?

Shed new light on old theories Investigate qualitative boundaries of

a theory, not just quantitative ones Show why a theory will not work in a

new application, not just that it does not

Focus on multiple elements of a theory, not just one inconsistency

8/6/2011

RESEARCH METHODS

7EVIDENCE REQUIRED

FOR REVISING A THEORY

Marshal compelling evidence. This evidence can be logical (e.g., the

theory is not internally consistent), empirical (its predictions are

inconsistent with the data accumulated from several studies), or

epistemological (its assumptions are invalid-given information from another field).

8/6/2011

RESEARCH METHODS

8PROVIDE REMEDIES,

ALTERNATIVES

Is the original actually inferior,

Or simply the best we can do?

8/6/2011

RESEARCH METHODS

9IS IT

PUBLISHABLE?

How radically new is the idea? Is it linked to evidence, will it change practice? Why so? Are the underlying logic and sup-

porting evidence compelling? Are the author's assumptions explicit? Are the author's views be-lievable?

Well rounded, broad, deep understanding? At professional standards? Who cares? Why now?

8/6/2011

RESEARCH METHODS

10WHAT THEORY IS NOT:

(1) REFERENCES

For example: "This pattern is consistent with findings that

industrialization produces growth (Marx 1880, Solow 1956)

This sentence lists publications that contain

conceptual arguments (and some findings).

But there is no theory because no logic is presented

to explain why industrialization produces growth

8/6/2011

RESEARCH METHODS

11WHAT THEORY IS NOT:

(2) DATA

Introduced to show what patterns are there Not why they are there

This approach relies on brute empiricism, where hypotheses are motivated by

prior data And not the relevance of theory

8/6/2011

RESEARCH METHODS

12(3) LISTS OF VARIABLES

AND DEFINITIONS

Lists of concepts, similarly, are not theory

Need connections between variables explained

8/6/2011

RESEARCH METHODS

13(4) DIAGRAMS

ALONE

What is needed is causal inference Time horizon and change over time Diagrams as Stage props, not the

performance Logic needs to be spelled out,

verbally

8/6/2011

RESEARCH METHODS

14(5) HYPOTHESES: WHAT IS EXPECTED TO OCCUR

Well-crafted conceptual argument includes hypotheses

They serve as crucial bridges between theory and data

Making explicit how the variables and relationships that follow from a logical argument will be operationalized.

These are statements about what is expected to occur, not why it is expected to

8/6/2011

RESEARCH METHODS

15IDEAS AND CONCEPTS

A theory generally stems from a small set of research ideas, not a list of testable hypotheses

8/6/2011

RESEARCH METHODS

16QUALITATIVE/QUANTITATIVE

DISTINCTIONS

Qualitative: Research Questions

Quantitative: Set of hypotheses (nul or directional)

8/6/2011

RESEARCH METHODS

17EVIDENCE FOR YOUR

HYPOTHESES/TESTING THEORY

When theories are particularly interesting or important, empirical support can be partial: a small set of interviews, a demonstration, experiment, a pilot survey, archival data

8/6/2011

RESEARCH METHODS

18EVIDENCE FOR YOUR

HYPOTHESES

May point to why a particular process might be true.

Subsequent research indicated

May show whether the theoretical statements hold up, or whether they can repeatedly be falsified

8/6/2011

RESEARCH METHODS 19

EXTERNAL VALIDITY

Economics

8/6/2011

RESEARCH METHODS

20

TWO CONCEPTS OF EXTERNAL VALIDITY,

AS APPLIED TO THEORY

1: Replicability, that the study would hold for other persons, settings, times, or places.

Only one form of validity when the objective of research is to test theory

8/6/2011

RESEARCH METHODS

21EXTERNAL

VALIDITY2: Applicability

It arises primarily through severe and rigorous tests of theory

rather than by attempts to incorporate "real world' variables into individual studies designed to test theory.

Such variables only become important in the context of evaluating interventions based on theory.

8/6/2011

RESEARCH METHODS

22 CONSTRUCT VALIDITY

One needs to make a distinction between (1) the construct validity of a concept, as reflected in the convergence (and discrimination) of some particular set of operationalizations of it, and (2) the construct validity of a relation between two concepts, as reflected in the "fit" of that relation within some nomological network.

The fit is linked to considerations of external validity.

8/6/2011

RESEARCH METHODS

23SAMPLING IS CRITICAL

FOR VALIDITY

Four major sampling strategies that might be adopted vis a vis any one aspect or facet of the events under study

1. Sampling homogeneously over the entire study

2. Sampling several subsets, each homogeneous within subset on the facet but differing on it between subsets, so that all the subsets together span the whole range

8/6/2011

RESEARCH METHODS

24 SAMPLING

3. Sampling heterogeneously. but in a way that yields an overall distribution of the facet among the cases within the study that reflects (is representative of) the distribution of the facet among cases "in nature”

4. Sampling heterogeneously on the facet but without regard to representativeness.

8/6/2011

RESEARCH METHODS

25SAMPLING THREATS

TO VALIDITY

These four strategies offer different opportunities for—and pose different threats to—the exploration of the external validity of any given set of findings with respect to the facet in question.

Define external validity: the deliberate and systematic search, on

a number of facets, for both the scope and the limits over which that given set of findings does and does not hold.

8/6/2011

RESEARCH METHODS

26 REPORTING

Validity as robustnesss Where are the boundaries? Can findings be replicated?

8/6/2011

RESEARCH METHODS

27 TESTS FOR VALIDITY

STAGE 1 STAGE 2

Design:

Instrument validity instrument use validity

Comparison validity execution validity

Hypothesis:

Experiment:

Construct validity Operational

Nomalogical validity Predictive8/6/2011

RESEARCH METHODS

28CHECKING

VALIDITY

Observation:

State validity Attribute

Pattern validity Process

8/6/2011

RESEARCH METHODS 29

THEORY

What is a good

theory?

8/6/2011

RESEARCH METHODS

30 A THEORY DOES NOT COME FIRST IN RESEARCH

Theory:

A reasoned and precise speculation about the answer to a research question, including a statement about why the proposed answer is correct

Implies several specific descriptive or causal hypotheses Is consistent with prior evidence

about a research question

8/6/2011

RESEARCH METHODS

31CHOOSING A

THEORY

Possibly, it will be wrong It must be falsifiable-what evidence would

convince us? Requires observable implications (many) Observe the principle of parsimony (a

judgment about the nature of the world, which is assumed to be simple)

These rules assume you have not yet collected the data, which can be used afterwards, to modify your theory

8/6/2011

RESEARCH METHODS

32AFTER YOUR

RESEARCH

What to do with a theory Expand it, dropping a condition or variable,

to, say, all countries, all regions Make it embrace a larger range of phenomena Do not do the opposite

If the theory does not work for some of your observations, do not squeeze new theory out of a revised and qualified research base

Ie, make it less restrictive, but not more restrictive—unless you go back and collect more data and observations beginning with the new hypothesis

8/6/2011

RESEARCH METHODS

33 DATA

Ensure reliability Maximize leverage

Explain as much as possible with as little as possible Increase numbers of observable implications

with confirmation Improve the theory, improve the data

From the beginning, list all the possible implications of your hypotheses—outcomes, responses, interviews, at all levels

8/6/2011

RESEARCH METHODS

34 SCEPTICISM

Report negative findings Report alternative

hypotheses

8/6/2011

RESEARCH METHODS 35

THEORY CONCEPT

How to build a

concept

8/6/2011

RESEARCH METHODS

36CONCEPT

FORMATION

Rigorous approach to concept formation (Osigweh) Defining meaning

boundaries (what the concepts do not include)

Minimizing concept misuse, confusion

8/6/2011

RESEARCH METHODS

37CONCEPT

PRECISION

General, yet precise Moving a concept from low

to high levels of abstraction Economic oncepts must

span several contexts Yet they must provide a

precise guide to what is not included

8/6/2011

RESEARCH METHODS

38CONCEPT

IMPRECISION

Useless: new contexts too easily fit

New topics can be suspect, vulnerable to herding by scholars (CR impact on performance)

8/6/2011

RESEARCH METHODS

39 CONCEPTS

Can be decomposed taxonomically

8/6/2011

RESEARCH METHODS

40 VARIABLES

A variable is a special

kind of concept.

It is a classification into two or more mutually exclusive and totally inclusive

categories (e.g., Hage, 1972; Smith, 1975).

8/6/2011

RESEARCH METHODS 41

CONCEPTS

Stretching and

Travelling

8/6/2011

RESEARCH METHODS

42CONCEPT TRAVELLING:

POSITIVE CONTRIBUTION

Concept travelling, in this sense, means that the concept is precise enough to allow researchers to define it in the same way, and so to test it in a wide range of situations—that is, that the concept is a universal.

8/6/2011

RESEARCH METHODS

43 EXAMPLE Puzzle -- Each use creates essentially the same

image anywhere: (a) a maze of activities that cannot be correctly

solved outside settings or mental frameworks that prescribe following certain paths or specific combination of paths;(jigsaw?Rubik’s cube)?

(b) an exercise of figures, numbers, or behaviours that cannot be engaged in or successfully disengaged in, except through a specific order or steps, procedures, or systems of thought processes;

(c) activities that when completed cannot fail to yield exact answers, known solutions, or ultimates that could be exactly predicted beforehand.

8/6/2011

RESEARCH METHODS

44CONCEPT STRETCHING

(TO BE AVOIDED)

Multi level governance

when refracted through the lens of lived political experiences-- notions of policy transfer and of the complex politics of scale of interventions in time, place and space--needed

8/6/2011

RESEARCH METHODS

45TOO VAGUE

(STRETCHING)

Decentralization Would be misapplied in

different societies Too broad

8/6/2011

RESEARCH METHODS

46TOO NARROW, AS WELL AS

VAGUE: OBSERVATIONAL CONCEPTS

Can be moved up or down the abstract ladder

Communication puzzle, group, decision, problem, conflict, and participation.

8/6/2011

RESEARCH METHODS

47

8/6/2011

Connotation (Depth, Intension) I HAL Low Middle High

DENOTATION

(EXTENSION,

BREADTH)

High abstract level • Traveling concept

•Stretched concept

domain domain

— Broad coverage, of — Broad coverage

distinct classes of — Too many classes

things of things lumped

— Few, but determinate together (with little

attributes of attention to the

distinct classes precision of their

— Precise at a many attributes)

universal level — Packed or too allembracing

RESEARCH METHODS

48

8/6/2011

Connotation (Depth, Intension) I I I I I I I I I I I I

Low Middle High

DENOTATION (EXTENSION,

BREADTH)

MAL

Mid abstract level • Generalizable

nonuniversal concept domain

— Breadth balanced

— Medium range concepts

— Similarities

RESEARCH METHODS

49 Connotation (Depth, Intension) I I I I I I I I I I I I HAL Low Middle High

DENOTATION (EXTENSION, BREADTH)

MAL LAL • Configurative • Taxonomic domain

Low abstract level

situational concept doman

— Narrow coverage

— Narrow coverage

— Many attributes

— Too few attributes for each class

— Configurative/ — Distinctive typologies

Imprecise taxonomies or precise taxonomies

specific generalities precise at a specific level

taxonomies — Precise at a specific level

8/6/2011

RESEARCH METHODS

50MOVING TO THE

UPPER LEFT CORNER

Move up the ladder by negating

Concepts with negation are precise

Those without are indeterminant

8/6/2011

RESEARCH METHODS

51EXAMPLE: STUDIES OF

“PROBLEM”

The concept of problem is studies without its meaning boundaries being distinctively delineated.

Generated by concrete, intuitive, ad hoc conceptualizations.

Studies of structured problems fall widely apart from a focus on conflict resolution to one on negotiation.

8/6/2011

RESEARCH METHODS

52THEORETICAL

CONCEPTS

Theoretical or universal concepts are defined by their systemic meaning in the sense that each meaning derives from the part that the concept plays in the theory.

Open concepts reflect the availability of different operational criteria for application to different contexts.

As a result, their meanings are not fully defined by reference to observable things and their characteristics.

8/6/2011

RESEARCH METHODS

53 EXAMPLES

Efficacy (Jones, 1986; Osigweh, 1983)

Synergy, feedback, adaptation (Osigweh 1985b, pp. 153- 157),

Homeostasis and isomorphism

8/6/2011

RESEARCH METHODS

54EMPIRICAL CONCEPTS

Observational Can be moved up or down

8/6/2011

RESEARCH METHODS

55 EXAMPLES

communication, puzzle, group, decision, conflict

8/6/2011

RESEARCH METHODS

56MOVING UP THE

LADDER

Requires that you keep it clear and not just extend meaning to other kinds of observations

8/6/2011

RESEARCH METHODS

57 ADVICE

Define a concept by saying what it is not; Give it precise boundaries Make it clear

8/6/2011

RESEARCH METHODS 58

THEORETICAL CONTRIBUTION

Definitions and

Approaches

8/6/2011

RESEARCH METHODS

59 COMPONENTS

Which factors (variables, constructs, concepts) logically should be considered as part of the explanation

Two criteria exist for judging the extent to which we have included the "right" factors: comprehensiveness (i. e., are all relevant

factors included?) and parsimony (i.e., should some factors be

deleted because they add little additional value to our under-standing?).

8/6/2011

RESEARCH METHODS

60 RELATIONSHIPS

Operationally this involves using "arrows" to connect the "boxes."

Such a step adds order to the conceptualization by explicitly delineating patterns.

It typically introduces causality.

8/6/2011

RESEARCH METHODS

61 WHY IMPORTANT? What justifies the selection of factors and the

proposed causal relationships? This rationale constitutes the theory's

assumptions It welds the model together. Why should colleagues give credence to this

particular representation of the phenomena? The answer lies in the logic underlying the

model. The soundness of fundamental views of

economics or processes

8/6/2011

RESEARCH METHODS

62FROM THE

EDITOR:

“The mission of a theory-development journal is to challenge and extend existing knowledge, not simply to rewrite it. Therefore, authors should push back the boundaries of our knowledge by providing compelling and logical justifications for altered views.”

8/6/2011

RESEARCH METHODS

63 SET CONTEXTUAL

LIMITS

These temporal and contextual factors set the boundaries of generalizability, and as such constitute the range of the theory.

Do theoretical effects vary over time, either because other time-dependent variables are theoretically important or because the theoretical effect is unstable for some reason.

8/6/2011

RESEARCH METHODS

64 SUMMARY

For purposes of publishing: Clarity in main ideas Currency in quality

8/6/2011

RESEARCH METHODS

65PUBLISHING IN

ECONOMICS

Papers are assumed to vary along two quality dimensions

The former is interpreted as the importance of a paper’s main ideas and

the latter as other aspects of quality. Observed trends are thought of as reflecting increases in this

8/6/2011

THE END