lecture 1 what is a good question?
Post on 23-Feb-2016
41 Views
Preview:
DESCRIPTION
TRANSCRIPT
Lecture 1What is a good question?
RESEARCH METHODSNATIONAL RESEARCH UNIVERSITYHSE ECONOMICS, PH. D PROGRAM DR C S LEONARD JUNE 2011
RESEARCH METHODS
2OUTLINE OF
LECTURE
Knowledge Claims Classical Approaches Post-positivism
Strategies of inquiry Data mining Design-based research
Good questions emerge from good research designs
8/06/2011
RESEARCH METHODS 3
KNOWLEDGE CLAIMS
8/06/2011
RESEARCH METHODS
4 KNOWLEDGE CLAIMS
What warrants knowledge?
How is scientific method applied?
8/06/2011
LEONARD GSOM PH.D. RESEARCH METHODS 2010
5 TWO POSITIONS
4/02/20010
Two scientific positions, inductivism and deductivism
From which method emerges
Deductive (logic) argumentation If premises are true and no fallacies in the argument,
then conclusion will be true
Not concerned with truth or falsity
Inductive arguments may have true premises, but we cannot be certain that conclusions will also be true (ampliative reasoning)
LEONARD GSOM PH.D. RESEARCH METHODS 2010
6INDUCTIVE, DEDUCTIVE
4/02/20010
Deductive, from general to particular, and inductive, from particulars to general
Inductive: Frances Bacon vs the medieval Church: purging ourselves of idols
Problem (Hume) Can we predict the future? No
Positivism is descended from Bacon
Research becomes historical, truth confined to a systematic empirical study, that might obtain general laws
Empirical findings worthless to some deductivists
LEONARD GSOM PH.D. RESEARCH METHODS 2010
7FALSIFICATION:
KARL POPPER (1902-1994)
4/02/20010
Karl Popper’s critical rationalism has generated much debate since the 1930s
Intellectual autobiography, Unended Quest
Background, early Marxist, training with Adler and Freudian theories convinced them that the theories were too broad
Later rejected psychologism
Favored theory of relativity, Einstein, could be tested, verified, falsified
RESEARCH METHODS
8LOGIC OF
FALSIFICATIONISM
Scientific theories are abstract
can be tested only indirectly, by reference to their implications.
Scientific theory, and human knowledge generally, is irreducibly conjectural or hypothetical
to solve problems that have arisen in specific historio-cultural settings
Logically, no number of positive outcomes at the level of experimental testing can confirm a scientific theory
A single counterexample is logically decisive: it shows the theory, from which the implication is derived, to be false.
8/06/2011
LEONARD GSOM PH.D. RESEARCH METHODS 2010
9 VS INDUCTIVISM
4/02/20010
Direct antithesis of inductivism
Growth of knowledge requires overturning previous beliefs
Have a theory, test it, falsify it and move on
Many economists seek to prove theories correct; the job of science is to disprove them
Don’t strive for certainty (verificationism)
Bans ad hoc adjustments to a theory to prevent it from being falsified
A new theory will possess greater empirical content than its predecessors
LEONARD GSOM PH.D. RESEARCH METHODS 2010
10 CRITICS
4/02/20010
This hinders not promotes science
Can’t reject theories so easily, some theories are better at some things than others
Marxism, accommodates business cycles and disequilibrium better then orthodox Keynsianism, but the Monetarists better understand inflationary processes than the Keynsians, who argue that their theories and policies are more effective against unemployment
Should not critique a new theory too rigorously, because it may have something good in it
RESEARCH METHODS
11DEFENDER:
BLAUG
Blaug (be taken seriously)
Have a prediction about the future
Require a formal model
Falsification is essential
Be scientific, or not
Falsificationism: much tougher
Lay down restrictions on what Popper calls immunizing strategems
8/06/2011
LEONARD GSOM PH.D. RESEARCH METHODS 2010
12ENORMOUS INFLUENCE
4/02/20010
Mark Blaug
Even econometricians, however, econometric results difficult to falsify
Plain fact Most economists tend to verify..
LEONARD GSOM PH.D. RESEARCH METHODS 2010
13 THOMAS KUHN
4/02/20010
Structure of Scientific Revolutions
Science is not good science unless it is working under the framework of a theory, makes no progress
It adopts a particular view of the world, and all subsequent research adds to that
Pre science—lots of theories hoping to explain the same thing
The paradigm: an achievement so important that it attracts an enduring group of adherents Commitment and consensus are prerequisites for normal
science
LEONARD GSOM PH.D. RESEARCH METHODS 2010
14 PARADIGMS
4/02/20010
Older generations stay with their paradigms, new ones acquire new paradigms
Releases scientists from the necessity of debating fundamentals
They can then concentrate on subtle, esoteric aspects of their subject
LEONARD GSOM PH.D. RESEARCH METHODS 2010
15KUHN: NORMAL
SCIENCE
4/02/20010
Mopping up
Determination of facts
Setting the facts within theory
Articulating the theory
Then, anomalies, followed by crisis, followed by fundamental change
LEONARD GSOM PH.D. RESEARCH METHODS 2010
16APPLIED
ECONOMETRICS
4/02/20010
Middle ground
Use theory, provide initial specification
Data exploration techniques to extend or refine it
Bridge theory and empirical data analysis
How do we know a theory is correct?
Different users have different tastes and beliefs
Complications with computation: large data sets numerous models possible
LEONARD GSOM PH.D. RESEARCH METHODS 2010
17 PARADIGMS
4/02/20010
How researchers will learn/what they will learn, assumptions
Philosophical assumptions, epistemologies (how we know something), ontologies (what is knowledge), axiology (what values go into knowledge), methodology (process for studying)
LEONARD GSOM PH.D. RESEARCH METHODS 2010
18 EXTENSIONS
4/02/20010
These debates shaped much social science theory about
Innovations
Science
Path dependence
Historical legacies
Nature of change
Nature of reform and timing
RESEARCH METHODS
19 POSITIVISM
Can we be positive about our claims of knowledge when studying behavior and actions (Comte, Mill, Durkheim, Newton and Locke)?
Causes probably determine effects? Reductionism: reduce ideas into
small discrete sets to be tested
8/06/2011
RESEARCH METHODS
20POSITIVISM VS
INTERPRETIVISM
Interpretivism: Weber (Verstehen)
8/06/2011
RESEARCH METHODS
21POSITIVISM AND ITS
OPPONENTS
Quantitative, positivist, post-positivist research, empirical science
Challenge to positivism: against the traditional notion of the absolute truth of knowledge; playing tennis with the net down
8/06/2011
RESEARCH METHODS
22METHODOLOGICAL
PLURALISM?
Bruce Caldwell (let 100 flowers bloom)
Little economics will survive if we take this seriously
Confirmationism Verification
Falsificationism is never practiced because it is unpracticeable
8/06/2011
RESEARCH METHODS
23BAYSIAN
METHODS
Test, verify and refine the laws and theories governing behavior
Baysian methods: from theory, to test, to revision
8/06/2011
RESEARCH METHODS
24(1) POST
POSITIVISM
Knowledge is conjectural
Research is to make claims and refine or abandon them
Data, evidence and rational considerations shape knowledge
Being objective is key
8/06/2011
RESEARCH METHODS
25(2) SOCIAL
CONSTRUCTION
Social construction
Mannheim, Burger, Luckmann, Neuman
Look at participants views
Judgments are subjective, meanings are varied and multiple
Interviews: open ended questioning, the more the interviewee talks spontaneously, the better
Participants allowed to construct meaning (rather than responding to concrete situations)
Process of interaction, context of work
8/06/2011
RESEARCH METHODS
26 (3) PRAGMATISM
Pierce, James and Dewey
Knowledge claims arise out of actions, situations, consequences, rather than ex ante conditions
Concern with what works
Pluralistic approach
Mixed methods, qualitative, quantitative
Research always occurs in social contexts
Stop asking questions about the laws of nature
8/06/2011
RESEARCH METHODS 27
LOGIC OF RESEARCH DESIGN
Influence
Research
Design
8/06/2011
RESEARCH METHODS
28“ACROSS MOST FIELDS… APPLIED
ECONOMISTS ARE NOW LESS LIKELY TO PIN A CAUSAL INTERPRETATION OF
THE RESULTS ON ECONOMETRIC METHODOLOGY ALONE.
DESIGN-BASED STUDIES ARE DISTINGUISHED BY THEIR PRIMA FACIE
CREDIBILITY AND BY THE ATTENTION INVESTIGATORS DEVOTE TO MAKING
BOTH AN INSTITUTIONAL AND A DATA-DRIVEN CASE FOR CAUSALITY.”
JOSHUA D. ANGRIST AND JÖRN-STEFFEN PISCHKE
8/06/2011
RESEARCH METHODS
29GENERAL TO SPECIFIC
METHODOLOGY
“LSE” tradition of time-series econometrics that began in the 1960s at the London School of Economics
Mizon (1995) a brief history The practitioners of LSE econometrics are now widely dispersed among academic institutions throughout Britain and the world.
The LSE approach is described sympathetically in Gilbert (1986), Hendry (1987,1995, esp. chs. 9-15), Pagan (1987), Phillips (1988), Ericsson, Campos and Tran(1990), and Mizon (1995). For more sceptical accounts, see Hansen (1996) and Faust and Whiteman (1995, 1997)
8/06/2011
RESEARCH METHODS
30GENERAL TO
SPECIFIC
Context: Linear (cross-country growth) Use: Time-SeriesStep 1. General regression will include every possible variable -- all the information about the true determinants. Step 2.The information content is then sharpened by a more parsimonious regression – the specific regression
(a) it is statistically well specified (for example, it has white noise errors);
(b) that it is a valid restriction of the general regression, and (c) that it encompasses every other parsimonious regression
that is a valid restriction of the general regression
Criticism: data-mining,
8/06/2011
RESEARCH METHODS
31 EXTREME BOUNDS ANALYSIS: THE NEW CRITIQUE Edward Leamer’s “extreme-bounds analysis” (1983, 1985).
A coefficient of interest is robust only to the degree that it displays a small variation to the presence or absence of other regressors. Leamer and Leonard (1983) define the extreme-bounds for the coefficient of a particular variable within a search universe as ranging between the lowest estimate of its value minus two times its standard error to the highest estimate of its value plus two times its standard error, where the extreme values are drawn from the set of every possible subset of regressors that include the variable of interest. A variable is said to be robust if its extreme bounds lie strictly to one side or the other of zero.
8/06/2011
RESEARCH METHODS
32 JUDGMENT CALLS
The main difference between structural and experimental (or ``atheoretic'') approaches is not in the number of assumptions but the extent to which they are made explicit. (Michael Keane)
8/06/2011
RESEARCH METHODS
33 SUMMARY
Quantitative (numbers)
Experimental design--controls
Non experimental design, surveys
Qualitative (words)
Narratives, phenomenologies, ethnographies, grounded theory, case studies
Mixed methods
Sequential, concurrent, transformative
8/06/2011
RESEARCH METHODS
34 QUANTITATIVE
Random assignment of subjects to treatment
Quasi experiments: non random designs
Surveys cross sectional and longitudinal, generalize from sample to population
8/06/2011
RESEARCH METHODS
35 QUALITATIVE
Ethnographies: researcher studies an intact cultural group in its setting over time (responses)
Grounded theory
Derive an abstract theory of a process, action or interaction, grounded in views of participants
Case Studies
Exploring in depth a program, event, activity, process, or individuals, bound in time, variety of procedures
Phenomenological: lived experiences
Narrative research: lives, stories, retellings
8/06/2011
RESEARCH METHODS
36 MIXED
Gets around biases in any one method used exclusively
Progress from one method to another
Illustrate
Determine what the concept is
Test assumptions on one case
8/06/2011
RESEARCH METHODS
37 LOGIC Qualitative:
Wider range of methods, non-numerical by definition
Small n, intensive interviews, depth analysis, discursive, account of event or unit
Focus on event, decision, institution, location, issue or legislation
Incident important in its own right (war, election, change in leadership, marketing strategy, community decision, etc)
Against bifurcation? mixed methods Systematic, scientific research of all kinds
Most research does not neatly fit one or other category
8/06/2011
RESEARCH METHODS
38GOAL OF SOCIAL
SCIENCE RESEARCH
Inference (descriptive, explanatory)
Attempting to infer beyond immediate facts to something broader
Learning about causal effects from data
Public procedures (explicit, codified)
Replication
Conclusions are uncertain
Observes rules of inference
8/06/2011
RESEARCH METHODS
39DESCRIPTIVE
INFERENCE
Distinguish systematic from non-systematic features
Systematic from stochastic
Counter-factuals (what would have happened, had meters not struck the earth 65 million yrs ago)
8/06/2011
RESEARCH METHODS
40RULES FOR RESEARCH
DESIGN
Intuition: Choice of better topics is idiosyncratic
Two ways to test if it is a good topic
Is it about something important in the real world
A research topic should make a specific contribution to an identifiable scholarly literature by increasing our ability to construct verified scientific explanations
ie: locating it within the framework of existing social science literature
This is the subject of the second lecture today—what makes a theory or theoretical contributions valuable to the community of editors of journals
8/06/2011
RESEARCH METHODS
41 CAUTION
There may be reasons a theory is practicable, even though its long term scientific value has been questioned
Theoretically incoherent models used to forecast the US economy—diversion of macroeconomic theory and applied macroeconomics (see Mankiw 1990)
New theories, however, remain speculative
8/06/2011
RESEARCH METHODS
42DESIGN BASED
RESEARCH
Leamer 1983 highlighted the benefits of sensitivity analysis, a procedure in which researchers show how their results change with changes in specification or functional form. Sensitivity analysis has had a salutary but not a revolutionary effect on econometric practice.
As we see it, the credibility revolution in empirical work can be traced to the rise of a design-based approach that emphasizes the identification of causal effects.
8/06/2011
RESEARCH METHODS
43 WOW-FACTOR
Design-based studies typically feature either real or natural experiments and are distinguished by their prima facie credibility and by the attention investigators devote to making the case for a causal interpretation of the findings their designs generate.
Design-based studies are most often found in the microeconomic fields of Development, Education, Environment, Labor, Health, and Public Finance, but are still rare in Industrial Organization and Macroeconomics.
8/06/2011
RESEARCH METHODS 44
LITERATURE REVIEW
8/06/2011
RESEARCH METHODS
45LITERATURE
REVIEW
Classical
Systematic review
Meta-analysis
Narrative review
Search issues
Presentation
8/06/2011
RESEARCH METHODS
46CLASSICAL
LITERATURE REVIEW
Large Disciplinary Differences Sociology, Psychology, Business Economics
8/06/2011
RESEARCH METHODS
47SYSTEMATIC
REVIEW
A systematic review aims to provide an exhaustive summary of literature relevant to a research questions
The first step of a systematic review is a thorough search of the literature for relevant papers.
The Methodology section of the review will list the databases and citation indexes searched as well as any individual journals. Next, the titles and the abstracts of the identified articles are checked against pre-determined criteria for eligibility and relevance.
8/06/2011
RESEARCH METHODS
48 MORE
Ability to control for between-study variation
Including moderators to explain variation
Deal with information overload: the high number of articles published each year.
It combines several studies and will therefore be less influenced by local findings than single studies will be.
Makes it possible to show if a bias for published works exists.
8/06/2011
RESEARCH METHODS
49SYSTEMATIC
ANALYSIS
Meta-Analysis
Meta-analysis leads to a shift of emphasis from single studies to multiple studies. It emphasizes the practical importance of the effect size instead of the statistical significance of individual studies. This shift in thinking has been termed "meta-analytic thinking"
8/06/2011
RESEARCH METHODS
50 META-ANALYSIS(AVERAGE EFFECT SIZE, FOREST PLOT)
8/06/2011
RESEARCH METHODS
51META-ANALYSIS:
CRIT
About science, but not science
Statistical examination of scientific studies
Cannot propose ways to falsify a theory
8/06/2011
RESEARCH METHODS
52META-NARRATIVE
REVIEW
Example: Connected Communities’,
a research programme of the Institute of Health and Human Development (IHHD) to investigate the meanings of community within and across research disciplines by adopting an innovative methodology based on a meta-narrative systematic review approach. Policy and academic interest in the concept of ‘community’ is longstanding and such interest has become central to policy making in the last two decades.
Meta-narrative review shows the diversity in the meaning of ‘communities’ --various conceptualisations and meanings of community across disciplines, over time, and within different cultures and contexts
(Greenhalgh et al, 2005) is a type of ‘systematic’ review rather than a traditional expert driven literature review
A focus on identifying the ‘storylines of research’ within and across disciplinary boundaries. Identifies the meta-narratives of each discipline and analyse the different ‘discourses’ and languages of ‘community’.
8/06/2011
Until Next HourTHE END
top related