3 march 2009 @ crawford school 1 empirical research design – 2 semester 1, 2009 pogo8096/8196:...

3 March 2009 @ Crawford School 1

Empirical Empirical Research Research Design – 2Design – 2

Semester 1, 2009Semester 1, 2009POGO8096/8196: Research MethPOGO8096/8196: Research Meth

odsods

Crawford School of Economics and Government Crawford School of Economics and Government

23 March 2009 @ Crawford School

This weekThis week

Research topics and questionsResearch topics and questions How to choose a research topic?How to choose a research topic? Normative, empirical, and analytical questionsNormative, empirical, and analytical questions What constitutes a “good” research question?What constitutes a “good” research question?

Theories, hypotheses and operational Theories, hypotheses and operational definitionsdefinitions What is a (causal and empirical) theory?What is a (causal and empirical) theory? What constitutes a “good” theory? What constitutes a “good” theory? What are hypotheses and operational What are hypotheses and operational

definitions?definitions?


Choosing a topic – 1Choosing a topic – 1

Picking a research topic is the critical Picking a research topic is the critical decision in doing research but is decision in doing research but is difficult to teach how to pick it.difficult to teach how to pick it.

This task requires not only that you are This task requires not only that you are familiar with as broad a range of familiar with as broad a range of existing studies and issues as possible, existing studies and issues as possible, but that you also have some idea of but that you also have some idea of where an existing body of research (or where an existing body of research (or an existing argument in policy debate) an existing argument in policy debate) is weakest.is weakest.


Choosing a topic – 2Choosing a topic – 2

Critically read and critically think. Critically read and critically think. You should identify what the bottom-line argument You should identify what the bottom-line argument

(i.e., (i.e., XX causes causes YY) is in each article, book or policy ) is in each article, book or policy debate.debate.

Can you think of any “alternative” argument?Can you think of any “alternative” argument? Do you think the argument is applicable to other Do you think the argument is applicable to other

data, which the author does not use (e.g., data data, which the author does not use (e.g., data from Asian countries)?from Asian countries)?

Any puzzle?Any puzzle? One often effective way of looking at choosing a One often effective way of looking at choosing a

topic is to work on something in our lives, which topic is to work on something in our lives, which puzzlespuzzles us. us.


Examples from my Examples from my researchresearch

Why do Japanese village elections often Why do Japanese village elections often record very high voter turnout (e.g., over record very high voter turnout (e.g., over 95%) even without a compulsory voting 95%) even without a compulsory voting system?system?

Do policy debates between parties promote Do policy debates between parties promote active civic participation in an election (as active civic participation in an election (as often advocated by the media)? often advocated by the media)?

Does natural-resource abundance truly Does natural-resource abundance truly hinder democratization (as argued by Ross hinder democratization (as argued by Ross and others)?and others)?


RemarksRemarks

Should we produce a “great work”? Should we produce a “great work”? There exists what is called “publication There exists what is called “publication

bias.” Intuitive and no-surprise findings bias.” Intuitive and no-surprise findings are often not published. are often not published.

““The contribution of even a minor The contribution of even a minor article is greater than that of the ‘great article is greater than that of the ‘great work’ that stays forever in a desk work’ that stays forever in a desk drawer or within the confines of a drawer or within the confines of a computer” (KKV, p. 9).computer” (KKV, p. 9).


Q & AQ & A

When you read existing studies, When you read existing studies, understand …understand … What is the author’s research What is the author’s research questionquestion?? What is the author’s What is the author’s answeranswer to the question. to the question.

When you conduct own research, explain When you conduct own research, explain …… What is your research What is your research questionquestion?? What is your What is your answeranswer to the question? to the question?

Is the author’s/your question, …Is the author’s/your question, … empirical, normative, or analytical?empirical, normative, or analytical?


Normative questionsNormative questions A A normativenormative question is a question question is a question

examining what is desirable and examining what is desirable and undesirable. The answers depend on the undesirable. The answers depend on the value judgments of the individual who value judgments of the individual who answers the question (i.e., of a answers the question (i.e., of a researcher).researcher).

Typical in normative philosophy or in Typical in normative philosophy or in policy debatespolicy debates

ExamplesExamples Is democracy the Is democracy the bestbest form of government? form of government? ShouldShould the Reserve Bank cut the interest rate? the Reserve Bank cut the interest rate?


Analytical questionsAnalytical questions

An An analyticalanalytical question is a question question is a question examining the validity of a proposition, examining the validity of a proposition, which is completely dependent on a which is completely dependent on a set of assumptions and definitions.set of assumptions and definitions.

Typical in formal theory (and Typical in formal theory (and mathematics)mathematics)

An exampleAn example Does the system of simple majority rule Does the system of simple majority rule

voting always give an unambiguous social voting always give an unambiguous social choice between alternatives?choice between alternatives?


Empirical questionsEmpirical questions

An An empiricalempirical question is a question question is a question examining what is true or untrue in the real examining what is true or untrue in the real world. The question can be supported or world. The question can be supported or rejected by observations and/or experience.rejected by observations and/or experience.

Typical in theory-oriented or policy-oriented Typical in theory-oriented or policy-oriented empirical researchempirical research

ExamplesExamples Does economic development cause Does economic development cause

democratization?democratization? Did the tax reform flatten income distribution?Did the tax reform flatten income distribution?


NotesNotes Your empirical question should be, in Your empirical question should be, in

principal, answerable without “your” principal, answerable without “your” value judgment and with value judgment and with observableobservable information (e.g., government statistics, information (e.g., government statistics, findings from a survey, findings from an findings from a survey, findings from an interview, your participant observations, interview, your participant observations, other data sources, etc.).other data sources, etc.).

Note that you never know what is true or Note that you never know what is true or untrue in the future: Your question untrue in the future: Your question should not be about something in the should not be about something in the future. future.


Should we always ask Should we always ask empirical questions?empirical questions?

Many important topics in social sciences Many important topics in social sciences are concerned with normative questions are concerned with normative questions about what about what ought to beought to be in societies, in societies, economies and politics. But most social economies and politics. But most social scientists today are concerned with scientists today are concerned with empirical questions. This does not mean empirical questions. This does not mean that social scientists think normative that social scientists think normative issues as unimportant.issues as unimportant.

We may be able to deal with normative We may be able to deal with normative issues issues indirectlyindirectly by examining empirical by examining empirical questions. How?questions. How?


Reformulating questionsReformulating questions

Method 1: change the frame of referenceMethod 1: change the frame of reference Is Kevin Rudd doing a good job?Is Kevin Rudd doing a good job? Does the public think that John Howard is Does the public think that John Howard is

doing a good job?doing a good job? Method 2: test the underlying argumentMethod 2: test the underlying argument

Should the Research Bank cut the interest Should the Research Bank cut the interest rate to boost economy?rate to boost economy?

Did an increase in the interest rate (under Did an increase in the interest rate (under similar conditions) boost economy?similar conditions) boost economy?


Good empirical questionsGood empirical questions

ClarityClarity: It must be clear and reasonably : It must be clear and reasonably specific.specific.

TestabilityTestability: It must be a question that can : It must be a question that can be investigated given available empirical be investigated given available empirical resources.resources.

Theoretical significanceTheoretical significance and/or and/or practical practical relevancerelevance: It must be important : It must be important theoretically and/or practically.theoretically and/or practically.

OriginalityOriginality: It should have at least some : It should have at least some degree of originality.degree of originality.


Examples: good or bad?Examples: good or bad?

What is today’s political and economic relatiWhat is today’s political and economic relationship between China and ASEAN?onship between China and ASEAN?

What fraction of people in North Korea suppWhat fraction of people in North Korea support the current Kim Chong-il regime?ort the current Kim Chong-il regime?

Should Australia’s state upper houses of parShould Australia’s state upper houses of parliament be abolished? liament be abolished?

Why did a mayor of City A in NZ resign? Why did a mayor of City A in NZ resign? Is the China and Taiwan reunification possiblIs the China and Taiwan reunification possibl

e?e?


RemarksRemarks

If you can formulate a “good” If you can formulate a “good” question (i.e., a clear, testable, question (i.e., a clear, testable, important, and original question), important, and original question), then a half of your research is done. then a half of your research is done. The rest of your research is The rest of your research is collecting pieces of information and collecting pieces of information and presenting your answer.presenting your answer.

Formulating a “good” question is, Formulating a “good” question is, however, not easy. however, not easy.


An empirical theoryAn empirical theory

A (causal and empirical) theory: “If A (causal and empirical) theory: “If XX happens, happens, YY will follow.” will follow.” DependentDependent variablevariable ( (YY) = some phenomenon ) = some phenomenon

that is to be explained. that is to be explained. IndependentIndependent variablevariable ( (XX) = one or more ) = one or more

factors that are thought to affect the factors that are thought to affect the dependent variable.dependent variable.

No variable is innately either independent or No variable is innately either independent or dependent.dependent.

DiscussionDiscussion: Why do we need such a theory?: Why do we need such a theory?


Examples – political Examples – political sciencescience

If a country uses the single-member If a country uses the single-member district system, the effective number of district system, the effective number of political parties is expected to shrink to political parties is expected to shrink to two.two.

If a country increases its military If a country increases its military expenditure, it is expected to go to war.expenditure, it is expected to go to war.

Countries with higher per capita Countries with higher per capita incomes are more likely to be incomes are more likely to be democraciesdemocracies. . DemocraciesDemocracies do not tend to do not tend to initiate wars. initiate wars. * * An example showing that no An example showing that no variable is innately dependent or independentvariable is innately dependent or independent..


Good theory – 1Good theory – 1

““Simplicity”Simplicity” A theory should give us as simple as a A theory should give us as simple as a

handle on the universe as possible. handle on the universe as possible. Note that theories are often stated in a Note that theories are often stated in a

“conditional” manner: “conditional” manner: YY is caused by is caused by XX, , when all the other variables are when all the other variables are controlledcontrolled. .

Independent variables = “key” causal Independent variables = “key” causal variable/s (often denoted as variable/s (often denoted as XX) + other ) + other “control” variables (often denoted as “control” variables (often denoted as ZZ).).



““Predictive accuracy”Predictive accuracy” Given available dataGiven available data, a theory should make , a theory should make

accurate prediction (i.e., accurate prediction (i.e., XX does, indeed, does, indeed, cause cause YY).).

Note: “Prediction” does not necessarily Note: “Prediction” does not necessarily mean that you have to predict something in mean that you have to predict something in the future. If a set of independent variables, the future. If a set of independent variables, to a considerable extent, explains the to a considerable extent, explains the dependent variable (i.e., events, actions, dependent variable (i.e., events, actions, opinions measured in the past), your theory opinions measured in the past), your theory has high predictive accuracy. has high predictive accuracy.



““Importance”Importance” A theory should be applicable broadly and A theory should be applicable broadly and

generally to a variety of situations and generally to a variety of situations and countries.countries.

A specific theory should also make A specific theory should also make contributions to a variety of preexisting contributions to a variety of preexisting theories that are affected by the new theory. theories that are affected by the new theory. For example, a theory on electoral change For example, a theory on electoral change would be more valuable if it could have would be more valuable if it could have implications of other broader areas of implications of other broader areas of theories, e.g., democratic theory.theories, e.g., democratic theory.


EleganceElegance

If a theory meets the three criteria, the If a theory meets the three criteria, the theory can be described as theory can be described as elegantelegant. .

An example: An example: Making Democracy WorkMaking Democracy Work What explains the effectiveness of What explains the effectiveness of

governmental institutions? governmental institutions? The root determinant of effective The root determinant of effective

government is civic community – government is civic community – participation by people in non-profit, non-participation by people in non-profit, non-governmental social activities.governmental social activities.

The theory is simple. It is important, and it The theory is simple. It is important, and it makes accurate prediction.makes accurate prediction.


Beautiful surpriseBeautiful surprise

A good theory gives us a “beautiful A good theory gives us a “beautiful surprise.” surprise.” A piece of research, which goes against our A piece of research, which goes against our

expectations or makes us rethink our expectations or makes us rethink our world, gives us a special kind of pleasure.world, gives us a special kind of pleasure.

ExamplesExamples Tougher border controls increase the Tougher border controls increase the

number of illegal immigrants.number of illegal immigrants. Participation by people in community Participation by people in community

activities enhance the effectiveness of local activities enhance the effectiveness of local government administration. government administration.


Theories and hypothesesTheories and hypotheses

A A theorytheory is a set of generalizations is a set of generalizations about a topic (often a set of about a topic (often a set of generalized causal relationships generalized causal relationships between phenomena: “between phenomena: “XX causes causes YY”). ”). It is broad and general.It is broad and general.

A A hypothesishypothesis is an empirical is an empirical statement derived from a theory (or statement derived from a theory (or an observable implication of a an observable implication of a theory). It is focused and specific.theory). It is focused and specific.


Variables and operational Variables and operational definitionsdefinitions

A A variablevariable is an empirical property that is an empirical property that can take on two or more values. can take on two or more values. A theory is a statement about A theory is a statement about generalgeneral (and (and

often conceptual) variables. often conceptual) variables. A hypothesis is a statement about A hypothesis is a statement about specificspecific

variables.variables. An An operational definitionoperational definition is a set of is a set of

directions as to how the variables in a directions as to how the variables in a hypothesis are to be observed and hypothesis are to be observed and measured.measured.


Next session – 31 MarchNext session – 31 March

Levels of research and unit of Levels of research and unit of analysisanalysis

Problems of measurement: Problems of measurement: The basic problem of measurementThe basic problem of measurement Reliability and validityReliability and validity Precision in measurement and measuresPrecision in measurement and measures Quantitative vs. qualitative measuresQuantitative vs. qualitative measures

* Guest lectures on 10, 17 and 24 March.* Guest lectures on 10, 17 and 24 March.

3 march 2009 @ crawford school 1 empirical research design – 2 semester 1, 2009 pogo8096/8196:...

Documents

existing body of research

questions research topics

empirical research design

existing argument

good research question

week research topics

bottomline argument

alternative argument