lecture 1 educational research design · a. ex post facto research b. survey research 2....
TRANSCRIPT
Lecture 1 Educational Research Design
Fang Chen East China Normal University
PART I: INTRODUCTION
2015/10/6 陈芳 华东师大英语系 2
1. Empirical 2. Systematic 3. Valid 4. Reliable 5. Variety of forms
General characteristics of educational research
陈芳
华东师大英语系
2015/10/6 3
Figure 1.2: Validity vs. reliability
2015/10/6 陈芳 华东师大英语系 4
• Classification of educational research – Basic vs. applied research
• Extension of knowledge vs. immediate practical problem
– Qualitative vs. quantitative • Words vs. numbers
– General methods of research
5. Variety of forms
2015/10/6 陈芳 华东师大英语系 5
• Theory/hypothesis – 理论/假设 – Substantive and statistical hypothesis
• Variables – Independent variable 自变量 – Dependent variable 因变量
• Research design – 研究设计
• Literature review guides you on all the above-mentioned points.
Some Terminologies
2015/10/6 陈芳 华东师大英语系 6
PART II:RESEARCH METHODS
2015/10/6 陈芳 华东师大英语系 7
1. Experimental research design 2. Quasi-experimental research design 3. Survey methodology 4. Historical research 5. Ethnographic research 6. Delphi method
2015/10/6 陈芳 华东师大英语系 8
① Posttest-Only Control Group Design: – 后测控制组
② Pretest-Posttest Control Group Design: – 前测后测控制组
③ Solomon Four-Group Design: – 所罗门四组
④ Factorial Designs: – 多因素设计
⑤ Repeated Measures Design: – 重复测量
⑥ Time-series Design: – 时间序列
1. Experimental research design
2015/10/6 陈芳 华东师大英语系 9
• We cannot control anything but we still want to answer a set of research questions. A. EX post facto Research B. Survey Research
2. Quasi-experimental research design
2015/10/6 陈芳 华东师大英语系 10
• Ex post facto research design ① Posttest-Only, Nonequivalent control group design
② Pretest-Posttest, Nonequivalent control group
design
③ Time Series design
④ Single-subject designs
⑤ Action research
2015/10/6 陈芳 华东师大英语系 11
• Possible issues: a. if pretest scores are different across
groups? b. This design makes the basic assumption
that the pre and post tests are equivalent. What if they are not? • Define subgroups based on the scores and
build the subgroup factor into research design. • Statistical control.
Lucky you…
2015/10/6 陈芳 华东师大英语系 12
a) Single group time series design G1 – O1 – O2 – O3 – X – O4 – O5
b) Multiple group time series design G1 – O1 – O2 – O3 – X – O4 – O5 G2 – O1 – O2 – O3 – – O4 – O5
③ Time series design
2015/10/6 陈芳 华东师大英语系 13
• Typically a repeated-measure design. • Single Subject Design
a) A-B Design b) A-B-A Design
④ Single-subject design
2015/10/6 陈芳 华东师大英语系 14
• Sounds qualitative but nothing prevents it from using quantitative measures.
• The key thing is that it is design to solve an immediate and local issue. Generalizability is not a concern.
⑤ Action research
2015/10/6 陈芳 华东师大英语系 15
3.SURVEY METHODOLOGY 调研法
QUANTITATIVE AND/OR QUALITATIVE
2015/10/6 陈芳 华东师大英语系 16
1. Longitudinal-(Given Multiple Times) a) Trend Study b) Cohort Study c) Panel Study
2. Cross-Sectional Design- (Given Once)
Survey as a research method
2015/10/6 陈芳 华东师大英语系 17
• Total survey design: – Quality of the sample : the frame, the size,
the design, and the response rate – Quality of the questions as measures – Quality of data collection: especially the use
of effective interviewer training and supervising procedures
Total survey design
2015/10/6 陈芳 华东师大英语系 18
Inference and errors
2015/10/6 陈芳 华东师大英语系 19
Characteristics of Population
Sample of population members who answer
questions
Answers respondents give
Validity Issue: How well are the
answers?
Generalizability Issue: Who answered the
questions?
• What? • Why?
– Selection framework – Question quality: 3rd lecture – Interviewer quality
• How to reduce? – Access: flexible, multi-mode – Cooperation
Nonresponse
2015/10/6 陈芳 华东师大英语系 20
QUALITATIVE RESEARCH IN GENERAL
Historical Ethnographic Delphi method
2015/10/6 陈芳 华东师大英语系 21
• 5 basic descriptors as the underlying epistemology of qualitative research.
① Holistic ② Natural Settings ③ Perceptions are Important ④ Favor Post Hoc conclusions ⑤ Phenomena are loosely modeled
2015/10/6 陈芳 华东师大英语系 22
Reliability
Quantitative research 定量 • External reliability
– Can replicate the study
• Internal reliability – Based on the internal
constancy of experiment
Qualitative research 定性
2015/10/6 陈芳 华东师大英语系 23
• External reliability: – specific to natural settings, but can
make a persuasive argument so that the reader can evaluate whether it can be replicated
• Internal reliability:
– Typically, there are observations from multiple observers so can compare the agreement rates or the agreement of conclusions.
• Interrater reliability
Validity
Quantitative research 定量 • External validity
– Generalizability • Internal validity
– Can make a reasonable conclusion
Qualitative research 定性
2015/10/6 陈芳 华东师大英语系 24
• External validity: – Comparability or translatability of
the research – Not a concern for the researcher
• Internal validity:
– Can the data provide a conclusion that still apply?
• Cross-validation
4. HISTORICAL RESEARCH 历史研究
Qualitative method
2015/10/6 陈芳 华东师大英语系 25
ETHNOGRAPHIC RESEARCH 人种志研究
2015/10/6 陈芳 华东师大英语系 26
• The process of ethnographic research is an integrated process in which procedures are conducted concurrently. ① Identify the Phenomena to be studied:
What? ② Identify Subjects: Who? ③ Hypotheses: What? ④ Data Collection: How? ⑤ Analysis: How and what? ⑥ Drawing Conclusions: How and why?
The process
2015/10/6 陈芳 华东师大英语系 27
• Field notes example
From data to hypothesis
2015/10/6 陈芳 华东师大英语系 28
• Triangulation.
2015/10/6 陈芳 华东师大英语系 29
DELPHI METHOD 德尔菲法 Good for policy-related research
2015/10/6 陈芳 华东师大英语系 30
• Groupthink tends to occur when a group strive to reach consensus despite organizational flaws, combined with a high degree of homogeneity of member's social background and ideology. Extreme pressure or stress exacerbates the situation!
• Historical lessons from:
2015/10/6 陈芳 华东师大英语系 31
1. Illusions of invulnerability : 2. Ignoring warnings 3. Unquestioned belief 4. Stereotyping 5. Direct pressure to conform. 6. Self censorship 7. Illusions of unanimity: 8. Mindguards –
8 Symptoms of group think
2015/10/6 陈芳 华东师大英语系 32
• Research category • Validity and reliability • Experimental and quasi-experimental
research design • Typical qualitative research methods • Next:
– Quantitative models and modeling: General approaches and examples in language-related research
– Data collection methods for language-related research
Summary for today
2015/10/6 陈芳 华东师大英语系 33