research methods of applied linguistics and statistics (3) types of research constructing research...
TRANSCRIPT
Research Methods of Research Methods of Applied Linguistics Applied Linguistics
and Statisticsand Statistics(3)(3)
Types of researchTypes of research
Constructing Research Constructing Research DesignsDesigns
Research TypesResearch Types
Data sources:Data sources: primary, secondaryprimary, secondary Objectives:Objectives: descriptive, exploratory, correlational, descriptive, exploratory, correlational,
explanatoryexplanatory Time for data collection:Time for data collection: cross-sectional, longitudinal (panel, trend)cross-sectional, longitudinal (panel, trend) Data types:Data types: qualitative, quantitative (survey, qualitative, quantitative (survey,
experimental)experimental)
Primary: studies based on primary, or Primary: studies based on primary, or original, data sources, such as original, data sources, such as classroom observations or real classroom observations or real students, or their test scores, or their students, or their test scores, or their responses to a questionnaire.responses to a questionnaire.
Secondary: studies based on secondary Secondary: studies based on secondary sources such as other researherssources such as other researhers’’ books and articles, including library books and articles, including library research and literature reviews. research and literature reviews.
Descriptive: studies involving the Descriptive: studies involving the collection of data in order to test collection of data in order to test hypotheses or to answer questions hypotheses or to answer questions concerning the current status of the concerning the current status of the subjects of the study. subjects of the study.
Exploratory: studies conducted into an Exploratory: studies conducted into an issue or problem where there are few or issue or problem where there are few or no earlier studies to refer to. The focus no earlier studies to refer to. The focus is on gaining insights and familiarity for is on gaining insights and familiarity for later investigation. later investigation.
Correlational: The basic question for Correlational: The basic question for descriptive research is - "What are the values descriptive research is - "What are the values of a number of variables for a given sample of of a number of variables for a given sample of subjects. The basic research question for subjects. The basic research question for correlation research is - What is the correlation research is - What is the relationship between two or more variables relationship between two or more variables for a given set of subjects. Notice that we said for a given set of subjects. Notice that we said relationship between variables and not the relationship between variables and not the effect of one variable on another variable. effect of one variable on another variable.
In descriptive research we are just describing In descriptive research we are just describing our subjects in terms of one or more our subjects in terms of one or more variables, while in correlational research we variables, while in correlational research we are looking at the relationship between the are looking at the relationship between the variables.variables.
Explanatory: studies conducted in Explanatory: studies conducted in order to explain any behaviour in L2 order to explain any behaviour in L2 learning and teaching. It could be learning and teaching. It could be done by using questionnaires, group done by using questionnaires, group discussions, interviews, etc.discussions, interviews, etc.
Cross-sectional: studies in which Cross-sectional: studies in which groups of participants of different groups of participants of different ages are observed and compared at ages are observed and compared at a given time a given time
Longitudinal: It is research that studies a Longitudinal: It is research that studies a person or group over a set period of time, person or group over a set period of time, normally to track the effect of some normally to track the effect of some variable and as such permits causal variable and as such permits causal pathways to be determined . For example, pathways to be determined . For example, when trying to discover the effect of when trying to discover the effect of language aptitude on L2 proficiency, a language aptitude on L2 proficiency, a researcher may track 50 children over a researcher may track 50 children over a 30 year period in order to find a common 30 year period in order to find a common factor between those children who are factor between those children who are quicker and effective L2 learners. quicker and effective L2 learners.
Constructing Research Constructing Research DesignsDesigns
Determining the designDetermining the design
Classification of designsClassification of designs
Research Design 1: Research Design 1: Quantitative / QualitativeQuantitative / Qualitative
Quantitative research is empirical research Quantitative research is empirical research involving analysis of numerical data Its aim is involving analysis of numerical data Its aim is to determine the relationship between one to determine the relationship between one thing (an independent variable) and another thing (an independent variable) and another (a dependent or outcome variable) in a (a dependent or outcome variable) in a population. Quantitative research designs are population. Quantitative research designs are either descriptive (subjects usually measured either descriptive (subjects usually measured once) or experimental (subjects measured once) or experimental (subjects measured before and after a treatment). A descriptive before and after a treatment). A descriptive study establishes only associations between study establishes only associations between variables. An experiment establishes causality. variables. An experiment establishes causality.
Qualitative research is empirical research Qualitative research is empirical research involving analysis of data such as words involving analysis of data such as words (e.g., from interviews), pictures (e.g., (e.g., from interviews), pictures (e.g., video), or objects (e.g., an artifact). Its aim video), or objects (e.g., an artifact). Its aim is building theory. It is generally inductive is building theory. It is generally inductive in approach, is based originally on the in approach, is based originally on the naturalistic assumption that reality is mind naturalistic assumption that reality is mind dependent (i.e., can only be known as it is dependent (i.e., can only be known as it is interpreted and has "meaning" for the interpreted and has "meaning" for the observer), is usually of single-subject observer), is usually of single-subject design, and generally deals with nominal design, and generally deals with nominal data data
Philosophical basePhilosophical base
Phenomenology (quali.) Logical positivism (quan.)
Data from personal involvement
Data from experiment
Subjective Objective
Empathy Distance
Data in the form of string
Data in form of number
MethodologyMethodology
Synthetic Analytic
From part to whole From whole to part
Internal structure External structure
Process-oriented Product-oriented
Dynamic reality hypothesis
Static reality hypothesis
PurposePurpose
Induction Deduction
Hypothesis generating Hypothesis confirming
Exploratory, descriptive
Inferential, confirmatory, explanatory
Result of pattern or hypothesis
Formation of theory
Features for both Features for both designsdesigns
High validity, low reliability
Low validity, high reliability
Preference of content to form
Preference of form to content
Low generalizing capacity
High generalizing capacity
High explanatory power
Low explanatory power
Mixed design: BalancedMixed design: Balanced
Mixed design: Mixed design: UnbalancedUnbalanced
Literature review of quantitative and qualitative studies by Lazaraton (2000)
Current Trends in Research Methodology and Statistics in Applied Linguistics
Research Design: Research Design: experimental researchexperimental research
An experimental study is a study in An experimental study is a study in which the researcher manipulates which the researcher manipulates one or more independent variables one or more independent variables and measures their effect(s) on one and measures their effect(s) on one or more dependent variables while or more dependent variables while controlling the effect of extraneous controlling the effect of extraneous variables.variables.
Major components of Major components of experimental researchexperimental research
The populationThe population A comparison which involves at least 2 A comparison which involves at least 2
groups or 2 conditions. groups or 2 conditions. The treatmentThe treatment A treatment in which one or more A treatment in which one or more
independent variables are manipulated.independent variables are manipulated. The measurement of the treatmentThe measurement of the treatment The measurement of dependent The measurement of dependent
variable(s) as the result of the variable(s) as the result of the treatment. treatment.
Letters standing for Letters standing for componentscomponents
X: an experimental treatmentX: an experimental treatment (teaching method, exposure to a (teaching method, exposure to a materialmaterial……))
-- : no treatment: no treatment O: measurement of the effects of the O: measurement of the effects of the
treatment or a testtreatment or a test R: randomization, or the random R: randomization, or the random
assignment of subjects to groups in assignment of subjects to groups in order to control for extraneous order to control for extraneous variables.variables.
Types of experimental Types of experimental studiesstudies
Pre-experimental Quasi-experimental True-experimentalOne-group posttest only study:
X O1
Posttest only nonequivalent groups:
EG: X O1
CG: O2
Posttest only equivalent groups
EG: R X O1
CG: R O2
One-group pre/post test study:
O1 X O2
Pre/post test nonequivalent groups:
EG: O1 X O2
CG: O3 O4
Pre/post test equivalent groups:
EG: R O1 X O2
CG: R O3 O4
No control group; no randomization
a control group; no randomization
a control group + randomization
Pre-experimental:Pre-experimental: single single group designgroup design
One-shot designOne-shot design One group pretest-posttest One group pretest-posttest
designdesign Time-sampling designTime-sampling design
Example for one-shot Example for one-shot designdesign
An English teacher used the An English teacher used the immersion teaching method for immersion teaching method for teaching English communicative teaching English communicative use. After 3 weeks, a test is use. After 3 weeks, a test is administered to the class. The class administered to the class. The class performs well on the test. performs well on the test.
X OX O It is also called a pilot study designIt is also called a pilot study design
One group pretest-One group pretest-posttest designposttest design
It is also called repeated It is also called repeated measures. measures. It uses the subjects as It uses the subjects as their own controls and eliminates their own controls and eliminates the need for a control group design.the need for a control group design.
StrengthStrength: It is efficient because : It is efficient because some extraneous variables, attrition some extraneous variables, attrition and loss of subjects controlled. and loss of subjects controlled.
WeaknessWeakness: validity might be affected : validity might be affected by history, attrition and maturation.by history, attrition and maturation.
Time-sampling designTime-sampling design
It is also referred to as time-series It is also referred to as time-series designs.designs.
OO11 O O22 O Onn X O X O44 O O55 O Onn
OO11 X O X O22 O O33 -- O -- O44 OO55 X O X O66 OO77 -- O -- O88
Quasi-experimental: posttest only
nonequivalent groups
• EG: X O1
CG: O2
• O1-O2
• The treatment effect could also be attributed to a series of extraneous variables.
Posttest/posttest nonequivalent groups
EG: O1 X O2
CG: O3 O4
(O(O22-O-O11) ) –– (O (O44-O-O33)) Without randomization, the Without randomization, the
confounding effects cannot be fully confounding effects cannot be fully controlled. controlled.
True-experimental: posttest only equivalent groups
EG: R X O1
CG: R O2
O1-O2
The design assumes that the 2 groups are identical on all relevant features, but it is still sensitive to selection bias, mortality, Hawthorn effects.
Pre/post test equivalent groups
EG: R O1 X O2
CG: R O3 O4
Factorial DesignsFactorial Designs
Factorial designs are similar to the Factorial designs are similar to the true experimental designs and true experimental designs and include all of the elements found in include all of the elements found in those designs, such as those designs, such as randomization, pre- and post-testing, randomization, pre- and post-testing, and treatments. The difference is and treatments. The difference is that the effects of independent that the effects of independent variables and a moderating variable variables and a moderating variable may be tested at the same time. may be tested at the same time.
Example for a factorial Example for a factorial designdesign
In a study to test for the effect of language In a study to test for the effect of language laboratory training on pronunciation, it is also laboratory training on pronunciation, it is also decided to measure for the effect of such decided to measure for the effect of such training on learners of different proficiency training on learners of different proficiency levels, as well as the effect of different kinds levels, as well as the effect of different kinds of language laboratory practice. of language laboratory practice.
Types of practice (contextualized / Types of practice (contextualized / decontextualized drills) , proficiency levels of decontextualized drills) , proficiency levels of the learner (beginners / advanced learners )the learner (beginners / advanced learners )
Exemplary Figure for Exemplary Figure for Factorial DesignFactorial Design
Experimental study by Experimental study by Robb, et al. (1986)Robb, et al. (1986)
Salience of feedback on error and its effect on EFL writing quality
Types of feedback: complete correction, Types of feedback: complete correction, coded feedback; yellow highlighting of coded feedback; yellow highlighting of errors, listing the number of errorserrors, listing the number of errors
The compositions were graded on 19 The compositions were graded on 19 features which were collapsed into 7 features which were collapsed into 7 categories. categories.