1330105791.13392.webmail191.mynet.com 8778184033 quantitative-experimental

8/2/2019 1330105791.13392.Webmail191.Mynet.com 8778184033 Quantitative-experimental

1/43

EXPERIMENTALRESEARCH


2/43

A very common way of research design

A good design to establish cause and effectrelationships among variables

It is the only type of research that

directly attempts to influence a particular variable

tests hypotheses about cause-and-effectrelationships


3/43

Major Characteristics of

Experimental Research

The researcher looks at the effect(s) of at leastone independent variable (experimental ortreatment variable) on one or more dependentvariables (criterion or outcome variable)

The researcher manipulatesthe independentvariable, which is not the case in other researchdesigns

Aim of the researcher is to see whether thetreatment/manipulation has made a difference.


4/43

Some independent variables: Methods of instruction

Types of assignment

Learning materials

Types of questions asked by teachers

Some dependent variables: Achievement

Interest

Motivation Attitudes

Now, write possible research questions and hypothesesconsidering these variables.


5/43

Comparison of Groups

Experimental research involves two groups ofsubjects

Experimental group

Control / comparison group

Experimental groupreceives the treatment;control groupreceives no treatment /comparison

groupreceives a different treatment

Comparison group rather than control group ineducational research


6/43

Manipulation of the

Independent Variable

The researcher actively manipulates theindependent variable: s/he decides the natureof the treatment, to whom and to what extent it

will be applied, and when, where and how itwill be done

Variables thatcanbe manipulated: teachingmethod, learning activities, materials, etc

Variables thatcannotbe manipulated: gender,ethnicity, age, etc


7/43

Ways of establishing

independent variables

A) one form of the variable versus another E.g. comparing the inquiry method vs the lecture

method in teaching listening

B) presence versus absence of a particular form E.g. comparing the use of videos vs no videos in

teaching speaking

C) varying degrees of the same form Comparing the effects of different amount of

homework on students motivation


8/43

Control of Variables

In experimental research, the researcher has thechance to control the variables for internal validitypurposes.

It is important for the researcher to be sure that NOTOTHER variables have affected the result.

E.g. studying students attitudes towards English: ifyou dont make sure that the groups are equivalent in

ability, then ability might be a reason for the cause.

There are some ways to control variables andminimize threats to internal validity.


9/43

a) Randomization

Random assignment: assigning the subjects togroups

Points to consider

Takes place before the experiment starts It is a process of assigning individuals to groups

It allows the researcher to assign two equivalentgroups at the beginning of the study

Equivalence is not possible if there are notenough number of subjects in a group (norules for the number, but generally not fewerthan 40 in a group)


10/43

b) Holding certain variables

constant

To eliminate its possible effects, the researchercan remove a variable from the study.

E.g. if gender is considered to influence thestudy, you can restrict the subjects to femalesor males only.

In that case, generalization is reduced!


11/43

c) Building the variable into the

design

Opposite of the previous idea

E.g. The researcher includes both females andmales (as distinct groups) and analyze theeffects of both gender and method onoutcomes


12/43

d) Matching

Pairs of subjects can be matched on certainvariables of interest.

E.g. if age is considered to be an effect, theresearcher can assign one member of eachage group to each of the comparison groups


13/43

e) Using subjects as their own

control

Using the same subjects for all the treatments

E.g.

teaching the same students by an inquiry methodand later by a lecture method

Assessing behavior during a period of time before

and after a treatment to see whether changes inbehavior occur


14/43

f) Using analysis of covariance

Equating groups statistically on the basis of apretest or other variables. The posttest scores

are adjusted accordingly


15/43

Internal Validity

Possible threats to internal validity

A) Selection Bias (subject characteristics: Might be differences in groups (age, gender,

intelligence)

B) Loss of Subjects (mortality) Losing some of the subjects (e.g. some subjects may

not return the questionnaires)

C) Location Place in which data are collected or an

implementation is conducted may affect the study


16/43

D) Instrumentation

instrument decay (e.g. essay scoring causes fatigueand can be score differently)

data collector characteristics (age, gender, experience

of data collector) data collector bias (e.g. if the collector knows the

hypotheses, s/he might unconsciously distract thedata)

E) Testing

Subjects can remember the test if the same is used inpre and post test at a short interval


17/43

F) History personal experiences during the course of the

study

G) Maturation E.g. young learners will change due to maturation

H) Attitude of Subjects The way subjects view the study (e.g.

experimental group might perform better becausethey know they are having a special treatment)


18/43

I) Regression

(e.g. if you include all good students in one group,they will improve no matter what you do)

J) Implementation

The researcher can give the experimental group

an advantage

Group Designs in Experimental


19/43

Group Designs in ExperimentalResearch:

WEAK Experimental Designs The One-Shot Case Study: A single group is exposed to a treatment and a

dependent variable is observed to assess the effect ofthe treatment

X O

treatment observation Why weak?

Absence of control: the researcher does not knowwhether the result is due to the treatment

Not possible to compare so the researcherdoesnt know whether the treatment had anyeffect at all

To remedy this, a comparison can be made with

another group who had the same course content


20/43

The One-Group Pretest-Posttest Design: A single group is observed before and after treatment

O X O

Pretest treatment posttest

Better than one-shot case study (we know whetherchange occurred), but it is still weak. Why?

Threats to internal validity: history, maturation,instrument decay, data collector characteristics, datacollector bias, testing, regression, attitude of subjectsand implementation (The researcher does not know ifthe difference is due to the treatment or any of these!)

To remedy this, a comparison group which does nothave the treatment can be added.


21/43

The Static-Group Comparison Design Two already existing groups (nonequivalent) are

usedX O..

O

Although it provides better results, it might stillhave problems. Why?

Some control on testing, regression threats; alittle control on history and maturation;problems of control on mortality and location


22/43

The Static-Group Pretest-Posttest Design Similar to static group comparison. The only

difference is to give a pretest to both groupsO X O

O O While analyzing, the pretest score is subtracted

from the posttest score, so change in eachstudent is analyzed.

Some possible threats? Subject characteristics (group scoring higher on

the pretest is likely to improve more)

Testing threat (pre and post test)


23/43

True Experimental Designs

The Randomized Posttest-Only Control GroupDesign Involves two groups which are formed by random

assignment: one receives the treatment; the other does not

Treatment group R (random) X O

Control group R (random) C O Control of some threats is excellent: subject

characteristics, maturation, regression

No testing threat because subjects are not testedtwice

Possible threats: Mortality (if more subjects drop out in one, then results

can be affected; the number of the subjects whodropped out should be reported)

Implementation, data collector bias, location andhistory threats may also exist


24/43

The Randomized Pretest-Posttest ControlGroup The difference is in using both pre and post test

for both groups

Treatment group R O XO

Control group R O X O

This design helps the researcher see if therandom assignment actually succeeded(pretest).

This is particularly important

if the number in each group is small


25/43

The Randomized Solomon Four-Group Design Used to eliminate the possible effect of a pretest

Treatment group R O X O

Control group R O C O

------------------------------------------------Treatment group R X O

Control group R C O

The first group is like pretest-posttest control group;the second is posttest only control group

Best control of threats Weakness: requires a large sample (subjects are

assigned to four groups); requires a lot of energy andeffort

QUASI EXPERIMENTAL


26/43

QUASI-EXPERIMENTALDESIGN

It is like experimental design but it doesnt userandom assignment.

It uses other techniques to control threats

There are many techniques used in this designbut the most commonly used one in socialresearch is nonequivalent groups design


27/43

Nonequivalent groups design

It is like pretest-posttest randomized

experiment but does not use randomassignment.

Instead similar groups are used as the

treatment and the control groups E.g. you can pick up two comparable

classrooms or schools.


28/43

CONCLUSION

In any type of experimental design, we create twogroups that are equivalent to each other.

One group gets the treatment; the other does not.

In all other respects the groups are treated the

same. If we observe differences in the outcome between

these groups, then we believe that the differenceis due to the treatment.

However, in all these designs, we are setting upan artificial situation, so we have to consider allthe situation.

If the situation is right, then an experiment can bea very strong design to use.


29/43

EXERCISES


30/43

Exercise 1. A researcher uses morning and afternoon historyclasses to test a new instructional strategy. In the morningclass the students are given a pretest to assess theirknowledge of WWI and then they are given the opportunity tolearn about this topic using a computerized program. In theafternoon class the students are also given a pretest; theylearn about WWI through discussion. At the end of six weeks,a final exam is given to both classes.

1. a) How is the research designed?

Quasi-experimental (non-equivalent pre-test/post-test)Gr. A (NR) O X1 O

Gr. B (NR) O X2 O


31/43

A researcher uses morning and afternoon history classes totest a new instructional strategy. In the morning class thestudents are given a pretest to assess their knowledge ofWWI and then they are given the opportunity to learn aboutthis topic using a computerized program. In the afternoonclass the students are also given a pretest; they learn aboutWWI through discussion. At the end of six weeks, a finalexam is given to both classes.

1. b) What are possible threats to internal validity?

Selection bias?

Location?

Instrumentation (data collector bias)?


32/43

A researcher uses morning and afternoon historyclasses to test a new instructional strategy. In themorning class the students are given a pretest toassess their knowledge of WWI and then they are

given the opportunity to learn about this topic using aprogram. In the afternoon class thestudecomputerized nts are also given a pretest; theylearn about WWI through discussion. At the end of sixweeks, a final exam is given to both classes.

1. c) How can you analyze the data?

t-test (comparison of the means in pre- and post-testsfor both groups)


33/43

Exercise 2. A social studies teacher isinterested in determining the effects of a newtextbook series, which has a multicultural

thematic approach. The teacher will teach aunit on colonization in America. After the unit,a multicultural attitude scale will beadministered to students.


one-shot case study


34/43

A social studies teacher is interested indetermining the effects of a new textbookseries, which has a multicultural thematic

approach. The teacher will teach a unit oncolonization in America. After the unit, amulticultural attitude scale will be administeredto students.

2. b) What are possible threats to internalvalidity?

Almost no control


35/43

A social studies teacher is interested indetermining the effects of a new textbookseries, which has a multicultural thematic

approach. The teacher will teach a unit oncolonization in America. After the unit, amulticultural attitude scale will be administeredto students.


Descriptive statistics


36/43

Exercise 3 A school administration wants to field-test aMethodology curriculum in the ELT course that is offered inthe summer. Two classes are to be offered during thesummer session. Students in both classes will be given apretest to measure their knowledge of Methodology. Oneclass will study Methodology using traditional textbookmaterials with a mostly lecture-style of delivery. Anotherclass, an experimental class, will study Methodology usingfilms and contemporary materials along with presentations byguest speakers. The final exam for the course, a parallelform of the pretest, will be given to both groups.


Quasi-experimental (pre-test/post-test nonequivalentcomparison groups)


37/43

A school administration wants to field-test a Methodologycurriculum in the ELT course that is offered in the summer.Two classes are to be offered during the summer session.Students in both classes will be given a pretest to measuretheir knowledge of Methodology. One class will studyMethodology using traditional textbook materials with amostly lecture-style of delivery. Another class, anexperimental class, will study Methodology using films andcontemporary materials along with presentations by guestspeakers. The final exam for the course, a parallel form ofthe pretest, will be given to both groups.

3. b) What are possible threats to internal validity?

Regression?

Selection bias?


38/43

A school administration wants to field-test aMethodology curriculum in the ELT course that isoffered in the summer. Two classes are to be offeredduring the summer session. Students in both classeswill be given a pretest to measure their knowledge of

Methodology. One class will study Methodology usingtraditional textbook materials with a mostly lecture-style of delivery. Another class, an experimentalclass, will study Methodology using films andcontemporary materials along with presentations by

guest speakers. The final exam for the course, aparallel form of the pretest, will be given to bothgroups.


t-test between and within groups


39/43

Exercise 4 With the current emphasis on parentalinvolvement in childrens academic endeavors, aresearcher is interested in assessing theconfidence level of parents who will read orally to

children in schools. He also wants to know ifthere is a difference in the confidence level ofparents of different education levels. From a listof volunteers who are interested in reading orallyto students, parents will be randomly selected and

assigned to one of two conditionstraining or notraining. Following training of parents in theexperimental condition, all parent volunteers willcomplete a confidence scale. Describe this studyusing a true experimental posttest-only control

group design.


40/43

Random assignment for two groups: give anumber to each and draw out a number for GroupA and then Group B until you finish all thenumbers.

Indep variable: training parents

Dep variable: parents confidence level

Group A will get training whereas Group B will not.Parents in both groups will read orally to childrenand then they will be given a confidence scale. T-test for the mean scores obtained from this scalefor each group will be found out to see the effectof training on confidence level.


41/43

Exercise 5 The dean of a college of education is interested indetermining if the education majors who take an optionalcourse in working with students with disabilities are using thestrategies learned in the course. He decides to study thisissue. Fifty education majors will be randomly assigned totake the course when they enter as freshman. The controlgroup of fifty education majors will be assigned a multiculturaleducation course. The junior field experience will occur priorto the courses, thus allowing monitoring for strategies prior totaking the course. Students will again be monitored for useof strategies during student teaching. In both the pre-courseand post-course field experiences, students will each teachten lessons in their classrooms. These lessons will bevideotaped. A researcher will view the videotapes andobserve the lessons to determine which strategies are used.Describe this study using a true experimental, pretest-posttest control group design.


42/43

For the following exercises, work in pairs anddescribe the research design answeringfollowing questions:

1. How will the subjects be assigned togroups?

2. Identify relevant variables

3. Describe the treatment design 4. Identify the way the data will be analyzed.


43/43

Please read a few article which used

experimental research method!!

1330105791.13392.webmail191.mynet.com 8778184033 quantitative-experimental

Documents