experimental validity

Comprehensive School ImprovementExperimental Validity

Comprehensive School Improvement What is the difference?

Reliability: Degree of consistency that the instrument or procedure demonstrate. Whatever it is measuring it does so consistently

Validity: quality of a data gathering instrument or procedure that enables it to measure what it is supposed to measure

Reliability and ValidityReliable but not validReliable but not valid

Reliability and ValidityValid but not reliableValid but not reliable

Reliability and ValidityNeither reliable nor valid: BiasedNeither reliable nor valid: Biased

Reliability and ValidityReliable and validReliable and valid

Comprehensive School Improvement Internal Validity: A project has internal

validity to the extent that the factors that have been manipulated actually have a genuine effect on the observed consequences in the experimental setting

The reflects the independent variable versus the dependent variables

Comprehensive School Improvement Variables

Independent?

Dependent?

Comprehensive School Improvment

Internal Validity: some factors that have the potential to influence the results of the project in ways that are difficult to evaluate

Comprehensive School Improvement

Internal Validity Maturation: Subjects change in maturity over a

period of time and these changes may be confused with the effect of the variable under consideration


Internal Validity History: Specific external events occurring

between the first and second measurement and beyond the control of the researcher may have an effect upon the performance of the subjects


Internal Validity Testing: The process of pretesting at the

beginning of the project can produce a change in subjects. Pretesting may produce a practice effect that can make subjects more proficient in subsequent testing


Internal Validity Unstable Instruments: If instrument or

observations are not accurate or consistent, a serious element of error is introduced. If human observers are used to describe behavior, changes in subjects or changes in the observers can introduce error


Internal Validity Statistical Regression: Operates when subjects

are selected on the basis of extremely high or low pretest scores and when the measurement device is not totally reliable. This occurs only when the subjects are selected as a group because of their extreme scores


Internal Validity Selection Bias: this is likely when upon invitation

volunteers are used as members of an experimental group. Their higher motivation may introduce bias. Selection bias may be introduced when intact classes are used as experimental and control groups


Internal Validity Experimental Mortality: Loss of subjects,

particularly likely in a long term project. The major concern here is whether the groups experience different loss rates or reasons for dropouts that might impact the results


Internal Validity Experimenter Bias: Researcher has some

previous knowledge about the subjects involved. This knowledge about the subjects involved may cause the researcher to provide some clue that affects the subjects performance or may affect the objectivity of judgment


External Validity: the extent to which the variable relationships can be generalized to other settings, other treatment variable, other measurement variables and other populations


External Validity Interference of Prior Treatments: Learning

produced by the first treatment in not completely erased and its influence may accrue to the advantage of the second treatment. This is a concern in a single-group where the same subjects serve as members of both the control and experimental group

Comprehensive School Improvemetn

External Validity Artificiality of the Setting: In an effort to control

the variable the researcher imposes careful controls which may introduce an artificial atmosphere that is not at all like the real life situation


External Validity Interaction Effect of Testing: The use of a

pretest at the beginning of a study may sensitize individuals by making them aware of the researcher purpose and may serve as a stimulus to change


External Validity Interaction of Selection and Treatment:

Samples used in most classroom experiments are usually composed of intact groups, not randomly selected individuals. They are based upon an accepted invitation to participate. Consequently generalizations from samples to populations is hazardous


External Validity Extent of Treatment Verification: Due to the

potential threat of experimenter bias or time most researchers have. Other individuals who are not directly involved in the formulation of the research deliver the treatment. Was the treatment administered as intended and described by the researcher?

experimental validity

Documents