copyright © allyn & bacon 2008 intelligent consumer chapter 14 this multimedia product and its...

Copyright © Allyn & Bacon 2008

Intelligent Consumer

Chapter 14

This multimedia product and its contents are protected under copyright law. The following are prohibited by law:• Any public performance or display, including transmission of any image over a network;• Preparation of any derivative work, including the extraction, in whole or in part, of any images;• Any rental, lease, or lending of the program.


Discussion Topics

The purpose of the text Questions for quantitative studies Questions for qualitative studies Reviewing three studies


Text Purpose

Present and explain the fundamental principles of educational research

– The quality of published educational research varies greatly– There is a need to have the knowledge and skills to critically

evaluate educational research

Rephrasing the consumer tips from each chapter into questions to provide guidelines for the evaluation of research articles


Quantitative Criteria

Topics for evaluation questions for quantitative studies

– Research problem– Review of the literature– Research hypotheses– Selection of participants– Instrumentation– Design– Results– Discussion and conclusions


Questions for Quantitative Studies

Research problem– What are the independent and dependent

variables?– Is the problem researchable?– Is the problem significant? Will the results have

practical or theoretical importance?



– Is the problem stated clearly and succinctly?– Does the problem communicate whether the

study is descriptive, experimental, or non-experimental?

– Does the problem indicate the population studied?

– Does the problem indicate the variables in the study?



Review of the literature– Does the review of literature seem

comprehensive? Are all important previous studies included?

– Are primary sources emphasized?– Is the review up-to-date?– Have studies been critically reviewed, flaws

noted, and results summarized?



– Does the review emphasize studies directly related to the problem?

– Does the review explicitly relate previous studies t the problem?

– If appropriate, does the review establish a basis for research hypotheses?

– Is the review well organized?



Research hypotheses– Is the hypothesis stated in declarative form?– Does the hypothesis follow from the literature?– Does the hypothesis state expected relationships

or differences?– Is the hypothesis testable?– Is the hypothesis clear and concise?



Selection of participants– Are the participants clearly described?– Is the population clearly defined?– Is the method of sampling used? If so, is it

proportional or disproportional?– What is the return rate in a survey study?– Are volunteers used?– Is there an adequate number of participants?



Instrumentation– Is evidence for validity and reliability clearly stated and

adequate? Is the instrument appropriate for the participants?

– Are the instruments clearly described? If an instrument is designed for a study by the researchers, is there a description of its development?

– Are the procedures for gathering data clearly described?– Are norms appropriate if norm-referenced tests are used?



– Are standard-setting procedures appropriate if criterion-referenced tests are used?

– Do the scores distort the reality of the findings?– Do response sets or faking influence the results?– Are observers and interviewers adequately trained?– Are there observer and interviewer effects?



Design– Non-experimental

If descriptive, are relationships inferred? Do graphic presentations distort the findings? If comparative, are criteria for identifying different groups

clear? Are causative conclusions reached from correlational

findings?



Is the correlation affected by restriction in the range and reliability of the instruments?

If predictions are made, are they based on a different sample?

Is the size of the correlation large enough? If causal-comparative, has the causal condition already

occurred? How comparable are the subjects in the groups being compared?



– Experimental Is there direct manipulation of an independent variable? Is the design clearly described? What extraneous variables are not controlled in the design? Are the treatments very different from one another? Is each replication of the treatment independent of the other

replications? Is the number of subjects equal to the number of treatment replications?



– Single subject Is the measurement of the target behavior reliable? Is the target behavior clearly defined? Are there enough measures of the behavior to establish

stability? Are procedures, subjects, and settings described in

detail? Is there a single treatment? Are there experimenter or observer effects?



Results– Is there an appropriately descriptive statistical

summary?– Is statistical significance confused with practical

significance?– Is statistical significance confused with internal or

external validity?– Are appropriate statistical tests used?



– Are levels of significance interpreted correctly?– How clearly are the results presented?– Is there a sufficient number of subjects to give

valid statistical results?– Are data clearly and accurately presented in

graphs and tables?



Discussion and conclusions– Is interpretation of the results separate from reporting of the

results?– Are the results discussed in relation to previous research,

methodology, and the research problem?– Do the conclusions address the research problem?– Do the conclusions follow from the interpretation of the

results?



– Are the conclusions appropriately limited by the nature of the subjects, treatments, and measures?

– Is lack of statistical significance properly interpreted?– Are the limitations of the findings reasonable? – Are the recommendations and implications specific?– Are the conclusions consistent with what is known from

previous research?


Qualitative Criteria

Topics for evaluation questions for qualitative studies– Introduction and problem– Review of the literature– Methodology– Discussion and conclusions


Questions for Qualitative Studies

Introduction and problem– Are the researcher’s background, interests, and

potential biases clear from the outset?– Does the researcher have the skill and training

needed to conduct the study?– Is the problem feasible?



– Is the problem significant?– Is there a clear conceptual and theoretical

framework for the problem?– Does the introduction include an overview of the

design of the study?– Is the purpose of the study clearly stated?



Review of the literature– Is the review preliminary? Does it indicate that the

researcher is knowledgeable about previous work in the area?

– Is the review up-to-date?– Does the review establish an adequate background and

theoretical framework for the study?– Is the review well organized?– Is the literature analyzed as well as summarized?



Methodology– Is the method of selecting participants clear?– Is the selection of participants biased?– Will the participants selected provide a credible

answer to the research question?– How involved is the researcher in the setting? Will

the researcher’s involvement affect the findings?



– How detailed are the field notes?– Are data collectors properly trained?– Are multiple methods of data collection used?– Is the study long enough?



Discussion and conclusions– Are descriptions clearly separate from

interpretations and researcher’s opinions?– Is the credibility of the findings addressed in terms

of reliability, internal validity, an external validity?– Are the results discussed in relation to previous

research?



– Do the conclusions follow from the interpretation of the results? Are the conclusions consistent with what is known from previous research?

– Are appropriate limitations indicated?– Are appropriate recommendations and

implications indicated?


Evaluation of Studies

Evaluation of a quantitative, non-experimental research article

– Article 1 – Investigating High School Students’ Science Experiences and Mechanics Understanding by Phillips, K. A. & Barrow, L. H.

– See the text Evaluation of a quantitative, experimental research

article– Article 2 – The Effect of Reciprocal Peer Counseling in the

Enhancement of Self-Concept Among Adolescents by Egbochuku, E.O., & Obiunu, J.J.

– See the text


Evaluation of Studies

Evaluation of a qualitative research article– Article 3 – How Elementary School Counselors

Can Meet the Needs of Students with Disabilities by Frye, H.N.

– See the text

copyright © allyn & bacon 2008 intelligent consumer chapter 14 this multimedia product and its...

Documents

quantitative studies

copyright allyn bacon

quantitative studies

qualitative studies

copyright law

evaluation questions

text questions

previous studies t