Originally developed by Dr. Jim DyerOriginally developed by Dr. Jim Dyerof the University of Floridaof the University of FloridaModified by Dr. David Agnew, September, 2006Modified by Dr. David Agnew, September, 2006Arkansas State UniversityArkansas State University

Research Applications in Career & Technical Education

Review

Overview of ResearchOverview of Research

What is research? (review) What are the types of studies? What terms are associated with

research? What is the purpose of research? What are the characteristics of

research?

What is Research?What is Research?

The systematic process of collecting and analyzing information to increase our understanding of phenomenon under consideration

Disciplined Inquiry

What Are the Types of Studies?What Are the Types of Studies?

Status variable cannot be manipulated by researcher.

Associational variable can be manipulated by researcher, but

is not. Experimental

variable is manipulated by researcher.

What terms are associated with What terms are associated with research?research?

Variable any factor having two or more mutually exclusive

properties or values. Dependent (criterion) variable

Outcome, cannot change. Independent variable

can manipulate.

TermsTerms

Extraneous variable may affect dependent variable and change

result. Antecedent variable

present at beginning. Covariate

antecedent or extraneous variable measured before and after treatment.

Terms Terms (continued)(continued)

Discrete one specific meaning--hair color, grade (A-F),

marital status (S,M,D,W) Continuous

can have only one value--IQ, age Dichotomous

discrete variable with only two distinct categories--yes/no, gender

TermsTerms Hypothesis

statement of expectation.

Null Hypothesis (HO) stated for no change.

Alternate Hypothesis (HA) stated for change.

Directional Hypothesis (H1) states direction of change.

TermsTerms Cronbach’s alpha

measures internal consistency based upon relationship to other statements on instrument.

a priori prior to

Population (target population) group to which we generalize. (N)

Sample smaller group of population. (n)

Terms Terms (continued)(continued)

Sampling frame list of accessible people.

Experimentally accessible population members of population we can access.

Parameter unknown characteristic of the population.

Statistic known characteristic of the sample.

What is the Purpose of What is the Purpose of Research?Research?

Describe -- Ex: settings Predict -- Ex: success based on ACT Improve--Ex: teaching methods Explain -- answers “why?”

What Are the Types of What Are the Types of Research -- by PurposeResearch -- by Purpose

Basic results in development of new knowledge

without concern for how or where it will be used. Applied

tests theories developed by basic research. Action

has specific application with the decision-maker involved in conducting the research.

Types of Research Types of Research -- by Philosophy-- by Philosophy

Quantitative -- (Positivistic) Things are meaningful only if we can verify

them with our five senses.

Qualitative -- (Post-positivistic) All research is value-laden. Can’t remove self

from research.

Types of Research Types of Research -- by Method-- by Method

Experimental Correlational Evaluation Historical Naturalistic Survey

Experimental

Correlational

Survey

Histo al

Naturalistic

Eva tion

ExperimentalExperimental

Definition Establishes cause and effect relationship. Systemmatically answers: Is there a difference

in characteristic X between and within groups and with and without characteristic Y? (Ex: productivity & background music)

ExperimentalExperimental

Purpose Seeks to identify differences between and/or

within groups to the extent they possess characteristics which can be measured in quantitative terms.

Example: “The Effect of PowerPoint Visuals on

Achievement in a Research Methods Course”

ExperimentalExperimental

Nature Involve systematic manipulation of some

characteristic Only way to identify cause and effect Starts with hypothesis (a priori) Write research problem as a question Write hypothesis as an answer to the question

(statement)

ExperimentalExperimental Strengths

rigorous; replicable can specify likelihood of errors of inference forces us to be explicit about variables to be studied

and hypotheses to be tested Limitations

tendency to disvalue qualitative information confuse statistical with logical/practical meaning results are “average,” individuals are nuisances

CorrelationalCorrelational

Definition Answers the question: “To what extent are

characteristics X and Y related?” (Ex: the louder the music, the higher the productivity).

Purpose determines relationship does NOT determine cause predict score on one variable from knowledge of

another

CorrelationalCorrelational

Nature involves collection of information about several

characteristics measures relationship

Example: The Relationship Between GRE Scores and

Graduate GPA of Agricultural Education Doctoral Students at Iowa State University

CorrelationalCorrelational Strengths

analyze relationships among large number of variables in single study

provides information on degree of relationship predict characteristic

Limitations not cause and effect difficult to identify all variables--best to use lit

review

EvaluationEvaluation

Definition Collection and use of information to facilitate

decision making or determination of worth Purpose

assists in decision making aids in planning assists in meeting accountability demands

EvaluationEvaluation

Nature intentions vary per study results for specific situation only audience is important for design & reporting usually done for a client

Example An Assessment of the Grading Practices in the

College of Agriculture at Iowa State University

EvaluationEvaluation

Strengths information for program improvement (action) provides accountability information looks at processes and products

Limitations danger of being too subjective difficult to evaluate and set criteria for some

processes & products (intangible, unintended)

HistoricalHistorical

Definition narration and description of “true” record of

past events, their developmental trends, and their interpretation

it is a longitudinal, genetic, or developmental approach in the past

attempts to discern actual meaning of primary and secondary sources by subjecting to external and internal criticism

HistoricalHistorical

Purpose construction of conceptual frameworks collection and analysis of historical evidence determination of meaning

Nature involves collection and analysis of primary and

secondary sources no variables

HistoricalHistorical

Example The Origin and Development of Agricultural

Education in America Strengths

understanding of history Limitations

past view is easily distorted in present setting difficult to remain objective--selective searching

NaturalisticNaturalistic

Definition attends to ordinary happenings and uses

common language reporting to provide experience for research audience

Purpose oriented to practice, not theory

Example Case Studies in Agricultural Education

NaturalisticNaturalistic

Nature observation and interview techniques used high attention to context narrative reporting with illustrations variables not used as conceptual structure

Strengths easy to comprehend by lay people, interesting resists some research oversimplifications

NaturalisticNaturalistic

Limitations requires special research skills to do properly researcher-biased, objectivity poorly controled difficult to summarize, lots of notes time-consuming to conduct

Survey (descriptive)Survey (descriptive)

Definition present-oriented methodology used to investigate

populations using samples to analyze interrelationships among variables

used to collect facts and assess beliefs, interests, and attitudes

Example The Career Interests of Secondary Agricultural

Education Students

Survey (descriptive)Survey (descriptive)

Purpose clarify and describe problems through data

gathering process Descriptive -- describes phenomena at point in time Comparative -- compares populations on some criteria Evaluative -- assesses aspects of setting by some criteria

Survey (descriptive)Survey (descriptive)

Nature involves identification of population, sampling

plan, & variables; data gathering instrument; natural setting

Strengths large amount of information can quickly get data well suited for extensive research

Survey (descriptive)Survey (descriptive)

Limitations may be superficial, not in-depth in longitudinal studies, difficult to remain in

contact with respondents

The Research ProblemThe Research Problem

Stating the Research ProblemStating the Research Problem Use a complete sentence with as few words as

possible. Limit/focus the statement. Examples:

Using learning teams = “What effect does learning teams have on achievement?”

Effect of problem solving approach = “What is the effect of the problem solving approach on student achievement, attitude, and retention of subject matter?”

Stating the Research ProblemStating the Research Problem Delimit research -- what will be done? Define terms -- define only those terms

which may be confusing or have special meaning.

State assumptions -- what do you assume? State research hypotheses and/or research

questions -- use these to guide study. Explain importance of study.

Reviewing LiteratureReviewing Literature

Purpose of a Lit ReviewPurpose of a Lit Review

What has been done? What was found? Were there problems with prior studies that

you can avoid? Where are the holes in the research base?

(What has yet to be found?)

Tools of Research Tools of Research (Leedy)(Leedy)

Library Measurement techniques--Design Computer Programs (SPSS, SAS) Statistics Writing ability

LibraryLibrary

Sources of information Primary sources Secondary sources

Conducting the literature review ERIC RIE DAI

Planning the Research DesignPlanning the Research Design

Criteria for Research ProjectCriteria for Research Project

Universality -- can be completed by anyone Replication -- can be repeated under same

conditions with same results Control -- use parameters to control as

many variables as possible Measurement -- important to quantify as

much as possible

Data CollectionData Collection

What data are needed? Where can we get it? How can we get it? How will it be interpreted?

Aggregate DataAggregate Data

Definition of:

To gather, collect or assemble. For example, "to aggregate data" means to gather separate sets of data. As a noun, "aggregate data" is data that has been collected from two or more sources.

Ethical StandardsEthical Standards

Personal integrity of researcher, fair, honest Right of privacy of participants Disclosure of methods Reason for research Informed willingness Respect for integritity of individual Acknowledge financial support

EffectsEffects

Hawthorne Effect -- any change in environment produces a change in response.

John Henry Effect -- Control group sees itself in competition with main group.

Pygmalian Effect -- We see what we want to see.

ContaminationContamination

Experimenter -- researcher knowingly or unknowingly influences application or observation of treatment.

Statistical -- related data are treated as being independent in the statistical analysis.

Bias in Research DesignBias in Research Design Deliberate -- check numbers Non-deliberate -- phrases, titles, etc. (“What’s

wrong with Ag. Ed.?) Selection

Volunteers Mortality (why did they leave) Groups that differ Learning time Teacher Quality

Selecting a Research ProblemSelecting a Research Problem

ID problem--What would you like to do? What are the variables? What are the possible results? How will you measure the results? Who will be involved? How will you conduct the study? What problems will you encounter?