carlo magno, phd lasallian institute for development and educational research de la salle...
Post on 14-Jan-2016
217 Views
Preview:
TRANSCRIPT
Developing Data Driven Research in School Counseling
Carlo Magno, PhDLasallian Institute for Development and Educational ResearchDe La Salle University, Manila
Acitivity 1
Work in a group List down in the manila paper four
ways on how you use test/assessment data in your school.
Discussion during presentation
School Setting
EVIDENCES
Improvement of the teaching and learning process
Improvement of counseling processes
What do we need to develop among our students?
Learning and innovation skills
Creativity and InnovationCritical Thinking and Problem SolvingCommunication and Collaboration
Information, media, and technology skills
Information LiteracyMedia LiteracyICT (Information, Communications and Technology) Literacy
Life and career skillsFlexibility and AdaptabilityInitiative and Self-DirectionSocial and Cross-Cultural SkillsProductivity and AccountabilityLeadership and Responsibility
21st century skills
How do you get evidence that your students develop 21st century skills?
21st century
skills
AssessmentInternational Level Assessment
National Level
AssessmentRegional/District
Level
AssessmentClassroom Level
Role of counselors in School
Help generate assessment information about learners.
ASCA Standards
Counseling standards in the RP
What can be done with assessment data?
Informs development of academic programs and special programs
Informs further improvement of the curriculum Identifies sections/students that needs further
help Reflection on how to teach or deliver the
curriculum better Decisions on the allocation of resources and
priorities Informs what is happening in the schools
(academic standards)
What to report Achievement gains per class/section Achievement gains per subject area Achievement gains per level Trends: Comparison across school
years Trends: Comparison with other
schools/countries/states/region
Sources of Data in schools
Standardized tests Standards-based test Results from
inventories Teacher-made tests Interview data Data from teachers Data from parents
Scope of Presentation
Approach: Quantitative Design
Correlational Comparative Experimental
Correlational Studies
Involves two variables where one increases with the other
Examples: Grades and motivation: Does student motivation
increase with students’ grades? Attitude in Math and Math performance: Does
students’ attitude in math increase with their performance in math achievement test?
Math anxiety and test in math: Does anxiety decrease math test scores?
The choice between the variables should be guided by a theory (theoretical or conceptual framework).
Both variables should be quantitatively measured.
Correlational Studies
Linear Regression There is a straight line relationship
between variables X and Y When X increases, Y also increases-
positive relationship When X increases, Y decreases or vice
versa – negative relationship
Correlational Studies
Problem: Is there a significant relationship between achievement and aptitude?
Hypothesis: There is a significant relationship between achievement and aptitude
Relationship between achievement and aptitude
Achievement (X) Aptitude (Y)
100 99
95 98
90 94
85 87
82 84
80 81
75 78
70 73
65 68
50 60
Regression Line between achievement and aptitude
Scatterplot: X vs. Y
Y = 14.379 + .85633 * XCorrelation: r = .98966
40 50 60 70 80 90 100 110
X
55
60
65
70
75
80
85
90
95
100
105
Y
95% confidence
Laziness Perseverance
100 35
95 40
90 45
85 50
75 55
70 60
65 64
60 70
55 76
50 80
Relationship between laziness and perspeverance
Relationship between Laziness and Perseverance
Scatterplot: Y vs. X
X = 139.94 - 1.138 * YCorrelation: r = -.9959
30 40 50 60 70 80 90
Y
40
50
60
70
80
90
100
110
X
95% confidence
Correlational Studies Analysis
2 variables that are interval or ratio: Pearson r 2 variables are ordinal: Spearman rho 2 variables and each is a dichotomy: phi coefficient
High Satisfaction in teaching
Low satisfaction in teaching
High teaching performance
50 21
Low teaching performance
12 48
• A significant relationship occurs if scores are extreme enough to surpass the probability of error.•If p value is < obtained value: reject the null hypothesis•If the obtained value > critical value : reject the null hypothesis
Interpreting the correlation coefficient
Direction or direction Strength Significance Variance
Group Comparison Studies
Involves group formed in categories (2 or more) and these categories are compared on an characteristic.
The groups are called as the independent variable The characteristics of where the groups are compared
on are called as the dependent variable. Examples:
Is there a significant difference between males and females on their math performance?
Is there a significant difference between public and private school students in their study habits?
Are there a significant differences among the school ability of students from across three years (2010, 2011, 2012)?
Are there significant differences among teachers, administrators, and staff on their attitude towards the RH bill?
Group Comparison Studies
Take note that the IV... is categorical can have two or more levels can also be more than one.... Example: Can gender and socio-
economic status differentiate students general intelligence?
A theoretical or conceptual framework is needed to justify the comparison.
Group Comparison Studies
Case: Third year high school males and females are tested in their Mathematical Ability
Males Females26 3824 2618 2417 2418 3020 2218
Group Comparison Studies
Males: Mean = 20.14 SD=3.48
Females: Mean = 27.33 SD = 5.89
Mean of Males and females in Math
Box & Whisker Plot: Var2
Mean ±SD ±1.96*SD Males Females
Var1
12
14
16
18
20
22
24
26
28
30
32
34
36
38
40
Var2
Group Comparison Studies
1. H1= There is a significant difference between males and females on their math scores
2. =.05 df = N1 + N2 –2 df = 7 + 6 –2 df = 11 t critical value = 2.201
Group Comparison Studies
3. Computation
t = X1 - X2
x12 + x2
2 1 + 1 N1 + N2 – 2 N1 N2
t = - 2.73
Group Comparison Studies
4. Decision and Interpretation
Since the t obtained which is – 2.73 is greater than the t-critical which is 2.201, the null hypothesis is rejected.
This means that there is a significant difference between males and females in their math scores.
Females (M=27.33) significantly scored higher on math as compared to the males (M=20.14)
Group Comparison Studies
4. Decision and Interpretation (another way using p values)
Since the p value obtained which is 0.0195 is less than the alpha level which is .05, the null hypothesis is rejected.
This means that there is a significant difference between males and females on their math scores.
Females (M=27.33) significantly scored higher in math as compared to the males (M=20.14)
Factorial Design
Independent Variable
B
A1 A2 A3
B1 A1 B1 A2 B1 A3 B1 B1 Mean
Main Effect for B
B2 A1 B2 A2 B2 A3 B2 B2 Mean
A1 Mean A2 Mean
A3 mean
Main Effect for A
Main effect of A
Main Effect of B
Interaction effect of A and B (A X B)
Talent
Achievement
Effect of Achievement and Type of school on Talent
Low Achievers
High Achievers
Type of school
Public school
Private School
H1: Achievement have a significant main effect on
talent (there is a significant difference between high
and low achievers on talent) Type of school have a significant main effect on
talent (there is a significant difference between public
and private school students in their talent) There is a significant interaction effect between
achievement and type of school (there are significant differences among high
achievers in public, high achievers in private, low achievers in public, and low achievers in private in their talent
Effect of Achievement and Type of school on Talent
Group Comparison Studies Analysis
If two categories are compared on one DV: t-test for two independent samples
If three or more categories (one IV) are compared on one DV: One way Analysis of Variance (ANOVA)
If two IV are investigated on one DV: two way ANOVA
If two or more IV are investigated on two or more DV: Multivariate Analysis of Variance (MANOVA)
Experimental Designs
Carlo Magno, PhDDe La Salle University, Manila
Effectiveness of an intervention on a set of measure (Experimental Study)
The effect of a treatment is tested on a specific change on a characteristic.
The treatment that is given to participants are called as the independent variable.
The independent variable should be manipulated. Ex. Groups are randomly assigned to listening and
watching stimulus to enhance their memory. Ex. Groups are randomly assigned to reading a text or
watching a news to enhance their recall of the information.
The characteristic that changes dues to the variation or manipulation of the IV is called as the dependent variable.
Experimental Study
How is the IV manipulated? Presence of absence Amount Type
Presence vs. absence
The effect of think-aloud reading on the reading comprehension of grade 8 students.
1st group: think-aloud while reading 2nd group: silent reading
Amount manipulation
The effect of cognitive load of concept on the recall of college students.
1st group: 200 words to study 2nd group: 500 words to study 3rd group: 800 words to study 4th group: 1,000 words to study 5th group: 1,200 words to study
Type manipulation
The effect of labeling on the teachers conduct assessment of students
ResultsTrouble makers low conductAverage Average conductIdeal students High conduct
Experimental Study
In an experiment done by dela Cruz, Cagandahan and Arciaga (2004), the effect of nonbehavioral intervention techniques was investigated on the computational abilities of fourth year high school students. The non-behavioral intervention techniques has three levels, bibliotherapy, small group interaction and games. These techniques were used as a teaching strategy in a lesson in a math class for three sections. Each of the strategy was used for each section. One section did not receive any strategy which served as the control group. After undergoing the strategy, the students were tested where they answered a series of computation items.
Experimental StudyBibliotherapy Small group
interactionGames Control Group
X1 X2 X3 X4
X1 X2 X3 X4
X1 X2 X3 X4
X1 X2 X3 X4
Experimental Study
1. H1: The non-behavioral intervention techniques have a significant effect on computational abilityH1: There are significant differences among the groups receiving bibliotherapy, small group interaction, games and control in their computational ability.
2. 2=.05 df between = groups – 1 = (4-1=3) df within = (N – 1) – df between ((209-1)-
3)=205 df total = df between + df within (3 + 205) F ratio critical value = 2.65
ANOVA Hypothesis Testing3. Computation
F ratio computed = 4.62
4. Decision and InterpretationSince the F ratio obtained which is 4.62 is
greater than the F ratio critical which is 2.65, the null hypothesis is rejected. The non-behavioral intervention techniques have a significant effect on computational ability.
ANOVA Hypothesis TestingIntervention techniques; LS Means
Current effect: F(3, 205)=4.6819, p=.00347Effective hypothesis decomposition
Vertical bars denote 0.95 confidence intervals
controlGames
BibliotherapySmall group interaction
Intervention techniques
3.5
4.0
4.5
5.0
5.5
6.0
6.5
7.0
7.5
8.0
8.5
com
puta
tion
The group who received the small group interaction significantly scored the highest among other intervention techniques.
Hypothesizing
The hypothesis needs to be backed up by a theory.
The effect of IV on the DV should be strictly controlled, no other factors should affect the DV other than the IV.
Extraeneous variables (show examples)
Techniques of constancy
Experimental Designs
Research Design – Refers to the outline, plan or strategy specifying the procedure to be used in seeking an answer to the research question
True Research Designs - Answers the research questions or adequately tests hypothesis. Extraneous variables are controlled Inclusion of a control group External validity - Generalizability
Experimental Designs
1. After-Only Design Dependent variable is measured only once and this
measurement occurs after the experimental conditions have been administered to the experimental group. Treatment Response Measure
Experimental Condition X YControl Condition Y
Between Subjects Design – If different subjects are used in each experimental treatment condition.
Within Subjects Design – If the same subjects are used in each experimental condition.
Experimental Designs 1.1 Between-Subjects After Only
Design subjects are randomly assigned to
the experimental and control group.
Simple Randomized Subjects Design Includes more than one level of the
independent variable
Experimental Designs
Factorial Design Two or more independent variables
are simultaneously studied to determine their independent and interactive effects on the dependent variables.
Main effect – influence of one independent variable
Interaction effect – Influence that one independent has on another
Experimental Designs
Within Subject After-Only Design Same subjects are repeatedly
assessed on the dependent variable after participating in all experimental treatment conditions
Experimental Designs
Combined Between- and Within-Subjects Designs
Factorial Design Based on a mixed Model Two independent variables have to be varied
in two different ways. One independent variable requires a
different group of subjects for each level of variation.
The other independent variable is constructed in such a way that all subjects have to take each level of variation.
Experimental Designs
Experimental Designs
2. Before-After Design The treatment effect is assessed by
comparing the difference between the experimental and control groups’ pre- and posttest scores.
The Solomon Four-Group Design - Designed to deal with a potential testing threat. - Testing threat occurs when the act of taking a test
affects how people score on a retest or posttest. - The design has four groups - Two of the groups receive the treatment and two
does not. - Two of the groups receive a pretest and two does
not. - By explicitly including testing as a factor in the
design, we are able to assess experimentally whether a testing threat is operating.
Experimental Designs
Switching Replications Design
- There is a need to deny the program to some participants through random assignment.
- A two group design with three waves of measurement. - The implementation of the treatment is repeated or
replicated. - In the repetition of the treatment, the two groups
switch roles: - The original control group becomes the treatment
group in phase 2 while the original treatment acts as the control. By the end of the study all participants have received
the treatment.
Experimental Designs
Randomized Block Design
Constructed to reduce noise or variance in the data Requires that the researcher to divide the sample
into relatively homogeneous subgroups or blocks. Then, the experimental design desired is
implemented within each block or homogeneous subgroup.
The key idea is that the variability within each block is less than the variability of the entire sample. Thus each estimate of the treatment effect within a block is more efficient than estimates across the entire sample
Experimental Designs
Recap
What are the three approaches in conducting a study?
Activity
Construct a plan for your classroom research Research Question Hypothesis What conceptual/theoretical framework will
be used? (be ready to explain) Method
▪ Experimental Design▪ Participants (who and how many)▪ Instruments used (how will you measure the DV?)▪ Procedure
top related