talking talons youth leadership evaluation report · talking talons youth leadership evaluation...
TRANSCRIPT
LEIDEN LLC
Talking Talons Youth Leadership Evaluation
Report School Year 2015
Dr. Carmen Sorge
1
Table of Contents
Guide to basic interpretation ........................................................................................................... 7
Summary of results ......................................................................................................................... 7
Data Collection and Research ....................................................................................................... 10
Research Design ........................................................................................................................ 10
Composite instrument ............................................................................................................ 10
The Composite instrument scoring .................................................................................... 12
Reliability of the instrument .............................................................................................. 13
Quizzes .................................................................................................................................. 14
Reliability and Scale development..................................................................................... 15
Talking Talons Quiz Knowledge ....................................................................................... 16
Talking Talons Quiz Attitudes ........................................................................................... 17
Teacher Feedback Form ........................................................................................................ 19
Qualitative Data Collection ................................................................................................... 19
Participants ................................................................................................................................ 19
Explanation of GLM Repeated Measures ................................................................................. 21
Examples ............................................................................................................................ 23
Science Attitudes .......................................................................................................................... 27
Research on Science Attitudes .................................................................................................. 27
Science attitude change during program for participants .......................................................... 29
Science Attitude change from pretest to posttest by group ....................................................... 32
Student Qualitative Feedback on Science Attitudes ................................................................. 37
Classroom Teacher feedback about science Attitudes .............................................................. 40
Hands on science experiments and Animals in the classroom ..................................................... 40
Research on Hands on Science and Animals in classroom ....................................................... 40
Classroom Teacher feedback .................................................................................................... 42
Student Feedback on Animals in classroom and Hands on science .......................................... 42
Buddy class ................................................................................................................................... 46
Research on peer tutoring .......................................................................................................... 46
2
Classroom teacher assessment of Buddy class .......................................................................... 48
Student feedback on Buddy class .............................................................................................. 49
Role Model for younger students .......................................................................................... 53
Science Knowledge ....................................................................................................................... 54
Change in Science Knowledge by Group ................................................................................. 54
Quiz Knowledge ........................................................................................................................ 57
Self-perceived science grade in school ..................................................................................... 65
Quiz Attitude subscales ............................................................................................................. 68
Feedback on Educator ................................................................................................................... 70
Student Quantitative Feedback.................................................................................................. 70
Student Qualitative Feedback.................................................................................................... 74
Classroom teacher feedback for Educator ................................................................................. 75
Classroom teacher feedback for Talking Talons Program ............................................................ 78
Teacher Attitude ........................................................................................................................ 78
Effectiveness by student ability: feedback by classroom teachers .......................................... 79
Classroom teacher subscales ..................................................................................................... 81
Other Results ................................................................................................................................. 83
Significant changes in pre posttest Composite .......................................................................... 83
Moral Attitude ........................................................................................................................... 83
Research on Moral Attitude ................................................................................................... 83
Overall significant Composite results ....................................................................................... 87
Overall Non significant composite results ................................................................................ 90
Summary ..................................................................................................................................... 101
References ................................................................................................................................... 103
3
Tables
Table 1 Pretest and Posttest Cronbach’s alpha for subscales ....................................................... 13
Table 2 Participants by Group ...................................................................................................... 20
Table 3 Participants by Gender .................................................................................................... 20
Table 4 Group by Class................................................................................................................. 21
Table 5 Science Attitude Change over Time ................................................................................ 30
Table 6 Science Attitude Change over Time Multivariate Tests .................................................. 30
Table 7 Science Attitude Change over Time within Subjects Effects .......................................... 30
Table 8 Science Attitude Change over Time Within subjects Contrasts ...................................... 31
Table 9 Science Attitude Change over Time between Subject Effects ........................................ 32
Table 10 Change in Science Attitude by Group ........................................................................... 33
Table 11 Change in Science Attitude by Group Multivariate Tests ............................................. 34
Table 12 Change in Science Attitude by Group Within subjects Effects ..................................... 34
Table 13 Change in Science Attitude by Group within Subjects Contrasts ................................. 35
Table 14 Change in Science Attitude by Group between Subject Effects .................................... 36
Table 15 Change in Science Attitude by Group Estimated Means ............................................... 36
Table 16 Difference Change in Science Attitude by Group ........................................................ 37
Table 17 Attitude toward Science from Quizzes Mean ............................................................... 37
Table 18 Classroom Teacher assessment of Science Attitude change ......................................... 40
Table 19 Classroom teacher assessment of program components ................................................ 42
Table 20 Classroom teacher assessment of buddy class impact .................................................. 49
Table 21 Student answers by group to "I am a good role model for younger students" .............. 54
4
Table 22 I am a good role model significance tests ...................................................................... 54
Table 23 Change in Science Knowledge by Group ...................................................................... 56
Table 24 Change in Science Knowledge by Group Multivariate tests ......................................... 56
Table 25 Change in Science Knowledge by Group Means ......................................................... 57
Table 26 Mean scores on Knowledge Quiz by topic and class ..................................................... 59
Table 27 ANOVA comparison on Quiz Knowledge by Class ..................................................... 60
Table 28 Quiz Knowledge Means by Gender .............................................................................. 62
Table 29 ANOVA comparison for Talking Talons Quizzes by Gender ....................................... 63
Table 30 Group Comparison for I will get a good grade in science class this year ...................... 66
Table 31 Group Comparison for I will get a good grade in science class this year Multivariate
tests ............................................................................................................................................... 67
Table 32 Quiz Attitude subscale descriptive statistics ................................................................. 69
Table 33 Student feedback on Educator Subscales ....................................................................... 71
Table 34 Student feedback on Educator from Quiz results .......................................................... 72
Table 35 Student Feedback on Individual Educator questions ..................................................... 73
Table 36 Classroom Teacher Feedback on Talking Talons Educator .......................................... 76
Table 37 Classroom Teacher Feedback Personal ......................................................................... 79
Table 38 Classroom teacher feedback program effectiveness by student ability ......................... 80
Table 39: Classroom teacher subscales for Talking Talons program ........................................... 82
Table 40 Change in Moral Attitude by Group .............................................................................. 85
Table 41 Change in Moral Attitude by Group Multivariate tests ................................................. 85
Table 42 Change in Moral Attitude by Group Within subjects .................................................... 86
Table 43 Change in Moral Attitude by Group Between subjects ................................................. 86
Table 44 Statistically significant Pre Post change by group ........................................................ 87
Table 45 ANOVA Statistically significant Pre Post change by group ......................................... 88
5
Table 46 Effect Size Statistically significant Pre Post change by group ...................................... 89
Table 47 Non Significant Pre Post change by Group .................................................................. 89
Table 48 Means Non Significant Pre Post change by Group........................................................ 90
Table 49 ANOVA Non Significant Pre Post change by Group .................................................... 91
Table 50 Effect size Non Significant Pre Post change by Group ................................................. 92
Table 51 Pre and Posttest means by group ................................................................................... 92
Table 52 Pre and Posttest multivariate tests by group .................................................................. 95
Table 53 Pre and Posttest Univariate tests by group..................................................................... 96
Table 54 Within Subjects contrasts Pre and Posttest means by group ......................................... 98
Table 55 Pre and Posttest means by group ................................................................................... 99
Table 56 Posttest Attitudes for Treatment .................................................................................. 100
Table 57 Student overall perception of program ........................................................................ 100
6
Figures
Figure 1 Longitudinal change in Science Attitude ....................................................................... 29
Figure 2 Change in Science Attitude from Pretesting to Posttesting by Group............................ 32
Figure 3 I am a good role model for younger students ................................................................. 53
Figure 4 Change in Science Knowledge by Group ....................................................................... 55
Figure 5 Quiz Knowledge section means ..................................................................................... 58
Figure 6 Self perceived change in science grade by group ........................................................... 65
Figure 7 Quiz Attitude subscales .................................................................................................. 69
Figure 8 Student Perception of Educator ...................................................................................... 70
Figure 9 Student Perception of Educator ...................................................................................... 72
Figure 10 Student Feedback on Educator questions ..................................................................... 73
Figure 11 Classroom teacher feedback on Talking Talons Educator ........................................... 75
Figure 12 Classroom Teacher subscales on Educator ................................................................... 77
Figure 13 Classroom teacher feedback on Talking Talons program ............................................ 78
Figure 14 Classroom teacher assessment of Effectiveness of program by student ability ........... 80
Figure 15 Classroom teacher perception ....................................................................................... 81
Figure 16 Change in Moral Attitude by group.............................................................................. 84
Figure 17 Statistically significant Pre Post change by group ....................................................... 87
Figure 18 Non Significant changes for Composite ...................................................................... 90
Figure 19 Pretest and Posttest Multivariate Tests by group ......................................................... 96
7
Guide to basic interpretation
The p value tells the researcher the probability that this data would be observed simply by
chance. A p value of .05 (5 percent chance that the difference was just due to random
fluctuation) is the statistical standard for the maximum while a p value of .01 (less than 1 percent
chance the difference is due to random fluctuation) is considered more conservative and thus a
better indication of actual results. All p values considered statistically significant for the Talking
Talons program are less than .01.
The effect size (η2) examines the magnitude of change or how MUCH difference
occurred. The standard in use for repeated measures is .02 is a small effect size, .13 is a medium
effect size and .26 is a large effect (Bakeman).
Summary of results
A statistically significant change in Attitude toward Science was seen in the treatment
group and not for the control group from pretesting to post testing. F(1,58)=12.25. p<.001
η2=.17 (medium effect size)
8
A statistically significant change in Knowledge was seen in the treatment group and not
for the control group from pretesting to post testing. F(1,55)=11.01. p<.01 η2=.45 (very
large effect size)
A statistically significant change in Moral Attitude was seen in the treatment group and
not for the control group from pretesting to post testing. F(1,58)=8.00. p<.01 η2=.12
(small/medium effect size)
A statistically significant change in self-reported anticipated grade in science was seen in
the treatment group and not for the control group from pretesting to post testing
F(1,57)=19.83. p<.001 η2=.26 (large effect size)
The treatment group exhibited a statistically significant positive change in attitude toward
science as the program progressed. F(3,9)=5.15. p<.01 η2=.15 (medium effect size). This
statistic uses only the treatment group, as the control group does not take the Talking
Talon’s quizzes.
Classroom teacher feedback was extremely positive and both teachers indicated the
program increased student science knowledge and attitudes, that the buddy class was
worthwhile and also indicated a strong willingness to have the program again. Mean
scores for all subscales measuring teacher feedback were all above 6 on a 7 point scale (a
9
higher number indicates more positive feedback) with a mean of 6.44 and a standard
deviation of .37.
Qualitative feedback from students was overwhelmingly positive, mentioning that the
program was a positive experience, that they enjoyed presenting to the buddy class and
found it both rewarding and educational. Many expressed that they relished “being the
teacher” stating that “It’s like my grownup mode just turns on.” The students also were
enthusiastic about that both the hands on and presentation sections of the program and
found them enjoyable and educational. As a student stated during interview “Talking
Talons is like unicorns, it’s amazing!”. The only suggestion for improvements by the
students was the inclusion of even more animals. Means scores for all student treatment
subscales were above 5.35 on a 6 point scale with a mean of 5.54 and a standard
deviation of .34.
10
Data Collection and Research
The Talking Talons program evaluation data collection consists of:
A control/treatment pretest and posttest called the Talking Talons Composite
A series of 10 quizzes measuring knowledge and attitude change for the
participants
Online teacher feedback
Qualitative interviews of a sample of the Talking Talons participants
Research Design
Composite instrument
Talking Talons program is evaluated using a pretest and posttest quasi- experimental
design with control and treatment groups for the composite instrument. As the program is
offered by classroom, true random assignment is not feasible. However, the control groups are
selected from the same school using teachers who are not currently receiving the program.
Although ideally a Solomon design would be used, the effort and expense involved in collecting
four sets of data points is prohibitive.
The use of a control group is imperative. Without control groups any changes cannot be
attributed to the program. Factors such as maturation, the effect of testing and other outside
factors cannot be eliminated as agents of change without the use of a control group.
11
Improvements seen in the treatment group therefore cannot be ascribed to the program unless a
control group is utilized.
For the composite instrument a General Linear Model Repeated measures utilizing SPSS
18 was run. The GLM repeated measures improves on the multivariate regression model by
allowing for linear transformations or linear combinations of multiple variables. This expansion
means that the GLM has important advantages over the multiple and the purported multivariate
regression models which are inherently univariate methods. The first advantage is that
multivariate test of significance may be use if the responses on multiple dependent variables are
correlated. This is helpful as separate univariate tests of significance for correlated dependent
variables (such as used in multiple t tests) are not independent and may not be appropriate.
Multivariate tests of significance of independent linear combinations of multiple dependent
variables may also yield information about which response variables are, and are not, actually
related to the predictor variables. A second advantage is the ability to analyze effects of repeated
measure factors. Linear combinations of responses reflecting a repeated measure effect can be
constructed and tested for significance using either the univariate or multivariate approach to
analyzing repeated measures in the general linear model. In this research, with pretest and
posttest measurements of control and treatment groups the GLM repeated measures is the most
appropriate test.
Effect sizes based on recent research on repeated measure designs based on Cohen (1988,
pp. 413–414), who did not consider repeated measures designs explicitly, defined an η2 of .02 as
small, one of .13 as medium, and one of .26 as large. “It seems appropriate
to apply the same guidelines to η2 as well.” (Bakeman)
12
The Composite instrument scoring
All composite questions are scored on a Likert scale from 1 to 6 ranging from Very False
to Very True. All questions and scales are renormed so that a higher value indicates a more
desirable outcome. Therefore a value of 5 indicates better self-control or less violent tendencies
than a score of 3. For all subscales a higher score is more desirable.
The Locus of Control measure reports a student’s self-reported ability to direct and
manage their own behavior. Ten questions addressing these issues such as “I do things I know
are wrong because my friends are doing them.” and “I am easily distracted.” are included.
The School Attitude scale consists of five self-reported feelings about school such as
“The work I do in school is important to me” and “I would skip school a lot if I knew that I
would not get caught”.
The Self-esteem attitude scale consists of twelve self-assessments of the student’s self-
value and perception of ability such as “I am proud of myself.” And “If I want to learn to do
something new I usually can learn it.”
The Attitude toward Violence scale consists of five self-reported actions or intentions of
violence such as “I fight a lot.” and “If people will not do what you want it is okay to threaten
that you will hurt them.” Questions 9, 11, 25, 42 and 20 from the composite instrument are
included in this subscale.
13
The Moral Judgment scale consists of eleven questions evaluating student attitudes
toward various ethical and unethical behaviors such as “It is okay to cheat on schoolwork if you
do not get caught.” And “It is wrong to lie in order to get what you want.”
Seventeen questions for the treatment group only are asked of the treatment
group. These questions address the participant’s evaluation of the program and the educator and
include questions such as “I looked forward to the days that Talking Talons came to my
classroom” and “My Talking Talons educator was easy to understand”. A full report of these
questions is included in the educator report.
Reliability of the instrument
Cronbach’s alpha was calculated for each subscale using the overall sample. Reliability
greater than .7 is considered good (Nunnaly, 1978). All subscales of the instrument have good to
excellent reliability. The 2015 evaluation has a much smaller sample size than previous years,
which does impact reliability. School Attitude reliability fell below the reasonable interpretation
threshold and thus any significant results are disregarded.
Table 1 Pretest and Posttest Cronbach’s alpha for subscales
2003-
2004
2003-
2004
2004-
2005
2004-
2005
2005-
2006
2005-
2006
2006-
2007
2006-
2007
2015* 2015
Pre Post Pre Post Pre Post Pre Post Pre Post
Locus of
Control
.797 .807 .78 .83 .79 .81 .81 .82 .78 .76
School
Attitude
.750 .779 .75 .75 .79 .78 .71 .74 .65 .52
14
2003-
2004
2003-
2004
2004-
2005
2004-
2005
2005-
2006
2005-
2006
2006-
2007
2006-
2007
2015* 2015
Self
Esteem
.666 .798 .81 .84 .83 .86 .82 .86 .91 .87
Attitude
toward
Violence
.717 .747 .73 .73 .80 .85 .77 .83 .77 .71
Moral
Scale
.776 .799 .83 .84 .86 .86 .80 .86 .70 .87
Science
Attitude
Scale
.957 .965 .95 .97 .96 .97 .94 .95 .90 .94
Talking
Talons
Scale
.680 .782 .76 .80 .71 .89 .77 .76 .77 .71
Posttest
only
Treatment
section
.94 .92 .76 .73
*note: Sample size for 2015 was much smaller than previous years, this tends to reduce
reliability
Quizzes
The treatment group students are also given a series of ten quizzes. These quizzes consist
of five knowledge questions and five attitude questions. The knowledge questions pertain to the
information taught by the program and the attitude questions cover attitudes towards the
program, the educator and science in general. Some science attitude questions are repeated
throughout the attitude section of the quizzes over the course of the program in order to measure
longitudinal changes in attitude scores.
15
These quizzes are used to determine both the changes in science knowledge and in attitudes
over the course of the program.
Reliability and Scale development
The quizzes were used to generate several subscales. For each subsection the internal
reliability was examined. Internal reliability is how much consistency is apparent in the answers
of one subtest. Internal consistency is measured with a scale called “Cronbach's Alpha”.
Technically speaking, Cronbach's alpha is not a statistical test - it is a coefficient of reliability (or
consistency). Alpha is calculated by one split-half reliability and then randomly dividing the
items into another set of split halves and recomputing until, all possible split half estimates of
reliability have been computed. In other words, each item is compared to the group in all
possible combinations of items. Cronbach's alpha measures how well a set of items (or variables)
measures a single unidimensional latent construct. When data have a multidimensional structure
(they are not all measuring the same construct) then Cronbach's alpha will usually be low. If the
number of items is increased and the consistency remains the same the Cronbach’s alpha will
increase. Additionally, if the average inter-item correlation is low, alpha will be low. As the
average inter-item correlation increases, Cronbach's alpha will also increase. This effect makes
sense intuitively - if the inter-item correlations are high, and then there is evidence that the items
are measuring the same underlying construct.
Recall that an alpha value of .70 is considered acceptable; however we would like it to be
higher. Reliabilities are presented in each section of the report.
16
Talking Talons Quiz Knowledge
All of the scales are mean scores. Thus if an educator did not give one of the quizzes
containing a question for that subset it is not entered into the mean. This can impact the results
because the score is then based on fewer data points. All attitude scales are recoded so that 6
represents the highest and best score and 1 is the lowest and worst score. For the quizzes 5 is the
highest possible score and 0 is the lowest. Remember that this is a very simple way at looking at
scores. With simple means one participant who has a negative change of 6 point will mask the
positive change of 1 point for 6 participants. This is especially true for small sample sizes (the
smaller classrooms). An educators with a score of 5.5 on the scale could have had half the kids
answer “6” and half answer “5” OR almost all kids answering 6 with a few answering “1”. Keep
this in mind when interpreting results.
Bar charts are good visual presentations of information. However, they may lead to over
interpretation. As the standard deviations are not listed on a bar chart it is difficult to ascertain
true significant differences. One must ask “Does that “3” on a bar chart mean that almost
everyone answered 3 or that half the participants answered “5” and half answered “2”? “
Significance tests indicate actually differences by utilizing the standard deviation as well.
Therefore two bars on a chart may look very different and not be statistically different or may
look similar but actually represent a statistical difference.
The knowledge sections relate to the topics studied in the program and includes the following
topics Introduction, Hawks, Raptors, Bats 1, Bats 2, Owls, Reptiles,Vocabulary General and
Energy. Five questions related to these topics are asked on each of the quizzes.
17
Talking Talons Quiz Attitudes
The back of each quiz includes an attitude section; these questions are then utilized for the
following subscales.
Overall perception of educator (Reliability α=.90)
My Talking Talons educator (teacher) is easy to understand.
My Talking Talons educator (teacher) understands how kids think.
My Talking Talons educator (teacher) knows a lot about the animals.
My Talking Talons educator (teacher) speaks clearly and loudly enough for me to hear
and understand.
My Talking Talons educator (teacher) pays attention to all of the kids in my class.
My Talking Talons educator (teacher) treats me kindly.
My Talking Talons educator (teacher) is prepared to teach us when he/she comes to class.
My Talking Talons educator (teacher) was fair to everyone in my class.
Bonding with Animals (Reliability α=.68)
In the future, I would like to be a Talking Talons Educator and hold a bird.
I would like the chance to hold a Talking Talons reptile (like a snake or lizard).
I think about the Talking Talons animals when I am not in science class
18
I feel comfortable around the Talking Talons animals (eliminated due to small sample
size)
Seeing the Talking Talons animals makes me feel happy.
Perceived Enjoyment of Talking Talons program (Reliability α=.78)
I like learning about animals.
I look forward to the days that Talking Talons comes to the classroom
I talk to kids who are not in the Talking Talons program about the program.
I would like the Talking Talons program to come to my science class next year.
Perceived Ability (Reliability α=.72)
The quizzes for Talking Talons were very hard.
I try my hardest to do well in school.
I am good at science.
I think I will get a better grade in science class this year than last year.
I understood most of the information in the Talking Talons program.
Attitude toward science (Reliability α=.82)
I like science class.
Science class is boring
19
I like science class.
Attitude toward environment (Reliability α=.68)
It is ok to shoot hawks because they kill smaller birds.
It is ok to throw trash out the window because the highway department picks it up.
I think teaching other kids about the environment is important
Teacher Feedback Form
At the end of the program an online form (to preserve teacher anonymity) is used to
collect data on the classroom teachers’ perceptions of the program, of changes in attitude of
his/her students and of the educator.
Qualitative Data Collection
A sample of five to six students from each group was interviewed by the evaluator for
feedback about the Talking Talons program. These interviews were recorded and transcribed.
Student names were removed from the transcription to preserve anonymity.
Participants
20
Participant demographic information is included below. For each subscale the total
number of student varies slightly due to outliers and participants who did not finish a
particular subscale. The total number can be found in the table of means for each subsection.
Because of the small sample size for the 2015 program, missing posttest data for the control
group was imputed from representative previously collected control test data for four cases in
order to reach a statistically useable sample size. Students were all between 10 and 11 years
old at post testing.
Table 2 Participants by Group
Frequency Percent Valid Percent Cumulative Percent
Valid Control 23 29.9 29.9 29.9
Treatment 54 70.1 70.1 100.0
Total 77 100.0 100.0
One student did not provide gender.
Table 3 Participants by Gender
Unknown Female Male
group Control 0 13 10 23
Treatment 1 21 32 54
Total 1 34 42 77
21
Table 4 Group by Class
class
Total Control Teacher A Teacher B
group Control 23 0 0 23
Treatment 0 29 25 54
Total 23 29 25 77
Explanation of GLM Repeated Measures
The data for the composite file were analyzed using SPSS General Linear Model
Repeated Measures (GLM). The GLM Repeated Measures procedure is based on the general
linear model, in which factors and covariates are assumed to have linear relationships to the
dependent variables.
This section explains how to interpret the graphs and data from the analysis of the
composite section. For simplicity of explanation sample graphs and data are used which make
the differences clear for interpretation.
A GLM repeated measures with control groups is used to examine the following possibilities:
22
QUESTION: Are the control and treatment groups similar to each other?
VISUAL ANSWER: Are the two lines far apart (be sure to look at the scale of the graph)
from each other, especially at pretesting?
STATISTICAL ANSWER: If the two groups are not similar to each other than the test
of between groups will be significant.
QUESTION: Does the score on the instrument change over time from pretesting to post testing
for one or both of the groups?
VISUAL ANSWER: Is the line flat or sloped for the groups?
STATISTICAL ANSWER: If the scores are changing overall in the same direction and
magnitude then the “Time” variable will be significant.
QUESTION: Is the change over time different for the two groups? (Either one group goes up
and one goes down or the magnitude of the change is different.)
VISUAL ANSWER: Is the slope of the line different (in direction or magnitude) for
each group?
STATISTICAL ANSWER: If the scores are changing, but differently for each group
then the “Time by Group” variable will significant overall. The difference can be either
in different directions or in different magnitudes.
23
These questions are answered statistically in the tables. However, for ease of interpretation the
statistical significance is listed under the graph showing change.
Examples
Groups are different but with no change over time
The first question is answered statistically in a table called “Between groups’ differences”
but can also be seen by the distance between the score values of the control and treatment
groups. Note the scale on the graph as well when interpreting the data.
The first graph shows the following
No change over time for either group (the slope of the lines are flat)
Groups are different from each other (there is a big gap between the Treatment and
Control Group lines)
24
No Change over Time
Groups are different
TIME
21
Estim
ate
d M
arg
ina
l M
ea
ns
4.5
4.0
3.5
3.0
2.5
2.0
1.5
1.0
.5
GROUP
Treatm ent
Control
Groups are similar and both change over time in the same way
The second graph shows the following
both change from pretesting to post testing (the slope of the line is not flat)
Both change in the same manner (the slope is in the same direction and at the same
angle).
25
Change over Time
TIME
21
Estim
ate
d M
arg
ina
l M
ea
ns
7.0
6.5
6.0
5.5
5.0
4.5
4.0
3.5
GROUP
Treatm ent
Control
Groups are the same but change differently over time
The third graph shows
change over time for both groups
Change is not in the same direction.
26
Different Changes Over Time
TIME
21
Estim
ate
d M
arg
ina
l M
ea
ns
6
5
4
3
2
1
0
GROUP
Treatm ent
Control
Obviously with real data the differences are not always so clear. However, this
explanation should make visual interpretation of the results simpler. Statistical results are
presented in the tables below the charts for those who are interested in the means, levels of
significance, effect size and power for the results. Type III Sums of squares were used as the
hypotheses being tested involve only marginal averages of population cell means. Greenhouse-
Geisser epsilon was used to adjust degrees of freedom if sphericity was violated as per
assessment with Mauchly's test. These statistics are available from the researcher if desired.
Estimated marginal means of the dependent variables (with covariates held at their mean value)
for specified between- or within-subjects factors in the model are provided. In this research these
predicted means are equivalent to observed means as the covariates are categorical.
27
Outlying scores due to extreme responses may influence results. In order to identify
outliers the score distributions for the participants for all observed variables were examined
univariately. Scores that were more than three standard deviations from cell means and were
also discontinuous from their closest neighboring scores were considered univariate outliers and
were removed on a subscale by subscale basis.
For each subsection the results were examined by GLM repeated measures using the
following group memberships.
Overall
By gender
The results are then presented for results that yielded significant or interesting result for
the difference between the groups if there was no difference then overall results are presented
unless a trend that merits consideration was present. Recall that as the group is subdivided the
membership within that group becomes smaller and significance is more difficult to detect.
Science Attitudes
Research on Science Attitudes
Attitudes toward science have long been examined as an important factor in student
science education. Researchers have found that attitudes toward science impact future course
selection (Farenga and Joyce), motivation to study science (Slate and Jones), achievement in
28
science (Oliver and Simpson; Kose et al.; Smith, Pasero, and McKenna; Baker; Oliver;
Reynolds and Walberg; Warburton, Jenkins, and Coxhead; Willson) and general attitudes toward
school (Jarvis and Pell) ). Improving student’s attitudes toward science could therefore be said
to have many positive outcomes.
However as students enter the middle school years, their generally positive attitudes
about science decrease (Mattern and Schau; Desy, Peterson, and Brockman; Catsambis;
Mattern; Sorge) especially for girls (Backes; Lee and Burkam; Oliver; Papanastasiou and
Zembylas; Warburton, Jenkins, and Coxhead) .
Of particular relevance is research examining changes in attitudes exhibited after
exposure to “hands on” or inquiry based science. These programs have been found to improve
attitudes toward science with increased frequency of hands on experiments leading to most
positive attitudes (Ornstein) and with hands on laboratory experiment increasing knowledge
(Freedman). Interestingly, girls were found to be impacted more (Teshome, Maushak, and
Athreya) by hands on experimentation than boys.
Specific research on involving students in environmental science projects in order to
increase both knowledge and attitudes toward science has statistically positive outcomes on both
factors (Al-Balushi and Al-Aamri).
Science attitudes for the Talking Talons students were examined in multiple forms.
Pretest and posttest attitudes were compared for control and treatment groups. For the treatment
groups the same question about attitudes towards science was asked throughout the program in
29
order to measure longitudinal change. Information was collected from the classroom teacher and
from interviews.
Science attitude change during program for participants
In order to examine longitudinal changes in science attitude for the treatment group
(those students who participated in the Talking Talons program), science attitude questions were
asked at pretesting, on quiz 3, on quiz 7 and at posttesting. The chart below delineates the
change over time. This change was statistically significant (see table for results)
Figure 1 Longitudinal change in Science Attitude
30
Table 5 Science Attitude Change over Time
Descriptive Statistics
Mean Std. Deviation N
Science Attitude Post. 4.9032 1.35043 31
Science Attitude Quiz 3 5.32 .945 31
Science Attitude Quiz 7 5.48 .811 31
Science Attitude Post. 5.6452 .70938 31
Table 6 Science Attitude Change over Time Multivariate Tests
Multivariate Testsc
Effect
Value F
Hypothesis
df
Error
df Sig.
Partial Eta
Squared
Noncent.
Parameter
Observed
Powerb
linsc Pillai's
Trace
.252 3.148a 3.000 28.000 .041 .252 9.443 .667
a. Exact statistic
b. Computed using alpha = .05
c. Design: Intercept
Within Subjects Design: linsc
Table 7 Science Attitude Change over Time within Subjects Effects
Tests of Within-Subjects Effects
Measure:MEASURE_1
31
Source Type III Sum
of Squares df
Mean
Square F Sig.
Partial Eta
Squared
Noncent.
Parameter
Observed
Powera
linsc Sphericity
Assumed
9.452 3 3.151 5.151 .002 .147 15.453 .913
Error(linsc) Sphericity
Assumed
55.048 90 .612
Lower-
bound
55.048 30.000 1.835
a. Computed using alpha = .05
Table 8 Science Attitude Change over Time Within subjects Contrasts
Tests of Within-Subjects Contrasts
Measure:MEASURE_1
Source linsc Type III Sum of
Squares df
Mean
Square F Sig.
Partial Eta
Squared
Noncent.
Parameter
Observed
Powera
linsc Linear 8.832 1 8.832 9.647 .004 .243 9.647 .852
Error(linsc) Linear 27.468 30 .916
a. Computed using alpha = .05
32
Table 9 Science Attitude Change over Time between Subject Effects
Tests of Between-Subjects Effects
Measure:MEASURE_1
Transformed Variable:Average
Source Type III Sum of
Squares df
Mean
Square F Sig.
Partial Eta
Squared
Noncent.
Parameter
Observed
Powera
Intercept 3534.226 1 3534.226 1730.366 .000 .983 1730.366 1.000
Error 61.274 30 2.042
a. Computed using alpha = .05
Science Attitude change from pretest to posttest by group
Both the treatment and control group were assessed for science attitude changes from
pretesting to postttesting. A statistically significant change in Attitude toward Science was seen
in the treatment group and not for the control group from pretesting to post testing.
F(1,58)=12.25. p<.001 η2=.17. This represents a medium effect size.
Figure 2 Change in Science Attitude from Pretesting to Posttesting by Group
33
Table 10 Change in Science Attitude by Group
Descriptive Statistics
group Mean Std. Deviation N
sci
dimension1
Control 5.2609 .87684 23
Treatment 5.0246 1.01946 37
Total 5.1152 .96657 60
psci
dimension1
Control 4.4314 1.49023 23
Treatment 5.2713 .96420 37
34
Descriptive Statistics
group Mean Std. Deviation N
Total 4.9494 1.25098 60
Table 11 Change in Science Attitude by Group Multivariate Tests
Multivariate Testsc
Effect
Value F
Hypothesis
df
Error
df Sig.
Partial Eta
Squared
Noncent.
Parameter
Observed
Powerb
science Pillai's
Trace
.058 3.592a 1.000 58.000 .063 .058 3.592 .462
science *
group
Pillai's
Trace
.174 12.251a 1.000 58.000 .001 .174 12.251 .931
a. Exact statistic
b. Computed using alpha = .05
c. Design: Intercept + group
Within Subjects Design: science
Table 12 Change in Science Attitude by Group Within subjects Effects
Tests of Within-Subjects Effects
Source Type III Sum
of Squares df
Mean
Square F Sig.
Partial Eta
Squared
Noncent.
Parameter
Observed
Powera
science Sphericity
Assumed
2.408 1 2.408 3.592 .063 .058 3.592 .462
35
Tests of Within-Subjects Effects
Source Type III Sum
of Squares df
Mean
Square F Sig.
Partial Eta
Squared
Noncent.
Parameter
Observed
Powera
science *
group
Sphericity
Assumed
8.213 1 8.213 12.251 .001 .174 12.251 .931
Error(science) Sphericity
Assumed
38.885 58 .670
a. Computed using alpha = .05
Table 13 Change in Science Attitude by Group within Subjects Contrasts
Tests of Within-Subjects Contrasts
Measure:MEASURE_1
Source science Type III Sum
of Squares df
Mean
Square F Sig.
Partial Eta
Squared
Noncent.
Parameter
Observed
Powera
science dimension2
Linear 2.408 1 2.408 3.592 .063 .058 3.592 .462
science *
group dimension2
Linear 8.213 1 8.213 12.251 .001 .174 12.251 .931
Error(science) dimension2
Linear 38.885 58 .670
a. Computed using alpha = .05
36
Table 14 Change in Science Attitude by Group between Subject Effects
Tests of Between-Subjects Effects
Measure:MEASURE_1
Transformed Variable:Average
Source Type III Sum of
Squares df
Mean
Square F Sig.
Partial Eta
Squared
Noncent.
Parameter
Observed
Powera
Intercept 2833.325 1 2833.325 1680.791 .000 .967 1680.791 1.000
group 2.584 1 2.584 1.533 .221 .026 1.533 .230
Error 97.771 58 1.686
a. Computed using alpha = .05
Table 15 Change in Science Attitude by Group Estimated Means
group * science
group science
Mean Std. Error
95% Confidence Interval
Lower Bound Upper Bound
dimension1
Control
dimension2
Pre 5.261 .202 4.857 5.665
Post 4.431 .248 3.934 4.929
Treatment
dimension2
Pre 5.025 .159 4.706 5.343
Post 5.271 .196 4.879 5.663
37
Table 16 Difference Change in Science Attitude by Group
Change in Science Attitude
group Mean N Std. Deviation
Control -.8295 23 1.46536
Treatment .2467 37 .92088
Total -.1658 60 1.26354
Student data from quizzes.
Table 17 Attitude toward Science from Quizzes Mean
Descriptive Statistics
N Minimum Maximum Mean Std. Deviation
Attitude science 47 3.67 6.00 5.3936 .69368
Valid N (listwise) 47
Student Qualitative Feedback on Science Attitudes
Interview sections pertaining to science attitudes for the treatment group are excerpted
below.
38
Student feedback on science from interviews
(Note: I=Interviewer, Students are indicated by a letter)
Classroom B:
I: Ok, so what about the TT program compared to your regular science program? Which do
you prefer?
[All talking over each other]
Student B: I definitely prefer TT.
Student A: Me too.
Student B: Talking Talons.
Student C: Talking Talons, definitely. Definitely.
Student A: I mean it’s cool that you get to learn about animals and like things like that, I’m
an animal lover. I ADORE animals and um.. and also I get to see them, so that’s even better.
I: And they are kind of unusual animals, right?
Student A: Yeah
Student B: Animals you won’t find in New Mexico.
Student A: Well actually. You find them, but you don’t get to get that close to them. They are
usually trying to get away from you.
I:Do you look forward to the days Talking Talons comes?
39
[All talking over each other] Yeah, Yeah, Yes
Student A: I really do. When I had a (redacted due to identifying information) test in here I
was really sad because it was Talking Talons day and I was like WHAT! I missed out.
Classroom A:
I: Do you guys look forward to the days talking talons comes to the classroom?
yes, yeah [all nodding heads]
I:So you are happy when TT is coming?
Student E: Sometimes it gives us a break from normal class and we can learn something
new about animals.
Student F: We learn something new every time we go, mammals reptiles it’s really
interesting
Student G: I like to tell my dad what happened at a school
I: Yeah it gives you something interesting to talk to your parents about?
Student G: it’s really interesting
I: What about you, what do you think? Are you happy on the days Talking Talons is coming to
school …
Student H: I’m happy about it
40
Classroom Teacher feedback about science Attitudes
The classroom teachers indicated that they perceived that the student’s both learned a lot
of science from the program (mean score 6.5/7) and that the program had a positive impact on
student attitudes toward science (mean score 6./7).
Table 18 Classroom Teacher assessment of Science Attitude change
The students learn a lot of science
from the TT program.
The TT program had a positive impact on the
student’s attitudes about science.
Mean 6.5000 6.50
Std.
Deviation
.70711 .707
Hands on science experiments and Animals in the classroom
Research on Hands on Science and Animals in classroom
Significant contribution of hands on learning to science knowledge has been identified as
making a significant contribution to peer interaction through cooperative learning, object-
mediated learning and embodied experience. (Satterthwait). The more hands on experience, the
more learning occurs “Specifically, students who engaged in hands-on activities every day or
41
once a week scored significantly higher on a standardized test of science achievement than
students who engaged in hands-on activities once a month, less than once a month, or never.”
(Stohr-Hunt)
When compared to textbook learning, hands on (inquiry) also improves achievement
“Students in the inquiry-based group reached significantly higher levels of achievement than
students experiencing commonplace instruction. This effect was consistent across a range of
learning goals (knowledge, reasoning, and argumentation) and time frames (immediately
following the instruction and 4 weeks later).” (C. D. Wilson et al.)
Hands on learning is more effective for student comprehension even when compared to
teacher demonstration “students in the hands- on laboratory class performed significantly better
on the procedural knowledge test than did students in the teacher demonstration class. These
results were unrelated to reasoning ability.” (Glasson). This impact has been shown to influence
traditionally underrepresented minorities attitudes toward science and career plans. (Kanter and
Konstantopoulos). This is especially important as the impact of No Child Left behind has led to
less teaching of science in the classroom. (Milner et al.)
Furthermore, the use of animals in the classroom has been found to teach humane values
(Zasloff et al.) while improving young students attitudes toward animals (Ellery Samuels, Nicoll,
and Trifone) and towards other humans (Arkow) . The use of animals also improves
environmental education for young children (Margadant-Van Arcken) and knowledge about
animals specifically (Kimble).
42
The specific use of more exotic animals (zoo outreach) found that students recorded more
observations, made more use of science facts and use more science vocabulary when writing
about more exotic animals rather than more ordinary pets. (Trainin et al.) and also used more
advanced literary concepts (K. Wilson et al.)
Classroom Teacher feedback
Classroom teachers gave the maximum score possible (7/7) to all factors relating to the
effectiveness of the hands on and animal sections of the Talking Talons program.
Table 19 Classroom teacher assessment of program components
The animals are
an important part
of the TT
program for the
students.
The public
speaking
component of TT
is an important part
of the program.
The hands on
activities are an
important part of
the TT program.
Presenting to the Buddy
class made the students
take the responsibility
of TT more seriously.
Mean 7.00 7.00 7.00 7.00
Std.
Deviation
.000 .000 .000 .000
Student Feedback on Animals in classroom and Hands on science
43
Sections of the interview pertaining to the animals and hands on learning are excerpted
below.
I:Ok, now I am going to ask you about what the best things were about Talking Talons and what
things could they do better? So what were the best things and what things would you improve?
What do you think Student B?
Student B: Hm, can I pass
I:Can you pass and I can come back to you?
Student B: To be honest I see no problems.
I:Would there be anything you would like to add.
Student C: More animals?
Student A: That would be awesome but like I don’t see much problems and like the animals
and everything and getting to hold them like the snake I was amazed by how it felt because
when it would like move it was just like plastic moving around it was like, this is really soft
Student B: and cold!
I: Yeah, cold! You hear cold blooded but you don’t know what that’s like until you touch
them, right?
Student A Because of their shiny skin you would think they would be like wet or something
and it’s like really fascinating. So I don’t see like much problem, I don’t see any problem at
all because everyone’s really nice and everything about it so it’s pretty cool
44
I: What about you Student C?
Student C: Actually, I actually want more activities to do.
I: Hands on activities? So which is more fun, doing the animal presentations or doing the
animal activities like the eyeballs?
Student B: I say both.
Student C: both
Student A: That’s really hard because they are both really fun.
I: What about you Student D? You like the hands on, or the animal part?
Student D: I like both
I: Were any of you afraid of any of the animals?
[Chorus of voices] No, no, nope (all indicated no and Student B and Student A were
shaking their heads no as well)
Student A: I understand that like some people leave because of like religious reasons.
Pretty much everyone thought it was fun.
I: Do you guys talk about Talking Talons like when you are not in class or not? So, do you go
home and tell your mom and dad if you got to hold the snake?
Student A: Yes, I do because it’s interesting and we do like at the dinner table…like how
was your day but not like always.
45
Student B: Like if I go to parks and recs and my dad picks me up and asks me how my
day was and I’m like good and then we get home and then I’m like hey I learned this
today at Talking Talons and I’ll say like this animal does like this and he will say oh
that’s cool
I: to Student C: Do you talk to your parents about it?
Student C: Oh yeah, my dad’s very annoyed with it
I:Annoyed?
Student C: Because I am talking about it too much.
I: to Student D: Do you talk about it?
Student D: No not really.
I: Do you talk to anybody else about it?
Student D: Oh, I do talk to my friends in (Teacher A) class because they have it; it’s not in
too many classes.
Student A: or if there is something like really exciting, Like when Animal A came, like
when that happens like that incident.
Student B: He pooped on the carpet
Student C: Right.
[All of them laughing]
Student C: Yeah like he literally pooped on the carpet
I:So if you could be in Talking Talons again you would do it?
[All talking at once] yes, yeah, of course
46
I: So even if you were older, like if they had Talking Talons in high school?
Student A: Probably
Student B: Maybe
Student D: Yeah
Student B: Yeah. Actually, to be honest I like Talking Talons because we don’t have to
do math.
[All laughing.]
Student D: That’s a plus
Buddy class
Research on peer tutoring
One aspect of the Talking Talons program is the peer interaction. Students in the Talking
Talons program learn the material with the goal of presenting to a younger group of students in
the school.
A meta-analysis of peer assisted learning found increases in achievement. Peer tutoring is
most effective with “PAL interventions were most effective with younger, urban, low-income,
and minority students.”(Rohrbeck et al.).
Peer tutoring helps reading fluency (Kamps et al.), reading comprehension (McMaster,
Fuchs, and Fuchs) mathematics learning (Fantuzzo, Polite, and Grayson) both mathematics and
47
social skills (Pigott, Fantuzzo, and Clement) English (Greenwood et al.) and social interaction
for rejected boys (Gumpel and Frank).
Meta analyses on impact on math found the effect is particular strong for elementary aged
students (Kunsch, Jitendra, and Sood). A hands on gardening program found improvement in
social relationships in the classroom (Kim, Park, and Son).
Specifically buddy lesson plans for healthy eating and exercise physical activity, healthy
eating, and self-esteem and body image where lesson plans were delivered by 9-12 year olds for
6 to 8 year olds found that “Reductions in waist circumference were particularly significant for
children who were younger, overweight or obese, or attending First Nations schools. No
difference in body mass index score was observed between groups. Self-efficacy, healthy living
knowledge, and dietary intake significantly improved in younger peers who received the
intervention compared with students from control schools.” (Santos et al.).
Meta-analysis also shows that this impact is positive for both the “teacher” and the
“student” in peer tutoring situations. “This review of the literature on peer and cross-age tutoring
emphasizes programs in mathematics and suggests that such programs have positive academic
outcomes for African American and other minority students as well as for White students who
participate as tutors, as tutees, or both”. This results was shown even with short programs
“tutors show academic gains even when they do not receive additional subject matter instruction,
why longer and/or more substantial tutoring programs may not foster greater immediate
academic gains than shorter programs,” (Robinson, Schofield, and Steers-Wentzell)
48
Teaching benefits the tutor specifically in science classes “Mental rehearsal of peer-tutoring
episodes helped them appreciate weaknesses in their own subject knowledge.” (Galbraith and
Winterbottom)
Meta analyses also supports that peer tutoring is effective regardless of many factors.
“This meta-analysis examined effects of peer tutoring across 26 single-case research experiments
for 938 students in Grades 1–12. The Tau effect size for 195 phase contrasts was 0.75 with a
confidence interval of 0.71 to 0.78, indicating that moderate to large academic benefits can be
attributed to peer tutoring. Five potential moderators of these effects were examined: dosage,
grade level, reward, disability status, and content area. This is the first peer tutoring meta-
analysis in nearly thirty years to examine outcomes for elementary and secondary students, and
extends previous peer tutoring meta-analyses by examining disability as a potential moderator.
”Findings suggest that peer tutoring is an effective intervention regardless of dosage, grade level,
or disability status. Among students with disabilities, those with emotional and behavioral
disorders benefitted most.” (Bowman-Perrott et al.)
Classroom teacher assessment of Buddy class
The classroom teachers all agreed very strongly (mean of 7/7) that having the buddy
class had a positive impact on their students.
49
Table 20 Classroom teacher assessment of buddy class impact
Presenting to the Buddy class made the students take the responsibility of TT more seriously.
N Valid 2
Missing 0
Mean 7.00
Std. Deviation .000
Student feedback on Buddy class
Students universally expressed that they enjoyed “being the teacher” and presenting to
younger students. They mentioned leadership directly as well. Sections related the buddy class
are excerpted from interviews below.
I: Did you like doing the presentations or not?
[All jump in and speak at once] yeah; I liked it [nodding heads]
Student E: I like doing it to the other class.
Student G: I like teaching the little kids.
Student G:Cause you like see (Talking Talons educator) come in and do it, and I always
wanted to do it like that
Student F: It’s like you are the teacher
50
Student E: So you get to be like them, it’s really good.
I: What do you think?
Student H: It teaches us how to be leaders, it’s kind of like nerve-wracking to talk to kids
though but its pretty fun, they get to see animals.
I: What about you, did you like doing presentations?
Student K: I have not done one yet.
2:05
I:You have not done one yet?
Student F: He goes somewhere during class sometimes.
I: What are the best things about Talking Talons?
Student H: You learn about new things and all sorts of animals.
I:What did you guys think about the buddy class?
Student E and F speaking alternatively: I love it because they are really focused…they are
quiet, they defiantly chose it…they defiantly chose a good class.
Student E: Both my cousins are in there
I: Are your cousins in there. So do you like having them come in?
Student E: I think it’s a good idea, it’s like Ms. Betsy and Ms. Laurie teaching us and then we
are teaching the buddy class.
51
Student F: It’s like my grownup mode just turns on.
[Laughter]
Student G: It’s like we are the teacher
Class B:
I:Ok, what about the presentations, did you like doing the presentations?
Student A: Yeah that was really fun, I had the snake we did too,
Student B: we were like first we had the snake and it was really cool experience that we got
to teach. : I like the presentations because we get to hold the animals.
Student A: Also its cool being like everyone watching you and they are like hi.
I: Oh, you liked that, you liked being in front of the class? What about you, did you like being in
front of the class?
Student B: Yeah, me and (pointing to Student A) got to be in front of the class, in front of
four thousand people before.
Student A: Yeah we were singing.
Student B: We went to a choir, like together, and my like there’s like 4000 people
Student A: and then I’ve also like done dancing
I: So you are used to being in front of the class. What about you?
Student D: I’m fine with people
I: You didn’t have any problem doing the presentations?
52
I: Did you liked doing them, what did you think of the having the buddy class? Was that a
good thing or a bad thing?
Student A” Well it’s pretty cool because you get to hear other people’s questions and
things. And like that and you know are like, like the teacher.
I:What did you think?
[Student B Laughs]
I: So Student C did you like having a buddy class come in or would you rather that there was not
a buddy class, be honest this does not affect the funding it just tells us whether you think it was
useful.
Student C : I would like the buddy class, I feel like it is in addition to our class so it’s kinda
cool how having third graders, seeing how third graders take on what we are learning.
Student A yeah Plus it’s cool because it gives other people a chance to like, experience
everything because I used to be in (lower grade teachers) and at one point in time she was
actually doing the buddy class kind of thing so it was really cool
I: Oh so you been a buddy class?
Student A: Yeah I’ve been a buddy class and I am doing that now
I: That is really cool, and so what do you remember from being a buddy class?
Student A: I remember Banana, she stood out a lot.
I:[ laughing] Banana?
Student A: I miss Banana! And it was like really cool, I just went in the classroom and I was
like, oh my gosh there’s animals in here. Cause you can’t even have a dog most of the time.
It’s like you guys get to bring in birds and snakes and bats and you can’t even bring in a dog!
I: That’s true. So you guys were good with the buddy class? You would keep the buddy class?
53
[All four]: Yeah, yes
Student B: It was really awesome.
Role Model for younger students
The change in role model was not significant, the power was low due to sample size.
However, the direction of change was positive for the treatment group and negative for the
control group. This information is included for trends, rather than statistical significance.
Figure 3 I am a good role model for younger students
54
Table 21 Student answers by group to "I am a good role model for younger students"
group Mean Std. Deviation N
I am a good role model for younger
students.
dimension1
Control 4.9130 1.50493 23
Treatment 5.0750 1.32795 40
Total 5.0159 1.38532 63
POST I am a good role model for
younger students.
dimension1
Control 4.4783 1.83079 23
Treatment 5.2750 1.06187 40
Total 4.9841 1.43113 63
Table 22 I am a good role model significance tests
Multivariate Testsc
Effect
Value F
Hypothesis
df
Error
df Sig.
Partial Eta
Squared
Noncent.
Parameter
Observed
Powerb
rolem Pillai's Trace .004 .264a 1.000 61.000 .609 .004 .264 .080
rolem *
group
Pillai's Trace .031 1.929a 1.000 61.000 .170 .031 1.929 .277
a. Exact statistic
b. Computed using alpha = .05
c. Design: Intercept + group
Within Subjects Design: rolem
Science Knowledge
Change in Science Knowledge by Group
55
A statistically significant change in Knowledge was seen in the treatment group and not
for the control group from pretesting to post testing. F(1,55)=11.01. p<.01 η2=.45. This
represents a very large effect size.
Figure 4 Change in Science Knowledge by Group
56
Table 23 Change in Science Knowledge by Group
Descriptive Statistics
group Mean Std. Deviation N
know
Control 4.3533 .71249 23
Treatment 3.7493 .51974 34
Total 3.9930 .66932 57
pknow
Control 4.0550 .43830 23
Treatment 4.6221 .61254 34
Total 4.3933 .61266 57
Table 24 Change in Science Knowledge by Group Multivariate tests
Multivariate Testsc
Effect
Value F
Hypothesis
df
Error
df Sig.
Partial Eta
Squared
Noncent.
Parameter
Observed
Powerb
knowled Pillai's
Trace
.167 11.010a 1.000 55.000 .002 .167 11.010 .903
knowled *
group
Pillai's
Trace
.454 45.737a 1.000 55.000 .000 .454 45.737 1.000
57
a. Exact statistic
b. Computed using alpha = .05
Control. Design: Intercept + group
Within Subjects Design: knowled
Table 25 Change in Science Knowledge by Group Means
group knowled
Mean Std. Error
Control
Pretest 4.353 .126
Posttest 4.055 .115
Treatment
Pretest 3.749 .104
Posttest 4.622 .094
Quiz Knowledge
The information below indicates the mean score for the treatment group on the
knowledge sections of the quizzes. A table with the standard deviation is also included. The
educators have been given the results by question in order to examine areas which need more
58
attention. This document is available upon request and is included on the CD. A comparison
was also done by class. For all quizzes except the Reptile quiz, the results were not different by
class. For the Reptile quiz one teacher had slightly lower scores than the other teacher. There
were no significant differences by gender.
Figure 5 Quiz Knowledge section means
59
Table 26 Mean scores on Knowledge Quiz by topic and class
Descriptive Statistics
N Range Minimum Maximum Mean Std. Deviation
Q1 Intro. 43 4.00 1.00 5.00 3.8605 1.08192
Q2 Hawks. 39 3.00 2.00 5.00 3.4359 1.04617
Q3 Raptors. 37 2.00 1.00 3.00 1.9459 .40455
Q4 Bats 1. 46 4.00 1.00 5.00 4.2174 1.05226
Q5 Bats 2. 42 4.00 1.00 5.00 4.0238 1.09295
Q6 Owls. 42 4.00 1.00 5.00 3.7619 1.05483
Q7 Reptiles . 40 4.00 1.00 5.00 4.1000 1.29694
Q8 Vocab. 39 3.00 2.00 5.00 3.9744 .98641
Q9 General. 47 4.00 1.00 5.00 4.0426 1.14127
Q10 Energy . 43 4.00 1.00 5.00 3.8605 1.24559
Valid N (listwise) 20
class Q1
Intro.
Q2
Hawks.
Q3
Raptors.
Q4 Bats
1.
Q5 Bats
2.
Q6
Owls.
Q7
Reptiles .
Q8
Vocab.
Q9
General.
Teacher B Mean 3.8696 3.4211 2.0000 4.3333 4.2381 4.1500 4.5714 4.0909 3.9615
N 23 19 25 24 21 20 21 22 26
Std.
Deviation
1.01374 .96124 .40825 1.04950 .99523 .87509 .81064 .92113 1.31090
Teacher A Mean 3.8500 3.4737 1.8333 4.0909 3.8095 3.4286 3.5789 3.8235 4.1429
N 20 19 12 22 21 21 19 17 21
60
Std.
Deviation
1.18210 1.17229 .38925 1.06499 1.16701 1.12122 1.53897 1.07444 .91026
Total Mean 3.8605 3.4359 1.9459 4.2174 4.0238 3.7619 4.1000 3.9744 4.0426
N 43 39 37 46 42 42 40 39 47
Std.
Deviation
1.08192 1.04617 .40455 1.05226 1.09295 1.05483 1.29694 .98641 1.14127
Table 27 ANOVA comparison on Quiz Knowledge by Class
ANOVA Table
Sum of
Squares df
Mean
Square F Sig.
Q1 Intro. * class Between
Groups
(Combined) .004 1 .004 .003 .954
Within Groups 49.159 41 1.199
Total 49.163 42
Q2 Hawks. * class Between
Groups
(Combined) .221 2 .111 .096 .908
Within Groups 41.368 36 1.149
Total 41.590 38
Q3 Raptors. *
class
Between
Groups
(Combined) .225 1 .225 1.391 .246
Within Groups 5.667 35 .162
Total 5.892 36
Q4 Bats 1. * class Between
Groups
(Combined) .675 1 .675 .604 .441
Within Groups 49.152 44 1.117
61
ANOVA Table
Sum of
Squares df
Mean
Square F Sig.
Total 49.826 45
Q5 Bats 2. * class Between
Groups
(Combined) 1.929 1 1.929 1.640 .208
Within Groups 47.048 40 1.176
Total 48.976 41
Q6 Owls. * class Between
Groups
(Combined) 5.926 2 2.963 2.911 .066
Within Groups 39.693 39 1.018
Total 45.619 41
Q7 Reptiles . *
class
Between
Groups
(Combined) 9.826 1 9.826 6.694 .014
Within Groups 55.774 38 1.468
Total 65.600 39
Q8 Vocab. * class Between
Groups
(Combined) .686 1 .686 .699 .408
Within Groups 36.289 37 .981
Total 36.974 38
Q9 General. * class Between
Groups
(Combined) .382 1 .382 .289 .594
Within Groups 59.533 45 1.323
Total 59.915 46
Q10 Energy . *
class
Between
Groups
(Combined) 1.058 1 1.058 .676 .416
62
ANOVA Table
Sum of
Squares df
Mean
Square F Sig.
Within Groups 64.105 41 1.564
Total 65.163 42
No significance difference was found by gender, males and females had equivalent
scores on the knowledge sections of the test.
Table 28 Quiz Knowledge Means by Gender
Report
gender
Q1
Intro.
Q2
Hawks.
Q3
Raptors.
Q4 Bats
1.
Q5 Bats
2.
Q6
Owls.
Q7
Reptiles
.
Q8
Vocab.
Q9
General.
Q10
Energy
.
Mean 3.0000 3.0000 2.0000
N 1 1 1
Std.
Deviation
. . .
f Mean 4.1579 3.3333 1.8333 4.0500 4.1579 4.0556 4.3889 4.1250 3.8500 3.5500
N 19 18 18 20 19 18 18 16 20 20
Std.
Deviation
.89834 1.08465 .38348 1.09904 .89834 .80237 1.09216 .95743 1.03999 1.19097
m Mean 3.6250 3.5500 2.0526 4.3462 3.9130 3.5652 3.8636 3.8696 4.2692 4.1304
N 24 20 19 26 23 23 22 23 26 23
Std.
Deviation
1.17260 1.05006 .40465 1.01754 1.23998 1.19947 1.42413 1.01374 1.15092 1.25424
63
Report
gender
Q1
Intro.
Q2
Hawks.
Q3
Raptors.
Q4 Bats
1.
Q5 Bats
2.
Q6
Owls.
Q7
Reptiles
.
Q8
Vocab.
Q9
General.
Q10
Energy
.
Total Mean 3.8605 3.4359 1.9459 4.2174 4.0238 3.7619 4.1000 3.9744 4.0426 3.8605
N 43 39 37 46 42 42 40 39 47 43
Std.
Deviation
1.08192 1.04617 .40455 1.05226 1.09295 1.05483 1.29694 .98641 1.14127 1.24559
Table 29 ANOVA comparison for Talking Talons Quizzes by Gender
ANOVA Table
Sum of
Squares df
Mean
Square F Sig.
Q1 Intro. * gender Between
Groups
(Combined) 3.011 1 3.011 2.675 .110
Within Groups 46.151 41 1.126
Total 49.163 42
Q2 Hawks. * gender Between
Groups
(Combined) .640 2 .320 .281 .757
Within Groups 40.950 36 1.138
Total 41.590 38
Q3 Raptors. *
gender
Between
Groups
(Combined) .445 1 .445 2.856 .100
Within Groups 5.447 35 .156
Total 5.892 36
64
ANOVA Table
Sum of
Squares df
Mean
Square F Sig.
Q4 Bats 1. * gender Between
Groups
(Combined) .991 1 .991 .893 .350
Within Groups 48.835 44 1.110
Total 49.826 45
Q5 Bats 2. * gender Between
Groups
(Combined) .624 1 .624 .516 .477
Within Groups 48.352 40 1.209
Total 48.976 41
Q6 Owls. * gender Between
Groups
(Combined) 3.022 2 1.511 1.384 .263
Within Groups 42.597 39 1.092
Total 45.619 41
Q7 Reptiles . *
gender
Between
Groups
(Combined) 2.731 1 2.731 1.651 .207
Within Groups 62.869 38 1.654
Total 65.600 39
Q8 Vocab. * gender Between
Groups
(Combined) .616 1 .616 .627 .434
Within Groups 36.359 37 .983
Total 36.974 38
Q9 General. *
gender
Between
Groups
(Combined) 6.250 2 3.125 2.562 .089
Within Groups 53.665 44 1.220
65
ANOVA Table
Sum of
Squares df
Mean
Square F Sig.
Total 59.915 46
Q10 Energy . *
gender
Between
Groups
(Combined) 3.604 1 3.604 2.400 .129
Within Groups 61.559 41 1.501
Total 65.163 42
Self-perceived science grade in school
A statistically significant change in self-reported anticipated grade in science was
seen in the treatment group and not for the control group from pretesting to post testing
F(1,57)=19.83. p<.001 η2=.26. This represents a large effect size.
Figure 6 Self perceived change in science grade by group
66
Table 30 Group Comparison for I will get a good grade in science class this year
Descriptive Statistics
group Mean Std. Deviation N
I will get a good grade in science class
this year
dimsion1
dimension1
Control 5.2609 1.28691 23
Treatment 5.1111 1.00791 36
Total 5.1695 1.11653 59
POST I will get a good grade in
science class this year
Control 4.1304 1.91417 23
Treatment 5.5833 .64918 36
67
Descriptive Statistics
group Mean Std. Deviation N
I will get a good grade in science class
this year
dimsion1
dimension1
Control 5.2609 1.28691 23
Treatment 5.1111 1.00791 36
Total 5.1695 1.11653 59
POST I will get a good grade in
science class this year
Control 4.1304 1.91417 23
Treatment 5.5833 .64918 36
Total 5.0169 1.46795 59
Table 31 Group Comparison for I will get a good grade in science class this year Multivariate tests
Multivariate Testsc
Effect
Value F
Hypothesis
df
Error
df Sig.
Partial Eta
Squared
Noncent.
Parameter
Observed
Powerb
gradesci Pillai's Trace .055 3.346a 1.000 57.000 .073 .055 3.346 .436
gradesci *
group
Pillai's Trace .258 19.836a 1.000 57.000 .000 .258 19.836 .992
a. Exact statistic
b. Computed using alpha = .05
c. Design: Intercept + group
Within Subjects Design: gradesci
68
Group Comparison for I will get a good grade in science class this year Within Subjects Contrasts
Tests of Within-Subjects Contrasts
Measure:MEASURE_1
Source gradesci Type III
Sum of
Squares df
Mean
Square F Sig.
Partial Eta
Squared
Noncent.
Parameter
Observed
Powera
gradesci dimension2
Linear 3.040 1 3.040 3.346 .073 .055 3.346 .436
gradesci *
group dimension2
Linear 18.023 1 18.023 19.836 .000 .258 19.836 .992
Error(gradesci) dimension2
Linear 51.790 57 .909
a. Computed using alpha = .05
Quiz Attitude subscales
Student mean attitudes about the program are presented below. The maximum possible
score is a 6/6 which indicates Strongly Agree.
69
Figure 7 Quiz Attitude subscales
Table 32 Quiz Attitude subscale descriptive statistics
N Minimum Maximum Mean Std. Deviation
Program enjoyment 48 3.00 6.00 5.3646 .76098
Reported understanding 49 4.25 6.00 5.3581 .46657
Attitude science 47 3.67 6.00 5.3936 .69368
70
N Minimum Maximum Mean Std. Deviation
Attitude Environment 46 4.00 6.00 5.7717 .42362
Feedback on Educator
Student mean attitudes about the Talking Talon’s educators are presented below. The
maximum possible score is a 6/6 which indicates Strongly Agree.
Student Quantitative Feedback Figure 8 Student Perception of Educator
71
Student Feedback
Table 33 Student feedback on Educator Subscales
N Minimum Maximum Mean Std. Deviation
Perception of Educator 52 4.36 6.00 5.6282 .39387
Bonding with Educator 50 4.00 6.00 5.5850 .53066
Educator Teaching Skills 51 4.38 6.00 5.6880 .37678
72
Figure 9 Student Perception of Educator
Table 34 Student feedback on Educator from Quiz results
N Minimum Maximum Mean Std. Deviation
POST My Talking Talons
educator (teacher) cared a lot
about what I learned.
45 1 6 5.56 1.119
POST My Talking Talons
educator (teacher) was interested
in what I had to say.
45 3 6 5.69 .668
POST My Talking Talons
educator (teacher) was fair about
giving everyone an equal chance
to do things.
45 1 6 5.58 1.215
73
N Minimum Maximum Mean Std. Deviation
POST My Talking Talons
educator knew a lot about the
animals.
45 5 6 5.89 .318
POST My Talking Talons
educator (teacher) was easy to
understand.
45 3 6 5.67 .739
Figure 10 Student Feedback on Educator questions
Table 35 Student Feedback on Individual Educator questions
74
Descriptive Statistics
N Minimum Maximum Mean Std. Deviation
POST I think my Talking Talons
educator (teacher) liked coming
to teach my class.
45 4 6 5.78 .471
POST My Talking Talons
educator understands how kids
my age think.
45 1 6 5.38 1.211
POST My Talking Talons
educator treated boys and girls
equally.
45 3 6 5.87 .505
POST My Talking Talons
educator helped me to learn how
to give speeches.
45 1 6 5.64 1.026
POST My Talking Talons
educator was usually on time for
my class.
45 4 6 5.71 .626
Valid N (listwise) 45
Student Qualitative Feedback
Portions of the interview relating to the Talking Talon’s educators are excerpted below.
Student E: I like our teachers Ms. Betsy and Ms. Laurie because they always call on everybody
and they are always like…
Student F: focused
Student E: they are always like focused and they know what to say, they are always prepared
75
Student F ( over the top of Student E) always prepared
Ok
Student G: They are not always serious and uh…they joke around times.
[Laughing]
Classroom teacher feedback for Educator
The classroom teachers mean perception of the educator is presented below. These
questions are on a scale with a maximum positive score of seven.
Figure 11 Classroom teacher feedback on Talking Talons Educator
76
Table 36 Classroom Teacher Feedback on Talking Talons Educator
N Minimum Maximum Mean Std. Deviation
The TT educator was generally
prepared to teach the class.
2 5 7 6.00 1.414
The TT educator worked with me
to make the program run
smoothly.
2 6 7 6.50 .707
The TT educator used classroom
time well.
2 7 7 7.00 .000
77
N Minimum Maximum Mean Std. Deviation
The TT educator appeared
generally knowledgeable about
content presented
2 6 7 6.50 .707
The TT educator appeared to
enjoy teaching the students.
2 7 7 7.00 .000
The students appeared
comfortable with the educator.
2 7 7 7.00 .000
Valid N (listwise) 0
Figure 12 Classroom Teacher subscales on Educator
78
Classroom teacher feedback for Talking Talons Program
The classroom teacher’s mean perceptions of the Talking Talon’s program are presented
below. These questions are on a scale with a maximum positive score of seven.
Teacher Attitude Figure 13 Classroom teacher feedback on Talking Talons program
79
Table 37 Classroom Teacher Feedback Personal
N Minimum Maximum Mean Std. Deviation
I was able to tie the TT program
into my curriculum.
2 7 7 7.00 .000
I was frustrated with the TT
program.
2 1 1 1.00 .000
I found the material presented by
the TT program to be interesting
to me personally.
2 6 7 6.50 .707
I would be willing to have the TT
program in my classroom again.
2 6 7 6.50 .707
Effectiveness by student ability: feedback by classroom teachers
The classroom teachers mean perception of the impact of the program by student ability
is presented below. These questions are on a scale with a maximum positive score of seven. The
teachers found the program generally very effective for average students and very effective to
effective for advanced students. The program was found to be effective for those with learning
disabilities. These results mirror previous years feedback and indicate that the program is
reaching the majority of the student’s abilities at the correct level.
80
Figure 14 Classroom teacher assessment of Effectiveness of program by student ability
Table 38 Classroom teacher feedback program effectiveness by student ability
N Minimum Maximum Mean Std. Deviation
The TT program was
effective for above average
students.
2 6 6 6.00 .000
81
N Minimum Maximum Mean Std. Deviation
The TT program was
effective for average
students.
2 6 7 6.50 .707
The program was effective
for students with learning
disabilities.
2 4 6 5.00 1.414
Valid N (listwise) 2
The teachers found the program generally very effective for average students and
effective for advanced students and those with learning disabilities.
Classroom teacher subscales
The classroom teacher’s mean perceptions for subscales are presented below. These
questions are on a scale with a maximum positive score of seven.
Figure 15 Classroom teacher perception
82
Table 39: Classroom teacher subscales for Talking Talons program
N Minimum Maximum Mean Std. Deviation
Teacher science for students 2 5.67 6.33 6.0000 .47140
Teacher effectiveness by
student ability
2 5.33 6.33 5.8333 .70711
Teacher components of TT 2 7.00 7.00 7.0000 .00000
Teachers personal feelings
about TT
2 6.20 6.80 6.5000 .42426
83
N Minimum Maximum Mean Std. Deviation
Teachers perception of
educator
2 6.33 7.00 6.6667 .47140
Mean overall teacher
assessment of educators
2 6.18 6.71 6.4412 .37435
Valid N (listwise) 2
Other Results
Significant changes in pre posttest Composite
Moral Attitude
Research on Moral Attitude
A higher moral attitude has been linked to the ability to judge aggressive tendencies in
oneself (Berkowitz et al.)(). It also has an impact on moral behavior (Aquino and Reed) and
serves to decrease delinquency( (Raaijmakers, Engels, and Van Hoof; Tarry and Emler^;
Arbuthnot and Gordon)). Increases in moral attitude are linked with increases in academic
performance (Maguin and Loeber; Arbuthnot and Gordon) and decreases in actual aggressive
tendencies (Guivernau and Duda). Moral attitude has also been found to impact moral behavior
and empathy (Tangney, Stuewig, and Mashek; Malti and Krettenauer; Reynolds and Ceranic)
84
A statistically significant change in Moral Attitude was seen in the treatment group and
not for the control group from pretesting to post testing. F(1,58)=8.00. p<.01 η2=.12. This
represents a small/medium to effect size.
Figure 16 Change in Moral Attitude by group
85
Table 40 Change in Moral Attitude by Group
group Mean Std. Deviation N
moral
dimension1
Control 5.5797 .46415 23
Treatment 5.3008 .60219 37
Total 5.4077 .56595 60
pmoral
dimension1
Control 5.2754 .99486 23
Treatment 5.5218 .55699 37
Total 5.4273 .75694 60
Table 41 Change in Moral Attitude by Group Multivariate tests
Multivariate Testsc
Effect
Value F
Hypothesis
df
Error
df Sig.
Partial Eta
Squared
Noncent.
Parameter
Observed
Powerb
dmoral1 Pillai's Trace .003 .202a 1.000 58.000 .655 .003 .202 .073
dmoral1 *
group
Pillai's Trace .121 8.006a 1.000 58.000 .006 .121 8.006 .795
a. Exact statistic
b. Computed using alpha = .05
c. Design: Intercept + group
Within Subjects Design: dmoral1
86
Table 42 Change in Moral Attitude by Group Within subjects
Tests of Within-Subjects Effects
Measure:MEASURE_1
Source Type III
Sum of
Squares df
Mean
Square F Sig.
Partial
Eta
Squared
Noncent.
Parameter
Observed
Powera
dmoral1 Sphericity
Assumed
.049 1 .049 .202 .655 .003 .202 .073
dmoral1 *
group
Sphericity
Assumed
1.957 1 1.957 8.006 .006 .121 8.006 .795
Error(dmoral1) Sphericity
Assumed
14.175 58 .244
a. Computed using alpha = .05
Table 43 Change in Moral Attitude by Group Between subjects
Tests of Between-Subjects Effects
Measure:MEASURE_1
Transformed Variable:Average
Source Type III Sum
of Squares df
Mean
Square F Sig.
Partial Eta
Squared
Noncent.
Parameter
Observed
Powera
Intercept 3332.529 1 3332.529 5286.484 .000 .989 5286.484 1.000
group .007 1 .007 .012 .914 .000 .012 .051
Error 36.562 58 .630
a. Computed using alpha = .05
87
Overall significant Composite results
Figure 17 Statistically significant Pre Post change by group
Table 44 Statistically significant Pre Post change by group
group Change in Moral
Attitude
Change in Science
Attitude Change in Knowledge
Control Mean -.3043 -.8295 -.2982
N 23 23 23
88
group Change in Moral
Attitude
Change in Science
Attitude Change in Knowledge
Std. Deviation .79717 1.46536 .73762
Treatment Mean .2209 .2467 .8728
N 37 37 34
Std. Deviation .63178 .92088 .56822
Total Mean .0196 -.1658 .4003
N 60 60 57
Std. Deviation .73948 1.26354 .86023
Table 45 ANOVA Statistically significant Pre Post change by group
ANOVA Table
Sum of
Squares df
Mean
Square F Sig.
Change in Moral
Attitude * group
Between
Groups
(Combined) 3.913 1 3.913 8.006 .006
Within Groups 28.350 58 .489
Total 32.263 59
Change in Science
Attitude * group
Between
Groups
(Combined) 16.426 1 16.426 12.251 .001
Within Groups 77.769 58 1.341
Total 94.195 59
Change in Knowledge
* group
Between
Groups
(Combined) 18.814 1 18.814 45.737 .000
Within Groups 22.625 55 .411
89
ANOVA Table
Sum of
Squares df
Mean
Square F Sig.
Change in Moral
Attitude * group
Between
Groups
(Combined) 3.913 1 3.913 8.006 .006
Within Groups 28.350 58 .489
Total 32.263 59
Change in Science
Attitude * group
Between
Groups
(Combined) 16.426 1 16.426 12.251 .001
Within Groups 77.769 58 1.341
Total 94.195 59
Change in Knowledge
* group
Between
Groups
(Combined) 18.814 1 18.814 45.737 .000
Within Groups 22.625 55 .411
Total 41.439 56
Table 46 Effect Size Statistically significant Pre Post change by group
Measures of Association
Eta Eta Squared
Change in Moral Attitude * group .348 .121
Change in Science Attitude * group .418 .174
Change in Knowledge * group .674 .454
Table 47 Non Significant Pre Post change by Group
90
Overall Non significant composite results
Note that the scale is small, and these bar charts represent small differences. These
results are not statistically significant and are presented to examine direction of change.
Figure 18 Non Significant changes for Composite
Table 48 Means Non Significant Pre Post change by Group
91
group Change in Self
Esteem
Change in Locus of
Control
Change in Attitude
toward School
Change in Attitude
toward Violence
Control Mean -.0755 -.4290 -.0413 -.3413
N 23 23 23 23
Std. Deviation 1.03937 .79424 .75388 .99780
Treatment Mean .1644 -.0626 .1194 .0635
N 37 37 37 37
Std. Deviation .69467 .74604 .63479 .90605
Total Mean .0725 -.2031 .0578 -.0917
N 60 60 60 60
Std. Deviation .84327 .77916 .68117 .95475
Table 49 ANOVA Non Significant Pre Post change by Group
ANOVA Table
Sum of
Squares df
Mean
Square F Sig.
Change in Self Esteem
* group
Between
Groups
(Combined) .816 1 .816 1.151 .288
Within Groups 41.139 58 .709
Total 41.955 59
Change in Locus of
Control * group
Between
Groups
(Combined) 1.904 1 1.904 3.256 .076
Within Groups 33.915 58 .585
Total 35.818 59
92
ANOVA Table
Sum of
Squares df
Mean
Square F Sig.
Change in Attitude
toward School * group
Between
Groups
(Combined) .366 1 .366 .786 .379
Within Groups 27.010 58 .466
Total 27.376 59
Change in Attitude
toward Violence *
group
Between
Groups
(Combined) 2.324 1 2.324 2.620 .111
Within Groups 51.457 58 .887
Total 53.781 59
Table 50 Effect size Non Significant Pre Post change by Group
Measures of Association
Eta Eta Squared
Change in Self Esteem * group .139 .019
Change in Locus of Control * group .231 .053
Change in Attitude toward School * group .116 .013
Change in Attitude toward Violence * group .208 .043
Table 51 Pre and Posttest means by group
Descriptive Statistics
group Mean Std. Deviation N
sci dimension1
Control 5.2609 .87684 23
93
Descriptive Statistics
group Mean Std. Deviation N
Treatment 5.1052 .91377 34
Total 5.1680 .89444 57
psci
dimension1
Control 4.4314 1.49023 23
Treatment 5.3379 .93918 34
Total 4.9721 1.26236 57
self
dimension1
Control 5.0242 1.22679 23
Treatment 5.1683 .70116 34
Total 5.1101 .94130 57
pself
dimension1
Control 4.9487 1.00337 23
Treatment 5.3276 .69226 34
Total 5.1747 .84445 57
moral
dimension1
Control 5.5797 .46415 23
Treatment 5.3652 .57605 34
Total 5.4518 .53986 57
pmoral
dimension1
Control 5.2754 .99486 23
Treatment 5.5645 .48619 34
Total 5.4479 .74068 57
viol
dimension1
Control 5.6261 .53361 23
Treatment 5.4235 .82941 34
Total 5.5053 .72615 57
pviol dimension1
Control 5.2848 .87431 23
94
Descriptive Statistics
group Mean Std. Deviation N
Treatment 5.5279 .68503 34
Total 5.4298 .76898 57
attsch
dimension1
Control 5.4239 .83583 23
Treatment 5.3926 .47068 34
Total 5.4053 .63658 57
pattsch
dimension1
Control 5.3826 .75297 23
Treatment 5.4235 .52746 34
Total 5.4070 .62217 57
know
dimension1
Control 4.3533 .71249 23
Treatment 3.7493 .51974 34
Total 3.9930 .66932 57
pknow
dimension1
Control 4.0550 .43830 23
Treatment 4.6221 .61254 34
Total 4.3933 .61266 57
locus
dimension1
Control 5.0362 .84560 23
Treatment 4.6961 .79463 34
Total 4.8333 .82544 57
plocus
dimension1
Control 4.6072 .96740 23
Treatment 4.6000 .89680 34
Total 4.6029 .91739 57
95
Table 52 Pre and Posttest multivariate tests by group
Multivariate Testsc
Effect
Value F
Hypothesis
df
Error
df Sig.
Partial
Eta
Squared
Noncent.
Parameter
Observed
Powerb
Between
Subjects
Intercept Pillai's
Trace
.993 1057.283a 7.000 49.000 .000 .993 7400.980 1.000
group Pillai's
Trace
.163 1.359a 7.000 49.000 .244 .163 9.514 .519
Within
Subjects
time Pillai's
Trace
.381 4.312a 7.000 49.000 .001 .381 30.186 .980
time *
group
Pillai's
Trace
.528 7.841a 7.000 49.000 .000 .528 54.887 1.000
a. Exact statistic
b. Computed using alpha = .05
c. Design: Intercept + group
Within Subjects Design: time
96
Figure 19 Pretest and Posttest Multivariate Tests by group
Multivariatec,d
Within Subjects Effect
Value F
Hypothesis
df
Error
df Sig.
Partial Eta
Squared
Noncent.
Parameter
Observed
Powerb
time Pillai's Trace .381 4.312a 7.000 49.000 .001 .381 30.186 .980
time *
group
Pillai's Trace .528 7.841a 7.000 49.000 .000 .528 54.887 1.000
a. Exact statistic
b. Computed using alpha = .05
c. Design: Intercept + group
Within Subjects Design: time
d. Tests are based on averaged variables.
Table 53 Pre and Posttest Univariate tests by group
Univariate Tests
Source Measure
Type III Sum of
Squares df
Mean
Square F Sig.
Partial
Eta
Squared
Noncent.
Parameter
Observed
Powera
time scig 2.443 1 2.443 3.655 .061 .062 3.655 .468
selfg .048 1 .048 .144 .706 .003 .144 .066
moralg .076 1 .076 .314 .578 .006 .314 .085
violg .385 1 .385 .850 .360 .015 .850 .148
schoolg .001 1 .001 .003 .953 .000 .003 .050
knowg 2.265 1 2.265 11.010 .002 .167 11.010 .903
97
Univariate Tests
Source Measure
Type III Sum of
Squares df
Mean
Square F Sig.
Partial
Eta
Squared
Noncent.
Parameter
Observed
Powera
locusg 1.891 1 1.891 6.442 .014 .105 6.442 .703
time *
group
scig 7.739 1 7.739 11.578 .001 .174 11.578 .917
selfg .378 1 .378 1.131 .292 .020 1.131 .181
moralg 1.740 1 1.740 7.219 .010 .116 7.219 .752
violg 1.363 1 1.363 3.010 .088 .052 3.010 .399
schoolg .036 1 .036 .167 .685 .003 .167 .069
knowg 9.407 1 9.407 45.737 .000 .454 45.737 1.000
locusg .760 1 .760 2.590 .113 .045 2.590 .353
Error(time) scig 36.764 55 .668
selfg 18.394 55 .334
moralg 13.260 55 .241
violg 24.898 55 .453
schoolg 11.792 55 .214
knowg 11.312 55 .206
locusg 16.147 55 .294
98
Table 54 Within Subjects contrasts Pre and Posttest means by group
Tests of Within-Subjects Contrasts
Source Measure time Type III
Sum of
Squares df
Mean
Square F Sig.
Partial Eta
Squared
Noncent.
Parameter
Observed
Powera
time
dimension1
scig Linear 2.443 1 2.443 3.655 .061 .062 3.655 .468
selfg Linear .048 1 .048 .144 .706 .003 .144 .066
moralg Linear .076 1 .076 .314 .578 .006 .314 .085
violg Linear .385 1 .385 .850 .360 .015 .850 .148
schoolg Linear .001 1 .001 .003 .953 .000 .003 .050
knowg Linear 2.265 1 2.265 11.010 .002 .167 11.010 .903
locusg Linear 1.891 1 1.891 6.442 .014 .105 6.442 .703
time *
group
dimension1
scig Linear 7.739 1 7.739 11.578 .001 .174 11.578 .917
selfg Linear .378 1 .378 1.131 .292 .020 1.131 .181
moralg Linear 1.740 1 1.740 7.219 .010 .116 7.219 .752
violg Linear 1.363 1 1.363 3.010 .088 .052 3.010 .399
schoolg Linear .036 1 .036 .167 .685 .003 .167 .069
knowg Linear 9.407 1 9.407 45.737 .000 .454 45.737 1.000
locusg Linear .760 1 .760 2.590 .113 .045 2.590 .353
Error(time)
dimension1
scig Linear 36.764 55 .668
selfg Linear 18.394 55 .334
moralg Linear 13.260 55 .241
violg Linear 24.898 55 .453
schoolg Linear 11.792 55 .214
99
Tests of Within-Subjects Contrasts
Source Measure time Type III
Sum of
Squares df
Mean
Square F Sig.
Partial Eta
Squared
Noncent.
Parameter
Observed
Powera
knowg Linear 11.312 55 .206
locusg Linear 16.147 55 .294
a. Computed using alpha = .05
Table 55 Pre and Posttest means by group
time
Measure time
Mean Std. Error
95% Confidence Interval
Lower Bound Upper Bound
dimension0
scig 1 5.183 .121 4.940 5.426
2 4.885 .161 4.563 5.207
selfg 1 5.096 .128 4.840 5.352
2 5.138 .112 4.913 5.363
moralg 1 5.472 .072 5.328 5.617
2 5.420 .099 5.222 5.618
violg 1 5.525 .098 5.328 5.721
2 5.406 .103 5.199 5.614
schoolg 1 5.408 .087 5.235 5.582
2 5.403 .085 5.233 5.573
knowg 1 4.051 .082 3.888 4.215
2 4.339 .074 4.190 4.487
100
locusg 1 4.866 .110 4.646 5.087
2 4.604 .125 4.353 4.854
Table 56 Posttest Attitudes for Treatment
Descriptive Statistics
N Minimum Maximum Mean Std. Deviation
Perception of Educator 51 4.36 6.00 5.6340 .39556
Animal Bonding 51 4.00 6.00 5.4627 .50595
Program enjoyment 47 3.00 6.00 5.3723 .76728
Reported understanding 49 4.25 6.00 5.3581 .46657
Attitude science 47 3.67 6.00 5.3936 .69368
Bonding with Educator 50 4.00 6.00 5.5850 .53066
Educator Teaching Skills 50 4.38 6.00 5.6951 .37715
Attitude Environment 45 4.00 6.00 5.7667 .42699
Valid N (listwise) 41
Table 57 Student overall perception of program
N Minimum Maximum Mean Std. Deviation
studenover 52 4.66 6.00 5.5366 .33736
Valid N (listwise) 52
101
Summary
A statistically significant change in Attitude toward Science was seen in the treatment group
and not for the control group from pretesting to post testing. F(1,58)=12.25. p<.001 η2=.17. This
represents a medium effect size. In order to examine the longitudinal change, the science
attitudes of the treatment group were examined twice during the program as well as during
pretesting and posttesting. The treatment group exhibited a statistically significant positive
change in attitude toward science as the program progressed. F(3,9)=5.15. p<.01 η2=.15
exhibiting a medium effect size. This statistic uses only the treatment group, as the control group
does not take the Talking Talon’s quizzes. Students in the program also exhibited a statistically
significant change in self-reported anticipated grade in science. This change was seen in the
treatment group and not for the control group from pretesting to post testing F(1,57)=19.83.
p<.001 η2=.26 which is a large effect size.
A statistically significant change in Knowledge was seen in the treatment group and not for
the control group from pretesting to post testing. F(1,55)=11.01. p<.01 η2=.45. This represents a
very large effect size. A breakdown of the treatment group Knowledge scores on quizzes also
was provided.
102
A statistically significant change in Moral Attitude was seen in the treatment group and not
for the control group from pretesting to post testing. F(1,58)=8.00. p<.01 η2=.12 with a small to
medium effect size.
Classroom teacher feedback was extremely positive and both teachers indicated the program
increased student science knowledge and attitudes, that the buddy class was worthwhile and also
indicated a strong willingness to have the program again. Mean scores for all subscales
measuring teacher feedback were all above 6 on a 7 point scale (a higher number indicates more
positive feedback) with a mean of 6.44 and a standard deviation of .37. The classroom teachers
rate the program as most effective for average students, with effectiveness for above average and
delayed students slightly lower.
Qualitative feedback from students was overwhelmingly positive, with students mentioning
that the program was a positive experience, that they enjoyed presenting to the buddy class and
found it both rewarding and educational. Many expressed that they relished “being the teacher”
stating that “It’s like my grownup mode just turns on”. The students also were enthusiastic about
that both the hands on and presentation sections of the program and found them enjoyable and
educational. As a student stated during interview “Talking Talons is like unicorns, it’s
amazing!”. The only suggestion for improvements by the students was the inclusion of even
more animals (they would like a llama and turtles). Means scores for all student treatment
subscales were above 5.35 on a 6 point scale with a mean of 5.54 and a standard deviation of .34.
Even with a small sample size, and thus less statistical power, it is clear that the program
positivity impacted the participants. Science attitudes and knowledge are increased and
classroom teachers find the program effective and enjoyable for their students.
103
References
Al-Balushi, Sulaiman M, and Shamsa S Al-Aamri. “The Effect of Environmental Science
Projects on Students’ Environmental Knowledge and Science Attitudes.” International
Research in Geographical and Environmental Education 2014: 213–227. Web.
Aquino, Karl, and Americus Reed. “The Self-Importance of Moral Identity.” Journal of
personality and social psychology 83.6 (2002): 1423–1440. Web.
Arbuthnot, J, and D a Gordon. “Behavioral and Cognitive Effects of a Moral Reasoning
Development Intervention for High-Risk Behavior-Disordered Adolescents.” Journal of
consulting and clinical psychology 54.2 (1986): 208–216. Web.
Arkow, Phil. “Animals as Facilitators of Social Capital and Character Education.” Society for the
Study of Social Problems 2006. Web.
Backes, John S. “Bridging the Gender Gap: Self-Concept in the Middle Grades.” Schools in the
Middle 3.3 (1994): 19–23. Print.
Bakeman, Roger. “Recommended Effect Size Statistics for Repeated Measures Designs.”
Behavior research methods 37.3 (2005): 379–384. Web.
Baker, Dale R. “Predictive Value of Attitude, Cognitive Ability, and Personality to Science
Achievement in the Middle School.” Journal of Research in Science Teaching 22.2 (1985):
103–113. Print.
Berkowitz, Marvin W. et al. “Moral Reasoning and Judgments of Aggression.” Journal of
104
Personality and Social Psychology 51.4 (1986): 885–891. Web.
Bowman-Perrott, Lisa et al. “Academic Benefits of Peer Tutoring : A Meta-Analytic Review of
Single-Case Research.” School Psychology Review 42.1 (2013): 39–55. Print.
Catsambis, Sophia. “Gender, Race, Ethnicity, and Science Education in the Middle Grades.”
Journal of Research in Science Teaching 32.3 (1995): 243–257. Print.
Desy, Elizabeth A, Scott A Peterson, and Vicky Brockman. “Gender Differences in Science-
Related Attitudes and Interests among Middle School and High School Students.” Science
Educator 20.2 (2011): 23–30. Web.
Ellery Samuels, William, Kate Nicoll, and Cindy Trifone. “An In-Class, Humane Education
Program Can Improve Young Students’ Attitudes Toward Animals.” Society & Animals
16.1 (2008): 45–60. Web.
Fantuzzo, John W., Kenneth Polite, and Natalie Grayson. “An Evaluation of Reciprocal Peer
Tutoring across Elementary School Settings.” Journal of School Psychology 28.4 (1990):
309–323. Web.
Farenga, Stephen, and Beverly Joyce. “Science-Related Attitudes and Science Course
Selection...” Roeper Review 20 (1998): 247. Print.
Freedman, Michael P. “The Influence of Laboratory Instruction on Science Achievement and
Attitude toward Science across Gender Differences.” Journal of Women and Minorities in
Science and Engineering 8.2 (2002): 191–200. Web.
Galbraith, Jonathan, and Mark Winterbottom. “Peer‐tutoring: What’s in It for the Tutor?”
105
Educational Studies 37.3 (2011): 321–332. Web.
Glasson, George E. “The Effects of Hands-on and Teacher Demonstration Laboratory Methods
on Science Achievement in Relation to Reasoning Ability and Prior Knowledge.” Journal
of Research in Science Teaching 26.2 (1989): 121–131. Web.
Greenwood, Charles R et al. “Classwide Peer Tutoring Learning Management System:
Applications with Elementary-Level English Language Learners.” Remedial and Special
Education 22.1 (2001): 34–47. Print.
Guivernau, Marta, and Joan L. Duda. “Moral Atmosphere and Athletic Aggressive Tendencies in
Young Soccer Players.” Journal of Moral Education 31.1 (2002): 67–85. Web.
Gumpel, T P, and R Frank. “An Expansion of the Peer-Tutoring Paradigm: Cross-Age Peer
Tutoring of Social Skills among Socially Rejected Boys.” Journal of applied behavior
analysis 32.1 (1999): 115–118. Web.
Jarvis, Tina, and Anthony Pell. “Changes in Primary Boys’ and Girls' Attitudes to School and
Science during a Two-Year Science in-Service Programme.” The Curriculum Journal 13.1
(2002): 43–69. Print.
Kamps, D M et al. “Classwide Peer Tutoring: An Integration Strategy to Improve Reading Skills
and Promote Peer Interactions among Students with Autism and General Education Peers.”
Journal of applied behavior analysis 27.1 (1994): 49–61. Web.
Kanter, David E., and Spyros Konstantopoulos. “The Impact of a Project-Based Science
Curriculum on Minority Student Achievement, Attitudes, and Careers: The Effects of
106
Teacher Content and Pedagogical Content Knowledge and Inquiry-Based Practices.”
Science Education 94.5 (2010): 855–887. Web.
Kim, SeongSil, SinAe Park, and KiCheol Son. “Improving Peer Relations of Elementary School
Students through a School Gardening Program.” HortTechnology 24.2 (2014): 181–187.
Web.
Kimble, G. “Children Learning about Biodiversity at an Environment Centre, a Museum and at
Live Animal Shows.” Studies in Educational Evaluation 41 (2014): 48–57. Web.
Kose, Sacit et al. “The Effects of Cooperative Learning Experience on Eighth Grade Students’
Achievement and Attitude toward Science.” Education 131.1 (2010): 169–180. Print.
Kunsch, Catherine A, Asha K Jitendra, and Sheetal Sood. “The Effects of Peer-Mediated
Instruction in Mathematics for Students with Learning Problems: A Research Synthesis.”
Learning Disabilities Research & Practice 22.1 (2007): 1–12. Web.
Lee, Valerie E, and David T Burkam. “Gender Differences in Middle Grade Science
Achievement: Subject Domain, Ability Level, and Course Emphasis.” Science Education
80.6 (1996): 613–650. Print.
Maguin, Eugene, and Rolf Loeber. “Academic Performance and Delinquency.” Crime and
Justice 20 (1996): 145–264. Web.
Malti, Tina, and Tobias Krettenauer. “The Relation of Moral Emotion Attributions to Prosocial
and Antisocial Behavior: A Meta-Analysis.” Child development 84.2 (2013): 397–412.
Web.
107
Margadant-Van Arcken, Marjan. “Environmental Education, Children, and Animals.”
Anthrozoos 3:1 (1989): 14–19. Print.
Mattern, Nancy Page Garland. “Relationships between Attitudes toward and Achievement in
Science for Rural Middle School Students: Patterns across Gender.” Dissertation Abstracts
International Section A: Humanities & Social Sciences 60.12-A (2000): 4319. Print.
Mattern, Nancy, and Candace Schau. “Gender Differences in Science Attitude-Achievement
Relationships over Time among White Middle-School Students.” Journal of Research in
Science Teaching 39.4 (2002): 324–340. Web.
McMaster, Kristen L., Douglas Fuchs, and Lynn S. Fuchs. “Research on Peer-Assisted Learning
Strategies: The Promise and Limitations of Peer-Mediated Instruction.” Reading & Writing
Quarterly 22.1 (2006): 5–25. Web.
Milner, Andrea R. et al. “Elementary Teachers’ Beliefs About Teaching Science and Classroom
Practice: An Examination of Pre/Post NCLB Testing in Science.” Journal of Science
Teacher Education 23.2 (2012): 111–132. Web.
Oliver, Joseph S. “A Longitudinal Study of Attitude, Motivation and Self Concept as Predictors
of Achievement in and Commitment to Science among Adolescent Students.” Dissertation
Abstracts International 47.8-A (1987): 2983. Print.
Oliver, Steve J, and Ronald D Simpson. “Influence of Attitude toward Science, Achievement
Motivation, and Science Self Concept on Achievement in Science: A Longitudinal Study.”
Science Education 72.2 (1988): 143–155. Web.
108
Ornstein, Avi. “The Frequency of Hands-on Experimentation and Student Attitudes toward
Science: A Statistically Significant Relation.” Journal of Science Education and
Technology 15.3/4 (2006): 285–297. Web.
Papanastasiou, Elena C, and Michalinos Zembylas. “The Effect of Attitudes on Science
Achievement: A Study Conducted Among High School Pupils in Cyprus.” International
review of education 48.6 (2002): 469–484. Print.
Pigott, H E, J W Fantuzzo, and P W Clement. “The Effects of Reciprocal Peer Tutoring and
Group Contingencies on the Academic Performance of Elementary School Children.”
Journal of applied behavior analysis 19.1 (1986): 93–8. Web.
Raaijmakers, Quinten, Rutger Engels, and Anne Van Hoof. “Delinquency and Moral Reasoning
in Adolescence and Young Adulthood.” International Journal of Behavioral Development
29 (2005): 247–258. Web.
Reynolds, Arthur J, and Herbert J Walberg. “A Structural Model of Science Achievement and
Attitude: An Extension to High School.” Journal of Educational Psychology 84.3 (1992):
371–382. Print.
Reynolds, Scott J, and Tara L Ceranic. “The Effects of Moral Judgment and Moral Identity on
Moral Behavior: An Empirical Examination of the Moral Individual.” The Journal of
applied psychology 92.6 (2007): 1610–1624. Web.
Robinson, Debbie R., Janet Ward Schofield, and Katrina L. Steers-Wentzell. “Peer and Cross-
Age Tutoring in Math: Outcomes and Their Design Implications.” Educational Psychology
Review 17.4 (2005): 327–362. Web.
109
Rohrbeck, Cynthia a. et al. “Peer-Assisted Learning Interventions with Elementary School
Students: A Meta-Analytic Review.” Journal of Educational Psychology 95.2 (2003): 240–
257. Web.
Santos, Robert G et al. “Effectiveness of Peer-Based Healthy Living Lesson Plans on
Anthropometric Measures and Physical Activity in Elementary School Students: A Cluster
Randomized Trial.” JAMA pediatrics 168 (2014): 330–7. Web.
Satterthwait, Donna. “Why Are ‘Hands-on’ Science Activities so Effective for Student
Learning?” Teaching Science: The Journal of the Australian Science Teachers Association
56.2 (2010): 7–10. Web.
Slate, John R, and Craig H Jones. “Fourth and Fifth Grade Students’ Attitudes toward Science:
Science Motivation and Science Importance as a Function of Grade Level, Gender, and
Race.” Research in the Schools 5.1 (1998): 27–32. Print.
Smith, T. J., S. L. Pasero, and C. M. McKenna. “Gender Effects on Student Attitude Toward
Science.” Bulletin of Science, Technology & Society 34.1-2 (2014): 7–12. Web.
Sorge, Carmen. “The Relationship Between Bonding with Nonhuman Animals and Students’
Attitudes Toward Science.” Society & Animals 16 (2008): 171–184. Web.
Stohr-Hunt, Patricia M. “An Analysis of Frequency of Hands-on Experience and Science
Achievement.” Journal of Research in Science Teaching 33.1 (1996): 101–109. Web.
Tangney, June Price, Jeff Stuewig, and Debra J Mashek. “Moral Emotions and Moral Behavior.”
Annual review of psychology 58 (2007): 345–372. Web.
110
Tarry, Hammond, and Nicholas Emler^. “Attitudes, Values and Moral Reasoning as Predictors
of Delinquency.” British Journal of Developmental Psychology 25 (2007): 169–183. Web.
Teshome, Yalem, Nancy Maushak, and Krishna Athreya. “Attitude toward Informal Science and
Math: A Survey of Boys and Girls Participating in Hands-On Science and Math
(FUNTIVITIES).” Journal of Women and Minorities in Science and Engineering 7.1
(2001): 59–74. Web.
Trainin, Guy et al. “Extraordinary Animals and Expository Writing: Zoo in the Classroom.”
Journal of Science Education and Technology 14.3 (2005): 299–304. Web.
Warburton, S J, W L Jenkins, and P Coxhead. “‘Science Achievement and Attitudes’ and the
Age of Transfer to Secondary School.” Educational Research 25.3 (1983): 177–183. Print.
Willson, Victor L. “A Meta-Analysis of the Relationship between Science Achievement and
Science Attitude: Kindergarten through College.” Journal of Research in Science Teaching
20.9 (1983): 839–850. Print.
Wilson, Christopher D. et al. “The Relative Effects and Equity of Inquiry-Based and
Commonplace Science Teaching on Students’ Knowledge, Reasoning, and Argumentation.”
Journal of Research in Science Teaching 47.3 (2010): 276–301. Web.
Wilson, K. et al. “Our Zoo To You: The Link between Zoo Animals in the Classroom and
Science and Literacy Concepts in First-Grade Journal Writing.” Journal of Early Childhood
Literacy 11.3 (2011): 275–306. Web.
Zasloff, R L et al. “Animals in Elementary School Education in California.” Journal of Applied
111
Animal Welfare Science 2.4 (1999): 347–357. Web.