talking talons youth leadership evaluation report · talking talons youth leadership report school...

Leiden LLC

Talking Talons Youth Leadership Report

School Year 2016

Dr. Carmen Sorge

The printed document includes:

Summary of Results

Talking Talons Evaluation Report

Talking Talons Structural Equation Model Report

In addition to digital copies of the above, the CD includes:

Quiz responses

Interviews (audio and transcript)

Photographs

Contact: Dr. Carmen Sorge

[email protected]

[email protected]

mailto:[email protected]

mailto:[email protected]

1

1

Summary of results

A statistically significant change in Attitude toward Science was seen in the treatment

group and not for the control group from pretesting to post testing. F(1,74)=4.62. p<.05 η2=.06.

This represents a small effect size.

A statistically significant change in Knowledge was seen in the treatment group and not

for the control group from pretesting to post testing. F(1,74)=4.63. p<.05 η2=.45. This

represents a very large effect size.

A statistically significant change in self-reported anticipated grade in science was seen in

the treatment group and not for the control group from pretesting to post testing

F(1,71)=4.05. p<.05 η2=.06. This represents a small effect size.

The treatment group exhibited a statistically significant positive change in attitude toward

science as the program progressed. F(3,30). p<.01 η2=.37 (large effect size). This

statistic uses only the treatment group, as the control group does not take the Talking

Talon’s quizzes.

2

2

Classroom teacher feedback was extremely positive and both teachers indicated the

program increased student science knowledge and attitudes, that the buddy class was

worthwhile and also indicated a strong willingness to have the program again. Mean

scores for all subscales measuring teacher feedback were all above 6 on a 7 point scale (a

higher number indicates more positive feedback).

Qualitative feedback from students was overwhelmingly positive, mentioning that the

program was a positive experience, that they enjoyed presenting to the buddy class and

found it both rewarding and educational and far useful than their standard science

curriculum, one student summarized by stating “I think it’s uh great because it teaches us,

well real-life actual skills and things we’ll need to know in the future unlike X-Cal which

is teaching us that we will probably never use again in my life. “. They pointed out that

the buddy class consisted of “them looking up to us and them actually learning something

new from someone else, cuz them might not pick it up from like a teacher or someone,

but they can probably pick it up from us” and “they also got to experience what we

experienced with the animals” The students also were enthusiastic about both the hands

on and presentation sections of the program and found them enjoyable and educational .

The main suggestion for improvements by the students (which is consistent across every

interview) a was the inclusion of even more animals, especially ones they can touch.

Students actually requested “dangerous” animals. Students indicated that the speech

practice was helpful because it “I got nervous but it got me over my stage fright.” All

students interviewed emphatically affirmed that they would like to participate in Talking

Talons again, and would like to do so even at higher grade levels.

3

3

Structural equation model

This Structural Equation Model is a preliminary investigation of the relative importance of

the educator and the animals upon the science attitudes of the students. The full report is

provided separately.

A theoretical structural model (the Saturated Model) had a good fit to the data The model fit

for this SEM is indicates that high level of confidence in the model.

4

4

Attitude toward Educator had

A large positive direct effect on Attitude towards Animals

A medium positive indirect effect on Science Attitudes

Attitude toward Animals had

5

5

A large positive direct effect Science Attitudes

Upstream variables in the Pruned Model predicted

42% for Science Attitude

65% for Attitude toward Animals

Students who had more positive Attitudes toward Animals had large statistically

significant impact on Science Attitudes. Students who had more positive viewpoints of Attitude

toward Educator had a large direct effect and a medium indirect effect on Science Attitudes for a

total large statistically significant impact on Science Attitudes. The direct impact of the Attitude

toward Educator on Science Attitudes was not significant. Attitude towards Animals has the

largest direct impact on Science Attitudes, however the Attitude toward Animals is directly

impacted by the Attitude toward Educators. Educators are changing Science Attitudes by

changing the student’s Attitudes toward Animals. This is a complex outcome, indicating

that the animals are the crux of the change in science attitudes for the students, but that the

educators are indirectly influencing science attitudes by working through impact on

attitudes toward animals.

6

6

LEIDEN LLC

Talking Talons Youth Leadership Evaluation Report

And

Structural Equation Model for Talking Talons Science Attitudes

School Year 2016

Dr. Carmen Sorge

[email protected]

7

7

Table of Contents

Summary of results ......................................................................................................................... 1

Guide to basic interpretation ......................................................................................................... 14

Summary of results ....................................................................................................................... 14

Data Collection and Research ....................................................................................................... 19

Research Design ........................................................................................................................ 19

Composite instrument ............................................................................................................ 19

The Composite instrument scoring .................................................................................... 21

Reliability of the instrument .............................................................................................. 22

Quizzes .................................................................................................................................. 23

Reliability and Scale development..................................................................................... 24

Talking Talons Quiz Knowledge ....................................................................................... 25

Talking Talons Quiz Attitudes ........................................................................................... 26

Teacher Feedback Form ........................................................................................................ 28

Qualitative Data Collection ................................................................................................... 29

Participants ................................................................................................................................ 29

Explanation of GLM Repeated Measures ................................................................................. 30

Examples ............................................................................................................................ 31

Science Attitudes .......................................................................................................................... 36

Research on Science Attitudes .................................................................................................. 36

Science attitude change during program for participants .......................................................... 37

Science Attitude change from pretest to posttest by group ....................................................... 43

Student Qualitative Feedback on Science Attitudes ................................................................. 49

Classroom Teacher feedback about Science Attitudes ............................................................. 50

Hands on science experiments and Animals in the classroom ..................................................... 50

Research on Hands on Science and Animals in classroom ....................................................... 50

8

8

Classroom Teacher feedback .................................................................................................... 52

Student Feedback on Animals in classroom and Hands on science .......................................... 53

Research on peer tutoring .......................................................................................................... 54

Classroom teacher assessment of Buddy class .......................................................................... 56

Student feedback on Buddy class .............................................................................................. 57

Role Model for younger students .......................................................................................... 58

Science Knowledge ....................................................................................................................... 62

Change in Science Knowledge by Group ................................................................................. 62

Quiz Knowledge ........................................................................................................................ 66

Self-perceived science grade in school ..................................................................................... 69

Quiz Attitude subscales ............................................................................................................. 73

Feedback on Educator ................................................................................................................... 76

Student Quantitative Feedback.................................................................................................. 76

Student Qualitative Feedback on Educators.............................................................................. 80

Classroom teacher feedback for Educator ................................................................................. 81

Classroom teacher feedback for Talking Talons Program ............................................................ 83

Teacher Attitude ........................................................................................................................ 83

Effectiveness by student ability: Feedback by Classroom teachers ........................................ 85

Classroom teacher subscales ..................................................................................................... 87

Other Results ................................................................................................................................. 89

Changes in pre posttest Composite ............................................................................................... 89

Structural Equation Model ............................................................................................................ 91

Summary ....................................................................................................................................... 94

References ..................................................................................................................................... 97

SEM

What is Structural equation modeling (SEM)? ....................................................................... 107

Study Design .............................................................................................................................. 110

Purpose of the Study ............................................................................................................... 110

Research Questions ................................................................................................................. 110

9

9

Limitations of the Study .......................................................................................................... 110

Definition of Constructs .......................................................................................................... 111

Attitude Theory ....................................................................................................................... 111

Method ....................................................................................................................................... 113

Analysis Plan ........................................................................................................................... 113

Analysis Sequence................................................................................................................... 113

Fit Indices ................................................................................................................................ 114

Models ........................................................................................................................................ 116

Measurement Model ................................................................................................................ 117

Examination of Latent Variables in the Measurement Model ......................................... 119

Impact of Outliers on the Measurement Model ................................................................... 119

Final Measurement Model ............................................................................................... 120

Saturated Model ...................................................................................................................... 121

Pruned Model .......................................................................................................................... 126

Comparison of Model Fit ........................................................................................................ 130

Direct and Indirect Effects ...................................................................................................... 132

Summary of Outcomes ............................................................................................................ 132

Variance of the Latent Variables ......................................................................................... 133

Global Summary ....................................................................................................................... 133

Bibliography .............................................................................................................................. 135

10

10

Tables

Table 1 Pretest and Posttest Cronbach’s alpha for subscales ....................................................... 22

Table 2 Participants by Group ...................................................................................................... 29

Table 3 Participants by Gender .................................................................................................... 29

Table 4 Science Attitude Change over Time ................................................................................ 39

Table 5 Science Attitude Change over Time Multivariate Tests .................................................. 39

Table 6 Science Attitude Change over Time within Subjects Effects .......................................... 40

Table 7 Science Attitude Change over Time Within subjects Contrasts ...................................... 42

Table 8 Science Attitude Change over Time between Subject Effects ........................................ 43

Table 9 Change in Science Attitude from Pretesting to Posttesting by Group ............................. 43

Table 10 Change in Science Attitude by Group Multivariate Tests ............................................. 44

Table 11 Change in Science Attitude by Group Within subjects Effects ..................................... 44

Table 12 Change in Science Attitude by Group within Subjects Contrasts ................................. 46

Table 13 Change in Science Attitude by Group between Subject Effects .................................... 46

Table 14 Change in Science Attitude by Group Estimated Means ............................................... 47

Table 15 Difference Change in Science Attitude by Group ........................................................ 47

Table 16 Attitude toward Science from Quizzes Mean ............................................................... 48

Table 17 Classroom Teacher assessment of Science Attitude change ......................................... 50

Table 18 Classroom teacher assessment of program components ................................................ 52

Table 19 Classroom teacher assessment of buddy class impact .................................................. 57

Table 20 Student answers by group to "I am a good role model for younger students" .............. 59

Table 21 I am a good role model significance tests ...................................................................... 60

11

11

Table 22 Change in Science Knowledge by Group ...................................................................... 63

Table 23 Change in Science Knowledge by Group Multivariate tests ......................................... 64

Table 24 Change in Science Knowledge by Group Means ......................................................... 65

Table 25 Mean scores on Knowledge Quiz .................................................................................. 68

Table 26 Group Comparison for I will get a good grade in science class this year ...................... 70

Table 27 Group Comparison for I will get a good grade in science class this year ...................... 71

Table 28 Quiz Attitude subscale descriptive statistics ................................................................. 75

Table 29 Student feedback on Educator Subscales ....................................................................... 77

Table 30 Student feedback on Educator from Quiz results .......................................................... 79

Table 32 Classroom Teacher Feedback on Talking Talons Educator .......................................... 82

Table 33 Classroom Teacher Feedback Personal ......................................................................... 84

Table 34 Classroom teacher feedback program effectiveness by student ability ......................... 86

Table 35: Classroom teacher subscales for Talking Talons program ........................................... 88

Table 36 Statistically significant Pre Post change by group ........................................................ 89

Table 37 ANOVA Statistically significant Pre Post change by group ......................................... 90

Table 38 Effect Size Statistically significant Pre Post change by group ...................................... 91

SEM

Table 1: Fit Indices ..................................................................................................................... 115

Table 2: Factor Path Loadings for Latents .................................................................................. 119

Table 3: Measurement Model Unstandardized Results** .......................................................... 120

Table 4: Correlations Between Latent Variables in the Measurement Model ............................ 121

Table 5: Measurement Model Covariances** ............................................................................ 121

Table 6: Regression Weights for Saturated Model** ................................................................. 122

Table 7: Squared Multiple Correlations for Saturated Model .................................................... 123

12

12

Table 8: Saturated Model Total Effects Standardized ............................................................... 124

Table 9:Saturated Model Direct Effects ..................................................................................... 124

Table 10: Saturated Model Indirect Effects ................................................................................ 125

Table 11: Fit Indices for Saturated Model .................................................................................. 125

Table 12: Unstandardized Regression Weights for Pruned Model** ........................................ 128

Table 13: Squared Multiple Correlations for Pruned Model ...................................................... 128

Table 14: Total Effects for Pruned Model .................................................................................. 128

Table 15: Direct Effects for Pruned Model................................................................................. 129

Table 16: Indirect Effects for Pruned Model .............................................................................. 129

Table 17: Fit Indices for Pruned Model ...................................................................................... 130

Table 18: Comparison of the Fit of the Models .......................................................................... 130

13

13

Figures

Figure 1 Longitudinal change in Science Attitude 2015 .............................................................. 37

Figure 2 Longitudinal change in Science Attitude 2016 .............................................................. 38

Figure 3 Change in Science Knowledge by Group ....................................................................... 62

Figure 4 Quiz Knowledge section means 2015 ............................................................................ 66

Figure 5 Quiz Knowledge means 2016 ......................................................................................... 67

Figure 6 Self perceived change in science grade by group ........................................................... 70

Figure 7 Quiz Attitude subscales .................................................................................................. 73

Figure 8 Student Perception of Educator ...................................................................................... 76

Figure 9 Student Perception of Educator ...................................................................................... 78

Figure 11 Classroom teacher feedback on Talking Talons Educator ........................................... 81

Figure 12 Classroom Teacher subscales on Educator ................................................................... 82

Figure 13 Classroom teacher feedback on Talking Talons program ............................................ 83

Figure 14 Classroom teacher assessment of Effectiveness of program by student ability ........... 85

Figure 15 Classroom teacher perception ....................................................................................... 87

Figure 16 Statistically significant Pre Post change by group ....................................................... 89

Figure 17: Final pruned model of impact of student attitude toward educator and animals on

science attitudes ............................................................................................................................ 91

SEM

Figure 1: Measurement Model .................................................................................................... 118

Figure 2: Saturated Model .......................................................................................................... 122

Figure 3: Direct and Indirect Effects .......................................................................................... 124

14

14

Guide to basic interpretation

The p value tells the researcher the probability that this data would be observed simply by

chance. A p value of .05 (5 percent chance that the difference was just due to random

fluctuation) is the statistical standard for the maximum while a p value of .01 (less than 1 percent

chance the difference is due to random fluctuation) is considered more conservative and thus a

better indication of actual results.

The effect size (η2) examines the magnitude of change or how MUCH difference

occurred. The standard in use for repeated measures is .02 is a small effect size, .13 is a medium

effect size and .26 is a large effect (Bakeman).

Summary of results






represents a very large effect size.

15

15

A statistically significant change in self-reported anticipated grade in science was seen in

the treatment group and not for the control group from pretesting to post testing


The treatment group exhibited a statistically significant positive change in attitude toward

science as the program progressed. F(3,30). p<.01 η2=.37 (large effect size). This

statistic uses only the treatment group, as the control group does not take the Talking

Talon’s quizzes.

Classroom teacher feedback was extremely positive and both teachers indicated the

program increased student science knowledge and attitudes, that the buddy class was

worthwhile and also indicated a strong willingness to have the program again. Mean

scores for all subscales measuring teacher feedback were all above 6 on a 7 point scale (a

higher number indicates more positive feedback).

Qualitative feedback from students was overwhelmingly positive, mentioning that the

program was a positive experience, that they enjoyed presenting to the buddy class and

found it both rewarding and educational and far useful than their standard science

curriculum, one student summarized by stating “I think it’s uh great because it teaches us,

well real-life actual skills and things we’ll need to know in the future unlike X-Cal which

is teaching us that we will probably never use again in my life. “. They pointed out that

the buddy class consisted of “them looking up to us and them actually learning something

16

16

new from someone else, cuz them might not pick it up from like a teacher or someone,

but they can probably pick it up from us” and “they also got to experience what we

experienced with the animals” The students also were enthusiastic about both the hands

on and presentation sections of the program and found them enjoyable and educational .

The main suggestion for improvements by the students (which is consistent across every

interview) a was the inclusion of even more animals, especially ones they can touch.

Students actually requested “dangerous” animals. Students indicated that the speech

practice was helpful because it “I got nervous but it got me over my stage fright.” All

students interviewed emphatically affirmed that they would like to participate in Talking

Talons again, and would like to do so even at higher grade levels.

Structural equation model




A theoretical structural model (the Saturated Model) had a good fit to the data The model fit






17

17
















18

18

19

19

Data Collection and Research

The Talking Talons program evaluation data collection consists of:

A control/treatment pretest and posttest called the Talking Talons Composite

A series of 10 quizzes measuring knowledge and attitude change for the

participants

Online teacher feedback

Qualitative interviews of a sample of the Talking Talons participants

Research Design

Composite instrument

Talking Talons program is evaluated using a pretest and posttest quasi- experimental

design with control and treatment groups for the composite instrument. As the program is

offered by classroom, true random assignment is not feasible. However, the control groups are

selected from the same school using teachers who are not currently receiving the program.

Although ideally a Solomon design would be used, the effort and expense involved in collecting

four sets of data points is prohibitive.

The use of a control group is imperative. Without control groups any changes cannot be

attributed to the program. Factors such as maturation, the effect of testing and other outside

factors cannot be eliminated as agents of change without the use of a control group.

20

20

Improvements seen in the treatment group therefore cannot be ascribed to the program unless a

control group is utilized.

For the composite instrument a General Linear Model Repeated measures utilizing SPSS

18 was run. The GLM repeated measures improves on the multivariate regression model by

allowing for linear transformations or linear combinations of multiple variables. This expansion

means that the GLM has important advantages over the multiple and the purported multivariate

regression models which are inherently univariate methods. The first advantage is that

multivariate test of significance may be use if the responses on multiple dependent variables are

correlated. This is helpful as separate univariate tests of significance for correlated dependent

variables (such as used in multiple t tests) are not independent and may not be appropriate.

Multivariate tests of significance of independent linear combinations of multiple dependent

variables may also yield information about which response variables are, and are not, actually

related to the predictor variables. A second advantage is the ability to analyze effects of repeated

measure factors. Linear combinations of responses reflecting a repeated measure effect can be

constructed and tested for significance using either the univariate or multivariate approach to

analyzing repeated measures in the general linear model. In this research, with pretest and

posttest measurements of control and treatment groups the GLM repeated measures is the most

appropriate test.

Effect sizes based on recent research on repeated measure designs based on Cohen (1988,

pp. 413–414), who did not consider repeated measures designs explicitly, defined an η2 of .02 as

small, one of .13 as medium, and one of .26 as large. “It seems appropriate

21

21

to apply the same guidelines to η2 as well.” (Bakeman)

The Composite instrument scoring

All composite questions are scored on a Likert scale from 1 to 6 ranging from Very False

to Very True. All questions and scales are renormed so that a higher value indicates a more

desirable outcome. Therefore a value of 5 indicates better self-control or less violent tendencies

than a score of 3. For all subscales a higher score is more desirable.

The Locus of Control measure reports a student’s self-reported ability to direct and

manage their own behavior. Ten questions addressing these issues such as “I do things I know

are wrong because my friends are doing them.” and “I am easily distracted.” are included.

The School Attitude scale consists of five self-reported feelings about school such as

“The work I do in school is important to me” and “I would skip school a lot if I knew that I

would not get caught”.

The Self-esteem attitude scale consists of twelve self-assessments of the student’s self-

value and perception of ability such as “I am proud of myself.” And “If I want to learn to do

something new I usually can learn it.”

The Attitude toward Violence scale consists of five self-reported actions or intentions of

violence such as “I fight a lot.” and “If people will not do what you want it is okay to threaten

that you will hurt them.” Questions 9, 11, 25, 42 and 20 from the composite instrument are

included in this subscale.

22

22

The Moral Judgment scale consists of eleven questions evaluating student attitudes

toward various ethical and unethical behaviors such as “It is okay to cheat on schoolwork if you

do not get caught.” And “It is wrong to lie in order to get what you want.”

Seventeen questions for the treatment group only are asked of the treatment

group. These questions address the participant’s evaluation of the program and the educator and

include questions such as “I looked forward to the days that Talking Talons came to my

classroom” and “My Talking Talons educator was easy to understand”. A full report of these

questions is included in the educator report.

Reliability of the instrument

Cronbach’s alpha was calculated for each subscale using the overall sample. Reliability

greater than .7 is considered good (Nunnaly, 1978). All subscales of the instrument have good to

excellent reliability. The 2015 and 2016 evaluations have a much smaller sample size than

previous years, which does impact reliability. School Attitude reliability fell below the

reasonable interpretation threshold and thus any significant results are disregarded.

Table 1 Pretest and Posttest Cronbach’s alpha for subscales

2003

-

2004

2003

-

2004

2004

-

2005

2004

-

2005

2005

-

2006

2005

-

2006

2006

-

2007

2006

-

2007

2015*

2015

2016 *

2016

Pre Post Pre Post Pre Post Pre Post Pre Post Pre Post

Locus of

Control

.797 .807 .78 .83 .79 .81 .81 .82 .78 .76 .80 .73

23

23

2003-

2004

2003-

2004

2004-

2005

2004-

2005

2005-

2006

2005-

2006

2006-

2007

2006-

2007

2015*

2015

2016 *

2016

School

Attitude

.750 .779 .75 .75 .79 .78 .71 .74 .65 .52 .67 .78

Self

Esteem

.666 .798 .81 .84 .83 .86 .82 .86 .91 .87 .89 .88

Attitude

toward

Violence

.717 .747 .73 .73 .80 .85 .77 .83 .77 .71 .78 .81

Moral

Scale

.776 .799 .83 .84 .86 .86 .80 .86 .70 .87 .81 .82

Science

Attitude

Scale

.957 .965 .95 .97 .96 .97 .94 .95 .90 .94 .95 .96

Talking

Talons

Scale

.680 .782 .76 .80 .71 .89 .77 .76 .77 .71 .71

Posttest

only

Treatmen

t section

.94 .92 .76 .73 .73

*note: Sample was much smaller than previous years, this tends to reduce reliability

Quizzes

The treatment group students are also given a series of ten quizzes. These quizzes consist

of five knowledge questions and five attitude questions. The knowledge questions pertain to the

information taught by the program and the attitude questions cover attitudes towards the

program, the educator and science in general. Some science attitude questions are repeated

24

24

throughout the attitude section of the quizzes over the course of the program in order to measure

longitudinal changes in attitude scores.

These quizzes are used to determine both the changes in science knowledge and in attitudes

over the course of the program.

Reliability and Scale development

The quizzes were used to generate several subscales. For each subsection the internal

reliability was examined. Internal reliability is how much consistency is apparent in the answers

of one subtest. Internal consistency is measured with a scale called “Cronbach's Alpha”.

Technically speaking, Cronbach's alpha is not a statistical test - it is a coefficient of reliability (or

consistency). Alpha is calculated by one split-half reliability and then randomly dividing the

items into another set of split halves and recomputing until, all possible split half estimates of

reliability have been computed. In other words, each item is compared to the group in all

possible combinations of items. Cronbach's alpha measures how well a set of items (or variables)

measures a single unidimensional latent construct. When data have a multidimensional structure

(they are not all measuring the same construct) then Cronbach's alpha will usually be low. If the

number of items is increased and the consistency remains the same the Cronbach’s alpha will

increase. Additionally, if the average inter-item correlation is low, alpha will be low. As the

average inter-item correlation increases, Cronbach's alpha will also increase. This effect makes

sense intuitively - if the inter-item correlations are high, and then there is evidence that the items

are measuring the same underlying construct.

25

25

Recall that an alpha value of .70 is considered acceptable; however we would like it to be

higher. Reliabilities are presented in each section of the report.

Talking Talons Quiz Knowledge

All of the scales are mean scores. Thus if an educator did not give one of the quizzes

containing a question for that subset it is not entered into the mean. This can impact the results

because the score is then based on fewer data points. All attitude scales are recoded so that 6

represents the highest and best score and 1 is the lowest and worst score. For the quizzes 5 is the

highest possible score and 0 is the lowest. Remember that this is a very simple way at looking at

scores. With simple means one participant who has a negative change of 6 point will mask the

positive change of 1 point for 6 participants. This is especially true for small sample sizes (the

smaller classrooms). An educators with a score of 5.5 on the scale could have had half the kids

answer “6” and half answer “5” OR almost all kids answering 6 with a few answering “1”. Keep

this in mind when interpreting results.

Bar charts are good visual presentations of information. However, they may lead to over

interpretation. As the standard deviations are not listed on a bar chart it is difficult to ascertain

true significant differences. One must ask “Does that “3” on a bar chart mean that almost

everyone answered 3 or that half the participants answered “5” and half answered “2”? “

Significance tests indicate actually differences by utilizing the standard deviation as well.

26

26

Therefore two bars on a chart may look very different and not be statistically different or may

look similar but actually represent a statistical difference.

The knowledge sections relate to the topics studied in the program and includes the following

topics Introduction, Hawks, Raptors, Bats 1, Bats 2, Owls, Reptiles,Vocabulary General and

Energy. Five questions related to these topics are asked on each of the quizzes.

Talking Talons Quiz Attitudes

The back of each quiz includes an attitude section; these questions are then utilized for the

following subscales.

Overall perception of educator (Reliability α=.90 (2015) α=.86 (2016))

My Talking Talons educator (teacher) is easy to understand.

My Talking Talons educator (teacher) understands how kids think.

My Talking Talons educator (teacher) knows a lot about the animals.

My Talking Talons educator (teacher) speaks clearly and loudly enough for me to hear

and understand.

My Talking Talons educator (teacher) pays attention to all of the kids in my class.

My Talking Talons educator (teacher) treats me kindly.

My Talking Talons educator (teacher) is prepared to teach us when he/she comes to class.

27

27

My Talking Talons educator (teacher) was fair to everyone in my class.

Bonding with Animals (Reliability α=.68 (2015) α=.67 (2016)))

In the future, I would like to be a Talking Talons Educator and hold a bird.

I would like the chance to hold a Talking Talons reptile (like a snake or lizard).

I think about the Talking Talons animals when I am not in science class

I feel comfortable around the Talking Talons animals (eliminated due to small sample

size)

Seeing the Talking Talons animals makes me feel happy.

Perceived Enjoyment of Talking Talons program (Reliability α=.78(2015) α=.78 (2016)))

I like learning about animals.

I look forward to the days that Talking Talons comes to the classroom

I talk to kids who are not in the Talking Talons program about the program.

I would like the Talking Talons program to come to my science class next year.

Perceived Ability (Reliability α=.72 (2015) α=.62 (2016)))

The quizzes for Talking Talons were very hard.

I try my hardest to do well in school.

I am good at science.

28

28

I think I will get a better grade in science class this year than last year.

I understood most of the information in the Talking Talons program.

Attitude toward science (Reliability α=.82 (2015) α=.90 (2016)))

I like science class.

Science class is boring

I like science class.

Attitude toward environment (Reliability α=.68(2015) α=.62 (2016))

It is ok to shoot hawks because they kill smaller birds.

It is ok to throw trash out the window because the highway department picks it up.

I think teaching other kids about the environment is important

Teacher Feedback Form

At the end of the program an online form (to preserve teacher anonymity) is used to

collect data on the classroom teachers’ perceptions of the program, of changes in attitude of

his/her students and of the educator.

29

29

Qualitative Data Collection

A sample of five to six students from each group was interviewed by the evaluator for

feedback about the Talking Talons program. These interviews were recorded and transcribed.

Student names were removed from the transcription to preserve anonymity.

Participants

Participant demographic information is included below. For each subscale the total

number of student varies slightly due to outliers and participants who did not finish a

particular subscale. The total number can be found in the table of means for each subsection.

Because of the small sample size for the 2015 and 2016 program, missing posttest data for

the control group was imputed from representative previously collected control test data in

order to reach a statistically useable sample size. Students were all between 10 and 11 years

old at post testing.

Table 2 Participants by Group

Frequency Percent Valid Percent Cumulative Percent

Valid Control 40 44.0 44.0 44.0

Treatment 51 56.0 56.0 100.0

Total 91 100.0 100.0

Table 3 Participants by Gender

group Total

30

30

Control Treatment

gender Female 20 26 46

Male 20 25 45

Total 40 51 91

Explanation of GLM Repeated Measures

The data for the composite file were analyzed using SPSS General Linear Model

Repeated Measures (GLM). The GLM Repeated Measures procedure is based on the general

linear model, in which factors and covariates are assumed to have linear relationships to the

dependent variables.

This section explains how to interpret the graphs and data from the analysis of the

composite section. For simplicity of explanation sample graphs and data are used which make

the differences clear for interpretation.

A GLM repeated measures with control groups is used to examine the following possibilities:

QUESTION: Are the control and treatment groups similar to each other?

VISUAL ANSWER: Are the two lines far apart (be sure to look at the scale of the graph)

from each other, especially at pretesting?

STATISTICAL ANSWER: If the two groups are not similar to each other than the test

of between groups will be significant.

31

31

QUESTION: Does the score on the instrument change over time from pretesting to post testing

for one or both of the groups?

VISUAL ANSWER: Is the line flat or sloped for the groups?

STATISTICAL ANSWER: If the scores are changing overall in the same direction and

magnitude then the “Time” variable will be significant.

QUESTION: Is the change over time different for the two groups? (Either one group goes up

and one goes down or the magnitude of the change is different.)

VISUAL ANSWER: Is the slope of the line different (in direction or magnitude) for

each group?

STATISTICAL ANSWER: If the scores are changing, but differently for each group

then the “Time by Group” variable will significant overall. The difference can be either

in different directions or in different magnitudes.

These questions are answered statistically in the tables. However, for ease of interpretation the

statistical significance is listed under the graph showing change.

Examples

Groups are different but with no change over time

32

32

The first question is answered statistically in a table called “Between groups’ differences”

but can also be seen by the distance between the score values of the control and treatment

groups. Note the scale on the graph as well when interpreting the data.

The first graph shows the following

No change over time for either group (the slope of the lines are flat)

Groups are different from each other (there is a big gap between the Treatment and

Control Group lines)

No Change over Time

Groups are different

TIME

21

Estim

ate

d M

arg

ina

l M

ea

ns

4.5

4.0

3.5

3.0

2.5

2.0

1.5

1.0

.5

GROUP

Treatm ent

Control

33

33

Groups are similar and both change over time in the same way

The second graph shows the following

both change from pretesting to post testing (the slope of the line is not flat)

Both change in the same manner (the slope is in the same direction and at the same

angle).

Change over Time

TIME

21

Estim

ate

d M

arg

ina

l M

ea

ns

7.0

6.5

6.0

5.5

5.0

4.5

4.0

3.5

GROUP

Treatm ent

Control

Groups are the same but change differently over time

34

34

The third graph shows

change over time for both groups

Change is not in the same direction.

Different Changes Over Time

TIME

21

Estim

ate

d M

arg

ina

l M

ea

ns

6

5

4

3

2

1

0

GROUP

Treatm ent

Control

Obviously with real data the differences are not always so clear. However, this

explanation should make visual interpretation of the results simpler. Statistical results are

presented in the tables below the charts for those who are interested in the means, levels of

significance, effect size and power for the results. Type III Sums of squares were used as the

35

35

hypotheses being tested involve only marginal averages of population cell means. Greenhouse-

Geisser epsilon was used to adjust degrees of freedom if sphericity was violated as per

assessment with Mauchly's test. These statistics are available from the researcher if desired.

Estimated marginal means of the dependent variables (with covariates held at their mean value)

for specified between- or within-subjects factors in the model are provided. In this research these

predicted means are equivalent to observed means as the covariates are categorical.

Outlying scores due to extreme responses may influence results. In order to identify

outliers the score distributions for the participants for all observed variables were examined

univariately. Scores that were more than three standard deviations from cell means and were

also discontinuous from their closest neighboring scores were considered univariate outliers and

were removed on a subscale by subscale basis.

For each subsection the results were examined by GLM repeated measures using the

following group memberships.

Overall

By gender

The results are then presented for results that yielded significant or interesting result for

the difference between the groups if there was no difference then overall results are presented

unless a trend that merits consideration was present. Recall that as the group is subdivided the

membership within that group becomes smaller and significance is more difficult to detect.

36

36

Science Attitudes

Research on Science Attitudes

Attitudes toward science have long been examined as an important factor in student

science education. Researchers have found that attitudes toward science impact future course

selection (Farenga and Joyce), motivation to study science (Slate and Jones), achievement in

science (Oliver and Simpson; Kose et al.; Smith, Pasero, and McKenna; Baker; Oliver;

Reynolds and Walberg; Warburton, Jenkins, and Coxhead; Willson) and general attitudes toward

school (Jarvis and Pell) ). Improving student’s attitudes toward science could therefore be said

to have many positive outcomes.

However as students enter the middle school years, their generally positive attitudes

about science decrease (Mattern and Schau; Desy, Peterson, and Brockman; Catsambis;

Mattern; Sorge) especially for girls (Backes; Lee and Burkam; Oliver; Papanastasiou and

Zembylas; Warburton, Jenkins, and Coxhead) .

Of particular relevance is research examining changes in attitudes exhibited after

exposure to “hands on” or inquiry based science. These programs have been found to improve

attitudes toward science with increased frequency of hands on experiments leading to most

positive attitudes (Ornstein) and with hands on laboratory experiment increasing knowledge

(Freedman). Interestingly, girls were found to be impacted more (Teshome, Maushak, and

Athreya) by hands on experimentation than boys.

37

37

Specific research on involving students in environmental science projects in order to

increase both knowledge and attitudes toward science has statistically positive outcomes on both

factors (Al-Balushi and Al-Aamri).

Science attitudes for the Talking Talons students were examined in multiple forms.

Pretest and posttest attitudes were compared for control and treatment groups. For the treatment

groups the same question about attitudes towards science was asked throughout the program in

order to measure longitudinal change. Information was collected from the classroom teacher and

from interviews.

Science attitude change during program for participants

In order to examine longitudinal changes in science attitude for the treatment group

(those students who participated in the Talking Talons program), science attitude questions were

asked at pretesting, on quiz 3, on quiz 7 and at posttesting. The chart below delineates the

change over time. This change was statistically significant (see table for results)

Figure 1 Longitudinal change in Science Attitude 2015

38

38

Figure 2 Longitudinal change in Science Attitude 2016

39

39

Table 4 Science Attitude Change over Time

Descriptive Statistics

Mean Std. Deviation N

Science is a topic which I enjoy studying. 4.818 1.1580 33

I like science class. (Quiz 3) 5.36 .822 33

I like science class. (Quiz 7) 5.39 .864 33

POST Science is fun 5.52 .667 33

Table 5 Science Attitude Change over Time Multivariate Tests

40

40

Multivariate Testsa E

ffec

t

Val

ue

F

Hypoth

esis

df

Err

or

df

Sig

.

Par

tial

Eta

Squar

ed

Nonce

nt.

Par

amet

er

Obse

rved

Pow

erc

Attitude

change

Pillai's Trace .372 5.924b 3.000 30.00 .003 .372 17.773 .927

Wilks'

Lambda

.628 5.924b 3.000 30.00 .003 .372 17.773 .927

Hotelling's

Trace

.592 5.924b 3.000 30.00 .003 .372 17.773 .927

Roy's Largest

Root

.592 5.924b 3.000 30.0 .003 .372 17.773 .927

Table 6 Science Attitude Change over Time within Subjects Effects

Tests of Within-Subjects Effects

41

41

Source

Typ

e II

I S

um

of

Sq

uare

s

df

Mea

n S

qu

are

F

Sig

.

Part

ial

Eta

Sq

uare

d

Non

cen

t. P

ara

met

er

Ob

serv

ed P

ow

era

Attitude

Change

Sphericity

Assumed

9.515 3 3.17

2

8.70

3

.000 .214 26.11

0

.993

Greenhouse

-Geisser

9.515 2.465 3.86

0

8.70

3

.000 .214 21.45

3

.983

Huynh-

Feldt

9.515 2.686 3.54

2

8.70

3

.000 .214 23.38

0

.989

Lower-

bound

9.515 1.000 9.51

5

8.70

3

.006 .214 8.703 .816

Error(Attitud

e Change)

Sphericity

Assumed

34.98

5

96 .364

Greenhouse

-Geisser

34.98

5

78.87

7

.444

Huynh-

Feldt

34.98

5

85.96

4

.407

Lower-

bound

34.98

5

32.00

0

1.09

3

42

42

a. Computed using alpha = .05

Table 7 Science Attitude Change over Time Within subjects Contrasts

Tests of Within-Subjects Contrasts

Measure: MEASURE_1

Sourc

e

Att

itude

Chan

ge

Type

III

Sum

of

Squar

es

df

Mea

n S

quar

e

F

Sig

.

Par

tial

Eta

Squar

ed

Nonce

nt.

Par

amet

er

Obse

rved

Pow

era

Attitude

Change

Linear 7.424 1 7.42

4

16.18

8

.00

0

.33

6

16.18

8

.97

4

Quadrati

c

1.485 1 1.48

5

4.314 .04

6

.11

9

4.314 .52

2

Cubic .606 1 .606 2.087 .15

8

.06

1

2.087 .28

9

Error(Attitud

e Change)

Linear 14.67

6

3

2

.459

Quadrati

c

11.01

5

3

2

.344

Cubic 9.294 3

2

.290


43

43

Table 8 Science Attitude Change over Time between Subject Effects

Tests of Between-Subjects Effects

Measure: MEASURE_1

Transformed Variable: Average

Source

Type

III

Sum

of

Squar

es

df

Mea

n S

quar

e

F

Sig

.

Par

tial

Eta

Squar

ed

Nonce

nt.

Par

amet

er

Obse

rved

Pow

era

Intercept 3669.818 1 3669.818 1735.092 .000 .982 1735.092 1.000

Error 67.682 32 2.115


Science Attitude change from pretest to posttest by group

Both the treatment and control group were assessed for science attitude changes from

pretesting to postttesting. A statistically significant change in Attitude toward Science was seen

in the treatment group and not for the control group from pretesting to post testing. F(1,74)=4.62.

p<.05 η2=.06. This represents a small effect size.

Table 9 Change in Science Attitude from Pretesting to Posttesting by Group

44

44


group Mean Std. Deviation N

sci Control 4.3137 1.24026 38

Treatment 5.0816 .90460 38

Total 4.6977 1.14539 76

psci Control 4.0690 1.32331 38

Treatment 5.2842 .70731 38

Total 4.6766 1.21853 76

Table 10 Change in Science Attitude by Group Multivariate Tests

Multivariate Testsa

Effect

Val

ue

F

Hypoth

e

sis

df

Err

or

df

Sig

.

Par

tial

Eta

Squar

ed

Nonce

nt.

Par

amet

e

r Obse

rve

d P

ow

erc

science Pillai's Trace .001 .041b 1.000 74.000 .840 .001 .041 .055

Wilks' Lambda .999 .041b 1.000 74.000 .840 .001 .041 .055

Hotelling's Trace .001 .041b 1.000 74.000 .840 .001 .041 .055

Roy's Largest

Root

.001 .041b 1.000 74.000 .840 .001 .041 .055

science *

group

Pillai's Trace .059 4.623b 1.000 74.000 .035 .059 4.623 .564

Wilks' Lambda .941 4.623b 1.000 74.000 .035 .059 4.623 .564

Hotelling's Trace .062 4.623b 1.000 74.000 .035 .059 4.623 .564

Roy's Largest

Root

.062 4.623b 1.000 74.000 .035 .059 4.623 .564

a. Design: Intercept + group

Within Subjects Design: science

b. Exact statistic

c. Computed using alpha = .05

Table 11 Change in Science Attitude by Group Within subjects Effects

45

45


Measure: MEASURE_1

Sourc

e

Type

III

Sum

of

Squar

es

df

Mea

n S

quar

e

F

Sig

.

Par

tial

Eta

Squar

ed

Nonce

nt.

Par

amet

er

Obse

rved

Pow

era

science Sphericity

Assumed

.017 1 .017 .041 .84

0

.00

1

.041 .05

5

Greenhous

e-Geisser

.017 1.000 .017 .041 .84

0

.00

1

.041 .05

5

Huynh-

Feldt

.017 1.000 .017 .041 .84

0

.00

1

.041 .05

5

Lower-

bound

.017 1.000 .017 .041 .84

0

.00

1

.041 .05

5

science *

group

Sphericity

Assumed

1.901 1 1.90

1

4.62

3

.03

5

.05

9

4.62

3

.56

4

Greenhous

e-Geisser

1.901 1.000 1.90

1

4.62

3

.03

5

.05

9

4.62

3

.56

4

Huynh-

Feldt

1.901 1.000 1.90

1

4.62

3

.03

5

.05

9

4.62

3

.56

4

Lower-

bound

1.901 1.000 1.90

1

4.62

3

.03

5

.05

9

4.62

3

.56

4

Error(scienc

e)

Sphericity

Assumed

30.43

6

74 .411

Greenhous

e-Geisser

30.43

6

74.00

0

.411

Huynh-

Feldt

30.43

6

74.00

0

.411

Lower-

bound

30.43

6

74.00

0

.411


46

46

Table 12 Change in Science Attitude by Group within Subjects Contrasts


Measure: MEASURE_1

Sourc

e

scie

nce

Type

III

Sum

of

Squar

es

df

Mea

n S

quar

e

F

Sig

.

Par

tial

Eta

Squar

ed

Nonce

nt.

Par

amet

er

Obse

rved

Pow

era

science Linear .017 1 .017 .041 .840 .001 .041 .055

science * group Linear 1.901 1 1.901 4.623 .035 .059 4.623 .564

Error(science) Linear 30.436 74 .411


Table 13 Change in Science Attitude by Group between Subject Effects


Measure: MEASURE_1


47

47

Sourc

e

Type

III

Sum

of

Squar

es

df

Mea

n S

quar

e

F

Sig

.

Par

tial

Eta

Squar

ed

Nonce

nt.

Par

amet

er

Obse

rved

Pow

era

Intercept 3339.323 1 3339.323 1764.335 .000 .960 1764.335 1.000

group 37.358 1 37.358 19.738 .000 .211 19.738 .992

Error 140.058 74 1.893


Table 14 Change in Science Attitude by Group Estimated Means

group * science

Measure: MEASURE_1

group science Mean Std. Error 95% Confidence Interval

Lower Bound Upper Bound

C 1 4.314 .176 3.963 4.665

2 4.069 .172 3.726 4.412

T 1 5.082 .176 4.731 5.432

2 5.284 .172 4.941 5.627

Table 15 Difference Change in Science Attitude by Group

48

48

Student data from quizzes.

Table 16 Attitude toward Science from Quizzes Mean

group * science

Measure: MEASURE_1



49

49

Control Pre 4.314 .176 3.963 4.665

Post 4.069 .172 3.726 4.412

Treatment Pre 5.082 .176 4.731 5.432

Post 5.284 .172 4.941 5.627

Student Qualitative Feedback on Science Attitudes

Interview sections pertaining to science attitudes for the treatment group are excerpted

below. The full transcript is also included in the evaluation information.

Students found the Talking Talons more enjoyable and educational when compared to the

standard science curriculum, which is called X-Cal.

Well uh, it’s definitely uh more educating, We have uh, X-Cal, which is kind of like

science, once a week and we have this two times so it’s probably double that.

Um, it’s really helping us because it explains more.

think it’s because, in X-Cal like Darren said, uh some of, Talking Talons doesn’t, uh

covers more than what X-Cal does like, we can actually use in the rest of our lives.

Um, I think Talking Talons is good cuz they let us experience like we’re gonna do a

presentation and they let us experience like, like if we were talking to actual people and

holding animals and all that stuff.

Uh I think it’s uh great because it teaches us, well real-life actual skills and things we’ll

need to know in the future unlike X-Cal which is teaching us that we will probably never

use again in my life.

When asked how Talking Talons compares to their regular science program.

It’s a lot better, Yeah.

50

50

Yeah we learn more about science and stuff. It’s more hands on too. Much-Yeah. I

prefer- Much more interesting, as in it puts, it makes it more fun than it usually is for me.

Classroom Teacher feedback about Science Attitudes

The classroom teachers indicated that they perceived that the student’s both learned a lot

of science from the program (mean score 6.5/7) and that the program had a positive impact on

student attitudes toward science (mean score 6.5/7).

Table 17 Classroom Teacher assessment of Science Attitude change

The students learn a lot of

science from the TT program.

The TT program had a positive impact on

the student’s attitudes about science.

Mean 6.5000 6.50

Std.

Deviation

.70711 .707

Hands on science experiments and Animals in the classroom

Research on Hands on Science and Animals in classroom

Significant contribution of hands on learning to science knowledge has been identified as

making a significant contribution to peer interaction through cooperative learning, object-

51

51

mediated learning and embodied experience. (Satterthwait). The more hands on experience, the

more learning occurs “Specifically, students who engaged in hands-on activities every day or

once a week scored significantly higher on a standardized test of science achievement than

students who engaged in hands-on activities once a month, less than once a month, or never.”

(Stohr-Hunt)

When compared to textbook learning, hands on (inquiry) also improves achievement

“Students in the inquiry-based group reached significantly higher levels of achievement than

students experiencing commonplace instruction. This effect was consistent across a range of

learning goals (knowledge, reasoning, and argumentation) and time frames (immediately

following the instruction and 4 weeks later).” (C. D. Wilson et al.)

Hands on learning is more effective for student comprehension even when compared to

teacher demonstration “students in the hands- on laboratory class performed significantly better

on the procedural knowledge test than did students in the teacher demonstration class. These

results were unrelated to reasoning ability.” (Glasson). This impact has been shown to influence

traditionally underrepresented minorities attitudes toward science and career plans. (Kanter and

Konstantopoulos). This is especially important as the impact of No Child Left behind has led to

less teaching of science in the classroom. (Milner et al.)

Furthermore, the use of animals in the classroom has been found to teach humane values

(Zasloff et al.) while improving young students attitudes toward animals (Ellery Samuels, Nicoll,

and Trifone) and towards other humans (Arkow) . The use of animals also improves

52

52

environmental education for young children (Margadant-Van Arcken) and knowledge about

animals specifically (Kimble).

The specific use of more exotic animals (zoo outreach) found that students recorded more

observations, made more use of science facts and use more science vocabulary when writing

about more exotic animals rather than more ordinary pets. (Trainin et al.) and also used more

advanced literary concepts (K. Wilson et al.)

Classroom Teacher feedback

Classroom teachers gave the maximum score possible (7/7) to all factors relating to the

effectiveness of the hands on and animal sections of the Talking Talons program.

Table 18 Classroom teacher assessment of program components

The animals

are an

important part

of the TT

program for

the students.

The public

speaking

component of

TT is an

important part of

the program.

The hands on

activities are

an important

part of the TT

program.

Presenting to the

Buddy class made

the students take the

responsibility of TT

more seriously.

Mean 7.00 7.00 7.00 7.00

Std.

Deviation

.000 .000 .000 .000

53

53

Student Feedback on Animals in classroom and Hands on science

Sections of the interview pertaining to the animals and hands on learning are excerpted below.

The students clearly found the animals to be an important aspect of the program, they universally

ask for more touchable animals.

Um, one thing that I liked is that they let us hold snakes and taught us about like, uh the

diseases and stuff. A thing I think they could improve in bringing more touchable

animals, I guess, because a lot of the times I have half the mind to just steal Hyde, the

owl I am going to be performing on today.

I really like how like uh their teaching us like if one day like we get lost in a forest and

there’s a snake like we can know that it’s poisonous or not poisonous. And I think they

could improve by, yeah having more touchable animals cuz like whenever we see that

birds we all wanna them.

I really like that they allow us to hold these snakes and show us like what to do if like

you see a snake or if you see a bird um injured, you know what to do, instead you just see

a bird injured and like go against the rules.

And you know, and that’s what I like about it, but I think that we can improve on like all

of them said is bringing in more, more touchable animals.

I like the activities because it actually gives us a more hands on experience of what we

are actually doing and what the animal does.

I actually like the activities better because it gives us more of an experience about what’s

around us in the world and stuff I guess.

54

54

I would change the fact that they don’t allow us to, to, to, see the more dangerous

animals. As in like, as in like rattlesnakes. I thought that, that they could do it by putting

it into a glass container, so that it won’t be able to damage us.

Well I, I was gonna say the same thing that he did but um, what I wanted them to do

more was that they should bring in more dangerous animals of course.

Was that I like how they taught us how to present it and, to let us touch the snakes and

lizards and such like that. And they taught us, taught me more about who’s endangered

and the continents and kind of that, yeah.

The same but like you know, if uh, if we would be able to hold the birds that would be

pretty cool.

It’s um, with the bats. Um, I felt like they’re not really that dangerous of animals because

even the vampire bats they don’t really suck any human blood, they do animal blood, as

in like cows and pigs, for animals that have lots of blood that they can spare, that they can

spare so I felt like that the bat was, I thought, I thought we would be able to like hold it.

Research on peer tutoring

One aspect of the Talking Talons program is the peer interaction. Students in the Talking

Talons program learn the material with the goal of presenting to a younger group of students in

the school.

55

55

A meta-analysis of peer assisted learning found increases in achievement. Peer tutoring is

most effective with “PAL interventions were most effective with younger, urban, low-income,

and minority students.”(Rohrbeck et al.).

Peer tutoring helps reading fluency (Kamps et al.), reading comprehension (McMaster,

Fuchs, and Fuchs) mathematics learning (Fantuzzo, Polite, and Grayson) both mathematics and

social skills (Pigott, Fantuzzo, and Clement) English (Greenwood et al.) and social interaction

for rejected boys (Gumpel and Frank).

Meta analyses on impact on math found the effect is particular strong for elementary aged

students (Kunsch, Jitendra, and Sood). A hands on gardening program found improvement in

social relationships in the classroom (Kim, Park, and Son).

Specifically buddy lesson plans for healthy eating and exercise physical activity, healthy

eating, and self-esteem and body image where lesson plans were delivered by 9-12 year olds for

6 to 8 year olds found that “Reductions in waist circumference were particularly significant for

children who were younger, overweight or obese, or attending First Nations schools. No

difference in body mass index score was observed between groups. Self-efficacy, healthy living

knowledge, and dietary intake significantly improved in younger peers who received the

intervention compared with students from control schools.” (Santos et al.).

Meta-analysis also shows that this impact is positive for both the “teacher” and the

“student” in peer tutoring situations. “This review of the literature on peer and cross-age tutoring

emphasizes programs in mathematics and suggests that such programs have positive academic

outcomes for African American and other minority students as well as for White students who

56

56

participate as tutors, as tutees, or both”. This results was shown even with short programs

“tutors show academic gains even when they do not receive additional subject matter instruction,

why longer and/or more substantial tutoring programs may not foster greater immediate

academic gains than shorter programs,” (Robinson, Schofield, and Steers-Wentzell)

Teaching benefits the tutor specifically in science classes “Mental rehearsal of peer-tutoring

episodes helped them appreciate weaknesses in their own subject knowledge.” (Galbraith and

Winterbottom)

Meta analyses also supports that peer tutoring is effective regardless of many factors.

“This meta-analysis examined effects of peer tutoring across 26 single-case research experiments

for 938 students in Grades 1–12. The Tau effect size for 195 phase contrasts was 0.75 with a

confidence interval of 0.71 to 0.78, indicating that moderate to large academic benefits can be

attributed to peer tutoring. Five potential moderators of these effects were examined: dosage,

grade level, reward, disability status, and content area. This is the first peer tutoring meta-

analysis in nearly thirty years to examine outcomes for elementary and secondary students, and

extends previous peer tutoring meta-analyses by examining disability as a potential moderator.

”Findings suggest that peer tutoring is an effective intervention regardless of dosage, grade level,

or disability status. Among students with disabilities, those with emotional and behavioral

disorders benefitted most.” (Bowman-Perrott et al.)

Classroom teacher assessment of Buddy class

The classroom teachers all agreed very strongly (mean of 7/7) that having the buddy

class had a positive impact on their students.

57

57

Table 19 Classroom teacher assessment of buddy class impact

Presenting to the Buddy class made the students take the responsibility of TT more seriously.

N Valid 2

Missing 0

Mean 7.00

Std. Deviation .000

Student feedback on Buddy class

Students universally expressed that they enjoyed “being the teacher” and presenting to

younger students. They mentioned leadership directly as well. Students were also irriated by the

fact that the younger students were not always quiet, this concern stemmed from the Talking

Talons students taking seriously that the animals were disturbed by excess noise. Sections related

the buddy class are excerpted from interviews below.

Um but one thing that was really nice about it is that they also got to experience what we

experienced with the animals. So yeah

Also they talked a lot during the presentations and they wouldn’t follow much of the

rules that we put them out, so they would scare the animals a lot.

Um I really like that they can um look up to us as their leaders and we can like educate

them and it was also kind of annoying cuz they did scare the animals a lot.

58

58

I kinda like how, them looking up to us and them actually learning something new from

someone else, cuz them might not pick it up from like a teacher or someone, but they can

probably pick it up from us. But then again they do kinda scare the animals.

Uh, it was a good thing for, to help us present that to other people because again at the

end we had to present to at least a hundred.

I, the buddy class was OK for me so like it prepared me for the presentations. Yes.

Role Model for younger students

The change in role model was not significant, the power was low due to sample size.

However, the direction of change was positive for the treatment group and negative for the

control group. This trend was seen in the 2015 data as well. This information is included for

trends, rather than statistical significance.

I am a good role model for younger students 2016

59

59

Table 20 Student answers by group to "I am a good role model for younger students"



I am a good role model for younger students. Control 4.472 1.4038 36

Treatment 4.611 1.2935 36

Total 4.542 1.3420 72

POST I am a good role model for younger students. Control 4.31 1.411 36

60

60

Treatment 4.86 1.175 36

Total 4.58 1.319 72

Table 21 I am a good role model significance tests

Multivariate Testsa

Eff

ect

Val

ue

F

Hypoth

esis

df

Err

or

df

Sig

.

Par

tial

Eta

Squar

ed

Nonce

nt.

Par

amet

er

Obse

rved

Pow

erc

Role Model Pillai's Trace .001 .052b 1.000 70.000 .821 .001 .052 .056

Wilks' Lambda .999 .052b 1.000 70.000 .821 .001 .052 .056

Hotelling's

Trace

.001 .052b 1.000 70.000 .821 .001 .052 .056

Roy's Largest

Root

.001 .052b 1.000 70.000 .821 .001 .052 .056

Role Model *

group

Pillai's Trace .018 1.289b 1.000 70.000 .260 .018 1.289 .201

Wilks' Lambda .982 1.289b 1.000 70.000 .260 .018 1.289 .201

Hotelling's

Trace

.018 1.289b 1.000 70.000 .260 .018 1.289 .201

Roy's Largest

Root

.018 1.289b 1.000 70.000 .260 .018 1.289 .201


Within Subjects Design: Role Model

b. Exact statistic


61

61


Measure: MEASURE_1

Sourc

e

Type

III

Sum

of

Squar

es

df

Mea

n S

quar

e

F

Sig

.

Par

tial

Eta

Squar

ed

Nonce

nt.

Par

amet

er

Obse

rved

Pow

era

Role Model Sphericity

Assumed

.062 1 .062 .052 .821 .001 .052 .056

Greenhouse-

Geisser

.062 1.000 .062 .052 .821 .001 .052 .056

Huynh-Feldt .062 1.000 .062 .052 .821 .001 .052 .056

Lower-bound .062 1.000 .062 .052 .821 .001 .052 .056

Role Model *

group

Sphericity

Assumed

1.562 1 1.562 1.289 .260 .018 1.289 .201

Greenhouse-

Geisser

1.562 1.000 1.562 1.289 .260 .018 1.289 .201

Huynh-Feldt 1.562 1.000 1.562 1.289 .260 .018 1.289 .201

Lower-bound 1.562 1.000 1.562 1.289 .260 .018 1.289 .201

Error(Role

Model)

Sphericity

Assumed

84.875 70 1.213

Greenhouse-

Geisser

84.875 70.000 1.213

Huynh-Feldt 84.875 70.000 1.213

Lower-bound 84.875 70.000 1.213


62

62

Science Knowledge

Change in Science Knowledge by Group



represents a small effect size.

Figure 3 Change in Science Knowledge by Group

63

63

Table 22 Change in Science Knowledge by Group



science Control 4.3137 1.24026 38

Treatment 5.0816 .90460 38

Total 4.6977 1.14539 76

pscience Control 4.0690 1.32331 38

64

64

Treatment 5.2842 .70731 38

Total 4.6766 1.21853 76

Table 23 Change in Science Knowledge by Group Multivariate tests

Multivariate Testsa

Eff

ect

Val

ue

F

Hypoth

esis

df

Err

or

df

Sig

.

Par

tial

Eta

Squar

ed

Nonce

nt.

Par

amet

er

Obse

rved

Pow

erc

science Pillai's Trace .001 .041b 1.000 74.000 .840 .001 .041 .055

Wilks'

Lambda

.999 .041b 1.000 74.000 .840 .001 .041 .055

Hotelling's

Trace

.001 .041b 1.000 74.000 .840 .001 .041 .055

Roy's Largest

Root

.001 .041b 1.000 74.000 .840 .001 .041 .055

science *

group

Pillai's Trace .059 4.623b 1.000 74.000 .035 .059 4.623 .564

Wilks'

Lambda

.941 4.623b 1.000 74.000 .035 .059 4.623 .564

65

65

Hotelling's

Trace

.062 4.623b 1.000 74.000 .035 .059 4.623 .564

Roy's Largest

Root

.062 4.623b 1.000 74.000 .035 .059 4.623 .564


Within Subjects Design: science

b. Exact statistic


Table 24 Change in Science Knowledge by Group Means


Measure: MEASURE_1


Source Type III

Sum of

Squares

df Mean

Square

F Sig. Partial

Eta

Squared

Noncent.

Parameter

Observed

Powera

Intercept 3339.323 1 3339.323 1764.335 .000 .960 1764.335 1.000

group 37.358 1 37.358 19.738 .000 .211 19.738 .992

Error 140.058 74 1.893


group * science

Measure: MEASURE_1



Control 1 4.314 .176 3.963 4.665

2 4.069 .172 3.726 4.412

Treatment 1 5.082 .176 4.731 5.432

2 5.284 .172 4.941 5.627

66

66

Quiz Knowledge

The information below indicates the mean score for the treatment group on the

knowledge sections of the quizzes. A table with the standard deviation is also included. The

educators have been given the results by question in order to examine areas which need more

Role Modelion. This document is available upon request and is included on the CD. A

comparison was also done by class. For all quizzes the results were not different by class. There

were no significant differences by gender. For both years student comprehension was lower for

the raptor quiz.

Figure 4 Quiz Knowledge section means 2015

67

67

Figure 5 Quiz Knowledge means 2016

68

68

Table 25 Mean scores on Knowledge Quiz


N Minimum Maximum Mean Std. Deviation

Q1 Intro. 43 1.00 5.00 4.1163 .95641

69

69

Q2 Hawks. 38 1.00 5.00 3.7105 1.13680

Q3 Raptors. 41 1.00 2.00 1.3902 .49386

Q4 Bats 1. 47 1.00 5.00 3.5957 1.20974

Q5 Bats 2. 47 1.00 5.00 3.2766 1.21050

Q6 Owls. 44 1.00 5.00 3.9773 1.10997

Q7 Reptiles . 44 1.00 5.00 4.1364 1.00211

Q8 Vocab. 42 2.00 5.00 4.3571 .98331

Q9 General. 42 1.00 5.00 3.4286 1.03930

Q10 Energy . 43 1.00 5.00 3.8605 1.18686

Valid N (listwise) 29

No significance difference was found by gender, males and females had equivalent

scores on the knowledge sections of the test.

Self-perceived science grade in school

A statistically significant change in self-reported anticipated grade in science was

seen in the treatment group and not for the control group from pretesting to post testing


70

70

Figure 6 Self perceived change in science grade by group

Table 26 Group Comparison for I will get a good grade in science class this year


group Mean Std.

Deviation

N

I will get a good grade in science class this year Control 4.946 .9985 37

71

71

Treatment 5.083 1.0247 36

Total 5.014 1.0068 73

POST I will get a good grade in science class this

year

Control 4.41 1.462 37

Treatment 5.25 .806 36

Total 4.82 1.251 73

Table 27 Group Comparison for I will get a good grade in science class this year

Multivariate Testsa

Eff

ect

Val

ue

F

Hypoth

esis

df

Err

or

df

Sig

.

Par

tial

Eta

Squar

ed

Nonce

nt.

Par

amet

er

Obse

rved

Pow

erc

scigrd

2

Pillai's

Trace

.016 1.130b 1.000 71.000 .291 .016 1.130 .182

Wilks'

Lambda

.984 1.130b 1.000 71.000 .291 .016 1.130 .182

Hotelling'

s Trace

.016 1.130b 1.000 71.000 .291 .016 1.130 .182

72

72

Roy's

Largest

Root

.016 1.130b 1.000 71.000 .291 .016 1.130 .182

scigrd

2 *

group

Pillai's

Trace

.054 4.045b 1.000 71.000 .048 .054 4.045 .510

Wilks'

Lambda

.946 4.045b 1.000 71.000 .048 .054 4.045 .510

Hotelling'

s Trace

.057 4.045b 1.000 71.000 .048 .054 4.045 .510

Roy's

Largest

Root

.057 4.045b 1.000 71.000 .048 .054 4.045 .510


Within Subjects Design: scigrd2

b. Exact statistic


Group Comparison for I will get a good grade in science class this year Within Subjects Contrasts

73

73


Measure: MEASURE_1

Sourc

e

scig

rd2

Type

III

Sum

of

Squar

es

df

Mea

n S

quar

e

F

Sig

.

Par

tial

Eta

Squar

ed

Nonce

nt.

Par

amet

er

Obse

rved

Pow

era

scigrd2 Linear 1.275 1 1.275 1.130 .291 .016 1.130 .182

scigrd2 * group Linear 4.563 1 4.563 4.045 .048 .054 4.045 .510

Error(scigrd2) Linear 80.095 71 1.128


Quiz Attitude subscales

Student mean attitudes about the program are presented below. The maximum possible

score is a 6/6 which indicates Strongly Agree.

Figure 7 Quiz Attitude subscales

74

74

75

75

Table 28 Quiz Attitude subscale descriptive statistics


N Minimum Maximum Mean Std.

Deviation

tt posttest student enjoyed TT 42 3.00 6.00 5.5952 .68880

TT postest overall student

perception of educator

42 3.50 6.00 5.6810 .50133

Program enjoyment 48 3.00 6.00 5.4062 .69246

Attitude Environment 45 4.00 6.00 5.7926 .40383


76

76

Feedback on Educator

Student mean attitudes about the Talking Talon’s educators are presented below. The

maximum possible score is a 6/6 which indicates Strongly Agree.

Student Quantitative Feedback Figure 8 Student Perception of Educator

77

77

Student Feedback

Table 29 Student feedback on Educator Subscales


N Minimum Maximum Mean Std. Deviation

Perception of Educator 50 4.43 6.00 5.7346 .34169

78

78

Bonding with Educator 49 4.67 6.00 5.7177 .41451

Educator Teaching Skills 50 4.33 6.00 5.7682 .36095


Figure 9 Student Perception of Educator

79

79

Table 30 Student feedback on Educator from Quiz results


N Minimum Maximum Mean Std.

Deviation

POST My Talking Talons educator

(teacher) cared a lot about what I learned.

41 5 6 5.76 .435


(teacher) was interested in what I had to

say.

42 4 6 5.71 .508


(teacher) was fair about giving everyone an

equal chance to do things.

42 2 6 5.60 .857

POST My Talking Talons educator knew a

lot about the animals.

42 5 6 5.86 .354


(teacher) was easy to understand.

42 2 6 5.48 .890

POST I think my Talking Talons educator

(teacher) liked coming to teach my class.

42 4 6 5.74 .544


understands how kids my age think.

42 1 6 5.50 .944

POST My Talking Talons educator treated

boys and girls equally.

42 2 6 5.74 .734

POST My Talking Talons educator helped

me to learn how to give speeches.

41 4 6 5.73 .549


80

80

Student Qualitative Feedback on Educators

The students enjoyed the Talking Talons educators and felt they were effective

instructors

Um, I like, like, because they know what we were thinking, or how we react to big birds,

they kind of break it down, or if we don’t know that word they break it down, or like

since they were our age they know how like to do stuff.

Well, that’s, they did, I, I can’t find, I don’t know what they can improve on but they did

pretty well.

kind of like uh, cuz Betsy and Lori they actually, they actually break it down in terms that

we can understand and they help us out in ways they know we would understand.

Um, I like that Lori is just like, whatever, and (Giggling and Laughing) and I like that

um, that Betsy like uh, she lets us understand, like when we’re learning about the animals

she like slowly, and then we catch on.

There’s nothing actually they can improve on they’re perfect.

Uh I like uh that Lori, well yeah I like that Lori she kind of like goes with the flow and

makes us laugh a lot, and I also like that how Betsy she explains to uh everything to us in

a way we can all understand.

Um, I thought that they did good about telling us how, how to hold it, how we could

change our speech to uh, improve it.

I think they did amazing.

81

81

They were fun, they, they were fun and they taught us a bunch of things.

They were really nice…and they did a good job

Classroom teacher feedback for Educator

The classroom teachers mean perception of the educator is presented below. These

questions are on a scale with a maximum positive score of seven.

Figure 10 Classroom teacher feedback on Talking Talons Educator

82

82

Table 31 Classroom Teacher Feedback on Talking Talons Educator

Statistics

The TT

educator

was

generally

prepared to

teach the

class.

The TT

educator

worked with

me to make

the program

run smoothly.

The TT

educator

used

classroom

time well.

The TT educator

appeared generally

knowledgeable

about content

presented

The TT

educator

appeared to

enjoy

teaching the

students.

N Valid 4 4 4 4 4

Missing 0 0 0 0 0

Mean 6.50 6.75 7.000 6.75 7.00

Median 7.00 7.00 7.000 7.00 7.00

Std.

Deviation

1.000 .500 .0000 .500 .000

Minimum 5 6 7.0 6 7

Maximum 7 7 7.0 7 7

Figure 11 Classroom Teacher subscales on Educator

83

83

Classroom teacher feedback for Talking Talons Program

The classroom teacher’s mean perceptions of the Talking Talon’s program are presented

below. These questions are on a scale with a maximum positive score of seven.

Teacher Attitude Figure 12 Classroom teacher feedback on Talking Talons program

84

84

Table 32 Classroom Teacher Feedback Personal

Statistics

Talking Talons is

a worthwhile use

of my classroom

time.

I was able to tie

the TT program

into my

curriculum.

I found the material

presented by the TT

program to be

interesting to me

personally.

I would be

willing to have

the TT program

in my classroom

again.

N Valid 4 4 4 4

Missing 0 0 0 0

Mean 6.75 6.25 6.25 6.75

Median 7.00 6.50 6.50 7.00

Std.

Deviation

.500 .957 .957 .500

85

85

Effectiveness by student ability: Feedback by Classroom teachers

The classroom teachers mean perception of the impact of the program by student ability

is presented below. These questions are on a scale with a maximum positive score of seven. The

teachers found the program generally very effective for average students and v for advanced

students. The program was found to be effective for those with learning disabilities. These

results mirror previous years feedback and indicate that the program is reaching the majority of

the student’s abilities at the correct level.

Figure 13 Classroom teacher assessment of Effectiveness of program by student ability

86

86

Table 33 Classroom teacher feedback program effectiveness by student ability

Statistics

The TT program was

effective for above

average students.

The TT program was

effective for average

students.

The program was effective

for students with learning

disabilities.

N Valid 4 4 4

Missing 0 0 0

Mean 6.25 6.50 5.50

Median 6.00 6.50 5.50

Std.

Deviation

.500 .577 1.291

Minimum 6 6 4

87

87

The teachers found the program generally very effective for average students and

effective for advanced students and those with learning disabilities.

Classroom teacher subscales

The classroom teacher’s mean perceptions for subscales are presented below. These

questions are on a scale with a maximum positive score of seven. Data for teachers for this year

and the previous year is included.

Figure 14 Classroom teacher perception

88

88

Table 34: Classroom teacher subscales for Talking Talons program

Statistics

Teacher

science

for

students

Teacher

effectiveness

by student

ability

Teacher

components

of TT

Teachers

personal

feelings

about TT

Teachers

perception

of educator

N Valid 4 4 4 4 4

Missing 0 0 0 0 0

Mean 6.3333 6.0833 7.0000 6.4000 6.8333

Median 6.3333 6.0000 7.0000 6.5000 7.0000

Std.

Deviation

.54433 .73912 .00000 .48990 .33333

89

89

Minimum 5.67 5.33 7.00 5.80 6.33

Other Results

Changes in pre posttest Composite

Figure 15 Statistically significant Pre Post change by group

Table 35 Statistically significant Pre Post change by group

Report

groupnum Change in Knowledge Change in Science Attitude

90

90

Treatment Mean .5758 .2026

N 37 38

Std. Deviation .61541 .70653

Control Mean .0255 -.2447

N 37 38

Std. Deviation .49282 1.07052

Total Mean .3007 -.0211

N 74 76

Std. Deviation .61911 .92862

Table 36 ANOVA Statistically significant Pre Post change by group

ANOVA Table

Sum of

Squares

df Mean

Square

F Sig.

Change in

Knowledge *

groupnum

Between

Groups

(Combined) 5.603 1 5.603 18.027 .000

Within Groups 22.378 72 .311

Total 27.980 73

Change in Science

Attitude * groupnum

Between

Groups

(Combined) 3.803 1 3.803 4.623 .035

Within Groups 60.873 74 .823

91

91

Total 64.675 75

Table 37 Effect Size Statistically significant Pre Post change by group

Measures of Association

Eta Eta Squared

Change in Knowledge * groupnum .447 .200

Change in Science Attitude * groupnum .242 .059

Structural Equation Model

A full separate report of a Structural equation model of the impact of student attitudes

toward the educator and the animals upon science attitudes is provided separately. This model

was designed to examine the impact of the educators and the animals upon science attitude. A

summary of the conclusions is presented below.

Figure 16: Final pruned model of impact of student attitude toward educator and animals on science attitudes

92

92

A theoretical structural model (the Saturated Model) had an good fit to the data The model fit

for this SEM is indicates that high level of confidence in the model. The sample size is small.

93

93

A second and simpler model (the Pruned Model) that eliminated the non-significant paths in

the Saturated Model also exhibited a close fit to the data
















changing the student’s Attitudes toward Animals. This is a complex outcome, indicating that

the animals are the crux of the change in science attitudes for the students, but that the



94

94

Summary




In order to examine the longitudinal change, the science attitudes of the treatment group

were examined twice during the program as well as during pretesting and posttesting. The

treatment group exhibited a statistically significant positive change in attitude toward science as

the program progressed. F(3,30). p<.01 η2=.37 (large effect size). This statistic uses only the

treatment group, as the control group does not take the Talking Talon’s quizzes. Students in the

program also exhibited a statistically significant change in self-reported anticipated grade in

science. This change was seen in the treatment group and not for the control group from

pretesting to post testing F(1,71)=4.05. p<.05 η2=.06. This represents a small effect size.

A statistically significant change in Knowledge was seen in the treatment group and not for

the control group from pretesting to post testing. F(1,74)=4.63. p<.05 η2=.45. This represents a

very large effect size. A breakdown of the treatment group Knowledge scores on quizzes also

was provided.

Classroom teacher feedback was extremely positive and both teachers indicated the program

increased student science knowledge and attitudes, that the buddy class was worthwhile and also

indicated a strong willingness to have the program again. Mean scores for all subscales

measuring teacher feedback were all above 6 on a 7 point scale (a higher number indicates more

positive feedback)

95

95

Qualitative feedback from students was overwhelmingly positive, with students mentioning

that the program was a positive experience, that they enjoyed presenting to the buddy class and

found it both rewarding and educational.
















96

96










Even with a small sample size, and thus less statistical power, it is clear that the program

positivity impacted the participants. Science attitudes and knowledge are increased and

classroom teachers find the program effective and enjoyable for their students.

97

97

References

Al-Balushi, Sulaiman M, and Shamsa S Al-Aamri. “The Effect of Environmental Science

Projects on Students’ Environmental Knowledge and Science Attitudes.” International

Research in Geographical and Environmental Education 2014: 213–227. Web.

Aquino, Karl, and Americus Reed. “The Self-Importance of Moral Identity.” Journal of

personality and social psychology 83.6 (2002): 1423–1440. Web.

Arbuthnot, J, and D a Gordon. “Behavioral and Cognitive Effects of a Moral Reasoning

Development Intervention for High-Risk Behavior-Disordered Adolescents.” Journal of

consulting and clinical psychology 54.2 (1986): 208–216. Web.

Arkow, Phil. “Animals as Facilitators of Social Capital and Character Education.” Society for the

Study of Social Problems 2006. Web.

Backes, John S. “Bridging the Gender Gap: Self-Concept in the Middle Grades.” Schools in the

Middle 3.3 (1994): 19–23. Print.

Bakeman, Roger. “Recommended Effect Size Statistics for Repeated Measures Designs.”

Behavior research methods 37.3 (2005): 379–384. Web.

Baker, Dale R. “Predictive Value of Attitude, Cognitive Ability, and Personality to Science

Achievement in the Middle School.” Journal of Research in Science Teaching 22.2 (1985):

98

98

103–113. Print.

Berkowitz, Marvin W. et al. “Moral Reasoning and Judgments of Aggression.” Journal of

Personality and Social Psychology 51.4 (1986): 885–891. Web.

Bowman-Perrott, Lisa et al. “Academic Benefits of Peer Tutoring : A Meta-Analytic Review of

Single-Case Research.” School Psychology Review 42.1 (2013): 39–55. Print.

Catsambis, Sophia. “Gender, Race, Ethnicity, and Science Education in the Middle Grades.”

Journal of Research in Science Teaching 32.3 (1995): 243–257. Print.

Desy, Elizabeth A, Scott A Peterson, and Vicky Brockman. “Gender Differences in Science-

Related Attitudes and Interests among Middle School and High School Students.” Science

Educator 20.2 (2011): 23–30. Web.

Ellery Samuels, William, Kate Nicoll, and Cindy Trifone. “An In-Class, Humane Education

Program Can Improve Young Students’ Attitudes Toward Animals.” Society & Animals

16.1 (2008): 45–60. Web.

Fantuzzo, John W., Kenneth Polite, and Natalie Grayson. “An Evaluation of Reciprocal Peer

Tutoring across Elementary School Settings.” Journal of School Psychology 28.4 (1990):

309–323. Web.

Farenga, Stephen, and Beverly Joyce. “Science-Related Attitudes and Science Course

Selection...” Roeper Review 20 (1998): 247. Print.

Freedman, Michael P. “The Influence of Laboratory Instruction on Science Achievement and

99

99

Attitude toward Science across Gender Differences.” Journal of Women and Minorities in

Science and Engineering 8.2 (2002): 191–200. Web.

Galbraith, Jonathan, and Mark Winterbottom. “Peer‐tutoring: What’s in It for the Tutor?”

Educational Studies 37.3 (2011): 321–332. Web.

Glasson, George E. “The Effects of Hands-on and Teacher Demonstration Laboratory Methods

on Science Achievement in Relation to Reasoning Ability and Prior Knowledge.” Journal

of Research in Science Teaching 26.2 (1989): 121–131. Web.

Greenwood, Charles R et al. “Classwide Peer Tutoring Learning Management System:

Applications with Elementary-Level English Language Learners.” Remedial and Special

Education 22.1 (2001): 34–47. Print.

Guivernau, Marta, and Joan L. Duda. “Moral Atmosphere and Athletic Aggressive Tendencies in

Young Soccer Players.” Journal of Moral Education 31.1 (2002): 67–85. Web.

Gumpel, T P, and R Frank. “An Expansion of the Peer-Tutoring Paradigm: Cross-Age Peer

Tutoring of Social Skills among Socially Rejected Boys.” Journal of applied behavior

analysis 32.1 (1999): 115–118. Web.

Jarvis, Tina, and Anthony Pell. “Changes in Primary Boys’ and Girls' Attitudes to School and

Science during a Two-Year Science in-Service Programme.” The Curriculum Journal 13.1

(2002): 43–69. Print.

Kamps, D M et al. “Classwide Peer Tutoring: An Integration Strategy to Improve Reading Skills

and Promote Peer Interactions among Students with Autism and General Education Peers.”

100

100

Journal of applied behavior analysis 27.1 (1994): 49–61. Web.

Kanter, David E., and Spyros Konstantopoulos. “The Impact of a Project-Based Science

Curriculum on Minority Student Achievement, Attitudes, and Careers: The Effects of

Teacher Content and Pedagogical Content Knowledge and Inquiry-Based Practices.”

Science Education 94.5 (2010): 855–887. Web.

Kim, SeongSil, SinAe Park, and KiCheol Son. “Improving Peer Relations of Elementary School

Students through a School Gardening Program.” HortTechnology 24.2 (2014): 181–187.

Web.

Kimble, G. “Children Learning about Biodiversity at an Environment Centre, a Museum and at

Live Animal Shows.” Studies in Educational Evaluation 41 (2014): 48–57. Web.

Kose, Sacit et al. “The Effects of Cooperative Learning Experience on Eighth Grade Students’

Achievement and Attitude toward Science.” Education 131.1 (2010): 169–180. Print.

Kunsch, Catherine A, Asha K Jitendra, and Sheetal Sood. “The Effects of Peer-Mediated

Instruction in Mathematics for Students with Learning Problems: A Research Synthesis.”

Learning Disabilities Research & Practice 22.1 (2007): 1–12. Web.

Lee, Valerie E, and David T Burkam. “Gender Differences in Middle Grade Science

Achievement: Subject Domain, Ability Level, and Course Emphasis.” Science Education

80.6 (1996): 613–650. Print.

Maguin, Eugene, and Rolf Loeber. “Academic Performance and Delinquency.” Crime and

Justice 20 (1996): 145–264. Web.

101

101

Malti, Tina, and Tobias Krettenauer. “The Relation of Moral Emotion Attributions to Prosocial

and Antisocial Behavior: A Meta-Analysis.” Child development 84.2 (2013): 397–412.

Web.

Margadant-Van Arcken, Marjan. “Environmental Education, Children, and Animals.”

Anthrozoos 3:1 (1989): 14–19. Print.

Mattern, Nancy Page Garland. “Relationships between Attitudes toward and Achievement in

Science for Rural Middle School Students: Patterns across Gender.” Dissertation Abstracts

International Section A: Humanities & Social Sciences 60.12-A (2000): 4319. Print.

Mattern, Nancy, and Candace Schau. “Gender Differences in Science Attitude-Achievement

Relationships over Time among White Middle-School Students.” Journal of Research in

Science Teaching 39.4 (2002): 324–340. Web.

McMaster, Kristen L., Douglas Fuchs, and Lynn S. Fuchs. “Research on Peer-Assisted Learning

Strategies: The Promise and Limitations of Peer-Mediated Instruction.” Reading & Writing

Quarterly 22.1 (2006): 5–25. Web.

Milner, Andrea R. et al. “Elementary Teachers’ Beliefs About Teaching Science and Classroom

Practice: An Examination of Pre/Post NCLB Testing in Science.” Journal of Science

Teacher Education 23.2 (2012): 111–132. Web.

Oliver, Joseph S. “A Longitudinal Study of Attitude, Motivation and Self Concept as Predictors

of Achievement in and Commitment to Science among Adolescent Students.” Dissertation

Abstracts International 47.8-A (1987): 2983. Print.

102

102

Oliver, Steve J, and Ronald D Simpson. “Influence of Attitude toward Science, Achievement

Motivation, and Science Self Concept on Achievement in Science: A Longitudinal Study.”

Science Education 72.2 (1988): 143–155. Web.

Ornstein, Avi. “The Frequency of Hands-on Experimentation and Student Attitudes toward

Science: A Statistically Significant Relation.” Journal of Science Education and

Technology 15.3/4 (2006): 285–297. Web.

Papanastasiou, Elena C, and Michalinos Zembylas. “The Effect of Attitudes on Science

Achievement: A Study Conducted Among High School Pupils in Cyprus.” International

review of education 48.6 (2002): 469–484. Print.

Pigott, H E, J W Fantuzzo, and P W Clement. “The Effects of Reciprocal Peer Tutoring and

Group Contingencies on the Academic Performance of Elementary School Children.”

Journal of applied behavior analysis 19.1 (1986): 93–8. Web.

Raaijmakers, Quinten, Rutger Engels, and Anne Van Hoof. “Delinquency and Moral Reasoning

in Adolescence and Young Adulthood.” International Journal of Behavioral Development

29 (2005): 247–258. Web.

Reynolds, Arthur J, and Herbert J Walberg. “A Structural Model of Science Achievement and

Attitude: An Extension to High School.” Journal of Educational Psychology 84.3 (1992):

371–382. Print.

Reynolds, Scott J, and Tara L Ceranic. “The Effects of Moral Judgment and Moral Identity on

Moral Behavior: An Empirical Examination of the Moral Individual.” The Journal of

103

103

applied psychology 92.6 (2007): 1610–1624. Web.

Robinson, Debbie R., Janet Ward Schofield, and Katrina L. Steers-Wentzell. “Peer and Cross-

Age Tutoring in Math: Outcomes and Their Design Implications.” Educational Psychology

Review 17.4 (2005): 327–362. Web.

Rohrbeck, Cynthia a. et al. “Peer-Assisted Learning Interventions with Elementary School

Students: A Meta-Analytic Review.” Journal of Educational Psychology 95.2 (2003): 240–

257. Web.

Santos, Robert G et al. “Effectiveness of Peer-Based Healthy Living Lesson Plans on

Anthropometric Measures and Physical Activity in Elementary School Students: A Cluster

Randomized Trial.” JAMA pediatrics 168 (2014): 330–7. Web.

Satterthwait, Donna. “Why Are ‘Hands-on’ Science Activities so Effective for Student

Learning?” Teaching Science: The Journal of the Australian Science Teachers Association

56.2 (2010): 7–10. Web.

Slate, John R, and Craig H Jones. “Fourth and Fifth Grade Students’ Attitudes toward Science:

Science Motivation and Science Importance as a Function of Grade Level, Gender, and

Race.” Research in the Schools 5.1 (1998): 27–32. Print.

Smith, T. J., S. L. Pasero, and C. M. McKenna. “Gender Effects on Student Attitude Toward

Science.” Bulletin of Science, Technology & Society 34.1-2 (2014): 7–12. Web.

Sorge, Carmen. “The Relationship Between Bonding with Nonhuman Animals and Students’

Attitudes Toward Science.” Society & Animals 16 (2008): 171–184. Web.

104

104

Stohr-Hunt, Patricia M. “An Analysis of Frequency of Hands-on Experience and Science

Achievement.” Journal of Research in Science Teaching 33.1 (1996): 101–109. Web.

Tangney, June Price, Jeff Stuewig, and Debra J Mashek. “Moral Emotions and Moral Behavior.”

Annual review of psychology 58 (2007): 345–372. Web.

Tarry, Hammond, and Nicholas Emler^. “Attitudes, Values and Moral Reasoning as Predictors

of Delinquency.” British Journal of Developmental Psychology 25 (2007): 169–183. Web.

Teshome, Yalem, Nancy Maushak, and Krishna Athreya. “Attitude toward Informal Science and

Math: A Survey of Boys and Girls Participating in Hands-On Science and Math

(FUNTIVITIES).” Journal of Women and Minorities in Science and Engineering 7.1

(2001): 59–74. Web.

Trainin, Guy et al. “Extraordinary Animals and Expository Writing: Zoo in the Classroom.”

Journal of Science Education and Technology 14.3 (2005): 299–304. Web.

Warburton, S J, W L Jenkins, and P Coxhead. “‘Science Achievement and Attitudes’ and the

Age of Transfer to Secondary School.” Educational Research 25.3 (1983): 177–183. Print.

Willson, Victor L. “A Meta-Analysis of the Relationship between Science Achievement and

Science Attitude: Kindergarten through College.” Journal of Research in Science Teaching

20.9 (1983): 839–850. Print.

Wilson, Christopher D. et al. “The Relative Effects and Equity of Inquiry-Based and

Commonplace Science Teaching on Students’ Knowledge, Reasoning, and Argumentation.”

Journal of Research in Science Teaching 47.3 (2010): 276–301. Web.

105

105

Wilson, K. et al. “Our Zoo To You: The Link between Zoo Animals in the Classroom and

Science and Literacy Concepts in First-Grade Journal Writing.” Journal of Early Childhood

Literacy 11.3 (2011): 275–306. Web.

Zasloff, R L et al. “Animals in Elementary School Education in California.” Journal of Applied

Animal Welfare Science 2.4 (1999): 347–357. Web.

106

106

Structural Equation Model of

Talking Talons Variables Attitudes toward

Animals and Educator upon

Science Outcome Attitudes

Dr. Carmen Sorge

Leiden LLC

Jan 2017

107

107

What is Structural equation modeling (SEM)?

Behavioral research often involves attempting to answer questions about how variables

relate to each other. For example, we might want to know what factors influence the grade a

student gets in algebra. We often go about trying to find the answer by examining factors

individually.

Perhaps we might look at whether the following factors influence grades:

Intelligence

Previous Experience with math

Attitudes toward math

Experiences with math teachers

In linear statistics the researcher looks at how each of these variables affects the grade in

math. So the researcher might find that IQ is a good predictor or that effort is not a good

predictor of what grade the student gets in algebra.

This research would look like this:

or

What is missing is how those variables might be interacting with each other complexly.

Often we know that the two variables are related (such as experiences with math teacher and

attitude about math). But does Intelligence affect Effort that then in turn affects Attitude? These

questions can be answered by structural equation modeling.

IQ Algebra grade

Effort Algebra grade

108

108

First the researcher makes a measurement model. This model looks at how the variables

interrelate with each other. The researcher then creates a model. For example the research might

use “Effort” as one of the variables. Several measures might be made of “Effort” such as the

hours the student spends studying and class attendance.

The variable called Effort, which is a “latent variable”, will then look like this:

Hours

Studying

The researcher continues with the other variables, say Intelligence, and Attitudes.

The strength of a Structural Equation model is that we can look at complex relationships

BETWEEN the variables. We can examine how one variable influences another that in turn

affects yet another variable. In the example above we might have model that looks like this:

Effort Class

Attendance

109

109

In the SEM the researcher can see if Intelligence affects Attitude that then in turn affects

Effort that predicts the grade in the class. Of course there are many more connections in this

model and with SEM it is possible to examine many of them and find which ones are the best

predictors. The researcher can also look at several outcomes or how one outcome affects another.

A SEM model gives weights to each of the relationships and allows the researcher to examine

the strength of these interactions. This is a powerful tool, which allows the researcher to give

more useful feedback to the program because it pinpoints the variables which lead to the most

change overall. However, it requires a greater investment of time and effort and a larger set of

participants than more linear research.

Intelligence

IQ Grades

Final Grade

in Algebra

Attitudes

Attitude

about

Learning

Perception

of Math

Difficulty

Effort

Hours

studying

Class Attendance

110

110

Study Design

Purpose of the Study

The purpose of this study was to examine possible causal relationships between the

attitudes of students towards the educator and animals and the impact upon science attitudes.

Throughout the program a significant change in science attitudes of treatment groups has been

observed which is not seen in the control groups. This SEM is a preliminary investigation of the

relative importance of the educator and the animals upon the science attitudes of the students.

All scales are designed so that a higher number is a more positive score. The models of these

relationships were drawn from theoretical constructs primarily related to Expectancy Value

theories.

Research Questions

1. Which model better describes the empirical relationships between the Talking Talons Factors

with the outcome variables?

2. Does the better model adequately describe the empirical relationships between the Talking

Talons Factors?

Limitations of the Study

1. The sample students in this study were drawn from one geographic area near Albuquerque.

The sample was not random. However, the sample is drawn from the population served by

the program and thus represents the target group

111

111

2. The attitude and demographic information was self-reported by participants.

3. No demographic differences were examined. It is possible that they exist and impacted the

models.

4. Data collection for this model is over a two year period and therefore and an ideal sample

size has not yet been achieved.

Definition of Constructs

Attitude toward Educator and Attitude toward Animals are latent variable constructed

form student feedback on their attitudes toward the educator taken from the Talking Talons

Posttest. Science Attitude outcome variable is constructed from responses on the Talking Talons

posttest and quizzes.

Attitude Theory

Theories of attitude and behavior have been entwined throughout their development.

Many attempts have been made to clarify the relationships between attitude and behavior in spite

of measurement difficulties. The use of attitude to predict behaviors, including achievement,

links these constructs both in practical application and in theory development.

Three components of attitude have been commonly identified from ancient times to the

present: affective, cognitive and behavioral. The affect component consists of feelings of like

and/or dislike held by the individual toward the attitude object. The cognitive component

consists of beliefs and ideas that the individual holds about the attitude object. The behavioral

component consists of tendencies to respond to the object.

112

112

Despite the long-standing interest in attitude, social psychologists have failed to provide a

single, universally accepted definition of the construct. Gordon W. Allport’s classic definition

(1935) is often cited as a starting point for attitude definition. He characterized attitude as “a

mental and neural state of readiness, organized through experience, exerting a directive or

dynamic influence upon the individual’s response to all objects and situations with which it is

related.” (Allport, 1935). Current literature suggests that attitude can be conceptualized as a

tendency to evaluate a stimulus with some degree of favor or disfavor. This evaluation is usually

expressed in the classical cognitive, affective and/or behavioral responses (Eagly & Chaiken,

1993; Manstead, 1996; Olson & Zanna, 1993).

Several schools of attitude theories have emerged from research in the area. These

theories can be broken down into at least four classes based on the assumptions that each makes

about attitude formation. These four categories are: Learning theories, Expectancy Value

theories, Consistency theories and Attribution theories. Each of these groups has a basic

underlying principle in common and contains variations on the main principle; each also

presumes the same underpinnings of attitude development. Most of the classes of theories have

modern offshoots that are derived from the 1960s model by Rosenberg and colleagues

(Rosenberg, Hovland, McGuire, Abelson, & Brehm, 1960). Their model added moderators

between attitude and objects. They theorized that affect, cognition and behavior all act as filters

on attitude.

Expectancy Value theories are based on the assumption that in making decisions people

try to maximize their reward potential. This is called subjective utility. Subjective utility is

defined as the product of 1) the value of a particular outcome and 2) the probability that this

113

113

alternative will produce that outcome. Examples of Expectancy Value theories are those by

Fishbein (Fishbein, 1979), Edwards (Edwards, 1954) and Rosenberg and colleagues (Rosenberg

et al., 1960).

The long term goal of this SEM research is to develop a model based on expectancy

theory that addresses the impact of various attitudes on risk and resiliency factors. This initial

research examines the impact on Science Attitudes. This section was chosen as a beginning

point due to the fact that five years of research indicated that the Talking Talons program has a

positive effect on Science Attitudes.

Method The participants were students enrolled in schools in a rural area near Albuquerque, New

Mexico.

Analysis Plan

SPSS 23 for Windows were used in the preliminary analysis of the data set to score the

composite instrument and quizzes, identify non-participants and examine score distributions.

AMOS 23 was used to estimate the Structural Equation Model (SEM) solutions.

Analysis Sequence

A two-step approach was used for the structural equation modeling. First, estimations of

the measurement models for the latent constructs were made. Second, the structural

relationships between the latent constructs were tested. This method is recommended by many

researchers (e.g. (Anderson & Gerbing, 1988; Joreskog & Sorbom, 1993). The following

analysis plan is based on that developed by Mattern (Mattern, 1999).

114

114

1. Analyzed preliminary sample for non-participants, outlying scores, missing data and

incomplete information.

2. Estimated solutions for the Measurement Model.

3. Estimated solutions for the Saturated Model.

4. Estimated solutions for the model nested within the full model (Pruned Model).

5. Compared chi-square value differences to determine the better fitting model.

6. Evaluated the fit of the better fitting model.

Fit Indices

Amos 23 provides a plethora of fit indices for each model tested. Fit indices measure the

fit of the data used to the hypothesized model. As different indices have been developed to

measure different parts of a model’s fit, it is customary to report a cross section of these indices

from different categories.

Fit indices can be classified as either absolute or incremental. Absolute indices assess

how well the model being tested fits the sample data. Incremental (or comparative) fit indices

measure improvement in fit for a model when compared to a second model. This second model

is usually some form of a baseline model and is often the null model.

The most universally used index of fit is the chi-square measure (or CMIN). Chi-square

is an absolute fit index and as such tests the extent to which the data fit all aspects of the model

together including factor loadings, factor variances/covariance’s and error variances (Byrne,

2001). Although commonly used, chi-square is problematic due to its sensitivity to sample size.

Large sample sizes are needed in SEM in order to have distributions that are well behaved.

115

115

Unfortunately, these same large sample sizes cause an inflation of chi-square (relative to the

sample size), thereby making it difficult to achieve a reasonable probability for accepting the

model fit. Due to these problems inherent with chi-square, researchers have developed a

multitude of goodness of fit indices (Byrne, 2001). A summary of these fit indices is presented

in Table 1.

Table 38: Fit Indices

Fit index Type of

index

Compares or tests Acceptable or good fit Sample size

issues

2

(Chi -square

or CMIN)

Absolute Likelihood ratio test

statistic

Small, non-significant

values

Large sample

sizes inflate

chi square and

increase

probability of

a Type II error

Cmin/df

(Relative chi-

square)

Absolute Chi square (taking into

account the df)

Lower values are better

values < 3 are

acceptable (Kline,

1998).

TLI

(Tucker-Lewis

fit index)

Comparative Fit compared to the null

model

Values closer to 1 are better

values close to or

greater than .95 (for

large samples)

indicative of good fit

(Hu & Bentler, 1999)

Less affected

by sample

size, penalizes

for model

complexity

116

116

Fit index Type of

index

Compares or tests Acceptable or good fit Sample size

issues

CFI

(Comparative

Fit index, also

called the

Bentler CFI)

Comparative Fit compared to the null

model

Values closer to 1 are better

values > .90 are

acceptable (Bentler,

1992)

values close to or

greater than .95 are

acceptable (Hu &

Bentler, 1999)

values should > .90 to

accept the model

(Garson, 2001)

Takes sample

size into

account

PRATIO

(Parsimony

Ratio)

Comparative

Ratio of the degrees of

freedom in the test

model to the degrees of

freedom in the null

model

Higher values are better

values > .5 are

reasonable fit

(Byrne, 2001)

Rewards

parsimonious

models

Models

Four models were of special interest in this research. They included the Measurement

Model, the Saturated Model, the Pruned Model and the Null Model. Descriptions of these

models are provided below.

117

117

Measurement Model

Measurement models are confirmatory factor analysis models that define the

relationships between the observed (indicator) variables and the unobserved (latent) variables.

The latent variables represent underlying constructs (for example, “Risk and Resiliency”) that

are measured by the observed variables (for example, “pself” and “pmoral”). Each indicator

variable has associated with it an error term which is indicated by the small circle containing an

“e” and a number attached to it (e.g. “e19” represents the error associated with the observed

variable “Risk and Resiliency”). This error consists of both random error and error which might

be part of the measurement. In these measurement models, a specific indicator variable’s error

variance is not allowed to covary with any of the errors associated with other indicator variables

for the latent corresponding construct. Measurement Models (unlike structural models) cannot

include an error term for the latent variable. In Measurement Models the latent variable is

considered perfectly reliable and errors are associated with the indicator variables.

Using Joreskog’s method the measurement model for each of the latent constructs for

attitude and achievement was tested for model fit separately. Factor loading patterns relating the

latent variables to their observed indicator variables were examined for size and direction. Each

latent construct model was estimated repeatedly (with each indicator’s path to the latent set to

“1” in turn) so that each indicator served as the scaling factor for one of the estimation runs.

A baseline Measurement Model was then estimated including all latent constructs, which

were allowed to intercorrelate. Each latent construct was connected by a regression path to its

observed indicators. The Measurement Model was then estimated repeatedly with a different

observed variable used to scale each latent in order to examine possible resulting differences (see

118

118

Figure 1). Because outlying scores can unduly affect statistical results, the baseline Measurement

Model was estimated three times: with outliers included, with outliers eliminated, and with

outliers reset to three standard deviations from their mean in the direction of their scores

Figure 17: Measurement Model

119

119

Examination of Latent Variables in the Measurement Model

The Measurement Model for each latent construct individually was estimated with the

participants including the original outlier scores. Some participants have missing data and are

not included in portions of the model. Unfortunately if the student is missing one part of the

data to be used in the SEM they will not be included in the model as the software needs full data

sets to fit the model.

The referent factor path loading (the path set to the value of 1) for each latent construct

was rotated among the indicators for that construct. The range of results obtained for all of the

indicator variables for each of the latents is given below in Table 2. Each factor loading was

positive, statistically significant, and at least moderate in size.

Table 39: Factor Path Loadings for Latents

Data Range of

Standardized

Factor path

loadings

Attitude

toward

Educator

.38-.64

Attitude

toward

Animals

.53-.66

Science

Attitude .58-.77

Impact of Outliers on the Measurement Model

The preliminary Measurement Model was specified with the three latent variables

intercorrelated. It was estimated three separate times; once with outliers included, once with

120

120

outliers removed, and once with outliers reset to three standard deviations from the distribution’s

mean in the direction of the original score. The results from the Measurement Model indicated

little difference in fit, regardless of the status of the outliers. Therefore, each subsequent model

was estimated including all participants and their original scores.

Final Measurement Model

The final Measurement Model also was specified with the three latent variables

intercorrelated. This model had an acceptable fit to the data. Each factor path loading was

positive, statistically greater than zero and at least moderate in size (see Figure 2). The

unstandardized results for the Measurement Model are provided in Table 3. The Measurement

Model was also reestimated using different individual variables as referents. Once again, all

factor path loadings were statistically greater than zero

Table 40: Measurement Model Unstandardized Results**

Regression Weights: (Group number 1 - Default model)

Estimate S.E. C.R. P

tt6q2 <--- Attitude toward Educator 1.000

tt5q1 <--- Attitude toward Educator .884 .268 3.294 ***

tt8q4 <--- Attitude toward Educator 1.627 .665 2.447 .014

tt3q4 <--- Attitude toward Animals 1.000

tt8q1 <--- Attitude toward Animals .881 .240 3.673 ***

tt9q5 <--- Attitude toward Animals 1.006 .252 3.988 ***

tt7q1 <--- Science Attitude 1.000

rtt5q3 <--- Science Attitude 1.321 .308 4.286 ***

tt10q1 <--- Science Attitude 1.035 .264 3.922 ***

*Path loadings of observed referent variables were set to 1.

**Amos output estimated values. The critical ratio (CR) is equal to the path value divided by its standard error. A

value greater than 1.96 indicates statistical significance at p<. 05.

121

121

Table 41: Correlations Between Latent Variables in the Measurement Model

Estimate

Attitude toward Educator <--> Attitude toward Animals .813

Science Attitude <--> Attitude toward Educator .517

Science Attitude <--> Attitude toward Animals .655

Table 42: Measurement Model Covariances**


Attitude toward Educator .050 .023 2.178 .029

Attitude toward Animals .145 .058 2.513 .012

Science Attitude .273 .099 2.772 .006

e1 .108 .022 4.869 ***

e2 .058 .014 4.075 ***

e3 .806 .136 5.911 ***

e4 .250 .051 4.912 ***

e5 .230 .044 5.235 ***

e6 .186 .041 4.509 ***

e7 .364 .080 4.552 ***

e8 .335 .107 3.139 .002

e9 .582 .111 5.237 ***

**Amos output estimated values. The critical ratio (CR) is equal to the path value divided by its

standard error. A value greater than 1.96 indicates statistical significance at p<. 05.

***Amos output estimated values. p<.001

Saturated Model

In this fully saturated model Attitudes toward Educator and Attitudes toward Animals

served as an exogenous variable which impacted Science Attitude. This model allowed for

each latent upstream to directly impact the latents downstream. The model assigned a latent

residual to each of the endogenous latent constructs. The theoretical structural model tested in

this project draws largely from Expectancy Value theory. However, many of the latents and

links between them have support in other theoretical models as well.

122

122

Figure 18: Saturated Model

The regression weights for the relationships between the latent variables and their

observed indicators and the direct relationships between the latents are presented in Figure 3 and

in Table 6. Squared multiple correlations for the Saturated Model are presented in Table 7

Table 43: Regression Weights for Saturated Model**


Attitude toward Animals <--- Attitude toward Educator 1.381 .481 2.874 .004

123

123

Science Attitude <--- Attitude toward Educator -.109 1.104 -.099 .921

Science Attitude <--- Attitude toward Animals .951 .681 1.396 .063









tt10q1 <--- Science Attitude 1.035 .264 3.922 *** **Amos output estimated values. The critical ratio (CR) is equal to the path value divided by its standard error. A



Table 44: Squared Multiple Correlations for Saturated Model

Estimate

Attitude toward Educator .000

Attitude toward Animals .660

Science Attitude .430

tt10q1 .335

rtt5q3 .587

tt7q1 .429

tt9q5 .441

tt8q1 .329

tt3q4 .368

tt8q4 .142

tt5q1 .404

tt6q2 .317

Several of the variables in the structural model can impact each other through multiple

routes or paths. For example, a subsection of the Saturated Model is shown in Figure 3. Note

that Attitude toward Educator can impact Attitude towards animals directly (the thick line) and

indirectly through Attitude towards animals (the thin line). In other words, Attitude toward

Educator affects the Attitude towards animals directly (the thick line) and by changing Attitude

124

124

towards animals (dashed thin line), which then in turn affects the Science Attitude (thin line).

Attitude toward Educator total effect on Science Attitude is the sum of these two effects, the

direct and the indirect effects.

Figure 19: Direct and Indirect Effects

Table 45: Saturated Model Total Effects Standardized

Attitude toward

Educator

Attitude toward

Animals

Science

Attitude

Attitude toward

Animals

.813 .000 .000

Science Attitude .517 .693 .000

tt10q1 .299 .401 .579

rtt5q3 .396 .531 .766

tt7q1 .338 .454 .655

tt9q5 .539 .664 .000

tt8q1 .466 .573 .000

tt3q4 .493 .606 .000

tt8q4 .376 .000 .000

tt5q1 .636 .000 .000

tt6q2 .563 .000 .000

Table 46:Saturated Model Direct Effects

Attitude toward

Educator

Attitude toward

Animals

Science

Attitude

Attitude

toward

Educator

Science

Attitude

Attitude

toward

Animals

125

125

Attitude toward

Animals

.813 .000 .000

Science Attitude -.047 .693 .000

tt10q1 .000 .000 .579

rtt5q3 .000 .000 .766

tt7q1 .000 .000 .655

tt9q5 .000 .664 .000

tt8q1 .000 .573 .000

tt3q4 .000 .606 .000

tt8q4 .376 .000 .000

tt5q1 .636 .000 .000

tt6q2 .563 .000 .000

Table 47: Saturated Model Indirect Effects

Attitude toward

Educator

Attitude toward

Animals

Science

Attitude

Attitude toward

Animals .000 .000 .000


tt10q1 .299 .401 .000

rtt5q3 .396 .531 .000

tt7q1 .338 .454 .000

tt9q5 .539 .000 .000

tt8q1 .466 .000 .000

tt3q4 .493 .000 .000

tt8q4 .000 .000 .000

tt5q1 .000 .000 .000

tt6q2 .000 .000 .000

The fit indices for the Saturated Model are presented in Table 11. These values indicate a

reasonably good (but not excellent) fit of the Saturated Model to the data (see Table 1 of fit

indices).

Table 48: Fit Indices for Saturated Model

Model 2 df p 2/df TLI CFI PRATIO

126

126

Saturated Model

40.80 24 .017 1.67 .73 .85 .53

Pruned Model

The Pruned Model is nested within the Saturated Model. This trimmed model eliminates

the paths from the Saturated Model that were not statistically significant or close to significant .

The following non-significant paths (from the Saturated Model) were eliminated in the Pruned

Model: Attitude toward Education impact on Science Attitude

The remainder of the paths remain the same as in the Saturated Model. Standardized estimates

are presented in Figure 6.

Figure 4 Pruned Model Standardized Estimates

127

127

The regression weights for the relationships between the latent variables and their

observed indicators as well as the direct relationships between the latents are presented in Table

10. All of the regression weights were statistically significant since the insignificant paths were

128

128

eliminated from the Pruned model. Squared multiple correlations for the Pruned Model are

presented in Table 10.

Table 49: Unstandardized Regression Weights for Pruned Model**


Attitude toward Animals <--- Attitude toward Educator 1.361 .472 2.886 .004

Science Attitude <--- Attitude toward Animals .897 .284 3.159 .002









tt10q1 <--- Science Attitude 1.038 .264 3.931 ***

**Amos output estimated values. The critical ratio (CR) is equal to the path value divided by its standard error. A



Table 50: Squared Multiple Correlations for Pruned Model

Estimate

Attitude toward Educator .000

Attitude toward Animals .652

Science Attitude .423

tt10q1 .337

rtt5q3 .585

tt7q1 .429

tt9q5 .446

tt8q1 .331

tt3q4 .365

tt8q4 .141

tt5q1 .401

tt6q2 .320

Table 51: Total Effects for Pruned Model

Standardized Total Effects (Group number 1 - Default model)

Attitude toward

Educator

Attitude toward

Animals

Science

Attitude

129

129

Attitude toward

Animals

.807 .000 .000


tt10q1 .305 .378 .581

rtt5q3 .402 .497 .765

tt7q1 .344 .426 .655

tt9q5 .539 .668 .000

tt8q1 .464 .575 .000

tt3q4 .487 .604 .000

tt8q4 .376 .000 .000

tt5q1 .633 .000 .000

tt6q2 .565 .000 .000

Table 52: Direct Effects for Pruned Model

Standardized Direct Effects (Group number 1 - Default model)

Attitude toward

Educator

Attitude toward

Animals

Science

Attitude

Attitude toward

Animals

.807 .000 .000


tt10q1 .000 .000 .581

rtt5q3 .000 .000 .765

tt7q1 .000 .000 .655

tt9q5 .000 .668 .000

tt8q1 .000 .575 .000

tt3q4 .000 .604 .000

tt8q4 .376 .000 .000

tt5q1 .633 .000 .000

tt6q2 .565 .000 .000

Table 53: Indirect Effects for Pruned Model

Standardized Indirect Effects (Group number 1 - Default model)

Attitude toward

Educator

Attitude toward

Animals

Science

Attitude

Attitude toward

Animals

.000 .000 .000


130

130

tt10q1 .305 .378 .000

rtt5q3 .402 .497 .000

tt7q1 .344 .426 .000

tt9q5 .539 .000 .000

tt8q1 .464 .000 .000

tt3q4 .487 .000 .000

tt8q4 .000 .000 .000

tt5q1 .000 .000 .000

tt6q2 .000 .000 .000

The fit indices for the Pruned Model for both years are presented in Table 13. These fit

indices indicate a good level of fit of the Model to the data. (see Table 1 of fit indices).

Table 54: Fit Indices for Pruned Model


Pruned Model

40.814 25 .024 .76 ..88 .56

Comparison of Model Fit

Table 18 presents the fit indices for the three Models from this research plus that of the

Null Model. The Null Model represents a model in which all of the observed variables are

uncorrelated and as such represent a “baseline” lower limit for the fit of the model to the data.

Table 55: Comparison of the Fit of the Models


Null Model 74.1 27 2.746 .24 .60 .32

Measurement Model 40.08 24 .017 1.67 .73 .85 .53

131

131

Saturated Model

40.80 24 .017 1.67 .73 .85 .53

Pruned Model

40.814 25 .024 1.67 .76 ..88 .56

Table 15 below compares the 2 and the df for the models used in this research. A

statistically significant 2 for the df (as indicated by p< .05) indicates that the fit of the two

models were significantly different from each other. As can be seen in Table 14, the

Measurement, the Saturated and the Pruned Models all fit the data significantly better than the

Null Model. There is no statistically significant difference in fit between the Saturated Model

and the Pruned Model. Because there is no difference in fit and the Pruned Model is simpler

than the Saturated Model, it is selected as the best fitting model.

Model fit for the Measurement Model was similar for all levels of inclusion for outliers.

Results from the comparison of model fit indicated that the Measurement, the Saturated and the

Pruned Models exhibited better fit than the Null Model. There was no significant difference in fit

between the Saturated and the Pruned Models. Therefore the answer to first research question

posed in this study is that the Pruned Model is the better fitting of the two models because it is

simpler.

132

132

Direct and Indirect Effects

This section discusses the standardized estimates of the path coefficients for the Pruned

Model. Standardized path coefficients estimate how much a downstream variable would change

assuming a change of one standard deviation in the upstream variable. For example, Attitude

towards Animals in the Pruned Model had a direct impact on Science Attitudes of .81. This

value means that a change of one standard deviation in Attitude towards Animals would produce

a change of .81 standard deviations in Science Attitudes (controlling for the rest of the upstream

latent variables). Attitude toward Educator had an indirect effect on Science Attitudes use of .53.

This value means that a change of one standard deviation in Attitude toward Educator would

produce a change of .53 standard deviations in Science Attitudes through its impact on other

latent variables that are upstream, in this case Attitudes toward Animals.

According to Kline (Kline, 1998); standardized path coefficients with values of less than

.10 can be interpreted as small effects, values of around .30 can be interpreted as medium effects

and values above .50 can be interpreted as large effects. All the effects in this model were larger

than .65, indicating large effects.

Summary of Outcomes

Every increase of one standard deviation in Attitude toward Animals produced a Direct effect

of .65 standard deviations in Science Attitude. Every increase of one standard deviation in

Attitude toward Educator factors produced a Direct effect of .81 standard deviations on Attitude

towards Animals and an Indirect effect of .53 standard deviations.

133

133

Variance of the Latent Variables

The magnitude of the variance for a latent variable indicates how much of the latent

variable's variance is accounted for by its upstream latent variables. The amount of variance for

each of the endogenous latent variables accounted for by its upstream latent predictors was 42%

for Science Attitude and 65% for Attitude toward Animals.

Global Summary


for this SEM is indicates that high level of confidence in the model. The sample size is small.

A second and simpler model (the Pruned Model) that eliminated the non-significant paths in

the Saturated Model also exhibited a close fit to the data









134

134








changing the student’s Attitudes toward Animals. This is a complex outcome, indicating that the

animals are the crux of the change in science attitudes for the students, but that the educators are

indirectly influencing science attitudes by working through impact on attitudes toward animals.

135

135

Bibliography

Allport, G. (1935). Attitudes. In C. Murchison (Ed.), A handbook of Social Psychology (pp. 789-

844.). Worcester, Mass.: Clark University Press.

Anderson, J. C., & Gerbing, D. W. (1988). Structural equation modeling in practice: A review

and recommended two-step approach. Psychological Bulletin, 103(3), 411-423.

Bentler, P. M. (1992). On the fit of models to covariances and methodology to the bulletin.

Psychological Bulletin, 107(238-246).

Byrne, B. (2001). Structural equation modeling with AMOS: Basic concepts, applications and

programming. New Jersey: Lawrence Erlbaum Associates, Inc.

Eagly, A. H., & Chaiken, S. (1993). The psychology of attitudes. Ft Worth, TX, USA: Harcourt

Brace Jovanovich College Publishers.

Edwards, W. (1954). The theory of decision making. Psychological Bulletin, 51, 380-417.

Fishbein, M. (1979). A theory of reasoned action: Some applications and implications. Nebraska

Symposium on Motivation, 27, 65-116.

Garson, G. (2001). PA 765 Statnotes: An Online Texbook. Available:

http://www2.chass.ncsu.edu/garson/pa765/structur.htm [2001, January 31, 2000].

Hu, L.-t., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis:

Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1-55.

Joreskog, K. G., & Sorbom, D. (1993). LISREL 8: Structural equation modeling with the

SIMPLIS command language. Chicago: Scientific Software International.

Kline, R. B. (1998). Principles and practice of structural equation modeling. New York: Guilford

Press.

136

136

Manstead, A. S. R. (1996). Attitudes and behaviour, Applied social psychology (pp. 3-29).

London, England UK: Sage Publications.

Mattern, N. (1999). Relationships between attitudes toward and achievement in science for rural

middle school students : patterns across gender., University of New Mexico,

Albuquerque.

Olson, J. M., & Zanna, M. P. (1993). Attitudes and attitude change. Annual Review of

Psychology, 44, 117-154.

Rosenberg, M. J., Hovland, C. I., McGuire, W. J., Abelson, R. P., & Brehm, J. W. (1960).

Attitude organization and change: An analysis of consistency among attitude components

(Yales studies in attitude and communication Vol III ). New Haven, CT: Yale Univer.

Press.

talking talons youth leadership evaluation report · talking talons youth leadership report school...

Documents