k-5 stem grantsosit.nv.gov/uploadedfiles/ositnvgov/content/reports...k-5 stem grants seek to...
TRANSCRIPT
K-5 STEM GrantsEvaluation and Outcomes Report
According to the research, one-third ofboys and girls lose an interest inscience by the fourth grade*
A child’s interest in STEM is largelyformed by the time he or she reachesupper elementary and middle school.*
Early exposure to STEM, especially for girls, makes children more likely to succeed in science and pursue STEM fields in college.*
… just 38% of Nevada’s elementary schools report offering STEM during the school day.**
… if the State’s goal is to increase the number of students participating in STEM programs in middle and high schools that prepare them for success in post-secondary STEM degrees, research suggests STEM concepts should first be introduced at the elementary level.***
*Daugherty, Michael K.; Carter, Vinson; and Swagerty, Lindsey (2016) "Elementary STEM Education: The Future for Technology and Engineering Education?," Journal of STEM Teacher Education: Vol. 49 : Iss. 1 , Article 7.**According to a statewide survey of STEM practices conducted in May, 2016 by the NV STEM Advisory Council.***DeJarnette, N. K. (2012). America’s children: Providing early exposure to STEM (science, technology, engineering and math) initiatives. Education, 133(1), 77–84.
K-5 STEM Grants seek to increase access to high-
quality STEM programs in elementary schools in
order to:
cultivate and maintain an interest in STEM in
younger grades, and
Lay the foundation for the skills needed for a
career pathway to success in the New Nevada.
This grant program aligns with four key
strategies identified in the Nevada State STEM
Strategic Plan:
To increase the prevalence of evidence-based, high-quality formal and informal STEM practices and programs in Nevada’s elementary schools.
To increase the use of hands-on, evidence-based, experiential STEM learning in grades K-5.
To increase the percentage of elementary schools that teach science for three-plus hours per week.
To increase interest in, awareness of, and achievement in the subjects of science, technology, engineering, and mathematics in grades K-5, particularly amongst demographic groups that are traditionally underrepresented in STEM.
osit.nv.gov | @sciNVtech | stemhub.nv.gov
K-5 STEM Grants
Program Goals
K-5 STEM
Classroom Grant
K-5 STEM
Program Grant
Targeted at individual classrooms Funds innovative, creative approaches
to teaching STEM concepts Funded with private dollars 12 Classroom Grants were awarded
Targeted at the grade-level or school-level Funds purchase of high quality, vetted
programs on the STEM Advisory Council’s List of Recommended STEM Programs
Funded with private dollars 6 Program Grants were awarded
osit.nv.gov | @sciNVtech | stemhub.nv.gov
K-5 STEM Grants
2 Grant Opportunities
Awarded Schools
K-5 STEM
Classroom Grant
K-5 STEM
Program Grant
Bordewich Bray Elementary School-Carson City
Mark Twain Elementary School-Carson City
Bordewich Bray Elementary School-Carson City
Oasis Academy- Fallon Fernley Elementary School- Fernley Riverview Elementary School- Dayton Spanish Springs Elementary School-
Reno Sepulveda Elementary School- Reno Peavine Elementary School- Reno Peavine Elementary School- Reno Gomes Elementary School- Reno Mt. Rose Elementary School- Reno
Douglas County Elementary Schools Carson City Elementary Schools Oasis Academy- Fallon Yerington Elementary Schools- Yerington Hugh Gallagher Elementary School-
Virginia City Coral Academy of Science- Reno
osit.nv.gov | @sciNVtech | stemhub.nv.gov
K-5 STEM Grants
Total Student Impact: 5,136 students attending 24 elementary schools will benefit annually from greater access to STEM curriculum and materials.
2,046 students will benefit each year from new and innovative STEM equipment and lessons plans.
3,090 students each year will have access to high-quality STEM curriculum from programs that increase interest in STEM careers and academic achievement.
82 teachers are benefiting from high-quality STEM teaching materials and equipment.
Total Funding: $134,129*
1.378
3.556
STUDENT CONTENT
KNOWLEDGEPretest Posttest
10.8
13.24
STUDENT INTEREST IN STEM (3-5)Pre-Survey Post-Survey
12.89
13.03
STUDENT INTEREST IN STEM (K-2)
Pretest Posttest
*External program evaluation conducted by the Raggio Research Center at the University of Nevada Reno: $15,871
Summary of Grant Outcomes and
Evaluation
osit.nv.gov | @sciNVtech | stemhub.nv.gov
K-5 STEM Grants
Full K-5 STEM Evaluation Report
K-5 STEM Grant
Evaluation Report
Final Report
Submitted to: Brian Mitchell, Office of
Science, Innovation, and Technology
University of Nevada, Reno
1664 N. Virginia, Reno, NV 89557-0299
Catherine Pozarski-
Connolly, Ph.D.
Camille T. Stegman, Ph.D.
Raggio Research Center
for STEM Education
12.31.18
2
Table of Contents
Introduction .................................................................................................................................................. 3
Summary of Results ...................................................................................................................................... 4
Data Collection and Statistical Analyses ....................................................................................................... 4
Student Interest in STEM .............................................................................................................................. 5
Kindergarten through Second Grade Survey ............................................................................................ 5
Third Through Fifth Grade Survey Results ................................................................................................ 8
Student Content Knowledge ....................................................................................................................... 14
Teacher Sense of Self-Efficacy .................................................................................................................... 15
Barriers to STEM Education in the Elementary Setting .............................................................................. 16
Teacher Perceptions of Changes to Classroom Culture .............................................................................. 20
Conclusions ................................................................................................................................................. 20
Appendix A .................................................................................................................................................. 22
3
K-5 STEM Evaluation Report
The K-5 STEM Grant from the Governor’s Office of Science, Innovation,
and Technology (OSIT) was a grant program, funded by the
Pennington Foundation to assist in getting STEM programs and
curriculum into classrooms across Nevada.
Introduction Drs. Catherine Pozarski-Connolly and Camille T. Stegman were Co-Investigators and Dr.
David Crowther supervised the evaluation of the K-5 STEM Education Program Grant (K-5
STEM). K-5 STEM was announced in November 2017. “The purpose of the program was
to increase the prevalence of evidence-based, high-quality formal and informal STEM
practices and programs within Nevada’s elementary schools.” (OSIT, 2017) The funding
came in two forms; 1) a classroom award for up to $1,500 and 2) a program award for up to
$20,000.
K-5 STEM had four over-arching goals that guide its purpose. The goals focused on
supporting equitable access to quality of STEM programs in elementary schools to prepare
students for STEM careers in the New Nevada. More specifically, the goals were:
Goal 1: To increase the prevalence of evidence-based, high-quality formal and informal STEM practices and programs in Nevada’s elementary schools.
Goal 2: To increase the use of hands-on, evidence-based, experiential STEM learning in Grades K-5.
Goal 3: To increase the percentage of elementary schools that teach science three-plus hours per week.
Goal 4: To increase interest in, awareness of, and achievement in the subjects of science, technology, engineering, and mathematics in grades K-5, particularly amongst demographic groups that are traditionally underrepresented in STEM. (OSIT, 2017)
Exhibit 1 provides an overview of the program and classroom awardees requirements to
receive K-5 STEM funds.
4
Exhibit 1: K-5 STEM Requirements
Program Awardees Classroom Awardees
o Pre-and post-STEBI-A
o Pre-and post-STEM Interest Student Survey
o Pre-and post-classroom Assessments from the
Program or teacher developed
o Pre-and post-STEBI-A
o Pre-and post-STEM Interest Student
Survey
Carson, Douglas, Storey, Lyon, and Churchill
County district schools received funding
Carson, Washoe, Churchill, and Lyon
County teachers received funding
Programs purchased include: STEM in Action,
Project Lead the Way, Desert Research Institute
(DRI) Green Boxes – STEM Snacks, Kinderlab
Robotics
Classroom materials purchased include:
Sierra Nevada Journeys Curriculum, DRI
Snow Collection Kits, Full Option Science
Systems (FOSS), ROK Blocks, Robotics and
Coding, STEM Supplies, Hydro Geology
Stream Table, DRI Green Boxes
Summary of Results
K-5 STEM provided funding to districts, schools, and classroom teachers. The evaluation
found increases in student interest in STEM subjects in both K-2 and 3-5, student
awareness of engineering as a useful skill at the 3-5 level, and student content knowledge.
The evaluation did not find evidence that teacher self-efficacy increased or that every
barrier to STEM implementation was removed; however, teachers expressed that there
were changes to their classroom culture, including amounts of time spent teaching STEM,
student engagement, student collaboration and discussion, problem solving skills, and
overall excitement about engaging in STEM. The following sections provide more
information about each of these results.
Data Collection and Statistical Analyses
Program implementation was less than a school year. In several cases, program materials
were not received until late spring of 2018 allowing for only six to eight weeks of program
use. Some implementations took place during the first two months of the 2018 school year.
This may have been a contributing factor to the lack of change in teacher self-efficacy.
Changes to self-efficacy require time, professional development, and high-quality materials
(Darling-Hammond, 1995; Garet, Porter, Desimone, Birman, & Yoon, 2001; Reiser, 2013;
Wilson, 2013).
Both statistical and teacher perceptions were used to conduct this evaluation. Two surveys
utilizing Likert-style data were created for the grant, one for Grades K-2 students and one
for Grades 3-5 students (Appendix A). Due to the nature of Likert data, an independent
samples Mann-Whitney U was applied to the data to determine if student attitudes changed.
A second analysis, chi-square, was applied to items requiring the student to select more than
one option and to items that could not be coded with a value, such as categories. The results
5
of the analysis are represented as a p-value, or probability value. Education tends to use a p-
value of 0.05 or less to show that the differences in the data sets were not the result of error,
but rather the result of some intervention. Additionally, the effect size that resulted from an
analysis aids in determining how large the difference was, or how big of an impact the
intervention made in respect to the scores received.
Teacher perceptions were gathered during informal interviews. Each of the five schools that
received program funding were required to participate in pre- and post- focus group
sessions. Each session lasted from 30-75 minutes and included nearly all participants that
received program supplies and/or training. On a couple of occasions an educator was
absent from a pre- or post- focus group session, but overall the rate of participation in the
pre- and post- focus groups was above 97%. All interviews were transcribed and coded
leading to several themes that were consistent across all the participants.
Student Interest in STEM
A STEM interest survey was created by the Raggio Research Center for the purpose of this
evaluation. Two levels of the survey were created, one for kindergarten through second
grade students and one for third through fifth grade students. The purpose of the survey was
to determine if student interest in school or in STEM subjects changed after classroom
instruction occurred utilizing the materials or kits purchased through grant funds. Teachers
who received funds or were part of the program level funding were asked to administer the
interest survey to all students in their classrooms before any instruction utilizing purchased
STEM materials and after instruction using the STEM materials. The survey for each grade
band was available on SurveyMonkey.com and as a paper copy. Teachers could administer
the survey in either format, and teachers in the kindergarten through second grade could
read the survey to their students. Student responses were numerically coded and entered into
a spreadsheet for statistical analysis. A copy of the student STEM interest surveys can be
found at the end of this report (Appendix A).
Kindergarten through Second Grade Survey
The kindergarten through second grade survey included six selected response questions
utilizing a Likert format for five questions and one single response item. The items used
pictures and simple verbiage to account for the age and developmental level of the students.
Two analyses were applied to the data. Questions 1-5 were scored on a 3-point Likert scale
with a total possible score of 15, indicating a student feeling the happiest about STEM-
related activities in class, and the lowest score of 5, representing a student who feels unhappy
about doing STEM-related activities in class. Students who colored in more than one answer
or who did not complete the question were eliminated from the analysis. STEM survey data
were collected and analyzed for 576 students in the pretest and for 522 students in the
posttest. Results of the assessment indicated an increased attitude towards STEM subjects
from pretest to posttest with a small effect size (mean pretest = 12.89; mean posttest =
6
13.03; p = 0.032; r = 0.073). A post hoc evaluation determined the power of this assessment
to predict changes was greater than 99%, meaning the results of the Mann Whitney U were
reliable. These results of the analysis suggest that student attitudes towards STEM subjects
and activities in school can be influenced by teacher actions. Students seemed to enjoy
school more as one teacher reported, “It’s great for the kids. It’s not like your goal [for]
reading, math. They love it and if we can expose them more, they’re actually learning when
they think they’re not, and it’s fun.” Since the surveys were given before and after instruction
using the STEM materials or kits purchased utilizing funds from the grant, it could be noted
that providing materials teachers need to teach STEM subjects in research-based, inquiry
formats can increase a student’s attitudes, interest, and enjoyment towards these disciplines.
Table 1: K-2 Attitudes Comparison
N Mean SD W p r
Pre- Post- Pre- Post- Pre- Post-
K-2 Attitudes 576 522 12.89 13.03 2.33 2.51 139394 0.032* 0.073
Figure 1: K-2 Attitudes Comparison
The second analysis was applied to Question 6 of the survey. The question asked students to
select the activity they liked best: science, computers, math, or building. A chi-square test
was applied to the data to determine if the number of students selecting one subject over
another changed from pretest to posttest. Results indicate no significant changes in which
activity students liked best from pre-survey to post-survey (N = 1045; X2 = 3.117; p =
0.374). See Table 2 for additional data resulting from the analysis.
The results of the analysis, however, indicated students prefer activities involving computers
the best and math the least, with both science and building in the middle. Teachers and
educational partners could capitalize on these natural interests by building technology pieces
12.89
13.03
PRE-SURVEY POST-SURVEY
K-2 Student Attitudes in STEM
Average Score
7
into their instruction in meaningful ways. Although students do prefer activities that involve
computers that is not to say that computer programs should take the place of authentic
investigations, but the programs should instead serve as a tool to complete investigations in
STEM subjects. Science and engineering are closely linked; the Next Generation Science
Standards (NGSS) require teachers at all grade levels K-12, to integrate them in authentic,
meaningful ways into daily instruction. The NGSS go beyond content acquisition by
requiring students to perform science and engineering tasks that become increasingly more
complex as students progress through grade levels. Students starting in kindergarten are
naturally inquisitive and providing materials, programs, and training to teachers that foster
that natural curiosity piques student interest and desires to solve problems and explore
STEM topics that would not otherwise be done through traditional didactic methods. The
contingency table below (Table 3) provides percentages of students who selected each
activity as their favorite for both pre-survey and post-survey.
Table 2: Contingency Table - Favorite STEM Subject
Percentage Within
Column Overall
Pre- Post-
Science 29.5 25.7 27.7
Computers 39.6 38.8 39.2
Math 9.9 10.8 10.3
Building 21.0 24.7 22.8
Figure 2: K-2 Favorite STEM Subject
29.5 29.6
9.9
21
25.7
38.8
10.8
24.727.7
39.2
10.3
22.8
SCIENCE COMPUTERS MATH BUILDING
K-2 Favorite STEM Subject
Pretest Posttest Overall
8
Third Through Fifth Grade Survey Results
Students in third through fifth grade received a Student Interest and Attitudes about Science
and STEM Survey consisting of 12 questions. The first seven items in the survey were single
response, Likert style items encompassing how students felt they could perform in school
and in STEM subjects. The remaining five items were select all that apply responses
consisting of topics surrounding what subjects and careers students may want to pursue in
secondary education settings and later in life when they consider career choices. Students
received the survey as a pretest before any instruction occurred utilizing materials purchased
with funding received from the grant, and as a posttest after instruction with the purchased
materials occurred. Results of the two testing administrations were compared to determine if
changes occurred, as a possible result from the use of STEM materials and kits purchased
through the grant.
As with the Grades K-2, a Mann Whitney U was applied to the data to determine if
differences in students’ perceptions exist between the pre-survey and post-survey. Data from
984 pre-surveys and 954 post-surveys were analyzed and resulted in a post hoc power greater
than 99% suggesting accurate results from the analysis. Results from Questions 1-7
surrounding students’ own beliefs about their abilities to succeed in school and in STEM
subjects indicate an increase in overall perceptions and interest in school and STEM subjects
or activities from 10.08 points to 13.24 points with a large effect size (mean pretest = 10.08;
mean posttest = 13.24; p < 0.001; r = 0.702). This increase of 31.5% in student self-
perceptions suggest that when students are engaged in STEM their interest in STEM careers
increases. STEM education is important for the next generation. As one teacher stated, “It’s
a circle and you always want to keep making things better so that when they [the student] go
into that profession, and thinking about STEM, they know that it's going to be a lot of fails
before there’s [sic] successes.” Students’ perceptions and interest toward their abilities in
school overall and in STEM subjects increased to the point that would suggest providing
these materials and funding STEM activities could make a lasting impact on student’s
interest in school and STEM careers. Additional data for this analysis can be found in Table
3.
Table 3: Student Perceptions Towards STEM and School Abilities N Mean SD
W p r Pre- Post- Pre- Post- Pre- Post-
3-5 Perceptions 984 954 10.08 13.24 2.66 1.43 139918 < 0001* 0.703
9
Figure 3: Student Perceptions About STEM and School
Questions 8-12 required students to select the statement and select all that applied. The
frequency of each choice being selected for each question was compared from pre-survey to
post-survey to determine if differences in what students selected changed over the time the
materials purchased through the grant were used. Each question was analyzed separately
using a chi-square analysis. A brief discussion of the results for each question follows with
supporting tables.
Question 8 asked students to select which STEM subjects interested them in school. Results
of the chi-square analysis suggested no changes were found between pre-survey and post-
survey (X2 = 2.046, p = 0.563). Overall students were most interested in science with
technology and math tied for second and engineering was last. This lack of interest in
engineering could be the result of a lack of understanding of what engineering is. The old
Nevada State Science Standards did not include engineering, and before the adoption of the
NGSS most teachers had no pedagogy or knowledge about engineering or how to teach it.
The adoption of the NGSS has clear performance expectations for students in engineering;
however, a lack of training in the NGSS and engineering specific pedagogy have left many
teachers struggling with how to implement engineering in their classrooms. See Table 4 and
Figure 4 for additional details.
10.8
13.24
PRE-SURVEY POST-SURVEY
Student Perceptions about STEM and School
Average Score
10
Table 4: Question 8
Percentage Within
Column Overall
Pre- Post-
Science 30.5 28.9 29.7
Technology 23.9 25.0 24.4
Engineering 20.4 21.6 21.0
Math 25.2 24.6 24.9
Figure 4: Question 8
Question 9 asked students to identify which STEM subjects helped them to be successful in
school. Results of the chi-square analysis suggested that by the post-survey significantly more
students thought engineering could help them succeed in school (X2 = 12.87, p = 0.005).
Although engineering was not the highest ranked STEM subject in either the pre-survey or
post-survey, it was the only subject that had significant increases in students selecting that
option from pre-survey to post-survey. This suggested students who encountered lessons
utilizing or teaching engineering might see the value it can bring to their school careers. As
one teacher stated, “Besides the engagement and the teamwork, they're learning what
engineering means, and I’m loving that they are transferring it to other genres.”
Additionally, a decrease in students who selected math as a subject that could help them
succeed was found from pretest to posttest. It is difficult to determine why less students
selected math in the post-survey compared to the pre-survey; however, the results may point
to a disconnect between math and its application in real-world situations.
30.5
23.9
20.4
25.2
28.9
25
21.624.6
29.7
24.4
21
24.9
SCIENCE TECHNOLOGY ENGINEERING MATH
Student Interest in STEM Subjects
Pre-survey Post-survey Overall
11
Table 5: Question 9
Percentage Within Column Overall
Pre- Post-
Science 27.7 26.0 26.8
Technology 17.6 19.6 18.6
Engineering 14.0* 17.4* 15.7*
Math 40.8* 37.0* 38.9*
Note.* Denotes significant differences between presurvey and postsurvey
Figure 5: Question 9
Question 10 asks students to select which STEM subjects they would like integrated into
their future careers. No differences in frequencies were found between pre-survey and post-
survey (X2 = 2.215; p = 0.529; r = 0.026). The most selected option for both pre-survey and
post-survey was technology and the least selected option was engineering.
Table 6: Question 10
Percentage Within
Column Overall
Pre- Post-
Science 25.3 23.7 24.5
Technology 27.1 29.3 28.2
Engineering 22.9 22.8 22.8
Math 24.7 24.3 24.5
27.7
17.614
40.8
26
19.617.4
37
26.8
18.615.7
38.9
SCIENCE TECHNOLOGY ENGINEERING MATH
Selected STEM Subjects for School Success
Pre-survey Post-survey Overall
12
Figure 6: Question 10
Question eleven asked students to select which professions they would like when they grow
up out of 12 options ranging from STEM professions to teachers, actors, and musicians. No
differences were found from pre-survey to post-survey in student selected professions (X2 =
6.202; p = 0.860, r = 0.040). The most selected options were athlete, engineer, teacher, and
doctor.
Table 7: Question 11
Percentage Within
Column Overall
Pre- Post-
Doctor 11.4 11.2 11.3
Actor 9.1 8.6 8.8
Biologist 4.0 4.3 4.2
Veterinarian 10.1 10.0 10.1
Musician 7.6 7.7 7.7
Computer Specialist 7.6 7.2 7.4
Athlete 15.0 14.6 14.8
Engineer 12.4 12.2 12.3
Teacher 11.5 11.1 11.3
Accountant 1.4 1.9 1.6
Physicist 1.9 2.8 2.3
Technology Specialist 7.9 8.3 8.1
25.327.1
22.924.723.7
29.3
22.824.324.5
28.2
22.824.5
SCIENCE TECHNOLOGY ENGINEERING MATH
STEM CareersPre-survey Post-survey Overall
13
Figure 7: Question 11
Question 12 asked student to select their favorite part or subject of the school day. No
significant differences were found from pre-survey to post-survey (X2 = 17.94; p = 0.056, r
= 0.047). P.E., art, and math were selected most often, and social studies and library were
selected least often.
Table 8: Question 12
Percentage Within
Column Overall
Pre- Post-
P.E. 13.2 12.1 12.6
Math 9.2 8.7 8.9
Science 9.0 9.0 9.0
Reading 7.8 8.4 8.1
Recess 11.1 9.4 10.2
Computers 9.1 8.6 8.8
Art 12.6 12.7 12.7
Writing 6.3 6.8 6.6
Library 5.7 7.1 6.4
Social Studies 5.6 5.7 5.7
Lunch 10.6 11.5 11.1
Career Choices
Pre-survey Post-survey Overall
14
Figure 8: Question 12
Student Content Knowledge Teachers who received program level funding were required to create or administer a content test to their classrooms as well as the survey. The assessment only needed to cover the content the students engaged in. Comparing results from different assessments required the data to be transformed into a Z score for comparison using an independent samples t-test. Results of the analysis suggest a significant improvement from pretest to posttest with a very large effect size (p < 0.001; d = 2.076). This suggests the methods employed by the teachers were sufficient for students to access and comprehend the content being covered in the purchased kits. One teacher stated that the program helped the students who frequently struggle “One of my lower kids that I was not expecting much growth [on the content test] from, zero to three, which would be considered, for me that I would consider that achieving what I wanted him to achieve. And they loved it, to me that was the best part.” See Table 9 for additional details.
Table 9: Content Results N Mean SD
t p D Pre- Post- Pre- Post- Pre- Post-
Content 532 478 1.378 3.556 0.866 1.220 34.35 < 0001* 2.076
Favorite Part of School Day
Pre-survey Post-survey Overall
15
Figure 9: Content Results
Teacher Sense of Self-Efficacy
The Science Teaching Efficacy and Beliefs Instrument for in-service teachers (STEBI-A)
was used to determine if teachers sense of self-efficacy or outcome expectancies for student
performance in science education changed from pretest to posttest. The assessment
measures two different constructs (1) self-efficacy, and (2) outcome expectancies. All
participating teachers were asked to take the STEBI-A anonymously through
SurveyMonkey.com before they received any materials or professional development
purchased with funds through the grant, and after they taught at least one lesson using
materials purchased through the grant. Eighty-two teachers took the pretest, of whom 56
were involved in program-level funding, and 26 received classroom level funding. Fifty-six
teachers took the posttest, of whom 24 were involved with program level funding and 32
received classroom level funding. A post hoc power analysis was conducted resulting in a
power of 0.88 for a Mann-Whitney U analysis. The high power suggested the results are
reliable and not due to error. No changes were found in either construct. This could mean
that teachers already held high expectations for their students and those expectations were
maintained throughout the program. One teacher stated, “I feel like they’ll just be really
awesome problem solvers in the future and being able to solve any problem they encounter”
when asked about the expectations of the program materials. It could also mean that
teacher expectations for their students were low and the implementation of the programs did
not change their ideas. “I personally don’t have a lot [of expectations] yet.” stated another
teacher. The lack of change could be the result of a lack of training in the materials or kits
purchased. Although materials were provided, an integral component to change self-efficacy
and beliefs about student performance revolves around preparing teachers to adequately use
the materials and feel comfortable diving into resources that may not have been used before.
1.378
3.556
PRETEST POSTTEST
Content Test
Score
16
In order for teachers to increase their self-efficacy in teaching science or increase their
expectancies for students in science education, teachers must be provided with more than
just materials; they also need the training as well as time to implement and use the resources
provided. However, even with the lack of training, teachers were still eager to implement the
programs. Many commented about their reason for teaching STEM. “I want to educate
myself as much as I can, so I feel my confidence build” stated one teacher. Many of the
teachers stated they did not feel comfortable teaching science or technology and tended to
gravitate toward training activities to increase confidence. Some teachers reported that they
were willing to restructure their day to fit STEM implementation into their busy schedules.
“My kids couldn’t wait for science. We chose to do science all week and usually I only do it
two to maybe three times during the week.” See Table 10 for additional data.
Table 10: STEBI-A N Mean SD
W p r Pre- Post- Pre- Post- Pre- Post-
Self-Efficacy 82 56 28.85 26.93 7.61 7.51 2661.5 0.113 0.159
Outcome Expectancies 82 56 29.11 27.82 5.57 5.50 2617.0 0.164 0.140
Figure 10: STEBI-A
Barriers to STEM Education in the Elementary Setting
The pre- and post- focus group sessions uncovered a few barriers that were consistent
amongst the groups. Issues discussed included lack of professional development (PD), lack
of time, and lack of materials. All three components seemed to be required to assist teachers
in becoming more comfortable with science and STEM teaching. The program funding
addressed the lack of materials and attempted to address the lack of PD, in that funds could
be used to pay for PD. Unfortunately, these barriers did not change from pre- to post-
sessions. Participants gave the following reasons for the lack of STEM education in the
28.8529.11
26.93
27.82
SELF-EFFICACY OUTCOME EXPECTANCIES
STEBI-A
Pretest Posttest
17
elementary setting. The perceived barriers were time, science/STEM content knowledge,
science/STEM professional development and training, technology availability, and student
abilities in technology and cooperative tasks.
Time was mentioned by nearly every participant in both the pre- and post- focus group
sessions. “There isn’t enough time” was stated by at least one member of every group with
multiple affirmations by others. One teacher reflected that there is a mandated time for
science in her district, but it does not allow enough time to really get into science problems.
Fellow participants noted that there is a mandated time for explicit subject instruction,
including science, and they agreed that the mandated science time was insufficient.
However, for teachers who had freedom to change the structure of their school day, the
materials assisted them with integrating their curriculum, and therefore, finding more time in
the school day. One teacher explained how she was able to integrate her subjects more
effectively, “We've been in [the materials] exploring it, talking about how to take care of the
earth in all different ways and with art and writing and different things, not just the kit.”
The structure of a school day and required time for other subjects meant that, for some
districts, science and STEM education time was often relegated to the end of the day or one
day a week. This led to situations where the allotted time could be cancelled or too much
time would go by between lessons, leading to wasted time refreshing previous information.
When discussing the use of the supplemental program materials, a teacher stated, “We don't
necessarily have to teach it [STEM]. It just kinda goes along with what we’re teaching.
Because we do it Friday, and then they don’t again until the next Friday, so sometimes
there’s a gap.” This statement reflected the teacher’s thinking that, not only is STEM
perceived as a supplemental activity, but also that only participating in STEM education once
per week does not allow students to develop a good understanding of the topic. She
continued to explain that though the students perform well on the STEM activity, because
so much time passed between lessons, there was confusion the following week on the same
topic. One difficulty cited by a group was that the program was only available to be used
during a specific time once per week because the school could not afford enough kits for all
teachers. The educators in this group perceived that there was not enough time in the day or
days in the week for science and STEM to be an everyday part of the curriculum. Many
districts require specific time amounts for explicit skill instruction per subject area. This
practice does not allow for truly integrated instruction which would provide the needed time
for the teaching of science and STEM.
Another aspect of the lack of time was found in the actual instructional time and preparation
time allotted for the program lessons. Many participants complained that the guides for the
lessons were frequently incorrect or inconsistent when giving directions on preparation or
instructional time. This complaint was most frequently cited among groups that had little
training in the materials. Some teachers felt that the lessons took much longer to prepare and
institute which led to some frustration. For example, one teacher stated, “I felt like I could
18
have done it [one lesson] in five minutes and then other lessons, it took us 45 minutes. And
then we still didn't get to all of what they wanted on the thing [lesson].” She continues with
“there was a lot of figuring things out still I feel like with them [students], but it's a new kit.”
These frustrations could have been eliminated with additional training and familiarity with
the programs. It should be noted that teachers tend to set aside specific amounts of time for
science, either mandated or by choice. When a lesson is shorter or longer than expected the
teacher must quickly adapt, this can lead to stress and unwillingness to use the materials.
Even though instructional and preparation time were still considered barriers to STEM
education, nearly all the educators agreed on the importance of science and STEM education
and did their best to make time for the program. One teacher’s positive remark was that the
program led to “more science in the classroom. Which is always great.” Other teachers made
positive comments regarding their students, such as, “They loved it” and “They're engaged;
they're absolutely engaged.” These types of positive feedback showed teachers’
understanding of the importance of STEM education and the willingness to teach STEM
when given the materials and time.
Nearly all the teachers perceived that their personal level of content knowledge and training
was insufficient to feel comfortable during STEM time. Some teachers felt that they had
received adequate training in science during their teaching programs and many felt they had
received some science in college. One teacher mentioned, “It was just one class [science]
and that was it. So I could have used more.” This general lack of content knowledge
appeared to impact teachers’ comfort level in teaching science/STEM. Most of the
educators perceived that their districts had done little to help them in science/STEM
training and were left on their own to figure it out. One teacher stated, “Not being a science
teacher,… I feel like sometimes there’s not enough to know the direction that [the program]
is going, and I Google some stuff.” When discussing science/STEM training “I haven’t had
any [training]” was a frequent answer. This feeling was less prevalent in districts or schools
that had a dedicated science coordinator or specialist, who were usually praised for providing
the participants with some assistance. However, when discussing training on the Nevada
Academic Content Standards for Science (NVACSS) nearly all the participants thought that
the Nevada Department of Education (NDE) had done a half-day of training a few years
ago. The underlying feeling seemed to be that more training needs to be provided by the
state or districts. The post-focus group answers did not change the perspective on STEM
training, as most of the equipment and materials did not utilize professional development
(PD) from the curriculum designers. Three different programs were chosen by schools. Only
two of the program grantees receive training from the curriculum developers. One program
grantee asked the Regional Professional Development Program (RPDP) to assist in PD,
although the coordinator was unfamiliar with the program. These teachers met with the
RPDP coordinator for an afternoon session. One district worked in collaboration with the
designers to improve premade STEM kits, and the other two relied on their science
teacher/coordinator(s) in their district for help with the materials. Many of the participants
reported frustration with the materials, feeling that the instructions and directions were
19
unclear or did not make sense. This feeling occurred more frequently with the programs that
did not purchase the professional development that was offered with the kits. One teacher
who received minimal training stated, “One time through it and you'll figure it out as a
teacher but [the] first time you open the kit and use it, [it felt] just a little disjointed in terms
of how to structure your lessons and how much time would be included in each chunk.”
Professional development and training were needed to reduce frustration and wasted time.
Although not as prominent in the pre-focus group interviews, lack of available technology
and technology training were considered barriers to STEM education during the post-focus
group interviews. This perceived barrier could be because the teachers who had these
struggles specifically had programs that related to technology. In particular, some of the
educators using a technology-based program stated there was no enough technology in the
school. A similar struggle occurred with another group that chose to work with the coding
and robotics lessons within their STEM kits. In this case, connection and conductivity were
a problem with in the buildings, compelling teachers to find ways to work around these
issues. As one teacher stated due to a connection issue, “the kids didn’t have fun.” She
continued with stating that once she got the technology (coding blocks) working, she only
had her phone that would connect to the blocks and that made the lesson difficult. Some
teachers felt that their personal level of technology understanding was lacking and therefore,
they were less able to solve the issues with the technology. Many teachers felt that student
understanding of technology, and other 21st century skills, were needed for their students to
be prepared for STEM careers.
Student preparation for collaborative learning was also cited by teachers. “The students need
a great deal of preparation in order to meet the expectations. You’ve gotta go [teach] from
the very minute, all the way up to the more complex, or else they're not gonna be able to do
it,” said one teacher who clearly understood that STEM education must start at the youngest
grades so that students are prepared for harder tasks in the upper levels. “They’ve got to be
trained on working collaboratively” and similar statements were frequently given when
asked, “What can teachers do to help train the next generation of STEM professionals?”
Another participant answered the question by stating, “I don't just want the answer I want to
know how you [the student] got to the answer because being able to explain yourself in that
math problem means you'll be able to explain when somebody else asks a question about
whatever else.” Many teachers felt that the students tried to give the teacher the expected
answer rather than thinking on the problem and finding a solution. The program materials
were cited, during post-focus group discussions, as helping students become more
collaborative and willing to figure things out. For example, regarding one particular lesson
using Morse code, a teacher stated, “What I noticed when I walked in and we have Morse
code going off was that everybody in the classroom was engaged; everybody was having
conversation [and] that wouldn’t have happened outside of that activity.” Another teacher
reflected that, “I think that the kids are beginning to understand that kind of thinking,
20
problem solving and collaborative thinking.” Using STEM-related materials in class allowed
students to improve their skills in problem-solving, collaboration, and discourse.
Teacher Perceptions of Changes to Classroom Culture
During the pre-focus group interviews, many of the program grantee participants were only
aware that their school had been selected to receive the funding, but they were not aware of
the program they were receiving. The teachers frequently responded to the question, “Why
did your group selected this program?” with the answer that they did not know how the
program was selected or that administration chose the program. However, even in the
schools where the program was chosen by administration or science/STEM coordinators,
teachers still opted to participate and frequently stated high expectations for student
engagement. As one enthusiastic teacher stated that she was very excited and looking
forward to the opportunity to teach the different parts of STEM, not just the science. “We
don't get to do the technology, engineering, and the math part of it and put[ting] it all
together so I'm hoping to have some really interactive kits from the end of the year for my
kids to be able to use and explore” she exclaims.
The post-focus group interviews showed that the expectations for student engagement were
met, even if a program was difficult for the teacher to institute in class. For the majority of
the teachers, the programs helped engage students in the building of things, which led to
improvements in sense making and asking good questions. One teacher stated that exposing
students to and prompting student questions through exploration were good ways to foster
creative thinking and the ability to make connections. Additionally, teachers felt that students
were associating school activities to STEM careers. For example, one teacher stated, “my
kids actually knew what an engineer was, after. That was kind of exciting. They were like,
‘Oh, maybe I could do that! I love building things.’” Teachers also expressed that
cooperation and collaboration increased as the students realized that failure was part of the
process, “that it’s okay for things to not go right, that they can test it and try again and keep
going.” Most participants reported that their students loved to work with the program
materials. “My kids really liked the hands-on [aspect], being able to be that engineer, being
able to use their thought process, being able to use it in a group and get up and walk around
and move and do things with it” reported a participant. Team-work improvement was also
cited from the participants. Students began to understand that their individual ideas might
not always be the best. For instance, “they had to be thinking like engineers and build on
each other's thoughts a little bit more and ideas and be respectful in their disagreements and
be okay with maybe their idea [sic] is not the one that’s used” stated one participant.
Conclusions The materials received by the program grantees resulted in students and teachers enjoying
STEM education. While significant barriers to STEM existed, including training, time, and
21
materials, teachers still wanted to work STEM into their classrooms. As one teacher stated,
“I chose to work with the STEM kit because we have great science kits, but it doesn’t cover
everything.” The participants of this study understood that STEM is important and needed
in today’s schools.
The evaluation showed that providing materials to teachers and their classrooms can
increase students’ interest and awareness of STEM professions. It should be noted, however
that simply providing funding for materials does not necessarily increase teacher comfort in
teaching STEM. The teacher self-efficacy and outcome expectancy did not change by just
receiving or using the materials. However, the teachers believed in the programs and
reported changes to their classroom even if their overall self-efficacy remained the same.
Changing teacher practice requires more than just materials; time and professional
development are essential.
Training must be a part of the solution to improve STEM education in the elementary
grades. Additionally, both instructional and preparation time continues to be the main
barrier when instituting STEM education. This cannot be solved through funding materials
alone. The school day is a set amount of time and teachers need training in how to create
interdisciplinary lessons so that there will be more time available for science and STEM
education. Furthermore, teachers need the freedom to structure these interdisciplinary
lessons in their classrooms without interference from the time requirements of siloed
subjects, which were frequently cited as the reason there was little time for STEM.
22
Appendix A
23
24
References
Darling-Hammond, L. (1995). Changing conceptions of teaching and teacher development. Teacher
Education Quarterly, 22(4), 9-26.
Garet, M.S., Porter, A.C., Desimone, L., Birman, B.F., & Yoon, K.S. (2001). What makes
professional development effective? Results from a national sample of teachers. American
Educational Research Journal, 38, 915-945.
Office of Science, Innovation, and Technology (OSIT) K-5 STEM Grants. (2017, December 19).
Retrieved November 1, 2018, from
http://osit.nv.gov/uploadedFiles/ositnvgov/Content/Grants/K-5 STEM Classroom Grant
RFA(1).pdf
Reiser, B. J. (2013, September). What professional development strategies are needed for successful
implementation of the Next Generation Science Standards. In Paper written for the
Invitational Research Symposium on Science Assessment (Vol. 24, p. 25).
Wilson, S.M. (2013). Professional development for science teachers. Science, 340(6130), 310-313.
25