john c. dunham stem partnership school - aurora … partnership school... · john c. dunham stem...
TRANSCRIPT
0
John C. Dunham STEM
Partnership School
External Evaluation: 2015-2016
5/31/2016
EvalSolutions Inc.
Dr. Elizabeth Oyer
www.evalsolutions.com
317-582-1925
This page intentionally left blank.
Disclosure of Potential Conflicts of Interest
The external evaluator, Dr. Elizabeth Oyer, is the President of EvalSolutions Inc, located in
Carmel, Indiana. She does not have financial interests that could be affected by findings from the
study.
This page intentionally left blank.
Table of Contents John C. Dunham STEM Partnership School: External Evaluation: 2015-2016 ............................. i
Executive Summary ......................................................................................................................... i
Summary of Progress Toward Short Term Outcomes ................................................................. i
Introduction ..................................................................................................................................... 1
Background ................................................................................................................................. 1
Partners ....................................................................................................................................... 3
Methods........................................................................................................................................... 7
Progress Report ............................................................................................................................. 10
Outstanding Logic Model Outputs............................................................................................ 10
Short Term Outcomes ............................................................................................................... 11
To what extent did STEM Partnership School teachers, administrators and university faculty
improve their mathematics and science content knowledge and practice? ........................... 11
To what extent did STEM Partnership School students increase achievement in STEM
areas?..................................................................................................................................... 13
To what extent is the STEM Partnership School used by educators outside the school as a
professional development opportunity for content knowledge and practice in STEM? ....... 17
To what extent have curriculum units become part of the approved grade three through eight
curricula at the STEM Partnership School? .......................................................................... 18
To what extent have courses become institutionalized professional development for STEM
Partnership School teachers and administrators? .................................................................. 19
Long Term Outcomes ............................................................................................................... 20
To what extent have STEM Partnership School teachers, administrators and university
faculty mentored teachers and faculty within the STEM Partnership School Districts and the
University? ............................................................................................................................ 20
To what extent have STEM Partnership School students sustained high achievement in
STEM areas and demonstrate STEM career interest through high school and university
course selections and career choices? ................................................................................... 20
To what extent have STEM Partnership School teachers, administrators and university
faculty become district and University leaders in STEM education? ................................... 20
To what extent have curriculum units been revised and sustained for the STEM Partnership
School aligned to national standards and currency with fidelity? ........................................ 21
To what extent have on-going professional development plans in STEM content been
revised and sustained for STEM Partnership School teachers, administrators and University
faculty with fidelity? ............................................................................................................. 21
To what extent have corporations, foundations, the university, school districts and the City
of Aurora sustain ongoing investment for STEM Partnership School operations and school
teachers, administrators, and university faculty professional development? ........................ 21
To what extent have STEM Partnership School governance policies and procedures been
revised and sustained with fidelity? ...................................................................................... 23
Recommendations ......................................................................................................................... 24
Figures Figure 1. Timeline ........................................................................................................................... 1
Figure 2. Logic Model .................................................................................................................... 1
Figure 3. Partner Structure for STEM School (Professional Development Plan, p. 4) ................ 11
Figure 4. Teachers’ Horizon Test Scores ...................................................................................... 12
Figure 5. Horizon Test Change ..................................................................................................... 13
Figure 6. Parent Perceptions of Achievement ............................................................................... 14
Figure 7. Student Performance Product ........................................................................................ 14
Figure 8. PARCC Data ................................................................................................................. 15
Figure 9. Composite Attitude Scores ............................................................................................ 15
Figure 10. Attitude Change ........................................................................................................... 16
Figure 11. Career Interest ............................................................................................................. 16
Figure 12. Tour Summary ............................................................................................................. 17
Tables
Table 1. Teacher DTAMS Results ................................................................................................ 12
Table 2. Field Trips 2015-2016 .................................................................................................... 19
Table 3. John C. Dunham STEM Partnership Teams ................................................................... 22
Chap
ter:
Exec
uti
ve
Sum
mar
y
i
John C. Dunham STEM Partnership School: External Evaluation:
2015-2016
Executive Summary
Summary of Progress Toward Short Term Outcomes Did STEM Partnership School teachers, administrators and university faculty improve their
mathematics and science content knowledge and practice?
Yes. Teachers’ posttest scores were higher for Forces and Motion (about 20 points) and
Ecosystems and Adaptations (about 16 points).
Did STEM Partnership School students increase achievement in STEM areas?
Yes. Elementary and middle school students’ scores showed statistically significant gains from
pre- to posttest in tests of Forces and Motion, Ecosystems and Populations, Diversity of Life.
Was the STEM Partnership School used by educators outside the school as a professional
development opportunity for content knowledge and practice in STEM?
Yes. In 2015-2016, Aurora University and STEM Partnership staff hosted tours of the school for
100 guests, reflecting nearly a four-fold increase from 2014-2015. In addition, participants in the
Exelon NEED program opportunity and the STEM Partnership Conference showed statistically
significant post-test gains and most participants reported in the survey they would recommend
the conference.
Are the curriculum units a part of the approved grade three through eight curricula at the STEM
Partnership School?
Yes. In 2015-2016, 36 STEM units were implemented in grades three through eight. There were
six units in six subject areas, including (1) matter and energy; (2) forces and motion; (3) geology
and space, (4) ecosystems and adaptations; (5) structure and function of organisms; and (6)
weather, climate and human impact.
To what extent have courses become institutionalized professional development for STEM
Partnership School teachers and administrators?
The STEM minor and the STEM Curriculum Instruction Doctorate are currently under
development through the Aurora University School of Education.
Ch
apte
r: I
ntr
odu
ctio
n
1
Introduction
Background The John C. Dunham STEM Partnership School on the Aurora University (AU) campus serves
200 students in third through eighth grades from the East Aurora, Indian Prairie, Batavia
(beginning 2015-2016), and West Aurora school districts, as well as AU undergraduate and
graduate students. The vision is rooted in a professional development strategy to train teachers
from the partner districts through AU graduate coursework to be leaders in mathematics and
science education. Teachers return to their home school districts to share their knowledge.
The development of the multi-generational STEM Partnership School is rooted in several STEM
initiatives supported by the Institute for Collaboration and community partners, including the
development of bachelor's and master's degree programs in mathematics and science education,
workshops and institutes for teachers, and after-school and summer programs for students (see
Figure 1).
Figure 1. Timeline
The vision is articulated in a logic model to guide planning and evaluation activities (see Figure
2). The long-term goal of the school is to create a sustainable model for mathematics and science
education in a diverse urban community to be replicated across the country.
Contextual Factors
Quality of collaboration
between stakeholders
Quality of STEM
course work
LEA, IHE,
Industry factors
Implementation
Fidelity
Inputs
Institute for Collaboration of Aurora University
Aurora University Faculty and Administration
School District Teachers and Administration (LEAs)
Community Partners
Corporate Partners
STEM Knowledge
Activities
Select leader to convene working committee of partners
to address teacher and administrator outcomes for STEM Partnership School
Select leader to convene working committee of partners to address student outcomes and develop selection criteria
for the STEM Partnership School
Convene committee of partners to meet with union leadership and craft contract
language related to the STEM Partnership School
Convene committee of STEM Partnership School partners to
(1) determine curriculum scope and sequence and
(2) design professional development in STEM to meet STEM Partnership School teacher, administrator and University faculty needs
Form committee to develop a fundraising plan for the STEM
Partnership School
AU President convenes superintendent meeting to discuss STEM Partnership
School governance
Outputs
24 teachers and an administrator selected to
participate in STEM professional development for the STEM Partnership School
200 students from grades three through eight selected
to attend the STEM Partnership School
Unions and school boards place language related to STEM Partnership School
appointments in negotiated agreements
Curriculum scope and sequence designed for the STEM Partnership School
Two courses created focusing on curriculum planning,
development, implementation and assessment of STEM
curriculum for STEM Partnership School teachers
and administrators
Fundraising plan and process is developed for the STEM
Partnership School
Governance agreements are drafted for the STEM Partnership School
Short-Term Outcomes
STEM Partnership School teachers, administrators and
university faculty improve their mathematics and
science content knowledge and practice
STEM Partnership School students increase
achievement in STEM areas
STEM Partnership School is used by educators outside the
school as a professional development opportunity for
content knowledge and practice in STEM
Curriculum units become part of the approved grade three through eight curriculum at
the STEM Partnership School
Courses become institutionalized professional
development for STEM Partnership School teachers
and administrators
Corporations, foundations and the university raise $12 million by 2013 for construction and
equipping of the STEM Partnership School
Boards of Education and University Board of Trustees
approve governance agreements for the STEM
Partnership School
Long-Term Outcomes
STEM Partnership School teachers, administrators and university faculty will mentor teachers and faculty within
the STEM Partnership School Districts and the University
STEM Partnership School students sustain high
achievement in STEM areas and demonstrate STEM
career interest through high school and university course
selections and career choices
STEM Partnership School teachers. administrators and
university faculty become district and University leaders
in STEM education
Revise and sustain curriculum units for the STEM
Partnership School aligned to national standards and currency with fidelity
Revise and sustain on-going professional development in
STEM content for STEM Partnership School teachers, administrators and Univeristy
faculty with fidelity
Corporations, foundations, the university, school districts
and the City of Aurora sustain ongoing investment for STEM Partnership School operations
and school teachers, administators, and university
faculty professional development
Revise and sustain STEM Partnership School
governance policies and procedures with fidelity
Figure 2. Logic Model
Under development
In progress
Completed
Ch
apte
r: I
ntr
odu
ctio
n
3
Partners The John C. Dunham STEM Partnership School opened in August 2014. The following fiscal
partners have contributed to the partnership school:
AT&T Illinois. AT&T Inc., a telecommunications corporation, is the second largest provider of
mobile telephone services and the largest provider of fixed telephone services in the United
States. They also provide broadband subscription television services. AT & T Illinois made a
fiscal contribution to support educational programming in the John C. Dunham STEM
Partnership School.
Argonne National Laboratory. Argonne National Laboratory is a multidisciplinary science and
engineering research center, where world-class researchers work alongside experts from
industry, academia and other government laboratories to address vital national challenges in
clean energy, environment, technology and national security. Argonne was a partner in the
creation of the curriculum for the John C. Dunham STEM Partnership School.
Cabot Microelectronics. Cabot Microelectronics is a leading supplier of advanced engineered
materials used in the production of advanced semi-conductor devices. The Aurora-based
company’s mission is to create value by developing reliable and innovative solutions, through
close customer collaboration, that solve today’s challenges and help enable tomorrow’s
technology. Cabot Microelectronics contributed fiscally to support construction of a material
sciences laboratory. Cabot professionals also were a partner in the creation of the curriculum for
the John C. Dunham STEM Partnership School and continue to support both students and
teachers.
Caterpillar Foundation. The Caterpillar Foundation made an investment in the John C.
Dunham STEM Partnership School to support the professional development of teachers in
designing and implementing curriculum focused on manufacturing for students in third through
eighth grades. Since 1952, the foundation has helped make sustainable progress possible around
the world by providing program support in the areas of environmental sustainability access to
education and basic human needs.
Chicago Zoological Society/Brookfield Zoo. The mission of the Chicago Zoological Society is
to inspire conservation leadership by connecting people with wildlife and nature. As the
operator of the Brookfield Zoo, they have an international reputation for taking a cutting-edge
role in animal care and conservation of the natural world. Their staff has provided professional
development for teachers and presentations to students at the John C. Dunham STEM
Partnership School.
Commonwealth Edison. Commonwealth Edison Company (ComEd) is a unit of the Chicago-
based Exelon Corporation. ComEd provides service to approximately 3.8 million customers
across northern Illinois, or 70% of the state’s population. ComEd provided fiscal support of
energy-saving features of the John C. Dunham STEM Partnership School.
Ch
apte
r: I
ntr
odu
ctio
n
4
Dart Foundation. The Dart Foundation is a private family foundation that supports youth
education programs, primarily in the STEM fields. The Dart Container Corporation, a
worldwide corporation with a facility in North Aurora, facilitated the awarding of two foundation
grants. The grants supported the construction of classroom space in the John C. Dunham STEM
Partnership School and the creation of interactive displays for the STEM Forum in the school.
Dunham Fund. The Dunham Fund, an early champion of the John C. Dunham STEM
Partnership School, has provided ongoing financial support for the planning and construction of
the school since its Challenge for Change grant competition in 2008. It continues to support
professional development in STEM for teachers in our partner school districts. The fund
supports organizations that work to make the world a safer and more comfortable place for
mankind to live and prosper, giving special consideration to Aurora-area education
organizations.
DuPage Children’s Museum. The mission of the DuPage Children’s Museum is to ignite the
potential of all children to learn through hands-on exploration by integrating art, mathematics,
and science. The DuPage Children’s Museum was one of the partners that created the corporate
framework, the foundation for the John C. Dunham STEM Partnership School’s curriculum, and
helped to design the curriculum for the school. The museum continues to work with teachers,
serving as a resource for the implementation of the curriculum.
Exelon Foundation. The Exelon Foundation became the first corporate foundation to support
the John C. Dunham STEM Partnership School by funding the Exelon Center for Energy
Exploration. They also sponsor an annual NEED (National Energy Education Development)
workshop for teachers from the STEM Partnership School and those from partner school
districts. Exelon also supported the school’s FTC (First Tech Challenge) Robotics Team,
providing funds and adult mentors to work with middle school students. The mission of the
independent, non-profit foundation is to encourage respect for the environment and strengthen
the social and economic fabric of the community by supporting programs in the areas of
environment and conservation, innovative math and science education, and diversity and
tolerance.
Fermi National Accelerator Laboratory. Fermilab is America's particle physics and
accelerator laboratory. Its mission is to drive discovery by building and operating world-leading
accelerator and detector facilities; performing pioneering research with national and global
partners; and developing new technologies for science that support U.S. industrial
competitiveness. Fermilab helped to create the corporate framework and curriculum for the John
C. Dunham STEM Partnership School. They continue to serve as a resource to teachers as they
implement the curriculum.
Ch
apte
r: I
ntr
odu
ctio
n
5
Illinois Mathematics and Science Academy. The Illinois Mathematics and Science Academy
(IMSA) was established by the Illinois General Assembly to offer a uniquely challenging
education for students talented in the areas of mathematics and science and serve the school
systems of the State as a catalyst and laboratory for the advancement of teaching. IMSA serves
as a resource to the teachers of the John C. Dunham STEM Partnership School as they
implement the school’s curriculum.
Morton Arboretum. The mission of The Morton Arboretum is to collect and study trees,
shrubs, and other plants from around the world. The Arboretum maintains living collections on
display across naturally beautiful landscapes for people to study and enjoy, and to learn how to
grow them in ways that enhance the environment. The Morton Arboretum serves as a resource
to John C. Dunham STEM Partnership School teachers as they implement the curriculum.
Students have also taken field trips to the Arboretum to further enhance classroom learning.
Nicor Gas. Nicor Gas, a subsidiary of AGL Resources, is one of the nation’s largest gas
distribution companies, serving more than two million customers in a service territory that
encompasses most of the northern third of Illinois, excluding Chicago. Nicor Gas provided fiscal
support for the Geology, Energy and Resources Sustainability (GEARS) Laboratory. Nicor Gas
also helped design the curriculum for the John C. Dunham STEM Partnership School. The
company continues to serve as a resource to teachers for the implementation of the curriculum
and provide presentations to students.
Scheck & Siress. Scheck & Siress provides services to patients to create functional solutions for
their orthotic, prosthetic or pedorthic needs, helping each patient achieve their physical goals
using the latest technology available. Professionals from Scheck & Siress donated an interactive
display for the STEM Partnership School and are supporting programming and professional
development in the areas of biomedicine and biotechnology.
Schmidgall Family Foundation. The mission of the Schmidgall Family Foundation is to make
financial investments in support of charitable and/or benevolent organizations enabling and
assisting individuals and families to break the cycle of poverty and enable self-sufficiency
through education and personal development. The Schmidgall Family Foundation has
underwritten professional development in computer coding for John C. Dunham STEM
Partnership School teachers and parents as well as teachers from partner school districts. They
also sponsor a coding club at the school.
Sci -Tech Hands On Museum. The Sci-Tech Hands On Museum is an interactive science
museum with over 150 exhibits exploring space, light, weather, magnets and sound. Sci-Tech
has worked with teachers to support curriculum implementation in the John C. Dunham STEM
Partnership School.
Ch
apte
r: I
ntr
odu
ctio
n
6
Tellabs Foundation. The Tellabs Foundation funded a global technology center as part of the
John C. Dunham STEM Partnership School, including wireless infrastructure and an educational
web portal. The Tellabs Foundation has also supported the writing of an Innovative Engineering
Program for STEM Partnership School graduates and high school students from other area
school districts. The foundation advances strategic programs in education, health and the
environment.
VVF. VVF, a worldwide manufacturing and marketing company of personal care products and
oleochemicals with a facility in Montgomery, contributed funds to support a manufacturing
laboratory in the John C. Dunham STEM Partnership School. The laboratory engages students
in activities related to production, logistics and quality control.
Waste Management. Waste Management, Inc. North America’s leading provider of integrated
environmental solutions, contributed funds to support the Geology, Energy and Resources
Sustainability Laboratory, focused on issues dealing with the environment. The company has
also provided on-going professional development to teachers and has facilitated problem-based
learning experiences for students. It partners with customers and communities to manage and
reduce waste from collection to disposal while recovering valuable resources and creating clean,
renewable energy.
William G. McGowan Charitable Fund. The McGowan Charitable Fund is a philanthropic
family foundation established in 1992 to perpetuate William McGowan’s tradition of
compassionate philanthropy and ethical leadership. The Fund promotes, nurtures, and funds
initiatives in three program areas: community initiatives for those most vulnerable; education;
and healthcare and medical research. The McGowan Charitable Fund supports teachers from the
John C. Dunham STEM Partnership School and partner districts by providing professional
develop to assist teachers in incorporating ethical decision-making into their curriculum.
This report documents the evidence of progress in 2015-16.
Ch
apte
r: M
etho
ds
7
Methods 1) Student and Parent surveys
Parent surveys were adapted by Aurora University partners from the Nine Characteristics of
High Performing Schools: (http://www.k12.wa.us/research/pubdocs/ninecharacteristics.pdf).
(Student surveys were online).
Aurora University partners adapted student surveys for online administration from the Friday
Institute for Educational Innovation which reports that the development of this survey was
partially supported by the National Science Foundation under Grant No. 1038154 and by The
Golden LEAF Foundation.
Friday Institute for Educational Innovation (2012). Middle and High School STEM-
Student Survey. Raleigh, NC: Author.
Friday Institute for Educational Innovation (2012). Upper Elementary School STEM-
Student Survey. Raleigh, NC: Author.
The survey measures attitudes toward mathematics, science, engineering and technology, as well
as 21st Century learning. In addition, the survey includes a career interest scale. NOTE: A mis-
key in the survey creating missing data for student survey. Additionally, one teacher did not
complete the posttest.
2) Student, Parent, Teacher, University, Corporate, and Non-Profit Partner Interview Protocols
EvalSolutions Inc. adapted the partnership protocol from Brinkerhoff (2002), HUD (2002),
Lewis (2000), and Borden and Perkins (1999).
The student protocol included five questions:
1) Let's talk about the way you learn at your school. Tell me about your experience this year at
the STEM Partnership School. 2) Do you find your math and science lessons interesting? Why or
why not? 3) Do you think you want to have a job using math or science when you grow up? Why
or why not? 4) What do you like the best about how you learn at the STEM Partnership School?
5) What do you wish was different about how you learn at the STEM Partnership STEM School?
Field notes from recorded interviews were summarized and presented to participants (not
including students). Member-checked field notes were analyzed to identify themes in the
feedback.
Ch
apte
r: M
etho
ds
8
3) Student and Teacher Content Tests
Horizon Research Inc. (HRI) developed assessments for elementary1 (25 items) and middle
school (30 items). Students completed all four assessments for elementary or middle school.
Teachers completed the middle school assessments for Force & Motion and Populations &
Ecosystems.
Assessment NAEP Framework Alignment Elementary
IRT
Reliability*
Middle IRT
Reliability*
Evolution &
Diversity
(1) differences and adaptations of
organisms, preferential survival and
relatedness of organisms
.90 .84
Force and Motion (1) motion at the macroscopic level
(2) forces affecting motion
.81 .66
Populations &
Ecosystems
(1) the interdependence of organisms
and specific types of interdependence .83 .82
*HRI advises that the reliability >.60 is acceptable for making judgements about groups
Content validity was established through cognitive interviews with students, content review by
subject matter experts, and factor analyses.
4) Classroom Observation –Observations by the external evaluator, Elizabeth Oyer, were guided
by the Collaborative for Excellence in Teacher Preparation Classroom Observation Protocol.
Field notes were summarized to contextualize the data collection and analyses for the evaluation.
5) Online STEM Units – web-based repository for units. Content includes:
1 Horizon Research Inc. (2013). Force and Motion Elementary School Student Assessment User Manual. The
assessment was developed by the Assessing the Impact of the MSPs: K-8 Science (AIM) project at Horizon
Research, Inc., funded by the National Science Foundation under grant number DU-0928177. Any opinions,
findings, and conclusions or recommendations expressed herein are those of the authors and do not necessarily
reflect the views of the National Science Foundation or Horizon Research, Inc.
NGSS
Major Science Projects
CCSS Math
Major Math Connections
CCSS ELA
Major ELA Activity
Science and Engineering Practices
Major Engineering Practices and Projects
Ch
apte
r: M
etho
ds
9
6) Teachers implemented a student performance product scored with a common rubric
(Nproducts=unknown Nstudents=136). The rubric was aligned with NGSS (2013) and NETS for
students (2007). Inter-rater agreement analyses were not completed. Previous analyses of rubric
demonstrated average percent of perfect agreement between raters was 69%. The average
percent across products for +/-1 point was 18%. The average agreement for perfect and/or +/-
1% was 87% of the work products. Analyses reflect missing data for some indicators that were
omitted by some teachers.
7) Content analysis of interview and observation field notes.
Fields notes from recorded interviews were sent to participants as part of a member-check
process. Field notes were updated based on feedback before analyses.
8) Extant documents
Curriculum Overview
Google STEM Unit Summary
Narrative for the PD Model, 2015-
2016
Partner List
Agendas (Partner, Faculty, Staff)
Partner Meeting Feedback
PTA Fundraising Report
STEM Conference Presentation and
Evaluation Documents
Visitor Log
Ch
apte
r: P
rogre
ss R
eport
10
Progress Report
Outstanding Logic Model Outputs
Output: Fundraising plan and process is developed for the STEM Partnership School
The fundraising process for the school includes three independent sources:
1. Aurora University Advancement Office – The Office of Advancement oversees all
fundraising activities for Aurora University. The Board of Trustees and Senior Management
team determines priorities, particularly for capital and programmatic fundraising. Each year,
priorities are reviewed and confirmed, and goals set.
2. Parent Teacher Student Association (PTSA) - Directed by the PTSA and the Director
The fundraising goal for the year was $10,000. The purpose of the funds was to run various
projects, activities and support school events/field trips. Activities included Kidstuff coupon
book and Krispy Kreme sales, PTSA membership dues, Luigi’s Family Night, Box Tops for
Education, Family Skate Night, Spirit wear, and Manna. Meeting minutes reported $4,134.19
received through March 2016. May minutes are available at the first PTSA meeting for 2016-
2017.
3. Grants - Directed by the University through the Institute for Collaboration.
The school received four awards in 2015-2016:
The Schmigdall Foundation Coding Grant ($10,000) that will help the school facilitate one-
hour long coding classes for teachers from partner districts, Aurora University Faculty, and
parents of STEM school.
AT&T Illinois ($10,000) Investing in Illinois Award that recognizes outstanding local
community commitment and service programs in AT&T communities.
NEED Project Grant which supports sending teachers to MSI.
Ms. Kapitanoff received full funding for the Donor Choose Classroom Literary Fundraiser
for approximately $2000.
At the time of this report, a formal articulation of the fundraising plan and supporting processes
is under development.
Ch
apte
r: P
rogre
ss R
eport
11
Short Term Outcomes
To what extent did STEM Partnership School teachers, administrators and university
faculty improve their mathematics and science content knowledge and practice?
Professional Development Plan
Professional development meetings during the 2015-16 school year involved a curricular
discussion with all partners (STEM teachers and staff, Aurora University (AU) faculty and staff,
corporate and not-for-profit partners). Partners and AU faculty helped lead the discussion
around different topics. Fundamental knowledge building, activities, design challenges, and
experiments, were linked to essential questions for each STEM unit. STEM teachers and staff
prepared questions.
Figure 3. Partner Structure for STEM School (Professional Development Plan, p. 4)
Topics published in the Professional Development Plan from 2015-2016 for professional
development sessions focused on 3D Printing, Microscopes, Supply Chains, Brookfield Zoo,
Caterpillar – Manufacturing Day, and Morton Arboretum.
AU faculty provide STEM teachers with knowledge to help align the curriculum. Contact
between AU and STEM school teachers varied. Some STEM teachers have weekly
contact with AU faculty to share ideas, learn content, new ideas, and training needs. Not
all AU faculty meet with teachers weekly (because it might not be needed), but STEM
teachers can reach out to any AU faculty as needed. STEM teachers can also reach out
beyond their specific mentor if needed and seem to do so freely. There is more
Ch
apte
r: P
rogre
ss R
eport
12
71
87
60
80
Pre Post
Teacher Pretest - Posttest
Percent Correct
Ecosystems Forces & Motion
coordination at the elementary level, so they may not need to meet as often with AU
faculty. The MS teachers’ members meet twice per week (Focus Group Field Notes).
Professional development meetings served as working meetings to support coordination between
stakeholders. These meetings provided updates from the STEM
school, from corporate, university, and not-for-profit partners, as
well as break-out sessions where all partners worked
collaboratively on curriculum units.
During meetings, stakeholders
identified strengths, improvements,
and insights identified in each meeting. The identified strengths
were the use of essential questions to guide curriculum
development, collaborative culture of the meetings, opportunity
for connectivity between partners, and the evolution of the
engagement model for 2016-2017. Improvements were noted throughout the year, including the
recommendation for teachers to take the lead in communicating with the corporate partners on
their decision about integrating ideas into the curriculum, and the usefulness of an “on-boarding
session” for new partners. Finally, new opportunities for STEM-focused resources, field trips,
and support opportunities were shared at each meeting.
STEM Partnership School Teacher Content Knowledge
Teachers completed the Horizon middle school assessment for Force and Motion and
Populations and Ecosystems (missing=1). Wilcoxon Signed Ranks statistical tests of the pre- and
posttest differences indicated statistically significant evidence that the median difference in test
scores across the teachers’ scores for both the Forces & Motion and Ecosystems tests are not
zero (Forces & Motion p = .018; Ecosystems p=.028). Descriptively, the pre- and posttest scores
were about 20 points higher for Forces and 16 points for Ecosystems (see Figure 4 and Table 1).
Table 1. Teacher DTAMS Results
Ecosystems &
Adaptions
Forces &
Motion
Pretest Posttest Pretest Posttest
Mean 71.2 86.7 60.0 80.0
Std Error 5.0 3.2 5.7 3.9
Median 77.0 86.5 53.0 77.0
Stand Dev 12.3 7.9 15.0 10.4
Minimum 50 77 43 65
Maximum 83 97 77 97
Count 6 6 7 7
Strengths:
Essential questions
Collaborative culture
Engagement model TEAM IMPROVEMENTS
TEACHERS LEAD
CURRICULUM
DEVELOPMENT PROCESS
ON-BOARDING SESSION
Figure 4. Teachers’ Horizon Test Scores
Ch
apte
r: P
rogre
ss R
eport
13
At the time of this report, there were not articulated goals for administrator or University faculty
improvement in content knowledge. The STEM Partnership School director is currently enrolled
in graduate classes.
To what extent did STEM Partnership School students increase achievement in STEM
areas?
Student and Feedback
Students (n= 155) completed three Horizon Tests before and immediately following their STEM
units: Forces and Motion, Ecosystems and Populations, Diversity of Life. A formal alignment of
the content for both tests with the curriculum was not completed. For these tests, elementary and
middle school students’ scores showed statistically significant gains from pre- to posttest
(FForces= 60.705, p=.001; FEcosystems=182.909, p=.001; FDiversity=57.112, p=.001; see Figure 5). The
effect sizes for the changes were small for Forces and Diversity (EtaForces=.284 and
EtaDiversity=.272) and moderate for Ecosystems (Eta=.545; see Figure 5).
Parent Feedback
In the fall survey, parents who completed surveys described their expectations for the school
year. At the spring Energy Fair, parents in attendance rated the degree to which they agreed that
the school had met their expectations. In terms of academic achievement, most responding
parents were satisfied with students’ progress in critical thinking (98%) and academics (90%). A
majority of responding parents were satisfied with the level of challenge (83%) and progress in
non-STEM activities like reading and writing (78%; see Figure 6).
Horizon Pretests - Posttests
Percent Correct
Elementary Middle
54
73
65
77
Pretest Posttest
Elementary Middle
52
6261
68
Pretest Posttest
Elementary Middle
50
59
49
56
Pretest Posttest
Figure 5. Horizon Test Change
Ch
apte
r: P
rogre
ss R
eport
14
In focus group interviews, parents described
their discussions with their STEM students as
deep. Their students spend time at home
researching topics of interest. Their students
do something with the learned content.
Student Products
Teachers assigned student performance
products and scored all products with a
common rubric aligned with NGSS (2013)
and NETS for students (2007). Student
performance product ratings were analyzed across five dimensions: Product Development,
Information Literacy, Higher Order Thinking, Problem Solving, and Decision Making, Digital
Citizenship, and
Learning Standards
(see rubric in
Appendix). Each
dimension included
several indicators of
quality. Teachers rated
the most student
products as meeting or
exceeding standards
for the use of
technology tools.
Teachers scored fewer
products as meeting or
exceeding standards
for problem definition
(Product shows clearly
defined authentic
problem or highly
significant questions
identified or defined by
student. Clear purpose for product is evident; see Figure 7).
65
66
70
81
84
86
87
87
88
91
91
93
96
Problem definition
Depth and breadth of content
Creativity and Innovation
Citing resources
Analysis and synthesis
Strategic use of design elements
Communication
Planning/designing product
Design
Meeting objectives for content/discipline
Quality of information/data use
Organization
Use of technology tools
Average Percent of Students Meeting or Exceeding
Minimum Standards on Performance Rubric Across
Indicators
Figure 7. Student Performance Product
78
83
90
98
Child is progressing in non-
STEM activities (eg., reading
and writing)
Child is being challenged at
school
Child is progressing
academically
Child is developing critical
thinking and problem-solving
skills
Parents' Feedback - Academic Achievement
% Agree or Strongly Agree
Figure 6. Parent Perceptions of Achievement
Ch
apte
r: P
rogre
ss R
eport
15
PARCC (Academic Year 2014-2015)
Student level PARCC scores (N=149) from three districts from spring 2015 administration were
merged into one data file for analyses.
The number of students whose
performance levels were “Met”
(Level 4) or “Exceeded” (Level 5) for
2015 were aggregated to generate
percentages for Mathematics and
English Language Arts (see Figure 8).
A majority of the STEM students’
scores from the spring 2015
administration exceeded cut-scores
for Mathematics (65%, n=96) and
English Language Arts (74%, n=110).
Composite scores were not computed by the state for the STEM Partnership School nor included
in this summary. No further statistical analyses have been completed at this time to investigate
any relationship between these data and instructional activities, student characteristics, or other
mediating factors that might explain differences in students’ scores.
Confidence in STEM
Students answered questions (n=164) that gauged their confidence in their skills in different
aspects of mathematics,
science, engineering &
technology, and 21st
Century learning
activities (e.g.,
leadership, collaboration,
and time management).
Consistent with 2014-
2015, students’ spring
post-survey responses
showed confidence
across all skills with the
highest ratings for 21st
Century Learning skills
(see Figure 9).
3.69 3.92 4.06 4.14
Math Science Engineering
& Technology
21st Century
Average Composite Attitude Scores (out of 5 pts)
vs. BenchmarksJCD STEM 2016
JCD STEM 2015
NSF MISO 2013
Figure 9. Composite Attitude Scores
Figure 8. PARCC Data
State of Illinois 2015 PARCC average for 3rd - 8th grades
73.90%
64.50%
ELA Percent Met or Exceeded Math Percent Met or Exceeded
PARCC
2014-2015
38.1%
29.4%
Ch
apte
r: P
rogre
ss R
eport
16
3.63.7
3.9
4.04.04.14.04.2
Pre Post
STEM Composite Attitude
Pretest - Posttest
21st Century
Engineering & Technology
Science
Math
STEM school students’ responses were
significantly different from pre- to posttest in
mathematics, science, and engineering
(FMath=6.32, p=.013; FScience=8.404, p=.004;
FEngineer=6.198, p=.014; F21st=20.795, p=.001;
see Figure 10). Students reported the largest
increase in their confidence in 21st Century
skills (for descriptive purposes – not tested for
statistical significance).
Interest in STEM Careers
Students indicated the degree to which they
were interested in
different STEM careers
in surveys. More
students responded they
were interested or very
interested in
Engineering (70%). For
the second year in a
row, the fewest
students were interested
in Physics (37%).
When compared to
benchmarks from last
year and a large survey
of students in the
Maximizing Impact of
STEM Outreach
(MISO) project in
North Carolina2, STEM
school students’ career
interest responses were
higher for most STEM
2 Wiebe, E.N., Faber, M., Corn, J., Collins, T. Unfried, A., Townsend, L. (2013). A Large-scale Survey of K-12
Students about STEM: Implications for Engineering Curriculum Development and Outreach Efforts. Paper
presented at the annual conferences of the American Society for Engineering Education in Atlanta, GA.
Figure 10. Attitude Change
Figure 11. Career Interest
Ch
apte
r: P
rogre
ss R
eport
17
careers. Note that the MISO student sample included high school students whose attitudes about
STEM were lower than elementary and middle school (for descriptive purposes – not tested for
statistical significance).
There is continued evidence that the STEM school student population may be more likely to
have positive attitudes toward STEM content and career choices (for descriptive purposes – not
tested for statistical significance).
The majority of STEM school students reported they know adults who work as engineers (77%;
MISO students=22%) or technologists (72%). About two thirds of students reported they know
adults who work as scientists (68%; MISO students=22%) and half know mathematicians (56%;
MISO students=38%).
To what extent is the STEM Partnership School used by educators outside the school
as a professional development opportunity for content knowledge and practice in
STEM?
School Tours
In 2015-2016, Aurora University and STEM Partnership School staff hosted tours of the school
for 100 guests, reflecting nearly a four-fold increase from 2014-2015 (see Figure 12).
Representatives were hosted from public and private schools and districts, higher education,
corporations, not-for-profit organizations, community organizations, and elected officers.
Figure 12. Tour Summary
3 3 2 2 1
12
57
911 11
13
44
Elected
Officials
Not-for-Profit Community Corporations Partners* Higher
Education
K-12
Educators
Dunham Tours
Number of Guests
2014-2015 2015-2016
Ch
apte
r: P
rogre
ss R
eport
18
NEED Conference
The Exelon NEED program opportunity and the STEM Partnership Conference were combined
into a one day-long professional development opportunity for district teachers and
administrators. The NEED portion of the day provided curriculum that aligns to Matter and
Energy, whereas the Partnership Conference connected home districts with the progress,
collaboration and roll out plan. The Institute Day was convened on March 4, 2016. There were
32 participants with pre- and posttest results. Gains in participants’ test scores were statistically
significant (t=3.228, p=.001) and 21 of the 32 teachers significantly increased their scores. Most
participants reported in the survey they would recommend the conference (91%) and materials
(94%) to others and will use the materials in their next unit (88%). The majority of the
participants believed that the workshop will allow them to increase student energy knowledge
(81%).
To what extent have curriculum units become part of the approved grade three
through eight curricula at the STEM Partnership School?
In 2015-2016, 36 STEM units were implemented in grades three through eight. There were six
units in six subject areas, including (1) matter and energy; (2) forces and motion; (3) geology and
space, (4) ecosystems and adaptations; (5) structure and function of organisms; and (6) weather,
climate and human impact.
The units created around these overarching themes include essential questions [intended to
support] students to investigate, analyze, solve problems and work in teams as they acquire,
integrate and apply new knowledge (Curriculum Overview).
The units were housed online and available for guest access, including the units, standards, and
essential questions (stemcurriculum.aurora.edu).
There were ten field trips for STEM students in 2015-2016 (see Table 2).
Ch
apte
r: P
rogre
ss R
eport
19
Table 2. Field Trips 2015-2016
Date Field Trip Grades Connection
10/2/15 CAT -
Manufacturing Day
3rd through
8th
Manufacturing
10/26/15
(MS)
& 11/3/15
(ES)
Morton Arboretum 3rd through
8th
Ecosystems and Adaptations
12/2/15 Author Presentations 3rd through
5th
ELA
1/18/16 Byron Power Plant 8th grade Energy and Matter
3/15/16 Ag Day 3rd through
5th
Structure and Function, Geology and
Space, Weather and Climate
3/22/16 MSI 5th and 7th Energy and Matter
3/24/16 Parkview Bowling
Lanes
3rd through
5th
Forces and Motion, mathematics
5/2/16 D204 Planetarium 3rd through
5th
Geology and Space
TBD Cougars Stadium 6th grade Forces and Motion
TBD Scheck & Siress 8th grade Forces and Motion
To what extent have courses become institutionalized professional development for
STEM Partnership School teachers and administrators?
The STEM minor and STEM Curriculum Instruction Doctorate are currently under development.
Ch
apte
r: P
rogre
ss R
eport
20
Long Term Outcomes
To what extent have STEM Partnership School teachers, administrators and
university faculty mentored teachers and faculty within the STEM Partnership
School Districts and the University?
The STEM Partnership school has a grant proposal under review to support professional
development for teachers in partner school districts. If accepted, veteran teachers will work with
faculty and partners to design and deliver the training. At the time of this report, there is no
articulated plan for connecting mentors with sending district teachers and new AU faculty. There
is a lack of clarity about the transition back to sending districts.
To what extent have STEM Partnership School students sustained high achievement
in STEM areas and demonstrate STEM career interest through high school and
university course selections and career choices?
At the time of this report, there is no data for analysis or an articulated plan for data exchange
available to address this goal. Corporate and not-for-profit partners expressed interest in the
transition to high school.
Partners believe that students at Dunham are doing well but are concerned about the
high school learning environments they are going into. They are wondering if teachers
and students are given the freedom to do what they’ve learned (Partner Focus Group
Field Notes).
The Innovative Engineering Bridge Program is currently under review for funding. This
program is intended to encourage students to haven an interest in and transferable skills for
STEM careers through viable internship experiences.
To what extent have STEM Partnership School teachers, administrators and
university faculty become district and University leaders in STEM education?
In March 2016, the STEM Partnership School hosted the Promoting Girls in STEM Professional
Development Workshop. The objective of the workshop was for participants to learn best
practices to engage young girls in STEM. The workshop featured corporations, not-for-profit
organizations and AU faculty who provided instructional support on learning theories, how girls
explore in STEM, mentoring strategies, collaboration techniques, science inquiry and problem
solving methods.
At the time of this report, three workshop series have been developed for teachers,
administrators, and university faculty. In the spring 2016 and fall 2016, a training for teaching
coding in STEM was implemented (with approximately $10,000 from Schmidgall Foundation).
Second, an ethics training series for ethical decision-making in STEM (funded by the McGowan
Fund and the Dunham Fund for $167,000) will be implemented with teachers and university
faculty in the fall and spring months of 2016-17. Finally, a STEM Professional Development
course is being designed with funding from the Dunham Fund. A plan for course implementation
Ch
apte
r: P
rogre
ss R
eport
21
for Summer 2017, Fall 2017, and Spring 2018 is under review for funding for $1.3M (from the
Dunham Fund) to serve 72 teachers from the four districts for three years.
To what extent have curriculum units been revised and sustained for the STEM
Partnership School aligned to national standards and currency with fidelity?
Revision of the curriculum for the John C. Dunham STEM Partnership School was a
collaborative process between school district teachers, university faculty, not-for-profit partners,
and corporate engineers and staff. The curriculum spans grades three through eight and uses the
design cycle and supply chain process to infuse manufacturing, engineering and design
principles into learning experiences (Curriculum Overview).
At monthly meetings, partners go through the curriculum that teachers are working on. The
agenda is sent out ahead of time. The partners share information in Google docs and at monthly
meetings, which enhances the curriculum and the partners’ involvement.
To what extent have on-going professional development plans in STEM content been
revised and sustained for STEM Partnership School teachers, administrators and
University faculty with fidelity?
The articulated Professional Development Plan for 2015-2016 focused on annual training goals
for Partnership School teachers only. At the time of this report, there was not an articulated
training sequence for administrators or University faculty, although five Partnership teachers and
the building director are completing graduate course work.
In the summer 2016, STEM Partnership School teachers, faculty and partners will revisit the
essential questions for each of the STEM units. They are aligning the essential questions for
each grade level and unit to Next Generation Science Standards for that grade level. Teachers,
faculty, and partners are also looking to ensure questions are higher level essential questions that
will require students to think deeper.
To what extent have corporations, foundations, the university, school districts and
the City of Aurora sustain ongoing investment for STEM Partnership School
operations and school teachers, administrators, and university faculty professional
development?
The quality of the partnership continues to be a strength of the STEM school and its
stakeholders. According to stakeholders who were interviewed, the partners’ enthusiasm has
continued this year. Several processes support continued focus, engagement, and commitment.
The STEM Partnership School Director and Teacher Leader and the Aurora University Partner
Liaison lead activities to purposefully engage partners, including regular visits to partners to
gauge their feedback and keep them engaged and on-going networking between stakeholders.
Involvement is self-perpetuating – working with the students is a positive experience which leads
to continued investment.
Ch
apte
r: P
rogre
ss R
eport
22
Districts provide funding, students, and support staff. On the day-to-day operations, districts
provide guidance on issues like discipline, test administration. There is constant contact with
someone in the districts from the STEM school. STEM Partnership School staff defer to districts
because Dunham students are their students. In addition, there are quarterly meetings with
administrators.
The University partner contributes faculty, physical space, professional development, and
programming. Dunham staff and students are free to participate in anything on campus. They
have full access to facilities. The school is fully a part of the university.
In terms of corporate/not-for-profit partners, the time of funding raising seems to be in the past.
The school is no longer seeking fundraising as before - now they are interested in human capitol,
speakers, experiments, field trips. Partners have really stepped up. With few exceptions they are
willing and present and engaged with the students. Since the school opened, regular monthly
meetings are held to keep partners involved and ask if there are other ways they can provide
assistance (see Table 3).
Table 3. John C. Dunham STEM Partnership Teams
Team Members Team Purpose Meeting
Times
STEM Staff Team STEM teachers, Director,
Teacher Leader Information sharing and
problem solving regarding
school needs
Monthly/as
needed
STEM Faculty
Team AU Faculty Liaison, Teacher
Leader, Director, Faculty Information sharing, problem
solving, decision making
regarding curriculum and
professional development
Weekly
STEM Leadership
Team AU Faculty, Director, Teacher
Leader, Institute for
Collaboration staff
Information sharing,
coordination of STEM
initiatives
Monthly
Elementary/Faculty
Team STEM Elementary Team, AU
Faculty members Curriculum Planning Weekly
District
Curriculum
Committee
District Curriculum Reps,
Director, Teacher Leader,
Faculty Liaison
Information sharing, problem
solving about school & district
needs
Quarterly
STEM/Partner
Committee STEM Teachers, Director, AU
Faculty, Teacher Leader,
Corporate Partners, Not for
profit partners, Institute for
Collaboration staff
Information sharing, feedback,
collaboration between AU
faculty, partners & STEM
teachers
6-8 times a
year
Middle School
Team Middle School Team,
SPED/RTI Teacher Information sharing, problem
solving regarding student
concerns
Weekly
Ch
apte
r: P
rogre
ss R
eport
23
Corporate and Not-for-Profit partners bring the business perspective which gives students
another anchor for content.
Students are learning a core curriculum, but it needs to be anchored in something else
[real world context]. Another angle. Partners also give a real-world context that takes it
to the next level and applies to future careers or just as informed citizens… From a
fundamental stand point, partners provide something concrete students can get their
hands on and relate to outside the classroom (Partner Focus Group Field Notes).
The structure for matching specific STEM Partnership School teachers with specific partners and
University faculty evolved this year and is reported to have improved connections in general.
All stakeholders (parents, corporate/not-for-profit partners, Partnership School teachers, and
Aurora University faculty) consistently report a strong sense of mutual trust and respect.
Partners find their relationships to be very collegial. Teachers do a good job of letting
partners give their perspectives and relating to that. Partners recognize teachers are
working hard. There is a very collaborative environment. Partners are also meeting with
AU faculty to work on a summer camp. There is a sense of a shared vision between
partners… Relationships are productive – there is mutual respect, cohesion, and
openness (Partner Focus Group Field Notes).
The structure of the partnership creates an overlapping web of smaller collaboration activities
that all intersect together through the larger partnership meetings. Coordination between the
needs of different stakeholder groups (e.g., shared lab space) requires patient leadership and
cooperation between groups to accommodate each other.
To what extent have STEM Partnership School governance policies and procedures
been revised and sustained with fidelity?
At the time of this report, there are no formally reported changes or deviations in governance
policies and procedures. For the 2016-17 school year, the Physical Education instructional time
will be reduced to align with the schedules of the sending districts.
Ch
apte
r: R
ecom
men
dat
ions
24
Recommendations Based on the evidence of progress for 2015-2016, several steps are recommended for 2016-2017
to continue or expand efforts that are already in progress but would benefit from more
development.
1. Institutionalize practices and procedures to improve fidelity. The following outputs are
recommended to guide practice:
Updated Logic Model after the third year to reflect evolution in short or long-term goals
Data Dissemination Plan to guide communication of outcomes to stakeholder groups as
well as to address long term student goals (including protocol for transfer of student data
in accordance with FERPA), including new data sharing plans for achievement in STEM
areas, course load, and STEM career interest through high school
Fundraising Plan to complete planning outputs for current logic model
Induction and transition plan, including documentation of roles and responsibilities to
support transitions of new partners, faculty, and staff
2. Further improve communication between stakeholder groups (Partnership faculty, University
faculty, districts, corporate/not-for-profit, and parents). Areas of focus for communication
coordinating with sending districts for setting dates for outreach events, setting protocols for
follow-up communication between teachers and partners about STEM unit activities,
resources, and field trips, and communication with partners who are not able to attend
monthly meetings.
3. Build and support teachers’ leadership outside classroom instruction, including reporting at
monthly meetings, partner outreach, and collaborating on staff meeting agendas.
4. Articulate an onboarding process, including expanded documentation for key roles (e.g.,
teacher leaders, corporate/not-for-profit partners, University faculty) and activities.
5. Create and implement formalized induction and transition plans for Partnership and
University faculty and staff as well as corporate and not-for-profit partners.
6. Reconsider external evaluation model to align evaluation activities with school processes to
inform data quality and implementation fidelity on an on-going basis rather than end-of-year
summative report. Consider alternative models such as developmental evaluation to provide
quarterly feedback rather than year-end reports. Review alignment of key evaluation
instruments with STEM model. Review data collection protocols for external evaluation data.
Create opportunity for early orientation with STEM Partnership School teachers and staff to
overall purpose of the external evaluation, timeline for data collection, importance of
adherence to evaluation protocols, and resources for trouble-shooting or questions.
Appendix A
Interview Protocols
2015-2016
Student Interview Protocol
1) Let's talk about the way you learn at your school. Tell me about your experience this year at
the STEM Partnership School.
2) Do you find your math and science lessons interesting? Why or why not?
3) Do you think you want to have a job using math or science when you grow up? Why or why
not?
4) What do you like the best about how you learn at the STEM Partnership School?
5) What do you wish was different about how you learn at the STEM Partnership STEM School?
Parent Interview Protocol
1) Let's talk about the way your child learns at Dunham. Tell me about your child’s experience
this year at the Dunham Partnership School.
2) To what extent do you believe the Dunham Partnership School has impacted your child’s
STEM career interest? Why do you believe so?
3) To what extent do you believe the Dunham Partnership School has impacted your child’s
interest in science, technology, engineering, and mathematics? Why do you believe so?
4) To what extent do you believe the Dunham Partnership School has impacted your child’s
content knowledge in science, technology, engineering, and mathematics? Why do you believe
so?
5) In what ways does the Dunham Partnership School serve your family’s needs the best?
6) In what ways could the Dunham Partnership School better serve your family’s needs?
Partner Protocol
1) Partnership Composition
What are the contributions of the partners in the Dunham STEM Partnership School?
PROBE: How has staffing changed this year? How has Dunham STEM Partnership School
environment changed over the course of the year?
2) Organizational Structure
How are decisions made?
PROBE: By whom? Describe the governance structure. Are community / school perspectives
valued and respected? Is there a sense of equity among the partners?
3) Qualities of the Partnering Relationship and Maintaining the Partnership
Mutuality & Trust. To what extent is there a mutual need, trust, equality in decision-making,
resource exchange, transparency, respect, representation, enthusiasm, and sustained
understanding between partners and stakeholders? To what extent is leadership collaborative and
transformational?
PROBE: Do the goals and objectives of the Dunham STEM Partnership School address mutual
needs across partners? What are the perceptions of trust across partners? Is there a sense of
safety for sharing of information and resources? What steps have partners taken to build trust?
What is the nature of most interactions between partners? How respectful is the leadership to
differences in cultural and organizational norms, values, and beliefs? How transparent are the
Dunham STEM Partnership School operations? Is their equality in decision-making? Is there
reciprocal accountability? Is there a balance in the representation of all partners? Does
leadership across partners work closely together? Is there enthusiasm surrounding school goals
and activities? What do you think contributes to the most successful aspects of your school?
What are the biggest barriers to effectively collaborating with your partners?
Leadership. Who are the leaders of the Dunham STEM Partnership School?
PROBE: Are there one or more persons taking leadership? What is their role? Was there
participation from the top levels of partner institutions?
Resources. Have the resources been sufficient to reach implementation goals?
PROBE: From what sources? How does this compare with the initial plans? Are there adequate
resources to accomplish goals? Are resources sufficient for all partners? What about resources
for managerial, instructional and technical support? What has contributed to any changes in the
resources needed to reach your goals? What lessons have you learned about managing your
resources that can inform the planning for future?
Communication. What are the guiding principles for communication at the Dunham STEM
Partnership School? Is there shared decision-making between partners? What are the primary
vehicles for communication? Is there a formal management and communication plan? How are
conflicts resolved?
5) Performance and Outcomes
Performance. What areas did the Dunham STEM Partnership School staff address most
successfully? Were there areas that the Dunham STEM Partnership School was not successful in
addressing?
6) Sustainability
What is the current understanding of the “costs” (time, effort, funding, etc.) or obstacles to the
community/industry/school partners working together?
PROBE: Are the community/ industry/school partners finding that the benefits outweigh their
costs?
Appendix B
Student Performance Product Rubric
Student Performance Product Rubric Criteria
Indicator 3 2 1 0
Product Development
1 Planning/ designing product
Planning for product development shows thorough, excellent consideration of the question/purpose, information/data needs. Steps are clearly and thoroughly outlined. All of the relevant, appropriate information sources or data needs are identified and documented.
Planning for product development shows good consideration of the question/purpose, information/data needs. Steps are clearly outlined and mostly complete. Most of the relevant, appropriate information sources or data needs are identified and documented.
Planning for product development shows little consideration of the question/purpose or information/data needs. Few steps are outlined or are unclear and out of order. Information sources, designs, plans, or data are not appropriate or relevant.
No planning activities documented.
2 Use of Technology Tools
Specific technology tools are used strategically and are clearly identifiable in the final project. Use of technology extends audiences’ understanding and exemplifies project’s purpose.
Specific technology tools are thoughtfully used and are identifiable in the final project. Use of technology is appropriate for audience and purpose.
Specific technology tools are inconsistently used or not explicitly identifiable in the final project. Appropriate choices for technology use are erratic for audience and purpose.
Technology tools are not evident or are inappropriate for the audience and purpose.
3 Design Theme applied enhances communication of content. There is exemplary balance, proportion, coherence, and clarity. The overall design takes communication to a superior level. Extremely engaging.
Theme is evident. There is consistent balance, proportion, coherence, and clarity. The design elements work together to reinforce the communication but not necessarily extend it. Engaging.
Theme is not evident. Design elements appear random. There is minimal consistency in visual design criteria such as balance, proportion, coherence, and clarity. There is random use of graphical elements that do not reinforce message. Minimally engaging.
Theme is distracting. Exaggerated emphasis on graphics, images and effects in the presentation weakens the message and interferes with the communication of content and ideas. Not at all engaging.
Student Performance Product Rubric Criteria
Indicator 3 2 1 0
4 Strategic Use of Design Elements
All graphics, video, audio, 3-D, or other enhancements are used effectively to enrich the learning experience. Enhancements contribute significantly to convey the intended meaning.
Most graphics, video, audio, 3-D, or other enhancements are used appropriately to enrich the experience. For example, clips are either too long or too short to be meaningful.
Limited graphics, video, audio, 3-D, or other enhancements are present and/or do not always enrich the learning experience. Some use of these enhancements is inappropriate.
No graphics, video, audio,3-D, or other enhancements are present or use of these tools is completely inappropriate.
Information Literacy
5 Depth and Breadth of Content
Excellent use of information from a variety of high quality sources and fully supports judgments or evaluations.
Use of information from a variety of sources. Insufficient evidence presented to fully support judgments or evaluations.
Use of information from only a few sources. End products apply relevant information but judgments or evaluations are not supported.
Use of information from few sources. Factual information stated but failed to make judgments or evaluations.
6 Quality of Information
/Data Use
All information/data sources are accurate, valid, relevant, comprehensive.
Most important Information/data sources are appropriate and valid but some sources may be missing or lacking rigor or quality.
Information/data sources are appropriate but incomplete.
Important information/data sources are inappropriate or invalid.
7 Organization The sequence of information is logical. Presentation of content is creative and inventive for audience and purpose.
The sequence of information is logical. Presentation of content is satisfactory for audience and purpose.
The sequence of information is incoherent in some areas. Presentation of content addresses audience and purpose inconsistently.
The sequence of information is completely incoherent. Presentation of content is inappropriate for audience and/or purpose.
Student Performance Product Rubric Criteria
Indicator 3 2 1 0
8 Communica- tion
Excellent communication of new understandings is effectively applied to authentic, real-world contexts and/or issues of concern. Information is organized and displays knowledge and understanding in ways that the audience can view, use, and assess.
Adequate communication of new understandings is applied to authentic, real- world contexts and/or issues of concern. Information is organized and displays knowledge and understanding in ways that somewhat limits ways that the audience can view, use, and assess.
Limited communication of new understandings applied to authentic, real- world contexts and/or issues of concern. Information is organized and displays knowledge and understanding in ways that obviously limits ways that the audience can view, use, and assess.
No communication of new understandings applied. Audience would not be able to gain new knowledge or understanding from viewing, using, and assessing this product.
Digital Citizenship
9 Citing Resources
All sources are properly cited within the project.
Most sources are properly cited within the project.
Few sources are properly cited within the project.
No sources are properly cited within the project.
Higher Order Thinking, Problem Solving, and Decision Making
10 Questions/ Purpose/ Problem Definition
Product shows clearly defined authentic problem or highly significant questions identified or defined by student. Clear purpose for product is evident.
Product shows clearly defined problem or questions identified or defined by student that show some depth. Purpose for product not consistently supported.
Problems, questions or purpose defined by student for product is narrow or superficial. Strong purpose for product is not evident.
No evidence of student-defined authentic problems or questions for investigation or purpose for product is unclear or unknown.
11 Creativity and Innovation
The project shows significant evidence of originality and inventiveness. The majority of the content and many of the ideas are fresh, original, and inventive. Work shows evidence of new ideas, products, or processes by applying existing knowledge. Extremely engaging.
The project shows some evidence of originality and inventiveness. While based on an extensive collection of other people's ideas, products, images and inventions, the work extends beyond that collection to offer a few new insights. High engagement.
The work is an extensive collection and rehash of other people's ideas, products, images and inventions. There is little evidence of new thought or inventiveness. Engagement is minimal.
The work is a minimal collection or rehash of other people's ideas, products, images and inventions. There is no evidence of new thought. Not engaging.
Student Performance Product Rubric Criteria
Indicator 3 2 1 0
12 Analysis and Synthesis
Project shows strong evidence that information and media were chosen and melded together in order to address the critical issues or problems and produce coherent, unique products. Synthesis is reflected through new ideas that were generated or assessed based on factual information.
Project shows adequate evidence that information and media were chosen and melded together in order to address the critical issues or problems and produce coherent products. New ideas were generated but not completely supported with factual information.
Project shows limited evidence that information and media were chosen and melded together in order to address the critical issues or problems and produce a product. Product is a collection of factual information and does not generate new ideas.
End product does not synthesize, analyze, or use information effectively or appropriately to address the critical issues or problems. No evidence of synthesis is present and factual information is not presented in a coherent fashion.
Learning Standards
13 Meeting Objectives for Content/ Discipline
Project clearly, comprehensively, and extensively addresses stated performance objectives for audience and purpose.
Project proficiently addresses stated performance objectives for audience and purpose.
Project minimally addresses stated performance objectives for audience and purpose.
Project does not address stated performance objectives for audience and purpose.