1

PISA2 Implementation Plan

Partnership to Improve Student Achievement in Physical Science:

Integrating STEM Approaches (PISA2)

www.stevens.edu/ciese/pisa2

MSP Targeted Partnership

Implementation Plan: June 1, 2010 – May 30, 2011

National Science Foundation Award DUE-0962772

December 2010

2

Overview

This Implementation Plan for the Targeted Partnership MSP: Partnership to

Improve Student Achievement in Physical Science: Integration STEM Approaches

is organized in the following sections:

1. Vision

2. Program Goals

A. Science Certification

B. Teachers’ STEM Content and Pedagogical Content

C. Student Outcomes

D. Leadership Development/Capacity Building

E. Institutionalization

F. Research on Teacher and Student Outcomes

G. Research on Program Efficacy and Organizational Climate

H. Data Management/Coordination

I. Project Management/Governance

J. Communications and External Relations

3. Program Activities A-J

3

Partnership to Improve Student Achievement in Physical Science: Integrating STEM Approaches (PISA2)

MSP Targeted Partnership Implementation Plan: June 1, 2010 – May 30, 2011

Vision PISA2 aims to significantly increase the quantity, quality and diversity of teachers who are prepared to teach physical science1

in grades 3-8 in order to realize measured gains in student achievement in physical science across all subgroups. This consortium, led by Stevens Institute of Technology and comprised of 12 diverse school/district partners, the NJ Department of Education, St. Peter’s College, Columbia University’s Teachers College, and the National Science Resources Center, further aims to increase students’ 21st century skills and strengthen science leadership and organizational capacity in partner districts and the lead IHE to institutionalize and sustain reform efforts.

Program Goals As articulated in the proposal, our overarching goals include A through G, below, and this implementation plan will describe our objectives, timeline, activities, and progress to date in meeting these goals. A. Science Certification: Increase the number of elementary teachers who possess the

Elementary Endorsement with Science Specialization in participating districts over the five-year program through a graduate certificate program for inservice and preservice teachers

B. Teachers’ STEM Content and Pedagogical Content Knowledge: Increase the professional preparation of 400 teachers Grades 3-8 in physical science content, learning sciences, awareness and use of research-based instructional resources, and assessment

C. Student Outcomes: Increase physical science content knowledge and 21st Century

Skills of students in Grades 3-8

D. Leadership Development/Capacity Building: Build leadership capacity among partner school and district administrators to implement a strategic and sustained science education reform agenda

1 Throughout this proposal Physical Science refers to the literacy maps of the American Association for the Advancement of Science Physical Setting cluster, which includes changes in the earth’s surface; weather and climate; use of earth’s resources; energy transformations; gravity; states of matter, etc. See http://strandmaps.nsdl.org/?chapter=SMS-CHP-0903. We have also incorporated the Designed World and the Nature of Technology clusters.

4

E. Institutionalization: Institutionalize new programs at the lead IHE, such that graduate teacher education programs, the number of faculty involved in K-12 STEM programs, the number of S&E undergraduates who pursue teaching certifications, and the use of research-based instructional strategies in university-level coursework increase

F. Research on Teacher and Student Outcomes: Contribute new knowledge on the impact of an integrated STEM approach on teacher and student science learning, motivation, and self-efficacy and on students’ acquisition of 21st Century Skills

G. Research on Program Efficacy and Organizational Climate: Employ and assess a

methodology, limiting factors analysis, to gauge the program’s effectiveness in meeting its long-term objectives in participating districts

In addition, H-J, below, identify implicit or associated program priorities which are integral to the attainment of PISA2 goals and objectives.

H. Data Management/Coordination: Collect, coordinate, manage, analyze and

disseminate project data to make improvements at the level of the project, the partner organizations, and at the individual classroom levels

I. Project Management/Governance: Manage and govern the project, its component programs and partner relationships through the creation of an Executive Committee, Project Advisory Board, Steering Committee and District Support Team

J. Communications and External Relations: Build awareness of PISA2 program goals

and activities among key internal and external stakeholder groups, including faculty, IHE students, parents, legislators, guidance counselors, and corporations. Utilize various media and technologies to communicate and publicize program goals, components, activities and events to all stakeholders and the public

5

Program Activities Goal A: Science Certification: Increase the number of elementary teachers who possess the Elementary Endorsement with Science Specialization in participating districts over the five-year program through a graduate certificate program for inservice and preservice teachers To achieve this goal, faculty in the Stevens Schaefer School of Engineering & Sciences, working collaboratively with representatives of the Center for Innovation in Engineering & Science Education (CIESE), conceptualized and are developing a new graduate certificate program through which PISA2 participating in-service teachers pursue five science courses to earn 15 graduate credits in science. These courses, together with other requirements (passing the Praxis exam, life sciences content, course in adolescent psychology) represent a critical component of the preparation necessary for teachers to earn the Elementary School Endorsement with Science Specialization. To date, Courses 1 and 2 have been developed and were offered in a hybrid (face-to-face + online coursework) mode in the summer and fall of 2010. Development of Courses 3-5 is underway, in preparation for delivery in summer and fall of 2011. Courses

Development Timeline

Course Delivery Schedule

Course Status

Notes

Course 1: Fundamentals of Physical Science (E. Whittaker, Physics & A. Macalalag, CIESE)

Spring 2010

June-Dec 2010 Initial development completed; course currently underway; course redesign to take place in Spring 2011.

Development timeline accelerated due to delayed award date.

Course 2: Fundamentals of Earth Science (T. Herrington, Civil, Environmental, Ocean Engineering & G. Bartus, CIESE)

Spring 2010 June-Dec 2010 Development and Delivery Completed

Development timeline accelerated due to delayed award date.

Course 3: Energy Production and Consumption (R. Besser, Chemical Engineering & A. Macalalag, CIESE)

Dec-Jan 2011 July-Aug 2011 Development In Progress

Course to be submitted to graduate committee 1.26.11

Course 4: Understanding Global Change (G. Bartus, CIESE)

Dec-Jan 2011 Aug-Dec 2011 Development In Progress

Course to be submitted to graduate committee 1.26.11

Course 5: Engineering Dec-Jan 2011 Jan-May 2012 Development In Course to be

6

Solutions to the Challenges of Energy and Global Change (F. Fisher, Mechanical Engineering & TBD)

Progress submitted to graduate committee 1.26.11

Faculty and CIESE personnel directly involved in the development and delivery of these courses are identified in column 1. In addition, other CIESE program representatives, listed below, have contributed in the areas of developing assessments, tutoring sessions, grading, and program and data analysis:

• D. Brockway, H. Harms, E. McGrath, B. Mammen, C. Jurado, P. Olesinka (CIESE)

The graduate courses:

• are/will be designed or redesigned to employ a problem-based learning approach that connects underlying science concepts to relevant societal topics such as climate change and energy consumption

• encompass experiential learning, inquiry, investigation, research, and collaborative reflection presented in a blended-learning format containing both face-to-face and online elements

• are/will be designed using the Understanding by Design (Wiggins & McTighe, 2005) approach

• utilize multiple forms of assessment to gauge student (teacher) progress toward learning objectives

Upon successful completion of the 5 graduate science courses, teachers may earn 15 graduate credits, a requirement for the Elementary School Science Specialization (grades 5-8) certification from the NJ Department of Education. We are also providing resources and guidance for teachers to complete other requirements (e.g., passing the Praxis exam given by the Educational Testing Service, completing a course in Adolescent Psychology) to those needing such assistance. Accomplishments, Challenges and Responses: • Program launched on accelerated schedule: course development/approval

completed; delivery of courses 1 and 2 underway; classroom visits underway; assessments developed, administered and preliminary data gathered.

• Very accelerated project start-up phase between grant approval of June 1, 2010 and start of first two courses in June 2010 (requested project start date was April 1, 2010). Implications included extremely condensed period for course development, teacher recruitment, issuance of subcontracts, course registration, technology training for teachers on online learning platform, and logistical efforts with course sites.

7

• Uneven preparation of teachers with online learning format and interface. An introductory orientation to the WebCT platform was held, but was insufficient for some participants. Classroom visits by CIESE staff, review during a scheduled PD day, and a “help line” were among the interventions that were employed to address this challenge.

• Uneven mathematics preparation among teachers. While we are still analyzing the data from the fall semester, it appears that elementary teachers (grades 3-5) had greater difficulty with the quantitative aspects of physical science course than many of the middle school teachers. The online text, Mastering Physics, required flexible use of algebra and trigonometry, and a level of mathematical precision not familiar to some participants. Approximately 18 of the original cohort of 96 teachers did not continue with the assignments presented in the online physics text and chose to drop out of the program. To address this issue, we arranged for and conducted on-site, face-to-face tutoring sessions, devoted a portion of a scheduled PD day to a review; offered a review sheet for the final exam; offered opportunities for teachers to take an Incomplete and take the final exam in January. Going forward, we envision a course redesign and format change that will provide the majority of the physical science course being offered in a face-to-face format; implementing an initial mathematics assessment for teachers and, if warranted, providing additional/prerequisite mathematics learning opportunities (such as the Intel Math program); and employing a backwards design approach to course redesign to analyze all components of the course.

8

Goal B: Teachers’ STEM Content and Pedagogical Content Knowledge: Increase the professional preparation of 400 teachers Grades 3-8 in physical science content, learning sciences, awareness and use of research-based instructional resources, and assessment PISA2 offers two pathways for practicing teachers to learn targeted science content, related pedagogical content knowledge, and use of research-based instructional materials and assessment tools through either (a) completion of a graduate certificate program (PISA2 Scholars) or (b) equivalent professional development experiences (PISA2 Teacher Leaders). Over five years, four cohorts averaging 100 teachers each will complete a five-course (two-year) certificate program or related PD experience, to deepen their understanding of physical and Earth sciences, and their classroom practice. Our proposal estimated that 40% of the first cohort would opt to enroll in graduate coursework and 60% would opt for the professional development option. However, after the grant was awarded and with the endorsement of the Stevens Provost, the option of tuition-free credit was offered to all Cohort 1 participants. As a result, most participants opted to pursue graduate credit, leading to the Elementary Endorsement with Science Specialization (74 Scholars and 4 Teacher Leaders). Two strands of activities are being implemented to enhance the science content knowledge and pedagogy of PISA2 teachers: • Professional development workshops: two professional days in year one (at which

teachers participate in hands-on science and engineering activities and strategies for implementing these STEM activities in the classroom); and,

• Monthly classroom visits which support teachers in adapting coursework and PD

resources to classroom needs. Area of Preparation

Activity Activity Components

Activity Timeline

Activity Status

Physical Science Content

Graduate Coursework

Face to face and online learning components of graduate courses

June-Dec 2010 Completed

Research-based Instructional Resources; Assessment

Classroom Visitations

Support implementation of STEM classroom activities: Intro to Electric Circuits, Designing Roller Coaster Rides, Correlating Fossil Evidence, Designing Earthquake Resistant Structures,

Sept 2010- June 2011

In Progress

9

Research-based Instructional Resources; Assessment; Learning Sciences

Professional Development Workshops

Engineering Design Process Activities: Designing Bridges, Alka Seltzer Rockets, Touchdown Designing a Space Shuttle; Designing Windmills, Ocean Current Models, 21st Century Skill Framework

Oct 2010 Jan 2011

Completed Scheduled

Learning Sciences; Assessment

Instructional Assignments and Readings

Lesson design study: Inquiry based lesson plan, examples of student work and assessment criteria; teaching philosophy paper; Ready, Set, Science; Understanding Models in Earth & Space Science

Sept-Dec 2010 Completed

Persons involved in these activities:

• Dr. Edward Whittaker, Dr. Thomas Herrington (Stevens Faculty) • C. Shields, Gregory Bartus, A. Macalalag, H. Harms, B. Mammen (CIESE)

The implementation of Goal B activities is proceeding as planned. Accomplishments, challenges and our responses are identified below. Accomplishments, Challenges and Responses:

• Majority of participants have opted for graduate credit option (PISA2 Scholars). • Aggressive schedule of classroom visits currently underway, with five CIESE

staff visiting classrooms on a regular basis. • PD days scheduled and partially completed for Year 1. • Logistical and scheduling issues are being addressed through building

relationships with teachers, school/district administrators • Classroom equipment and materials constraints are being addressed through the

provision of project-sponsored classroom materials • Misalignment of teachers’ science curricula (grade levels, subjects) with program

content. This challenge is a result of the acceptance into the program of teachers not currently teaching physical or earth science in their Grades 3-8 classrooms. We are currently working with district administrators to develop an approach that allows for accountability in implementation of PISA2 materials in the classroom, without major disruption to district curriculum and pacing.

10

Goal C: Student Outcomes: Increase physical science content knowledge and 21st Century Skills of students in Grades 3-8 The achievement of this goal is linked to teacher success in Goal A & Goal B activities. The content of the graduate-level courses, the classroom visitations, PD workshops, research-based instructional resources, and the professional development days address key 21st century skills, such as communication and collaboration, problem-solving and critical thinking, and creativity and innovation. More specifically, problem-based learning (PBL), the engineering design process (EDP), and inquiry-based are embedded in the Goal A & B courses and PD for teachers and translate to classroom activities impacting students.

21st Century Skill

Teacher Instructional Design

Student Tasks

Activity Timeline

Activity Status

Critical Thinking & Problem Solving; Communication & Collaboration; Creativity & Innovation

Problem-based Learning

Defining the problem, proposing solutions, gathering &evaluating information, synthesizing & forming solutions

Sept-June 2010

In Progress

Critical Thinking & Problem Solving; Communication & Collaboration; Creativity & Innovation

Engineering Design Process

Asking questions, brainstorming solutions, planning, investigating, creating prototypes, improving prototypes

Sept-June 2010

In Progress

Critical Thinking & Problem Solving; Communication & Collaboration

Inquiry-based Learning

Conducting observations, formulating questions, constructing/testing/ revising models, experimentations, argumentation

Sept-June 2010

In Progress

Persons involved in these activities:

• Dr. Edward Whittaker, Dr. Thomas Herrington (Stevens Faculty) • C. Shields, Gregory Bartus, A. Macalalag, H. Harms, B. Mammen (CIESE)

The implementation of Goal C will be measured by pre and post-tests of student’s 21st century skills. This activity is referenced under Goal H, Assessment & Evaluation. There are no Challenges identified at this time.

11

Goal D: Leadership Development/Capacity Building: Build leadership capacity among partner school and district administrators to implement a strategic and sustained science education reform agenda To achieve this goal, Stevens Institute partnered with the National Science Resources Center (NSRC) to co-sponsor three synergistic programs to involve PISA2 school, district, and community stakeholders in a collaborative, strategic planning process for science education reform. Activities Attendees Activity

Timeline Activity Status

PISA2 Building Awareness for Science Reform Symposium

District Representatives Corporate Leaders Community Leaders Parent Representatives Stevens’ Professors PISA2 Team Members

Nov 12, 2010

Completed

PISA2 SWOT Analysis and Strategic Planning Meeting

District Representatives Corporate Representative Parent Representative CIESE Exec. Director PISA2 Program Manager NSRC Executive Director & Staff Members

Dec 8-10, 2010

Completed

PISA2 Leadership Development & Strategic Planning Institute

District Teams PISA2 Program Manager NSRC Executive Director & NSRC Staff Members

May 23-27, 2011

Scheduled

Persons involved in these activities include: • E. Whittaker, R. Besser, T. Herrington (Stevens Faculty) • A. Macalalag, B. Mammen, E. McGrath, C. Shields, G. Bartus (CIESE) • District representatives and other and external partners. Accomplishments, Challenges and Responses: • All districts have been represented and actively participated at events/programs held

to date. • Congressman Rush Holt (D-NJ 12) visited recent program and received

overview/orientation • Districts have been challenged with budget cuts and other state mandates resulting

in retirements and personnel/leadership changes in project liaisons and other key leadership to be involved in these programs. These personnel changes have required additional outreach and contact with district partners, school boards and other community stakeholders to engage a broad representation at Strategic Planning programs. These efforts are underway.

12

• Key partner, the NJ Department of Education, has not been represented, due to a change in leadership in the Department and among a majority of senior staff. We are awaiting the appointment of an Assistant Commissioner-level staff member responsible for STEM education before re-engaging NJDOE.

13

Goal E: Institutionalization: Institutionalize new programs at the lead IHE, such that graduate teacher education programs, the number of faculty involved in K-12 STEM programs, the number of S&E undergraduates who pursue teaching certifications, and the use of research-based instructional strategies in university-level coursework increase These programs impact: • Current Stevens students • Stevens faculty currently involved in PISA2 programs • Stevens faculty not currently involved in PISA2 programs The PISA2 team, in cooperation the Stevens Director of Career Development and St. Peter’s College have planned and are executing several programs to: • provide pathways for current Stevens undergraduate and graduate students to

obtain teaching experience and credentials (coursework toward a minor in education or an alternate route certification)

• enhance knowledge and use of research-based instructional practices and

assessment strategies in undergraduate coursework

• provide ongoing faculty development to learn and practice research-based STEM pedagogy

• identify opportunities for tuition-supported programs to continue courses and

associated programs with districts beyond grant funding Programs completed, planned, or underway are listed below: Activity Activity

Participants Activity Components Activity

Timeline Activity Status

PISA2 Graduate Courses

PISA2 Faculty CIESE PISA2 Team

Design and teach courses incorporating Problem-based Learning, Inquiry-based Learning, Engineering by Design

Start year : June 2010-May 2011

In Progress

14

RIEE* PISA2 Faculty Invited Stevens’ Faculty

Workshop Series: Understanding by Design; STEM Education Research & Publications; Writing Assessment Questions; Sharing Research Findings; Developing Student Centered Learning Environments

Start year: June 2010-May 2011

In Progress

NSRC Program PISA2 Faculty Participate in Building Awareness for Science Education Reform Symposium

Nov 2010 Completed

Pathways to Teaching Options**

St. Peter’s College School of Education; PISA2 Program Manager; Stevens’ Career Development Office Stevens’ Undergrad/Grad Students;

Advertise thru flyers and Student Life Newsletter; Provide information sessions for interested students; Design course** sequence charts for students Register for and attend 1st course, Educational Psychology; Register for and attend Alternate Route 24 Hour Course

Start year: June 2010-May 2011 Dec-May 2011

Completed In Progress

*RIEE – Research and Innovation in Engineering Education: An initiative established in 2004 to catalyze culture change toward awareness, adoption and adaptation of research based instruction in university teaching and the development of engineering education innovations. **Pathways to Teaching Options - Through a partnership with St. Peter’s College, Stevens Institute of Technology designed this opportunity for Stevens’ science and engineering students to obtain NJ teaching certification for high school math or science through preparatory education coursework or the State’s Alternate Route Program. The coursework provided by St. Peter’s College includes:

• Educational Psychology • The Improvement of Reading in the Secondary School • Principles and Techniques of Teaching in Elementary and Secondary Schools • High School Curriculum Organization • Tests and Measurements • Sophomore Field Experience • Junior Field Experience

15

• Student Teaching: Secondary

The Alternate Route Program allows a student with 30 credits in math or science and passes the Praxis exam to complete 24 hours of formal instruction through St. Peter’s College prior to receiving provisional certification. These students are not required to pursue the teacher preparation courses shown above. Persons involved in these activities:

• E. Whittaker, R. Besser, F. Frank Fisher, T. Herrington, K. Sheppard (Stevens Faculty)

• Prof. J. Jacobson, St. Peter’s College, School of Education • L. Insley, Stevens Institute of Technology Director of Career Development • S. Lowes, Columbia University, Institute for Learning Technologies • C. Shields, G. Bartus, A. Macalalag, D. Brockway, H. Harms, E. McGrath, B.

Mammen (CIESE)

The implementation of these activities to date shows successful progress. No challenges have been identified.

16

Goal F: Research on Teacher and Student Outcomes: Contribute new knowledge on the impact of an integrated STEM approach on teacher and student science learning, motivation, and self-efficacy and on students’ acquisition of 21st Century Skills The achievement of this goal is directly tied to the collection, management and analysis of the PISA2 assessment and evaluation data. Stevens’ faculty, members of the CIESE/PISA2 team, external evaluators and assessment vendors are or will be involved in the development, administration and analysis of the data. PISA2 Assessments

Activity

Activity Timeline

Activity Status

Pre & Post Physical Science & Earth Science Content Tests for Teachers

Development of tests Administration of pre-tests Administration of post-tests Administration of Incomplete Grade Extension post-tests Scoring and Analysis

Spring 2010 June 2010 (first day of course) Dec 2010 (last day of course) Feb 2011 Spring 2011

Completed Completed Completed Scheduled Planned

Pre & Post Pedagogical Content Knowledge, Engineering Design Process Test for Teachers

Development of tests Administration of pre-test Administration of post-test Scoring and Analysis

Spring 2010 September 2010 May 2011 Spring 2011

Completed Completed Scheduled Planned

Pre & Post Science Content and Engineering Test for PISA2 and Comparison Students

Development of tests Administration of pre-test Administration of post-tests Scoring and Analysis

Spring 2010 Sept- Oct 2010 April-May 2011 Summer 2011

Completed Completed Completed Not Yet Done

17

Pre & Post Attitudes and Beliefs Survey for Teachers

Development of Survey Administration of initial survey Administration of final survey Scoring and Analysis

June 2010 Sept 2010 May 2011 Summer 2011

Completed Completed Scheduled Planned

Pre & Post 21st Century Skills Assessment of PISA2 and Comparison Students

Development of Assessment

Jan- Aug 2011 Planned

Survey of Student Attitudes Toward STEM Education and Careers

Development of Survey Jan- Aug 2011 Planned

Pre-Program Interview of PISA2 Faculty

Conduct Interview July 2010 Completed

Physical Science & Earth Science Course Evaluations by PISA2 Teachers

Development of Evaluations Submission of Evaluations

Dec 2010 Completed

Interview & Observation Instruments

Development of Instruments Conduct Interviews & Observations

Nov-Dec 2010 Jan-May 2011

Completed Pending Award of New Evaluator Contract

CIESE at Stevens Institute of Technology is responsible for the collection, management and storage of all pre/post assessments, surveys, evaluations, and interviews. A member of the CIESE staff ensures that data is well-scrubbed and in good form before turning it over to the evaluator. Assessments and processed data are stored in a secure cabinet or password-protected computer. Access is restricted to senior members of the researcher, evaluator and PI team. Names are replaced by codes to ensure confidentiality. Schools and individual students and teachers will not be reported in any public manner. Persons involved in these activities:

• E. Whittaker, T. Herrington (Stevens Faculty) • C. Shields, G. Bartus, A. Macalalag, D. Brockway, H. Harms, E. McGrath

(CIESE) • G. Nyre, Nyre & Associates

Accomplishments, Challenges, and Responses:

18

• Development administration of instruments: pre-/post-content assessments for Courses 1 and 2 (teachers and students); development of attitudes and beliefs survey; The 6 month contract with external evaluator Nyre & Associates was not renewed due to the evaluator’s inability to submit data, analysis, and reports according to agreed timeline dates.

• The 21st Century Skills Assessment delivered by Learning.com did not align with the skills identified as integral in the PISA2 program. As a result, an rfp for a new 21st century skills assessment was released and a new vendor will be selected. The 21st century skills assessment will be administered to students and baseline data will be collected in year two.

• Evaluation subcontractor did not meet project requirements. An RFP was released in November 2010 and proposals have been submitted (as of December 15, 2010). The new evaluator will be selected in January, 2011 and participation in the MSP Learning Network Conference is required.

• An RFP seeking vendors/teams interested in developing 21st century skills assessments for PISA2 was released in November 2010. Proposals have been received (as of December 15, 2010); the vendor/team will be selected in January, 2011.

• Due to the Incomplete Grade Extensions granted to teachers, additional post-test dates in February were established.

19

Goal G: Research on Program Efficacy and Organizational Climate: Employ and assess a methodology, limiting factors analysis, to gauge the program’s effectiveness in meeting its long-term objectives in participating districts The achievement of this goal involves the PISA2 research question: What immediate and contextual factors limit or facilitate a teacher’s success in changing classroom practice? Data to answer this question is gathered by situating the individual teacher in the context of all the forces that impact on his/her classroom practice and examining in detail those factors that limit or facilitate success in changing classroom practice. Limiting Factors Analysis determines ahead of time those factors that are likely to limit the project’s ability to reach its goals and then focuses on how these foster or limit success.

Example Limiting Factors: Lack of adequate institutional capacity to complete project activities • Lack of project management capacity • Lack of provider capacity Lack of adequate scientific understanding • Insufficient content knowledge among teachers to master graduate-

level course • Insufficient PCK among faculty to make course content useful for

teachers • Insufficient PCK among teachers to make course content useful to

students • Lack of learning capacity among teachers (lack of commitment, lack of

time) Public policies that do not support the project • Focus on standardized testing and test prep • Scripted or otherwise mandated curriculum Economic pressures that lead to the destruction of the project

Activity Activity Timeline Activity Status Identify limiting factors * Sept 2011 Completed Review content pre-test data Nov 2011 Completed Review data from Teacher Attitude and Belief surveys

Nov 2011 Completed

Select initial 10 teachers Nov 2011 Completed Review content post-test data Feb 2011 Planned Select final 10 teachers Feb 2011 Planned Interview final 10 teachers and teachers who have dropped out of program

Mar-May 2011 Planned

20

• District- and school-level budget cuts and staff cutbacks .Teachers retiring early, leaving current school for other districts

Inadequate stakeholder support for the project • District and/or school leaders who do not support the project • Lack of PD time being given to teachers • Lack of in-class time for curriculum Insufficient short-term funding for the project, including insufficient capital expenditure on equipment and infrastructure • NSF budget reductions for future years • School district budget reductions Insufficient long-term funding to support recurrent costs • Lack of funding for post-course PD Persons involved in these activities: • Columbia University, Institute for Learning Technologies, Director of Research and

Evaluation: Dr. Susan Lowes

The implementation of this goal to date shows targeted progress. Data Management: Digital and paper-based transcripts, notes, and files of interviews conducted by Dr. Lowes or her research assistants are kept on a secure server or computers at Teachers College, Columbia University. No names of individuals, schools or districts will be reported in and public document.

21

Goal H: Data Management/Coordination: Collect, coordinate, manage, analyze and disseminate project data to make improvements at the level of the project, the partner organizations, and at the individual classroom levels

CIESE at Stevens Institute of Technology is responsible for the collection, management and storage of all pre/post assessments, surveys, evaluations, and interviews. A member of the CIESE staff ensures that data is well-scrubbed and in good form before turning it over to the evaluator. Assessments and processed data are stored in a secure cabinet or password-protected computer. Access is restricted to senior members of the researcher, evaluator and PI team. Names are replaced by codes to ensure confidentiality. Schools and individual students and teachers will not be reported in any public manner. Data collected and analyzed will be disseminated to district partners and other key stakeholders for use in program improvement at the program level, the district level, the school level, and the teacher level. Persons involved in this goal include: • A. Macalalag oversees data management; E. Whittaker, T. Herrington, C. Shields,

H. Harms, B. Mammen, G. Bartus, D. Brockway, J. Sayres, C. Jurado, (CIESE) participate in data collection, processing and analysis, along with S. Lowes (Columbia Teachers College) and G. Nyre (Nyre & Associates).

22

Goal I: Project Management/Governance: Manage and govern the project, its component programs and partner relationships through the creation of an Executive Committee, Project Advisory Board, Steering Committee and District Support Team

To achieve this goal, several collaborative groups were created, representing various and diverse stakeholders involved in the PISA2 project. The project is complex, involving participation across a diverse set of partners, roles, and individuals. Each group collaborates through a combination of face to face meetings, phone and email correspondence and conference calls.

Activity

Activity & Components

Activity Timeline

Activity Status

Executive Committee: PISA2 PIs PISA2 Faculty PISA2 Team

Initial Project Planning of PISA2 Program Components Ongoing Project Management of PISA2 Program Components

Develop plans, processes & procedures for: Publicity, recruitment, applications & acceptance of districts; Project personnel; Course Development (syllabi, delivery, materials, logistics, scheduling), Assessment & evaluation; NSRC Leadership/Science Reform Program; RIEE Workshops Monitor course delivery, partner participation & feedback, assessments (surveys, pre-post testing), personnel issues; Identify RIEE workshops; Initiate Pathways to Teaching Options; Redesign PISA2 components for Year 2

Spring 2010 June 2010- May2011 Jan-May 2011

Completed In Progress Planned

23

Steering Committee: PISA2 Faculty, PISA2 Team, District Partners NJ Dept of Ed, External Evaluator

Initial Project Planning Ongoing Project Implementation

Collaborate to: recruit Cohort 1 teachers, disseminate start-up information to teachers, identify comparison teachers, supply district data to evaluator Collaborate to ensure district and teacher commitment, attend Steering Committee meetings, identify comparison teachers, attend PISA2 events and NSRC conferences, communicate with Program Manager Recruit Cohort 2 teachers

Spring 2010 June 2010- May2011 Jan-April 2011

Completed In Progress Scheduled

PISA2 District Support Team: Curriculum & PD Specialists, Program Mgr.

Administration of pre/post assessments & surveys Conduct 1st PD day activities Conduct classroom visitations Address day to day issues Conduct 2nd PD day

June-Dec 2010 Oct 2010 Jan 2011

Completed In Progress Planned

Project Advisory Board: PISA2 PIs NJ Dept of Ed Evaluator K-20 Educators

Guide course development, evaluation & research on an as needed basis Attend Project Advisory Board meeting

June 2010-May 2011 Jan 2011

In Progress Scheduled

Persons included in these committee activities: Executive Committee:

• E. Whittaker, R. Besser, F. Fisher, T. Herrington, K. Sheppard (Stevens Faculty) • C. Shields, G. Bartus, A. Macalalag, D. Brockway, H. Harms, E. McGrath, B.

Mammen (CIESE) • M. Amendola (District Representative) • TBD (NJ Department of Education Representative) • J. Jacobson, St. Peter’s College

Steering Committee:

• Executive Committee members • District Liaisons: David Minnard, Bayonne Public Schools, Robin Wyche,

Camden Public Schools, Howard McKenzie, Hoboken Public Schools, Manisha Shah, Jersey City Public Schools, Laura Winters, Lakewood Public Schools, Audrey Becker, Margate Public Schools, Michael Amendola, Morris Public Schools, Kathleen Hanson, Mustard Seed School, Hoboken, Cherry Sprague, Princeton Regional Schools, John Bombardier, Red Bank Public Schools, Robert

24

Onorato, Saddle Brook Public Schools, Mary Ann Cinque, West New York Public Schools

• External Evaluator

PISA2 District Support Team: • C. Shields, G. Bartus, A. Macalalag, H. Harms, B. Mammen (CIESE)

Project Advisory Board:

• Dr. Richard Duschl, Penn State University • Dr. Anthony Petrosino, University of Texas at Austin • Dr. Robert Goodman, Bergen County Technical High School, NJ • Dr. Johannes Strobel, Purdue University • Dr. Christine Cunningham, Museum of Science, Boston, MA • Dr. Carlo Parravano, Merck Institute of Science Education, Rahway, NJ

Accomplishments, Challenges, and Responses:

• Job transitions (retirements, advancement, changes) have led to personnel shifts in the project, notably from the NJ Department of Education and the School District coPI. In some cases, new contacts have been identified, but in others, they have not yet.

o Dr. Patricia Camp, Morris School District, left the district and has been succeeded by Michael Amendola, Supervisor of Math & Science, Morris School District, whom we will nominate to serve as coPI.

o Dr. Sandra Alberti, NJ Department of Education has left NJDOE and no successor has been named.

o Due to the increased interest by Stevens undergraduate and graduate students in Pathways to Teaching programs and increased opportunities for faculty participation in PISA2, we would like to add Dr. Keith Sheppard, Associate Dean of Science & Engineering as a coPI

• The following Steering Committee members have been replaced by new district liaisons:

o Dr. Eva Ogens, Science Supervisor of Jersey City Public Schools, left the district and has been succeeded by Manisha Shah, Supervisor of Science, Jersey City Public Schools

o Debra Mercora, Supervisor of Curriculum, Lakewood Public Schools has been replaced by Laura Winters, Supervisor of STEM

• The following Project Advisory Board member has not been replaced as yet o Ms. JoAnn Dow, Public Service Electric & Gas, has retired. We aim to

invite her successor, Ms. Shauwea Hamilton, to serve in her place.

25

Goal J: Communications and External Relations: Build awareness of PISA2 program goals and activities among key internal and external stakeholder groups, including faculty, IHE students, parents, legislators, guidance counselors, and corporations. Utilize various media and technologies to communicate and publicize program goals, components, activities and events to all stakeholders and the public

The achievement of this goal contributes to the effective implementation of many program components since PISA2 involves geographically diverse districts across NJ, many stakeholders/partners in other states and PISA2 team members who are out in the field. Media/Technology

Activity

Activity Timeline Activity Status

PISA2 Web Site Development of website Post course information, evaluation/research questions, events, photos of participants, Instructional videos, news media articles

Summer 2010 Sept-May 2011

Completed In Progress

eLearn & Web Campus Register Teachers Post course schedules, assignments, message board, discussion questions, grades

June 2010 Sept-May 2011

Completed In Progress

Email Communicate PISA2

information to stakeholders on a timely basis

June 2010-May 2011

In Progress

Webinars Communicate with teachers, vendors, other remote stakeholders

June 2010-May 2011

In Progress

Conference Calls Communicate with teachers, vendors, public relation firm, NSRC, other remote stakeholders

June 2010-May 2011

In Progress

Brochures & Flyers Create and Distribute: PISA2

recruitment brochure, Pathways to Teaching Options flyer, NSRC/PISA2 conference brochure Redesign recruitment brochure

Sept-Dec 2010 Jan 2011

Completed In Progress

26

Events PR/Media Outreach

PISA2 Celebration Event: Congressman Albio Sires keynoted event for all district partners, including superintendents, administrators, teachers, and students. NSRC Building Capacity & Leadership Conference Various: Grant announcement; district recognition; classroom implementation; Congressional representation at Project Launch Meeting

Oct 2010 Nov 2010-May 2011 June 2010 and ongoing

Completed In Progress In Progress

Persons involved in these activities:

• E. McGrath, B. Mammen, A. Macalalag, C. Shields, H. Harms, J. Sayres, G. Bartus, H. Diaz, P. Slater, R. Cully (CIESE)

• E. Whittaker, T. Herrington (Stevens Faculty) • J. Childress, S. Shuler, A. DeLa Roche, M.Raucci (NSRC)

Accomplishments, Challenges and Responses: • October 4, 2010 project celebration attended by 75 district stakeholders and Stevens

representatives; keynoted by Congressman Albio Sires (D-NJ13). • November 10 and December 8-10, 2010 Building Awareness and Strategic Planning

Meetings attended by approximately 100 stakeholders and visited by Congressman Rush Holt (D-NJ12).

• Articles in major daily newspapers and news story on WABC-7 on PISA2 program

and participants.

Note: This document represents activities planned and underway as of December 2010. It will be revisited and updated quarterly, and submitted to NSF for review annually or as requested.