Sophie FranklinAZSTT sponsored postgraduate student

AimsWhat is impact?

What do we measure?

How do we measure it?

Case study of successful impact

What is dissemination?

Pathways of dissemination

What is impact? Impact is seen as the positive and negative, intended or

unintended long-term results of a project. Impact is a measure of change. Impact is not quick to see.

Importance of measuring:

Learn more from projects by assessing impact. Measure the ‘profit’ of an investment. If impact is credible, measurable and positive, it acts a positive

promotion for a project – this can lead to influence of policy development.

Activity

Total Spending £5,575,82

2

122 funded projects

2805 Teache

rs

1504Schools

How is the impact of a project measured?We use certain indicators to measure impact.

These are normally defined by the projects initial goals and targets.

They can be qualitative or quantitative. Achievement, progress, enthusiasm of students. Before and after questionnaires measure changes

in understanding of individuals Subjective views from individuals. Focus groups. Positive changes in approaches to teaching.

How can the Trust measure its Impact?

1.Papers published and cited

2.Website statistics – use of CPDUs and resources

3.Case Studies

Ways of looking at impact of the Trusts work:

International Journal of Science Education.

British Educational Research Journal.

Educational Research.

Journal of Research in Science Teaching.

Curriculum Journal.

Teacher Development.

Innovations in Education and Teaching International.

Title Authors Journal Cited Year

Developing attitude to science scales for use with

children of ages from five to eleven years

Tony Pell &

Tina Jarvis

International Journal of

Science Education42

200

1

Effect of the Challenger Experience on Elementary

Children’s Attitudes to Science

Tony Pell &

Tina Jarvis

Journal of Research in

Science Teaching15

200

2

Developing attitude to science education scales for

use with primary teachers

Tony Pell &

Tina Jarvis

International Journal of

Science Education11

200

3

Primary teachers' changing attitudes and cognition

during a two-year science in-service programme

and their effect on pupils

Tony Pell &

Tina Jarvis

International Journal of

Science Education12

200

4

Factors influencing elementary school children's

attitudes toward science before, during, and after

a visit to the UK National Space Centre

Tony Pell &

Tina Jarvis

Journal of Research in

Science Teaching17

200

5

What can web stats tell us about the popularity of the AZSTT website?

How can impact be measured?

The AZSTT website contains information on funded projects, published papers and resources produced through the funded projects.

Google analytics can be used to see interesting patterns of the use of the website.

Feedback on CPDUs can also be accessed via the content management system.

Lowest numbers in August....

Best time to reach the AZSTT audience? Oct/Nov & Jan/Feb. Why? Percentage of new visits is also lowest in these months.

Popularity of resources on the AZSTT website

Resources produced by AZSTT projects are now split into two categories on our new website:

Curriculum Materials CPDUs

Sc1 Scientific Enquiry -Focussed assessment sheets from Bath Spa’s Improving Science Together project.

STRATA – Science To Raise And Track Achievement, from a 2001 cluster project in Cambridgeshire. Special needs across all key stages and abilities were addressed

Popularity across the world.....

CPDU FeedbackTalking Science

“I found this CPD session very relevant and interesting. It really reinforces the importance of using dialogue and other principles of AfL in science lessons.”

Fostering Curiosity In Early Years Science “It was Fantastic .”

Science Clubs “Very useful. Am thinking of using the resources for girl guides. Girls need encouragement to gain confidence in science.”

Top rated CPDUs are:

Talking Science - 5Fostering Curiosity In Early Years Science - 5STAY - 5

Thinking Beyond the Classroom“I enjoyed this and will use some of the activities in my own area, Horticulture.”

Case studies allow us to go into greater detail to find out more about

the impact of a project. We can learn from this to make future

projects more sustainable.

Improving Science Together

Bath Spa, 2000-2002. 20 Primary schools and 4 Secondary schools

in Bristol and South Gloucestershire. Focus on Assessment; developing lesson

plans and accompanying assessment sheets. Huge focus on subject leadership. Produced two major resources for

dissemination.

Resources from the Project

1. A set of online materials on the AZSTT website to support primary teachers in planning for focussed assessment of scientific enquiry.

2. Online CPDU for primary teachers on the AZSTT website on the use of ‘floorbooks’ as a formative assessment strategy in science.

Strategy for looking at the Impact of the projectImpact on practice in science enquiry

assessment in the project schools.Impact on the subsequent careers and practice

of the project teachers involved.Impact on practice of teachers outside the IST

project.Data from AZSTT website on wider

dissemination.

ObjectivesOne:Tracing and interviewing of teachers, past and

current.

Two:Use of IST resources with new teachers.

Three:AZSTT archive and website data.

Objective One Tracing and interviewing of IST

teachers, past and current. Headteachers : 5 – Still Headteacher

1 – Advisor for workforce and governance

1 – Priest

3 – Retired

10 – Unsure

SSLs: 3 – Still at same school

4 - @ different schools in the area

1 - @ Graduate School of Education UoB

3 – Unknown whereabouts but contactable

8 – Unsure

Key teachers:

4 - @different schools in the area

5 - @ same school

1 – Unknown whereabouts but contactable

There are a number of ways of looking at the impact just within the tracing of past and current teachers:

Teachers who have remained at the same school in the same position – did the project have longevity?

Teachers who have stayed at the same school and been promoted – how did the project impact on professional development?

Teachers who have moved to another school in the area which was also an IST project school – did they find the resources produced from the project were being used differently?

Teachers who have moved to another school not associated with the project – could what was learnt be implemented into a new school?

Teachers who were not involved in the project, but are at the project schools – was there a legacy of the project left behind?

Cath Foote Avon Primary Original IST Key Teacher still at the same school

Abbie Cowland Bankleaze Primary New science coordinator at original project school

Sam Nunn Bowsland Green New science coordinator, currently trailing new science assessment

strategies.Erica Cheary Callicroft Junior

Was key teacher at the time of the project, now SSL at same school.Liz Doorbar Deputy Head at Westbury-on-Trym.

Was key teacher at Charborough Road Primary school, now deputy head at new school.

Eric Rydon Charborough Road Primary New Science Coordinator at original project school.

Nicola Bailey Headteacher at Charborough Road Primary. Still Headteacher at the same school.

Emma Gundy St Michaels Primary New science coordinator.

Kirsteen Craig Sea Mills Junior Current Headteacher

Amy Easterbrook Infant teacher at Highdown Was subject leader at Wheatfield Primary during project.

Dr Angela Greenwood Still head at Little Stoke Primary, (but leaving end of year).Heather Allen Little Stoke Primary (still there)Jo Davey Shield Road Primary (still there)

Objective Two Using IST resources with

new teachers

Teachers external to the project who have been IST resources:

Floorbooks:Emma Rigarlsford Primary Science Leading Teacher – AZSTT projectKay Coverdale Member of College and has been using

Floorbooks.Linda James Has been using FloorbooksAnna Peart Has been using FloorbooksRachel Brooks Recently introduced to them on Science Subject

Leaders course , now using them in schools

Focussed assessment sheets:Sarah Doneghan Member of College and has been using the sheets

across her cluster.Helen Grainger Uses both Floorbooks and Focussed assessment

sheets

Sarah on Planning Sheets“I absolutely love them.”

“Great way of developing progression and development of skills.”

“As County Durham AST, I recommend them to all schools and in a recent Ofsted science subject inspection a school using them got outstanding in all areas.”

Rachel on Floorbooks“They encourage learners to contribute their ideas and we all know science is a continual assessment from initial thoughts to testing and challenging ideas, Floorbooks allow me to do this.”

“Swiftly provides evidence. They are easy and efficient.”

“Interactive and engaging.”

“So easily shared with parents and carers.”

Why have Floorbooks been such a popular resource?

Over 12 years the basic idea hasn't

changed, or needed to change.

Integrity of the idea never lost even if context

changes.

Not a tip or a trick, very open

and straightforward approach which

doesn’t need lots of time to

set up.

Very valuable approach

which works from early

years to early secondary.

Easy to see usefulness in

AfL and evidence for

APP.

Has value and

validity.

Natural part of

Assessment.

Further Impact

Primary Science Quality Mark submissions – what can we learn about science in primary schools from the PSQM?

Teachers mentioned they were using AZSTT assessment frameworks.

Objective Three AZSTT archive and

website data.

Again through the use of Google Analytics, the popularity of the resources produced by the Improving Science Together project can be looked at.

Only for Sc1

Scientific Enquiry

Floorbooks CPDUFloorbooks page views per month are smaller than that of the Sc1 Scientific Enquiry section of the website.

However, a much longer time is spent on these pages.

Floorbooks are not so popular across the world, mainly only in Australia, Netherlands, USA, UAE, Guernsey and New Zealand. But have a huge audience in the UK.

Why such a wide impact?Dissemination strategy worked very well.Routes of dissemination:ASE South West Region MeetingDissemination Day to Primary Science

subject Leaders from other South Gloucestershire and Bristol LEA schools

ASE Annual Meeting via the AZSST websiteThrough the work of university tutors and

academic publications

Future of DisseminationThe role of key people.

How can we make sure that dissemination occurs through other routes?

The COLLEGE!

Final thoughts....

For project leaders just finishing up...

For project leaders just starting...

For the College...

Collaboration is key, this conference is a prime example of that.