DATA COLLECTION – TO CURRICULAR CHANGE

USING YOUR ASSESSMENT DATA TO DRIVE CURRICULUM

CHANGES

ASSESSMENT “…the systematic collection,

review, and use of information about educational programs undertaken for the purpose of improving student learning and development.”

Palomba & Banta, 1999, pg. 4

THE 50,000 FOOT LEVEL Start with the culture because faculty

must be comfortable with the assessment culture before they feel comfortable with assessment!

Determine the role that assessment will play in college processes – and what it WON’T be used for! Improving Student LearningProgram EvaluationBudgetingNOT Faculty Evaluation

START WITH THE CULTURE

Value Campus Culture & History

(Assessment is not a one size fits all)

Respect and Empower People – especially

Faculty!

Value Assessment by providing

appropriate resources and infrastructure

Value innovation & risk taking to improve

teaching (even if it fails)

VALUE INNOVATION

“I have not failed. I've just found 10,000 ways that won't work.” ― Thomas A. Edison

Assessment

I think we are about to turn the corner on this whole “Assessment” thing…

EFFECTIVE ASSESSMENT Is linked to decision making about the

curriculum – Paloma & Banta

Measures real-life gaps in desired skills & performance – Swing, et. al.

Leads to reflection and action by faculty – Paloma & Banta

WE’RE COLLECTING DATA BUT…

Institutions and faculty are good at collecting data – just not as good at using data to drive curricular changes

This part of the assessment cycle is often called:“Closing the loop”What loop – and who left it open?

HELPING FACULTY GET TO CLOSING THE LOOP Start with asking the right questions

Questions to help faculty focus their assessment activitiesWhat should students learn?How well are they learning it?What evidence do you have?What are you doing with the evidence?

ASSESSMENT TRAIL

Question

Plan

Collect & Score

Analyze/Reflect

Report/Act

Assessment Cycle

ASK THE “RIGHT” QUESTION The right question is:

Meaningful – it is a question that faculty want to know the answer to, and that knowing the answer will help them impact student learning

Measurable – work at asking a question that faculty can answer – usually that means narrowing down the question

Manageable – it is important to keep the question and the process of collecting data manageable – this isn’t the only or primary job that faculty have

ASSESSMENT FOR LEARNING Once faculty has determined the actual

question that is going to be answered this will drive the methodology of assessmentPre/Post TestEmbedded test questionsProject Based Assessment PortfoliosSurveysPerformance, etc., etc.

How often will they collect the data, in what classes – What makes sense? (Remember keep it manageable)

DATA COLLECTION – NOW WHAT First encourage faculty to take time to

organize the dataExcel is an easy accessible tool available to

99% of all facultyOffer Data management workshops

What does on a row, column, what are some analyses you can run in Excel

QUALITATIVE DATA ANALYSIS Qualitative Data is an appropriate tool

for many disciplinesMap out the requirements of the

assignmentSearch for themes in student responsesTrack how often key curricular themes

appear

STATISTICS - SMATISTICS If you are a IR person – or big on Stats –

close your ears for a minute… Rather than worry so much about

reliability, validity, statistical significance, think about these assessment projects as Action Research

Action Research focuses on getting information that will enable faculty to change conditions in a particular situation in which they personally involved Seeks to solve a problem, or Inform local practice – specifically –

classroom/course practice!

STEPS IN ACTION RESEARCH Identify the Research Question Gather the necessary information Analyze and interpret the information Develop an action plan

Sound familiar?? – Think Cycle of Assessment

LOOKING AT THE DATA Move beyond “Averages” Look at the Spread of the Data

What does the spread indicate? Is the data evenly distributed?Are there large gaps? Where do they exist?

Has the faculty member or department decided what is an acceptable level of performance?

WHEN TO CHANGE CURRICULUM – WHEN TO MAKE OTHER CHANGE?Curricular Change Follow the data trail – and then talk to

invested participantsData leads to students not understanding a

concept What is the benchmark of performance? Curriculum mapping – where does the concept

occur? How is the concept taught (pedagogy) Where is the concept reinforced? (Scaffolding) What changes can we (faculty/department)

make to the curriculum to help students understand and apply concept

How will we measure this curricular change to see if it is successful

WHEN THE DATA SUPPORTS OTHER CHANGES

Case Study – Visual CommunicationsGeneral Education Curriculum Assessment

Cycle Visual Communications

Mass Exodus of Visual Communications Classes

Faculty couldn’t come up with acceptable assignments to show how they were assessing visual communications

Focused discussion with department chairs/faculty groups on the issueFaculty felt unqualified to “teach” visual

communications Professional Development, in-service workshops,

teaching circles, etc.

MAKING THE CURRICULAR CHANGE If the focus throughout the process has

been on student learning (versus report writing), then faculty will be more open to making curricular changes

Next Assessment Cycle – what difference did the change make? Was there a difference in performance?Make sure appropriate time has elapsed for

changes to be in effectMake sure the measurement is parallel to

the previous assessment

ARE WE DONE YET?“CLOSING THE LOOP”

“When are we done with a learning outcome?”

Did you see improvement?

Did you meet your benchmark

performance?

Are you satisfied?

Do you see a greater need/question that

needs to be asked?

RANDOM THOUGHTS

KEEP THE FOCUS ON WHAT MATTERS MOST – STUDENT LEARNING

Reports and data analysis that don’t focus on student learning are a waste of paper

Faculty must be engaged in making sense of and interpreting assessment results – administration can’t do it for them

Share SuccessesWith permission – share assessment results

from other departments/disciplinesGet the faculty to tell their success stories –

it carries more weight with their peers

MAKE REALISTIC PLANS TO AVOID PITFALLS Effective Assessment Takes time to Plan,

Implement and SustainDon’t expect instant results – reliable data

takes time to gather

Make sure the assessment is asking a question that can be answeredThis means being narrow in scope – rather

than throwing as much stuff as possible at a wall and seeing what sticks!

DON’T FORGET TO ENGAGE ADMINISTRATION While it is important for faculty to “own”

assessment, support needs to come from administration:Recognizing faculty effortsAttending faculty functionsProviding appropriate resourcesConsidering policy implications

Questions??

Sheri H. Barrett, EdDDirector of Outcomes AssessmentJohnson County Community Collegesbarre13@jccc.edu

913-469-7607