gathering, analyzing, using & reporting assessment data university of wisconsin parkside january...
TRANSCRIPT
Gathering, Analyzing, Using & Reporting Assessment Data
University of Wisconsin Parkside January 20, 2012
Susan HatfieldWinona State University
1. Gathering Information
Learning Objects• Standardized Exam, abstract, advertisement,
annotated bibliography, biography, briefing, brochure, budget, care plan, case analysis, chart, cognitive map, court brief, debate, definition, description, diagram, dialogue, diary, essay, executive summary, exam, flow chart, group discussion, instruction manual, inventory, lab notes, letter to the editor, matching test, mathematical problem, memo, micro theme, multiple choice test, narrative, news story, notes, oral report, outline, performance review, plan, presentation, process analysis, proposal, regulation, research proposal, review of literature, taxonomy, technical report, term paper, thesis, word problem, work of art. (Walvoord Anderson 1998).
Learning Objects
• Test• Project• Assignment• Portfolio• Recital• Performance• Presentation• Exhibit• Internship
Student LearningOutcome
3
2
1
4
Components Test Questions
1 2 3 4
5 6 7 8
10 11 12
1713 14 15 16
9
18
Student LearningOutcome
3
2
1
4
Components Assignment
1
2
3
4
Student LearningOutcome
3
2
1
4
Components Assignment
1
2
Student LearningOutcome
3
2
1
4
Components Experience
Assessing Learning
• Test• Project• Case Study• Portfolio• Recital• Performance• Presentation• Exhibit• Internship
• Number or percentage correct
Student LearningOutcome
3
2
1
4
Test Questions
1 2 3 4
5 6 7 8
10 11 12
1713 14 15 16
9
18
73%
82%
64%
87%
% correct
Assessing Learning
• Test• Project• Case Study• Portfolio• Recital• Performance• Presentation• Exhibit• Internship
• Number or percentage correct
• Scale
Scale
• Semantic Differential• Repeat mode
• effective / ineffective
Student LearningOutcome
2
1
Components
1 2 3 41 2 3 41 2 3 4
1 2 3 41 2 3 41 2 3 4
effective ineffective
Scale
• Semantic Differential• Repeat mode
• effective / ineffective
• unique descriptors enhances / distractsoriginal / commonalways / neverengaged / disconnected
Student LearningOutcome
2
1
Components
1 2 3 41 2 3 41 2 3 4
1 2 3 41 2 3 41 2 3 4
Assessing Learning
• Test• Project• Case Study• Portfolio• Recital• Performance• Presentation• Exhibit• Internship
• Number or percentage correct
• Scale
• Rubric
Rubrics
• Analytical• Numerous components which you want to
assess individually• Summative
Components
Analytical / Summative Rubric
Does notmeet
expectations
Outcome:
Components
Meetsexpectations
Analytical / Summative RubricOutcome:
Components
Exceedsexpectations
Outcome:Analytical / Summative Rubric
Components
Analytical / Developmental RubricOutcome:
complexity
Components
Analytical / Developmental RubricOutcome:
detail
Components
Analytical / Developmental RubricOutcome:
sophistication
Components
Analytical / Developmental Rubric
Fourth year
Outcome:
Rubrics
• Analytical
• Holistic• - generally smaller assignments
with fewer criteria, shorter turn around time.
Exceptionalcomponentcomponentcomponent
Level
Holistic Rubric
Above Average
componentcomponentcomponent
Averagecomponentcomponentcomponent
Below Average
componentcomponentcomponent
Resource
www.winona.edu/air/rubrics.htm
PRACTICAL ADVICE:
Pilot test your rubrics!
VerbalDelivery
NonverbalDelivery
Organization
Does notmeet
Meets Exceeds
1%
3%
0%
15%
5%
3%
84%
92%
97%
Evidence 1% 6% 93%
Transitions 2% 8% 90%
2. Analyzing Data
Interpreting Data
Consistency - over time
Consensus - different populations
Distinctiveness - different situations/ variables / items
Consistency
• Examines the same practice of and individual or group over time
• Key question: • Has this person or group
acted, felt, or performed this way in the past / over time?
Consistency
How well are students performing on thelearning outcome?
Highperformance
Lowperformance
04 05 06 07 08 09 10 11
Consensus
• Comparison to or among groups of students • Variation between disciplines, gender,
other demographic variables • Key questions: • What is the general feeling, outcome,
attitude, behavior? • Do other groups of people act, perform
or feel this way?
Consensus
How well are students performing on thelearning outcome?
Females Males Transfers OTA
Highperformance
Low performance
Distinctiveness
• Examines individual or cohort perspectives across different situations, categories
• Key Question: • Does a person or group respond differently
based upon the situation, item, issue?
Distinctiveness
How well are our students achieving ourCore Learning Outcomes?
HighPerformance
Low Performance
3. Using Data
How Assessment Works
componentcomponentcomponentcomponentBASELINE
CompareAgainst
Benchmarks,Standards,
Targets,Past
Performance
New / Revisedlearning event New / Revisedlearning eventNew / Revisedlearning event
Year 2
Acting on Results
• Development • Faculty, Staff, Student
• Infrastructure• Policy, Process, Planning
• Curriculum• Learning Opportunities
The Seven Principles for Good Practice in Undergraduate Education
• 1. Student-Faculty Contact• 2. Cooperative Learning• 3. Active Learning• 4. (Prompt) Feedback • 5. Time on Task• 6. High Expectations• 7. Respect for Diverse Talents and
Ways of Learning
IncreaseReduce
DetermineMaintain
the
NumberAmount
PercentageRatioLevel
of
of
whothatwith
SomeoneMajorsFaculty
StaffCommunity members
Employers <<do or have>>Something
Something
Can analyze
Performance
data
IncreaseReduceImprove
DetermineMaintain
the
SomeoneStudentsFaculty
StaffCommunity members
Employers
Somethingapplications
grantsInterviews
requestworkshops
reportssatisfactionengagement
byof
of
whothatwith
Action
Action
Action
NumberAmount
PercentageRatioLevel
<<do or have>>Something
Educating…..Assisting …..
Explaining …..Assisting …..
Developing …..Consulting …..Delivering …..Facilitating …..Creating …..Nurturing …..
Negotiating …..Sponsoring …..
Training …..Translating …..Validating …..Updating …..Enforcing …..Testing …..
Assessing …..Reporting …..
Issuing …..Communicating …..
Integrating …..Responding …..Arranging …..
Coordinating …..Providing …..
Organizing …..Producing …..Training …..
Maintaining …..Supporting …..Directing …..Designing …..Collecting …..
Developing …..Responding …..Reviewing …..Studying …..
Distributing …..Executing …..Recruiting …..Generating …..
Implementing …..Investigating …..Designing …..
Collaborating …..Revising …..
Scheduling …..Sharing …..
IncreaseReduceImprove
DetermineMaintain
the
NumberAmount
PercentageRatioLevel Something
applicationsgrants
Interviewsrequest
workshopsreports
satisfactionengagement
byof
of
whothatwith
SomeoneStudentsFaculty
StaffCommunity members
Employers <<do or have>>Something
IncreaseReduce
DetermineMaintain
the
NumberAmount
PercentageRatioLevel
of
of
whothatwith
SomeoneSecond year
studentsFaculty
StaffCommunity members
Employers<<do or have>>
Something
Demonstratecompetence
in criticalthinkingSomething
Action
Action
Action
by
IncreaseReduce
DetermineMaintain
the
NumberAmount
PercentageRatioLevel
of
of
whothatwith
SomeoneSecond year
studentsFaculty
StaffCommunity members
Employers<<do or have>>
Something
Demonstratecompetence
in criticalthinkingSomething
Action
Action
Action
by
4. Reporting Data
Outcome outcome
What did you look at? object
What did you find? results
What does it mean? analysis
What are you going to do
about it?action
What happened? reaction
Spring 2012
Summer 2012
Fall 2012
Spring 2013
Summer 2013
After That
Gathering, Analyzing, Using & Reporting Assessment Data
University of Wisconsin Parkside January 20, 2012
Susan HatfieldWinona State University