the linkages of learning outcomes, learning … › users › tqf-hed › news › filesnews ›...

1

THE LINKAGES OF LEARNING OUTCOMES,

Workshop on

LEARNING ACTIVITIES AND ASSESSMENTS IN HIGHER EDUCATION

Australian and Thai Perspectives

THAILAND 19 – 28 June 2012THAILAND 19 28 June 2012

Thailand Qualification Framework

Nattavud Pimpa

Tim Moore

Learning Outcomes, Assessment and Feedback

2

Sessions 3 & 4: Learning Objective

By the end of these two sessions the participants will be able to link learningparticipants will be able to link learning outcomes with assessment, and strengthenteaching practice through effective feedback.

TQF

What are the purposes of assessment?

3

There isThere is nothing to fear from changes in assessment practices

What are the purposes of assessment?• to measure students’ preparedness for further study or professional accreditation

• to rank students, relative to one another, for the , ,purposes of competitive scholarships or other opportunities

• to provide feedback on student learning for both students and staff

4

What are the purposes of assessment?

• to provide feedback• to provide feedback on teaching for staff

• to define and protect academic standards

• to direct students’ learning

What are your experiences in assessment?

5

Step 5: Determining Assessment

•Fair, equitable and transparent.

Steps in Designing Learning Outcomes

• Must reflect the learning outcomes

•Incorporate a range of types or modes of assessment appropriate to the nature of the unit, method of delivery and the students involved

•Design appropriate assessment strategies that ideally engage the learners in activities they can relate to real‐life or workplace situations

TQF

Thinking critically and making judgments (Developing arguments, reflecting, evaluating, assessing, judging)

Steps in Designing Learning OutcomesExample

•Essay

•Report

• Journal

•Letter of advice to … (about policy, public health matters … )

•Present a case for an interest group

•Prepare a committee briefing paper for a specific meeting

•Book review (or article) for a particular journal

•Write a newspaper article for a foreign newspaper TQF

6

Solving problems and developing plans (Identifying problems, posing problems, defining problems, analysing data, reviewing, designing experiments planning applying information)


designing experiments, planning, applying information)

•Problem scenario

•Group work

•Work‐based problem p

•Prepare a committee of enquiry report

•Draft a research bid to a realistic brief

•Analyse a case

TQF

Demonstrating knowledge and understanding (Recalling, describing, reporting, recounting, recognising, identifying, relating and interrelating)


relating and interrelating)

•Written examination

•Oral examination

•Essay

•Report p

•Devise an encyclopaedia entry (Wiki)

•Short‐answer questions: true/false/ multiple‐choice questions (paper‐based or computer‐aided assessment

TQF

7

Designing, creating, performing (Imagining, visualising, designing, producing, creating, innovating,


performing)

•Portfolio

•Performance

•Presentation•Presentation

•‘Hypothetical’

•Projects

School/Department/Area

Check the Alignment

Checking to see that assessment and learning outcomes align requires determining whether:

Steps in Designing Learning Outcomes

outcomes align requires determining whether:

• the assessment includes knowledge, understanding or skills not in the learning outcomes then review the learning outcomes. If after review you are still happy with the learning outcomes then you will have to change the assessment;change the assessment;

• the assessment misses knowledge, understanding or skills in the learning outcomes then review the assessment.

TQF 14

8

Core Principles of Effective Assessment

Objectives for higher education assessment

1. Assessment that guides and encourages effective approaches to learning

2. Assessment that validly and reliably measures expected learning outcomes, in particular the higher‐order learning that characterises higher education

3. Assessment and grading that define and protect academic standards

Adapted from the Centre for the Study of Higher Education, University of Melbourne


1. set clear expectations

Well designed assessment should …

2. establish a reasonable workload (one that does not push students into rote reproductive approaches to study)

3 id t iti f

TQF

3. provide opportunities for students to self‐monitor, rehearse, practise and receive feedback.

9



10

16 INDICATORS OF EFFECTIVE ASSESSMENT IN HIGHER EDUCATION

A h kli f li i d


A checklist for quality in student assessment

1. Assessment is treated by staff and students as an integral component of the entire teaching and learning process.

2. The multiple roles of assessment are recognised. The powerful motivating effect of assessment requirements on students is understood and assessment tasks are designed to foster valuedunderstood and assessment tasks are designed to foster valued study habits.

3. There is a faculty/departmental policy that guides assessment practices. Subject assessment is integrated into an overall plan for course assessment.


4. There is a clear alignment between expected learning outcomes, what is taught and learnt, and the knowledge, skills (and attitudes) assessed.

5. Assessment tasks assess the capacity to analyse and synthesis new information and concepts rather than simply recall information which h b dhas been presented.

6. A variety of assessment methods is employed so that the limitations of particular methods are minimised.

7. Assessment tasks are designed to assess relevant generic skills as well as subject‐specific knowledge and skills.

11

8. There is a steady progression in the complexity and demands of assessment requirements in the later years of courses.

9. There is provision for student choice in assessment tasks and weighting at certain times


certain times.

10. Student and staff workloads are considered in the scheduling and design of assessment tasks.

11. Excessive assessment is avoided. Assessment tasks are designed to sample student learning.

12. Assessment tasks are weighted to balance the developmental (‘formative’) and judgemental (‘summative’) roles of assessment Early low stakes lowand judgemental (‘summative’) roles of assessment. Early low‐stakes, low‐weight assessment is used to provide students with feedback.

13. Grades are calculated and reported on the basis of clearly articulated learning outcomes and criteria for levels of achievement.

14. Students receive explanatory and diagnostic feedback as well as grades.


p y g g

15. Assessment tasks are checked to ensure there are no inherent biases that may disadvantage particular student groups.

16. Plagiarism is minimised through careful task design, explicit education and appropriate monitoring of academic honesty.

12


What students value in assessment

• Unambiguous expectations

• ‘Authentic’ tasks

• Choice and flexibility

Why Feedback?

Feedback

•Without feedback, mistakes go uncorrected and good performance is not reinforced.

•Without feedback, our learners may not know how they are doing, if they are doing well, or if there are l t f th i f th t d t belements of their performance that need to be improved so that they can be competent.

13

Mi i l F db k/R i f t

Feedback

•Minimal Feedback/Reinforcement

•Evaluation

•Feedback

How do They Differ?

Reinforcement/Minimal Feedback

•Statements expressing positive (or

Feedback

•Statements expressing positive (or negative) reaction to a behaviourwhich aims to increase (or decrease) the likelihood of that behaviour happening again

–“That was a great presentation”–“You need to work on your

i kill ”presentation skills”

•Often mistaken for feedback–Timing is similar ‐ immediate

14

Evaluation

Q lit ti j d t hi h k l ’

Feedback

•Qualitative judgement which ranks a learner’s performance in comparison to other learners

–3.7 for professionalism competence

•Often the only measure of performance visible to the learner

•Usually given after the performance is over

Feedback

•(Reinforcement or correction) + Explanation

Feedback

•(Reinforcement or correction) + Explanation

•Keeps you on course to meet goals

•Allows you to adjust your course to meet goals

•Given immediately after the performance or at some time soon after, when the learner still has time totime soon after, when the learner still has time to demonstrate improvement

15

…as it is part of other training cultures

Barriers to Feedback

Barriers

•It is uncomfortable


•It is uncomfortable–Avoid confrontation –Learner will not be receptive

–No one ever gave me feedback

–Not quite sure how to do it–I hated getting feedback

16

•I’m not sure of the goals or the


gexpected behaviour

•Not sure that the observed behaviour is really a problem

–Need to see it twice

•It’s not my job

•It’s not that important

•There’s not enough time

What are they?

Essential Components of Feedback

•What was done well

•What could be done better

•What could be done to improve next timep

17

•Well timed and expected

Characteristics of Feedback

•Based on first hand data

•Phrased in descriptive language, based on specific remediable behaviours

•Should be undertaken withShould be undertaken with teacher and learner working as allies, with common goals

Ende J. Feedback in Clinical Medical Education. JAMA 1983;250:777-781.

The Old Feedback Sandwich

Praise

Criticism

Praise

l bl ?Is it more palatable?

18

The NEW Feedback Sandwich

Ask

TellTell

AskAdaptation of “The New Feedback Sandwich,’ common in patient-physician communication literature; adapted by Lyuba Konopasek, MD, for use in feedback settings.

Ask

The New ‘Feedback Sandwich’

•Ask learner to assess own performance first–What went well and what could have gone better?

–What were their goals?

–Have they ever seen a problem like this before?

•Begins a conversationg

•Assesses learner’s level of insight

•Useful for second‐hand feedback

19

Tell


• Tell what you observed: diagnosis and explanation–React to the learner’s observation–Include both positive and corrective elements

–“I observed….”–Give reasons in the context of well-defined shared goals

Ask (again)


•Ask about recipients understanding and strategies for improvement–What could you do differently?

–Give own suggestionsGive own suggestions

–Perhaps even replay parts of the encounter: “show me”

–Commit to monitoring improvement together

20

Limit the Quantity!!!

The ‘Feedback Sandwich’

Ask learner to assess own performance firstWhat went well and what could have gone better?

Checklist: The Feedback Dialogue

Tell what you observed:diagnosis and explanationReact to the learner’s observationInclude both positive and constructive elementsGive reasons in the context of well‐defined shared goalsRegulate quantity

Ask about recipients understanding and strategies for improvementWhat could you do differently?What could you do differently?Give own suggestionsPerhaps even replay parts of the encounter ‐ show meCommit to monitoring improvement together

Adaptation of “The New Feedback Sandwich,’ common in patient-physician communication literature; adapted by Lyuba Konopasek, MD, for use in medical resident feedback settings.

21

1. Return to your groups and yesterday’s scenarios

ACTIVITY

2. Look at your learner outcome.

Consider assessment needs and opportunities, then write down suitable assessment activities

3 Wh f db k ill id d f3. What feedback will you provide to students of the subject?

Adaptation of “The New Feedback Sandwich,’ common in patient-physician communication literature; adapted by Lyuba Konopasek, MD, for use in medical resident feedback settings.

Scenario 2.

Degree: Graduate Diploma of Business

Subject: AEC and Thailand

Learning Outcome: By the completion of this subject students will be able to analyse the political, social and economic impacts of the AECsocial and economic impacts of the AEC

TQF 42

22

Scenario 3.

Degree: Master of Arts in Political Science

Subject: Globalisation and Diversity

Learning Outcome: By the completion of this subject students will be able to effectively apply understanding of globalisation concepts tounderstanding of globalisation concepts to issues of cultural diversity, social justice and ethical accountability

TQF 43

Scenario 4.

Degree: Bachelor of Science in Multimedia Studies

Subject: Website Development

Learning Outcome: By the completion of this subject students will be able to design user‐friendly and interactive web sitesinteractive web sites

TQF 44

23

Scenario 5.

Degree: PhD

Subject: Research Methodology

Learning Outcome: By the completion of this subject students will be able to critically evaluate a research project and select appropriate researchproject and select appropriate research methods to undertake the study

TQF 45

Scenario 6.

Degree: Doctor of Medicine (MD)

Subject: Treatment of Bacterial Infection

Learning Outcome: By the completion of this subject students will be able to correctly choose and deliver effective treatment for bacterial infectionseffective treatment for bacterial infections

TQF 46

24

What about feedback to teachers…?

Feedback

Afterthoughts

48

25

Evaluations

49

THANK YOU

the linkages of learning outcomes, learning … › users › tqf-hed › news › filesnews ›...

Documents