10 Dec 2010

NSS Assessing Student Learning Series for the HMSC Curriculum (4) -

Designing Internal Assessment

Part 2: Basic Question Setting SkillsPart 2: Basic Question Setting Skills

Mr. KWOK Kai-yin, Bill

2

Tools for setting HMSC questions

• HMSC Curriculum and Assessment Guide Curriculum Structure and Organization Assessment Objective

• SRR Information Package Assessment Framework Level Descriptors Sample papers and Marking Schemes Annotated Exemplars (for illustration of performance standards in various level)

• HMSC Resource Kits

• References (Websites, News, books, Journals, Publications etc.)

3

What are measured?

demonstrate comprehensive knowledge and understanding of issues, concepts and principles in health and social care across all systems and from all perspectives

apply concepts of health and social care to unfamiliar situations in a wide range of contexts

demonstrate excellent critical thinking and problem solving skills, making inferences, judgements and evaluations when analysing interconnected HMSC issues

effectively communicate ideas in a coherent and logical manner through extensive and sophisticated use of terminology and in an appropriate form

design and conduct creative and focused investigations, synthesise and evaluate data collected, and draw valid conclusions in authentic health/ social care situations Level Descriptors (L5)

4

Paper 1

• Questions will be set on the compulsory part of the curriculum.

Paper format and question types

Part Marks Remarks

Paper 1

(2 hrs)

Max. marks: 80

Section A 32 marks Short Questions (Compulsory)

Content-based. Mainly focus on factual recall; Simple and direct answers are needed.

Section B 48 marks Structured Questions

Data-response questions. Data analysis is required. Data may be presented in the form of figures, tables, charts etc. Focus on assessing critical thinking and reasoning skills when interpreting phenomenon/ trends/ patterns of issues. Higher cognitive level is required.

5

Paper 2

• Questions will be set on both the compulsory and elective parts of the curriculum.

Paper format and question types

Part Marks Remarks

Paper 2

(11/2 hrs)

Max. marks: 60

Section A 24 marks Short Questions (Compulsory)

Scenario-based. Focus on assessing the students’ ability on the application of knowledge in authentic situation.

Section B 36 marks Issue-based Questions (2 out of 3)

2 Case study questions. Focus on studying HMSC related case from different perspectives and/ or across different levels. Assembling knowledge from different topics is required. Problem solving and evaluation skills are assessed.

1 Essay type question. Focus on a health/ social care issue.

6

Paper 1: Section A

Question 6

Short question: Mainly focus on factual recall

7

Paper 1: Section B (1)

Question 10

Data response question: Answers should make reference to the raw

figures/ trends/ patterns of the data provided.

8

Paper 1: Section B (2)

Question 11

Data response question: Answers should make reference to the raw

figures/ trends/ patterns of the data provided.

9

Paper 1: Section B (3)

Question 13

Data response question: Answers should make reference to the

content of the information provided

10

Paper 2: Section A

Question 4

Scenario-based : Answers are identified and interpreted from the information provided (the scenario); no mark will be given to answers purely based on factual recall. High quality distractors may be included for this type of question.

11

Paper 2: Section B (1)

Question 6

Essays: limited response with suggested answers (accept alternative reasonable answers)

12

Paper 2: Section B (2)

Question 8

Essays: open-end question under pre-set framework

Marking scheme should include clear assessment guidelines and mark allocation for each areas.

13

Activity One: Classification

• Teachers in groups, classify the questions designed by your group members into various sections of the HMSC Papers.

14

Action Verbs in Question Setting (1)

15

Action Verbs in Question Setting (2)

16

Action Verbs in Question Setting (3)

17

Teachers in groups,

identify the “action verbs” that are commonly used in question setting of the subject.

Activity Two: Action verbs

* Different action/ command words may have different interpretation and cognitive requirements in different contexts

18

Common action/ command words

Identify …

State…

Describe

Suggest……

Explain…..

Discuss …..

Compare and contrast…

Relate … to……

Illustrate

Elaborate……

Analyse….

Criticize …

Propose…..(a new idea)

Comment on ….

Evaluate..

Justify your views…..

19

….from various perspectives

As a ….., what is your opinion (authenticity)

Suggest the ways

With reference to ….

….. the pros and cons of …..

According to…

Relate your answer to……

Identify/ describe the role(s) of …

Comment on …. and justify your views…..

To what extent….

Compare the relative importance…

Comment on the validity of ….

Common question words

20

Support viewpoints/ arguments with relevant concepts/ facts/ information.

Analyse and answer the question from various perspectives/ across different levels

Strive hard to prevent factual errors when quoting evidence to support viewpoints

Pay attention to the focus of the question – do not miss the key word(s) of the question

Avoid contradictory assertions

In essays, introductory and concluding paragraphs as well as topic sentences signaling the content of each paragraph are needed.

In answering HMSC questions

21

Quality Analysis of HMSC question - Case study (1)

22

Quality Analysis of HMSC question - Case (2)

23

Quality Analysis of HMSC question - Case (3)

24

Quality Analysis of HMSC question - Case (4)

25

Quality Analysis of HMSC question - Case (5)

26

Major Concerns in question setting

1. Cognitive requirement of the question

2. Skill coverage

3. Validity of the question

4. Sufficiency of the information

5. Quality of the question – question is clear and is easy to follow with proper question words

6. Open-end or restricted response

7. Misconception check

8. Avoid Errors/ ambiguities – proofreading is needed• Typographical/ spelling mistakes• Grammatical mistakes• Ambiguity in meaning/ instructions• Inconsistency with the paper• Punctuations…..

9. Arrangement of distractors

10. Avoid the acceptance of layman answers

27

Try to identify…

Can you …..

Do you think …..

All the ….

Based on your personal experience/ own thought,…

Poor questions - examples

28

Activity 4: Proofreading

Identify the errors and ambiguities of the following question within 15 minutes.

29

Objectives– Quality audit of INTERNAL examination paper

s/ test/ quiz from school teachers (On demand)

– Provision of extra HMSC questions in form of DVDs and booklets

– Provision of professional development programmes on basic question setting skills for HMSC teachers

– Sharing of extra HMSC questions and the development of web-based question bank

Assessment Partnership Scheme 「評核伙伴計劃」

30

– Working group for setting HMSC questions will be established for this scheme.

– A maximum of 10 meetings (3hrs sessions) with 4 members (or equivalent) will be held for question setting and moderation.

– The deliverables, including 80 extra HMSC questions and suggested answers for HMSC, will be disseminated to teachers in form of DVDs.

– Please contact Mr. Bill KWOK if you are interested. (kykwok@hkeaa.edu.hk)

Assessment Partnership Scheme 「評核伙伴計劃」