belal m. hijji, rn. phd, arwa owies, rn. phd faculty of nursing, philadelphia university, jordan...

Belal M. Hijji, RN. PhD,Arwa Owies, RN. PhD

Faculty of Nursing, Philadelphia University, Jordan

Presented inPresented inThe second International Conference for Allied Health Care

Professionals, Physicians, Physiotherapists, and Technicians, April 7-8 2010, Alexandria, Egypt

An Examination of the Routine Blood Transfusion Knowledge

Questionnaire Through Item Analysis

• Background• Aim and objectives• Methods• Results• Summary of results and discussion• Conclusion • Acknowledgment• References

Contents

8.04.10The second International Conference for Allied Health Care Professionals

3

Background


4

The Routine Blood Transfusion Knowledge Questionnaire (RBTKQ) was developed (Hijji, 2007) as a measure to investigate nurses’ level of

knowledge and practice of blood transfusion

The RBTKQ includes has true-false items (three), multiple-choice (MC) (20), and multiple-response (MR) (10)

For the purpose of this analysis, five sections of the RBTKQ that are relevant for inclusion: (A) is about issues related to

patient preparation before blood bag collection, (B) is concerned with blood bag collection section, (C) is related to

pre-transfusion initiation nursing activities, (D) addresses post-transfusion initiation activities, and (E) is related to

complications of blood transfusion


5

An examination of empirical item response data maybe appropriate in detecting item flaws (Crocker and Algina, 1986), and provides useful information

about test-item quality (Oermann & Gaberson, 1998).

Since its inception, the RBTKQ has not undergone any form of item analysis to evaluate its psychometric properties


6

AIMAIM


7

Principal Aim

The main aim of this study was to determine the overall quality of the items and the test

Objectives:A. Identify the items’ difficulty “p” and discrimination “D” levelsB. Investigating the performance of distractorsC. Find out whether the items discriminate between nurses whose scores place them in the top 30% of all nurses and those whose scores place them in the lower 30% of all nurses. This figure provides a stable index of differences between high and low ability groups (Crocker & Algina, 1986)


8

MethodsMethods


9

Data Collection

Responses to the item were taken from the RBTKQ that was filled out by a random sample of 305 nurses (95.3%). Analysis was performed for p, D, and for the distractors. Because the p

value is unavailable with MR items (http://www.cat.ilstu.edu/services/opscan/opFAQStatQ.php), each one

of these was broken down to a number of sub-items equal to the number of its options. Based on a nurse’s selection of an

option, a label of “yes” or “no”, or “true” or “false” was assigned. The outcome of this procedure culminated in a total of 89 items

for the whole questionnaire.

http://www.cat.ilstu.edu/services/opscan/opFAQStatQ.php

http://www.cat.ilstu.edu/services/opscan/opFAQStatQ.php


10

Data Analysis

scoringFor most items, one point was awarded for the correct

answer and 0 for the incorrect one. The maximum possible score was 57 (100%) points

The number of nurses’ subpopulations was 91 for each group. The upper group’s scores ranged from 32 (56%) to 40 (70%); lower group from 8 (14%) to 27 (48%). The mean knowledge score for the sample is 29.6 (SD = 4.14). A passing score of 34 (60%) (one standard deviation above the mean) was set up.

Scores were arranged in descending order from the highest to the lowest


11

For each item, data analysis details:A. the p value (whole sample). The p value is set at 0.5

to 0.85 (Lin et al., 1999). B. the p value of each option selected by both groupsC. the difference in proportions “D” between nurses’

subgroups selecting each option. A D value of 0.20 or above is appropriate (Crocker and Algina, 1986; Brown, 1983) for the correct option.

D. the number of omits


12

ResultsResults


13

Due to time restrictions, there will be partial presentation of the results


14

Table 1. Computations of item analysis indices for sections A and C PercentagesUpper 30% Lower 30%

Section A

Options

ItemKeyTFOmitp1F35-6255-281-1.47

%.40.-69.60.-31

D (h – l).-29.292F53-6535-243-2.30

%.61.-71.38.-26

d.-13.123F49-6441-221-5.33

%.55.-76.45.-24

d.-21.21

Section C1T87-724-190-0.89

%.96.-79.04.-21

d.17.-172F13-2578-660-0.81

%.14.-27.86.-73

d.-13.13


15

Table 2. Computations of item analysis indices for section D Percentages:Upper 30%Lower 30%

Options

ItemKeyTFOmitp1T39-1952-710-1.33

%.43.-21.57.-79

d.22.-222T78-7513-150-1.83

%.86.-82.14.-18

d.04-0.043F23-3868-520-1.68

%.25.-43.75.-57

d.-18.184F41-5450-360-1.47

%.45.-60.55-40

d- .15.155T77-4614-440-1.71

%.85.-51.15.-49

d.34.-34


16

Table 4. Computations of item analysis for sections A, B, and C

PercentagesUpper 30%

Lower 30%

Section A

Options

ItemKeyABCDEOmitp4D57-665-77-1322-4--0-1.12

%.63.-73.05.-08.08.-14.24.-05

d .-10.-03.-06.19

Section B1A56-261-1031-472-3-1-5.51

%.62.-29.01.-11.34.-52.03.-09

d.33.-10.-18.-062A48-351-62-829-2511-160-1.46

%.53.-38.01.-08.02.-09.32-27.12.-18

d.15.-07.-07.05.-063C36-406-611-710-376-12-1.09

%.40.-44.09.-08.12.-08.11.-41.07.-01

d.-040.01.04.-30.06


17

Table 4. (Continued…….)

Section COptions

ItemKeyABCDEOmitp1C35-518-1639-154-35-60-0.30

%.38.-56.09.-18.43.-16 .04.-03.05.-07

d.-18.-09.27.01.-02

2A51-2219-2521-320-90-30-1.38%.56.-24.21.-27.23.-350.-110.-03

d.32.-06.-12.-11.-03

3D56-4519-302-813-70-01-1.08%.62.-49.21.-33.03.-10.14.-08

d.13.-12.-07.064E1-60-71-50-388-701-0.90

%.01.-070.-08.01.-050.-03.97.-77

d.-06.-07.-04.-03.20

5C30-3316-2225-147-13-13-10.27%.33.-36.18.-24.27.-15.22.-25

d.-03.-06.12.-036C6-223-1581-500-4-1-0.70

%.07.-24.03.-16.89.-550.-04

d.-17.-13.34.-04


18

Table 5. Computations of item analysis indices for sections D PercentagesUpper 30%

Lower 30%

Options

ItemKeyABCDEOmitp1B37-3731-218-154-12-3-1.32

%.41.-41.34.-23.09.-16.08.-14

d0.11.-07.-062E74-644-122-51-39-21-5.05

%.81.-70.04.-13.02.-05.02.-09.10.-02

d.11.-09.-03.-07.08

3C48-4220-178-104-6-1-2.14%.59.-55.25.-22.10.-13.05.-09

d.04.03.-03.-044C6-3415-1761-308-8-1-2.45

%.07.-37.16.-19.67.-33.10.-11

d.-30.-03.34-0.015B9-1541-3728-2313-12-0-4.45

%.10.-21.45.-41.31.-25.14.-13

d.-11.04.06.01


19

Table 5. Computations of item analysis indices for sections E ItemPercentagesUpper 30% Lower 30%

Options

KeyABCDEOmitp1C16-135-2959-3411-15-0-0.50

%.18.-14.05.-32.65.-37.12.-16

d.04.-27.28.-042C8-511-1551-317-2114-170-2.45

%.09.-08.12.-16.56.-34.08.-23.15.-19

d.01.-04.22.-15.-04

3C75-633-139-54-60-10-3.09%.82.-69.03.-14.10.-05.04.-100.-01

d.13.-11.05.-06.-01

4D2-1215-2127-4144-15-3-2.29%.05.-15.16.-23.30.-45.48.-16

d.-10.-07.-15.325B5-1279-396-311-6-0-3.65

%.05.-13.87.-43.07.-34.01.-10

d.-08.44.-27.-09


20

Summary of Results and Discussion

• Item difficulty– Eight items (27%) had p value of .50 to .85 (neither too difficult

nor too easy)– Two items (6%) had p value of >.85 (easy)– 20 items (67%) had p value of < .50 (difficult). This could be,

most probably, because nurses did not master the content

• All distractors were selected by nurses and only a few of them, sometimes, omitted responding to some items.

• Sixteen items (53%) had a D value of ≥ .20. This indicates that the items were especially easier for the top group and more difficult to the lower group. – The last two results are good indicators of the quality of item

(Osterland, 1998, Oermann & Gaberson, 1998). Functional distractors decrease the chances that low ability nurses select the correct response by guessing which would introduce measurement error (Crocker & Algina, 1986)


21

• Fourteen items (47%) had a D value of < .20. – The test was not designed to maximise variance among groups.

In essence, D relies on a fundamental assumption, which is that examinees who exhibit mastery on the subject matter are presumed to be more likely to respond correctly to an item than low ability examinees (Osterland, 1998). In this study in fact, none of the nurses per se mastered the content of interest.

• The differences between both groups for the distractor were, mostly, negative values. This indicates that fewer upper group nurses selected the distractor than did the lower group nurses. – This is another positive feature of items (Osterland, 1998)

• However, the differences for some distractors in eight items were scant (≥ -.04). For two items, the differences were positive indicating that higher ability nurses selected distractors more often than low ability nurses. – In this case, Osterland (1998) recommends reviewing the

options for possible improvement.


22

• For a number of items (n = 13, 43%), nurses responses were widely scattered among the responses’ alternatives, and/ or the correct responses attracted less attention than the distractor(s). – This result may indicate that nurses could not understand the

items or that the items covered unfamiliar content. Therefore, these items may need to be reviewed (Crocker & Algina, 1986).


23

Conclusion

• This study has highlighted the need for:– Jordanian nurses to master the content related knowledge and

practice of blood transfusion. – Examining the questionnaire items for possible improvement

• However, no educator should retain, revise, or eliminate an item on the basis on item statistics alone (Oermann & Gaberson, 1998).


24

AcknowledgementsAcknowledgements


25

We are immensely grateful to the Ministry of Health, all nursing staff, and

hospitals’ management.We express our gratitude to

Philadelphia University Deanship of Scientific Research and Postgraduate

Studies for funding the study.


26

ReferencesReferences


27

Brown, F. (1983). Principles of educational and psychological testing (3rd ed.). New York: Holt, Rinehart and Winston.

Crocker, L. and Algina, J. (1986). Introduction to classical & modern test theory. Belmont CA: Wadsworth.

Hijji, B. (2007). Knowledge and practice of blood transfusion: A survey of nurses in Abu Dhabi, United Arab Emirates. Unpublished PhD thesis, the University of Ulster.

Lin, L.; Tseng, H. and Wu, S. (1999). Item analysis of the registered nurse licensure exam taken by nurse candidates from vocational nursing high schools in Taiwan. Proc. Natl. Sci. counc. ROC (D). 9(1): 24-31.

Oermann, M. and Gaberson, K. (1998). Evaluation and testing in nursing education. New York: Springer.

Osterland, S. (1998). Constructing Test Items: Multiple-Choice, Constructed-Response, Perfromance, and Other Formats. 2nd edition. London: Kluwer.


28

Thank You

Principal Investigator

[email protected]@gmail.com

mailto:[email protected]

mailto:[email protected]

belal m. hijji, rn. phd, arwa owies, rn. phd faculty of nursing, philadelphia university, jordan...

Documents

international conference

practice of blood transfusion

form of item analysis

items difficulty p

truefalse items

nurses selection

mr items http

nurses level of knowledge