Radiography (2009) 15, 70e76

ava i lab le a t www.sc iencedi rec t .com

journa l homepage : www.e lsev i er . com/ loca te / rad i

Defining and assessing critical thinking skills forstudent radiographers

Alan Castle*

University of Portsmouth, School of Health Sciences and Social Work, Centre for Radiography Education,James Watson Building, 2 King Richard 1st Road, Portsmouth PO1 2FR, United Kingdom

Received 15 June 2007; revised 18 October 2007; accepted 30 October 2007Available online 25 July 2008

KEYWORDSEducation;Critical thinking skills;Defining;Assessing;Professional practice;Curriculum design

* Tel.: þ44 02392 845392; fax: þ44E-mail address: alan.castle@port.a

1078-8174/$ - see front matter ª 200doi:10.1016/j.radi.2007.10.007

Abstract Developing critical thinking skills is a key aim of higher education and is importantin preparing student radiographers for their future careers in clinical practice. The aim of thispaper was to attempt to devise and assess six key components of critical thinking appropriatefor radiographic practice.

Each of the six components was divided into three dimensions and a Critical Thinking SkillsScoring Chart (CTSSC) devised to assess students’ written performance against each dimension.Scores revealed that approximately 30% of students were rated as good and approximately 10%of students were rated as poor in each component, although there was some variability be-tween different dimensions.

It is suggested that educators need to encourage and support students to develop their crit-ical thinking skills by reviewing their curriculum to clearly define specific skills and ensure thatthey are appropriately taught and assessed.ª 2008 The College of Radiographers. Published by Elsevier Ltd. All rights reserved.

Introduction

Critical thinking is a high level intellectual activity thatgoes beyond the acquisition of knowledge, and is seen asa key skill for all health professionals to master in an agewhere they are expected to be reflective practitioners andbase their practice on sound evidence.1e3

However, many students are unsure of what beinga critical thinker means but are unwilling to say so becausethey assume they are expected to already know.

02392 844402.c.uk

8 The College of Radiographers.

Consequently, they areoften unwilling to challenge evidencepresented by academics and practitioners in their field.4

The exact meaning of the term critical thinking has beenthe subject of much academic debate in a variety ofeducational contexts, but a consensus definition has beendeveloped which suggests that it is at the level wherea student is able to interpret, analyse, evaluate, explainand infer concepts and ideas.5 It would seem logical there-fore, that student radiographers should be encouraged to de-velop critical thinking skills during their undergraduatetraining. However, it is suggested that whilst students are ex-pected to demonstrate critical thinking skills these are notadequately defined, taught or assessed.6 It is importantthat curriculum design ensures that critical thinking skills

Published by Elsevier Ltd. All rights reserved.

Defining and assessing critical thinking skills 71

are clearly defined and that learning activities and assess-ment tasks are undertaken to achieve the desired outcomes.

Consequently, the aim of this paper is to define appro-priate components of critical thinking for student radiog-raphers and to assess to what extent these skills have beenacquired.

Methodology

Developing methods that can assist in defining the compo-nents of critical thinking and assessing to what extent bothdiagnostic and therapeutic radiography students demon-strate such skills, is an essential prerequisite in designinga more explicit curriculum linking teaching, learning andassessment.7 The methodology involved development inthree areas: components of critical thinking, assessmentschedule and critical thinking skills scoring chart.

Components of critical thinking

The 17 consensus components of critical thinking in nursingthat have been proposed by Scheffer and Rubenfeld8 weremodified into the six components chosen for this study be-cause they were thought to represent a range of importantcritical thinking skills important in the practice of radiogra-phy, namely:

� investigation;� discrimination;

Table 1 Schedule of the components and dimensions of critica

Component Dimensions (learning outcomes) Idenkey

Investigation Locate appropriate literature LocaOffer appropriate explanation fordefinitions and terms

Expla

Record an accurate reference list RecoDiscrimination Recognise strengths and weaknesses of

methodology chosenReco

Select appropriate results to display SelecIdentify any inconsistencies betweenresults obtained/expected

Iden

Judgement Score each element of the article ScorSummarise the evidence presented SummAppraise the evidence presented Appr

Inference Formulate questions/hypotheses FormReview literature ReviReflect on methodological approach Refle

Evaluation Consider the background andindications for examination/treatment

Cons

Explore factors that determine thechoice of examination/treatment

Explo

Justify the basis for the examination/treatment

Justi

Analysis Assess the evidence presented AsseDraw appropriate conclusions based onevidence presented

Conc

Suggest recommendations/implicationsbased on evidence presented

Sugg

� judgement;� inference;� evaluation;� analysis.

Having identified the six components of critical thinking,each component was divided into three dimensions alongthe lines of sub-skills suggested by Scheffer and Ruben-feld,8 to clearly define appropriate learning outcomes.Table 1 illustrates the six components and, in each case,the three dimensions devised for each component (includ-ing identifying key words subsequently used in the text),and when and how each dimension were assessed.

Assessment schedule

As identified in Table 1, two components of critical thinkingwere assessed in each year of the course to allow forbreadth of coverage.

In year 1 the following components were assessed:

� investigation, which involved locating evidence, factsor knowledge by identifying relevant sources and gath-ering data,8 via a written physics assignment;� discrimination, which involved demonstrating an ability

to assess the strengths and weakness of evidence andrecognise differences and similarities in data,5 bymeans of writing-up a laboratory report of a physicsexperiment.

l thinking used to assess student performance

tifyingwords

Yearassessed

Assessment

te 1 Physics assignment requiring thefinding, defining and explaining ofscientific terms and recordingreferences

in

rdgnise 1 Laboratory report of a physics

experimentt

tify

e 2 Appraisal of published articlearise

aiseulate 2 Project proposal

ewctider 3 Review of a radiographic examination/

treatmentre

fy

ss 3 Project reportlude

est

Table 2 CTSSC used to assess the written laboratory report of a year 1 physics experiment

Assignment Critical thinking skill component:discrimination

3 2 1 0

Written laboratory report ofa physics experiment

Evidence of an ability to: Strongly agree Agree Disagree Strongly disagreeRecognise strengths andweaknesses of chosenmethodologySelect appropriate results todisplayIdentify any inconsistenciesbetween results obtained/expected

72 A. Castle

In year 2 the following components were assessed:

� judgement, which involved summarising and appraisingevidence and making sound decisions based on theevidence,9 following reading of a published article;� inference, which involved forming hypotheses, review-

ing evidence and reflecting on assumptions and views,5

by presenting a written project proposal.

In year 3 the following components were assessed:

� evaluation, which involved examining ideas, detectingarguments and clarifying issues in order to assess thelogical strength, validity and relevance of the evi-dence,10 by writing a review of a radiographic examina-tion/treatment;� analysis, which involved comparing and contrasting

ideas and drawing appropriate conclusions to supportarguments and the extent to which these have implica-tions for future practice,10 by means of a written pro-ject report.

8090100

en

ts Investigation

Discrimination

Critical thinking skills scoring chart (CTSSC)

The CTSSC was devised to assess each of the six compo-nents of critical thinking, and was used by the assignmentmarker to judge the written evidence presented by stu-dents. Each CTSSC included the three dimensions of thecomponent of critical thinking, providing the learningoutcomes against which the evidence was judged. TheCTSSC uses a four point scale ranging from 0 Z strongly dis-agree to 3 Z strongly agree. Thus the maximum score

Table 3 Overall scores for the six assessed components ofcritical thinking

Year Component n c SD

1 Investigation 132 5.86 1.771 Discrimination 119 5.51 1.892 Judgement 111 6.30 1.452 Inference 115 5.22 1.973 Evaluation 61 5.63 1.783 Analysis 96 6.36 1.62

attainable Z 9 and the minimum score attainable Z 0.Table 2 is an example of the CTSSC used to assess the dis-crimination component of critical thinking in year 1.

The number of students assessed in each of thesix components of critical thinking varied due to theavailability of scripts and the diligence of the assignmentmarkers. However, the sample size in each component isconsidered sufficient to draw some general conclusions andrecommendations.

Results

The overall scores for each of the six components of criticalthinking are shown in Table 3.

Each student performance was rated as good (where thetotal score was between 7 and 9, so it must include at leastone strongly agree mark), average (where the total scorewas between 4 and 6, so it must include at least one agreemark) and poor (where the total score was between 0 and3, so it must include all disagree marks). The overall scoressuggest that most students were rated as average for eachcomponent of critical thinking (mean scores between 5.22and 6.36), although a closer analysis of the componentscores reveals a more complex story, where some dimen-sions proved more challenging than others as shown inFigs. 2, 4 and 6.

010203040506070

Poor Average GoodRated Scores

Percen

tag

e o

f S

tu

d

Figure 1 Student performance in the two components ofcritical thinking assessed in year 1.

0

10

20

30

40

50

60

70

80

90

100

Identify Explain Recognise Select Record LocateDimensions (Keywords)

Percen

tag

e o

f S

tu

den

ts

Figure 2 Problem dimensions for students in year 1.

0

10

20

30

40

50

60

70

80

90

100

Reflect Review Appraise Formulate Summarise ScoreDimensions (Keywords)

Percen

tag

e o

f S

tu

den

ts

Figure 4 Problem dimensions for students in year 2.

Defining and assessing critical thinking skills 73

Year 1

Fig. 1 illustrates the scores for student performance in thetwo components of critical thinking assessed in year 1.

InvestigationForty percent (n Z 53) of students were rated as good, 10%(n Z 13) were rated as poor and 50% (n Z 66) were rated asaverage. Of the 43% (n Z 57) of students who scored either1 or 0 on at least one of the dimensions, 82% (n Z 47) demon-strated problems in the dimension requiring them to offer ap-propriate explanations for definitions and terms (Fig. 2).

DiscriminationThirty percent (n Z 35) of students were rated as good, 15%(n Z 18) were rated as poor and 55% (n Z 66) were rated asaverage. Of the 59% (n Z 70) of students who scored either1 or 0 on at least one of the dimensions, 86% (n Z 60), dem-onstrated problems in the dimension requiring them toidentify any inconsistencies between results obtained/ex-pected (Fig. 2).

Year 2

Fig. 3 illustrates the scores for student performance in thetwo components of critical thinking assessed in year 2.

0102030405060708090100

Poor Average Good

Rated Scores

Percen

tag

e o

f S

tu

den

ts Judgement

Inference

Figure 3 Student performance in the two components ofcritical thinking assessed in year 2.

JudgementForty percent (n Z 45) of students were rated as good, 4%(n Z 4) were rated as poor and 56% (n Z 62) were ratedas average. Of the 36% (n Z 40) of students who scored ei-ther 1 or 0 on at least one of the dimensions, 45% (n Z 18)demonstrated problems in the dimension requiring them toappraise the evidence presented (Fig. 4).

InferenceTwenty-one percent (n Z 24) of students were rated asgood, 20% (n Z 23) were rated as poor and 59% (n Z 68)were rated as average. Of the 57% (n Z 66) of studentswho scored either 1 or 0 on at least one of the dimensions,74% (n Z 49) demonstrated problems in the dimension re-quiring them to reflect on their methodological approachsuggested in the project proposal (Fig. 4).

Year 3

Fig. 5 illustrates the scores for student performance in thetwo components of critical thinking assessed in year 3.

EvaluationThirty-three percent (n Z 20) of students were rated asgood, 11% (n Z 7) were rated as poor and 56% (n Z 34)

0102030405060708090100

Poor Average GoodRated Scores

Percen

tag

e o

f S

tu

den

ts Evaluation

Analysis

Figure 5 Student performance in the two components ofcritical thinking assessed in year 3.

0

10

20

30

40

50

60

70

80

Suggest Justify Conclude Explore Consider AssessDimensions (Keywords)

Percen

tag

e o

f S

tu

den

ts

Figure 6 Problem dimensions for students in year 3.

74 A. Castle

were rated as average. Of the 56% (n Z 34) of students whoscored either 1 or 0 on at least one of the dimensions, 61%(n Z 21) demonstrated problems in the dimension requiringthem to justify the basis for the examination/treatment(Fig. 6).

AnalysisThirty-five percent (n Z 34) of students were rated as good,2% (n Z 2) were rated as poor and 63% (n Z 60) were ratedas average. Of the 26% (n Z 25) of students who scored ei-ther 1 or 0 on at least one of the dimensions, 72% (n Z 18)demonstrated problems in the dimension requiring them tosuggest recommendations/implications based on the evi-dence presented (Fig. 6).

Discussion

Whilst the mean scores for each component suggests thatmost students were able to demonstrate satisfactorycritical thinking skills in each of the six componentsassessed (Table 3), a closer analysis reveals a more complexstory where some dimensions within each componentcaused more problems than others. Thus, for example, inthe discrimination component in year 1, whilst 50% of stu-dents had problems identifying any inconsistencies be-tween results obtained/expected, only 21% of studentshad problems selecting appropriate results to display.Fig. 7 ranks the extent to which each dimension of critical

0102030405060708090100

Identify

Reflect

Review

ExplainJustify

Recognise

Explore

FormulateSelect

Dimensions of Critic

Perc

en

tag

e o

f S

tu

den

ts

Figure 7 Extent of problems reported for each o

thinking caused problems to all students, and these are dis-cussed below under each of the six components.

Year 1

InvestigationThe main dimension of concern in this component of criticalthinking is the fact that over a third of all students were un-able to explain accurately key scientific definitions andterms (Fig. 7), a fact emphasised following analysis of wherestudents who scored poorly in this component had the mostproblems (Fig. 2). This may be due to a failure of understand-ing or it may be that such issues are not seen as particularlyrelevant to the practice of radiography, a view that is some-times reinforced by such messages such as ‘you don’t need tounderstand the physics to operate this equipment effec-tively.’ However, for example, it is of paramount importancethat students are able to explain how and why changing thepotential difference across the X-ray tube and altering thetube current affects radiographic image quality.

DiscriminationThe main dimension of concern in this component of criticalthinking is the fact that half of all students were unable tosatisfactorily identify inconsistencies between results ob-tained/expected (Fig. 7), a fact emphasised following anal-ysis of where students who scored poorly in this componenthad the most problems (Fig. 2). For a profession that isbased on the application of physical principles and con-cerned with delivering ionising radiation to the generalpublic, this apparent lack of scientific scrutiny and rigourmay compromise their future role as a practitioner witha responsibility to justify medical exposures and to havea full knowledge of the potential benefits and risks as iden-tified in the ionising radiation (medical exposure) regula-tions.11 For example, it is important that students learnto be accurate using appropriate units for measuring radia-tion, such as exposure (C-Kg), absorbed dose (Gy) or effec-tive dose (Sv) and the range of doses used from a typicalchest X-ray to a radical radiotherapy treatment.

Year 2

JudgementGenerally student performance in this component ofcritical thinking was satisfactory, but the main dimensions

Consider

Suggest

Appraise

Conclude

Record

Summarise

Locate

AssessScore

al Thinking (Keywords)

f the assessed dimensions of critical thinking.

Table 4 Main component and dimensions presenting problems

Year Component Dimensions presenting main problems Suggested remedy

1 Investigation Offer appropriate explanation for definitionsand terms

Teaching students the importance of using andexplaining appropriate scientific terminology intheir writing of scientific assignments. Forexample, explaining the significance of thedifferent processes that take place when x-radiation interacts with matter.

1 Discrimination Identify any inconsistencies between resultsobtained/expected

Teaching students to identify and consider theimplications of their findings, so that they canrecognise inconsistencies in results and explorepossible reasons for them. For example, incorrectcalculation of a dose prescription in radiotherapy.

2 Inference Reflect on methodological approach Teaching students to reflect more effectively bywriting reflective accounts of, for example, theirclinical practice or interprofessional learningactivities.

2 Inference Review literature Teaching students to review evidence with acritical eye. For example, being able to scrutiniseliterature to assess the advantages and limitationsof one particular technique/treatment comparedwith another.

2 Judgement Appraise evidence presented Teaching students to read literature with a criticaleye, appraise its value and make sound decisionsbased on the evidence presented. For example,reading a published article and scoring eachsection based on reasoned judgement.

3 Evaluation Justify the basis for the examination/treatment Teaching students to take a more holisticapproach to their work so that they are able tojustify their actions. For example, analysis of risk/benefit and optimisation when undertaking aradiographic examination.

3 Analysis Suggest recommendations/implications basedon the evidence presented

Teaching students to consider the meaning of theevidence they have presented. For example,suggesting changes in practice as a result of areview of the literature a clinical audit.

3 Analysis Draw appropriate conclusions based on theevidence presented

Teaching students that the take home message isimportant. For example, stating the contributiontheir work has to answer the central projectquestion and avoiding over-generalisation offindings.

Defining and assessing critical thinking skills 75

of concern are that a number of students were unable tosatisfactorily appraise or summarise evidence presented(Figs. 4 and 7). If best practice is based on best evidence,then students must be able to assess the strengths andweaknesses of arguments and supporting evidence. For ex-ample, when appraising a published article, it is importantto identify key claims, the strength of evidence presentedto support such claims, and to what extent the claims aresupported or challenged by other work.

InferenceThe main dimension of concern in this component of criticalthinking is the fact that over 40% of all students wereunable to adequately reflect on the methodological ap-proach used in their project proposal (Fig. 7), a fact em-phasised following analysis of where students who scoredpoorly in this component had the most problems (Fig. 4).Despite the emphasis on reflection in the radiographic

curriculum, some students still find this as a difficult skillto master. More explicit teaching in this area may be re-quired so that students are able, for example, to discuss al-ternative methodological approaches that might have beentaken in order to try and answer their project question andto justify the methodological paradigm within which theyare working. An inability to formulate a reasonable ques-tion/hypothesis demonstrates a lack of clarity that is likelyto lead to the production of work that is unfocused and in-coherent. ‘Defining a topic, a central question, a set of re-search aims and research questions contribute todeveloping a critical frame of mind (p. 45)’.4

Year 3

EvaluationThe main dimension of concern in this component of criticalthinking is the fact that over a third of students were

76 A. Castle

unable to justify the basis for an examination/treatment(Fig. 7), a fact emphasised following analysis of where thestudents who scored poorly in this component had the mostproblems (Fig. 6). The requirement to take a more holisticview of the patient journey and to consider issues such aswhy the patient was being referred and what would bethe most effective diagnostic/treatment regime in a partic-ular context, thus moving away from a pure description oftechnique, is fundamental to professional development.Many students are still too focussed on the task to considerthe responsibilities they will have as a practitioner who isexpected to justify medical exposures to ionising radiation.

AnalysisThe main dimensions of concern in this component ofcritical thinking is the fact that whilst only 20% of allstudents were unable to suggest recommendations/impli-cations based on the evidence presented and 15% unable todraw appropriate conclusions based on the evidence(Fig. 7), this becomes more problematic following analysisof data which revealed that over 70% and 50% of students,respectively, who scored poorly in this component had themost problems with these dimensions (Fig. 6). Informingpractice through becoming a critical consumer of literatureis important in attempting to develop a practice domain(p. 19).4 Whilst the ability to express clear project aims, re-view literature and collect and analyse data are importantskills, the conclusions should ‘pull all aspects of the worktogether (and) consider with what degree of certainty thestudy answered your research question’ (p. 54).4

Conclusions

This study has demonstrated that some students were unableto perform satisfactorily in each of the dimensions assessed,and some dimensions caused more of a problem than others.A summary of the main dimensions causing problems,together with a suggested remedy is shown in Table 4.

It is acknowledged that there are issues of sensitivity,validity and reliability of the CTSSC, which need to beaddressed for future studies. The four point rating scalemay not be sensitive enough to discriminate effectivelybetween students and work needs to be done on producingvalidity and reliability data. Whilst the CTSSC only allowsthe assignment marker to make a subjective, impression-istic rating of an individual students’ performance in eachof the three dimensions, it is a judgement made on theevidence presented in the written assignment. Also, be-cause different individuals mark each of the writtenassignments, the issue of inter-rater reliability arises. Allmarkers were coached in the completion of the CTSSCbefore they rated any written work, but nevertheless it isacknowledged that having only one marker would improvereliability. However, the CTSSC allows for the collection of

a range of data relating to different dimensions of criticalthinking and thus provides a simple, practical tool that canbe adapted and modified to suit individual curriculaactivities.

Whilst the task is important, so too is the ability of thepractitioner to review evidence, choose appropriate op-tions and reflect on past performance if they effectivelyassess and judge the value of knowledge and informationand relate it to their professional practice. If students areto be encouraged and supported in developing their criticalthinking skills, then more emphasis needs to be placed ondefining, teaching, assessing and giving feedback on spe-cific dimensions of critical thinking. This may help thosestudents who find the concept of critical thinking difficultto understand, to realise its importance in professionalpractice and help them to perform more effectively in thefuture. Evidence suggests that effective critical thinkingskills can be learnt by students in a wide variety of contextswhen they receive training using appropriate teachingactivities.12 Effort needs to be put into developing a curric-ulum that overtly and systematically focuses on the im-provement of critical thinking as these skills will notautomatically result as a by-product of standard teachingin a content area.13

References

1. Quality Assurance Agency for Higher Education. Benchmarkstatement: health care programmes. London: Quality Assur-ance Agency; 2001.

2. College of Radiographers. A curriculum framework for radiog-raphy. London: College of Radiographers; 2003.

3. Health Professions Council. Standards of proficiency: radiogra-phers. London: Health Professions Council; 2003.

4. Poulson L, Wallace M, editors. Learning to read critically inteaching and learning. London: SAGE Publications; 2004.

5. Facione PA. Critical thinking: a statement of expert consensusfor purposes of educational assessment and instruction. TheDelphi Report. Milbrae: California Academic Press; 1990.

6. Castle A. Assessment of critical thinking skills of student ra-diographers. Radiography 2006;12:88e95.

7. Biggs J. Teaching for quality at university. Buckingham: SRHEand Open University Press; 1990.

8. Scheffer BK, Rubenfeld MG. A consensus statement on criticalthinking in nursing. J Nurs Educ 2000;39:352e9.

9. Thomas G, Smoot G. Critical thinking: a vital work skill. Trustfor Educational Leadership 1994;23:34e8.

10. Facione PA. Critical thinking: what it is and why it counts. Mil-brae: California AcademicPress; 1998.

11. Ionising radiation (medical exposure) regulations. London: IR-MER Legislation; 2000.

12. Bowkett S. 100þ ideas for teaching thinking skills. London:Continuum International Publishing Group; 2007.

13. Halpern DF. Thought and knowledge: an introduction to criti-cal thinking. 3rd ed. Mahwah: Lawrence Erlbaum Associates;1996.