appraising my teaching skills using msf
DESCRIPTION
TRANSCRIPT
![Page 1: Appraising my Teaching Skills using MSF](https://reader033.vdocuments.mx/reader033/viewer/2022061201/547942d9b4af9fe2158b47a2/html5/thumbnails/1.jpg)
APPRAISING MY ‘IMPROMPTU WARD-BASED TEACHING’
Dr Colin MitchellMSc Geriatric MedicineJune 2009
![Page 2: Appraising my Teaching Skills using MSF](https://reader033.vdocuments.mx/reader033/viewer/2022061201/547942d9b4af9fe2158b47a2/html5/thumbnails/2.jpg)
Choosing a Topic
I know some of my strengths as a teacher: Can explain complex concepts Keen to teach
And I’m aware of my weaknesses Prone to long-windedness Can over-elaborate
I already know to concentrate on the known weaknesses But are there any issues that I’m not fully aware of?
![Page 3: Appraising my Teaching Skills using MSF](https://reader033.vdocuments.mx/reader033/viewer/2022061201/547942d9b4af9fe2158b47a2/html5/thumbnails/3.jpg)
Is this right?
Previous 360o feedback ‘Can be intimidating’
Really? Jars with my self-image Could interfere with a conducive learning
environment
![Page 4: Appraising my Teaching Skills using MSF](https://reader033.vdocuments.mx/reader033/viewer/2022061201/547942d9b4af9fe2158b47a2/html5/thumbnails/4.jpg)
The MSF - Keeping it simple
Aim to focus on the learning environment Is the environment conducive to learning?
Relaxed / Intimidating Is some pressure a good thing?
Further focus on one skill: ward-based teaching (WBT) Multiple exposures, multiple sources of feedback
Just F1s
![Page 5: Appraising my Teaching Skills using MSF](https://reader033.vdocuments.mx/reader033/viewer/2022061201/547942d9b4af9fe2158b47a2/html5/thumbnails/5.jpg)
Assessing Educational Environment
DREEM - Dundee Ready Educational Environment Measure*
50 Questions – agree / disagree
Based on instruments in use for 40 years Designed by 80+ experts using the Delphi process Statistically validated to focus on reliable / discriminatory
elements
Used worldwide to assess medical school teaching (PHEEM)
*Roff et al, 1997. “Development and validation of the DREEM. Medical Teacher 19:4, 205-209
![Page 6: Appraising my Teaching Skills using MSF](https://reader033.vdocuments.mx/reader033/viewer/2022061201/547942d9b4af9fe2158b47a2/html5/thumbnails/6.jpg)
Refining the Quantitative
Principles of choosing the items*: Relevant Clear, concise, focused and simple Some balance of +ve and –ve (DREEM 82% +ve)
Chose 7 from DREEM (slightly adapted) Added 4 of my own Some concordance, not repetition 8 +ve, 3 -ve
*Adapted from Berk (2006), 13 Strategies to Measure College Teaching. Stylus: Virginia
![Page 7: Appraising my Teaching Skills using MSF](https://reader033.vdocuments.mx/reader033/viewer/2022061201/547942d9b4af9fe2158b47a2/html5/thumbnails/7.jpg)
Choosing the scale
DREEM uses 5-point bipolar ‘intensity’ anchors: Strongly Agree Agree Uncertain Disagree Strongly disagree
Simple but allows a breadth of response Still sufficiently granular & simple Balanced (✔) and has a midpoint (?✔)
![Page 8: Appraising my Teaching Skills using MSF](https://reader033.vdocuments.mx/reader033/viewer/2022061201/547942d9b4af9fe2158b47a2/html5/thumbnails/8.jpg)
Defining the Qualitative
Opportunity to explore particular issues more fully
Correlate with quantitative themes
Concepts:
Questions:
Strengths
Weaknesses
Context
Suggestions
![Page 9: Appraising my Teaching Skills using MSF](https://reader033.vdocuments.mx/reader033/viewer/2022061201/547942d9b4af9fe2158b47a2/html5/thumbnails/9.jpg)
Defining the Qualitative
Opportunity to explore particular issues more fully
Correlate with quantitative themes
Concepts:
Questions:
Strengths What are the strengths of my WBT?
Weaknesses
What do you like LEAST about my WBT?
Context How does the pressure of on-the-spot quizzing affect you?
Suggestions
Every teacher can improve his skills – how would you suggest I improve my WBT?
![Page 10: Appraising my Teaching Skills using MSF](https://reader033.vdocuments.mx/reader033/viewer/2022061201/547942d9b4af9fe2158b47a2/html5/thumbnails/10.jpg)
Making it anonymous - online
MSF in UK training – anonymous. In industry – 97% anonymous “Maintaining the anonymity of the rater’s feedback is key to the
process. Just as a tie to compensation can lead raters to soften their ratings, so can a lack of anonymity”*
“…when peer assessment is used in a high stakes setting, it results in inflated estimates of performance... There is no way to completely avoid this issue but it may help to ensure the anonymity of evaluators”**
Online forms preserve anonymity - This can be made clear in the MSF
Rogers E et al, 2002. “Improving the Payoff from 360-Degree Feedback”. Human Resource Development. 25: 44-54**Norcini J, 2003. “Peer assessment of competence”. Medical Education. 37(6): 539-543
![Page 11: Appraising my Teaching Skills using MSF](https://reader033.vdocuments.mx/reader033/viewer/2022061201/547942d9b4af9fe2158b47a2/html5/thumbnails/11.jpg)
![Page 12: Appraising my Teaching Skills using MSF](https://reader033.vdocuments.mx/reader033/viewer/2022061201/547942d9b4af9fe2158b47a2/html5/thumbnails/12.jpg)
Validation
Cannot empirically validate without significant resources Using 7 items from the DREEM – pre-validated
My own questions must be logically validated Representative pilot Sent to 5 SHOs. 4 replied with comments
(unstructured) Minor adjustments made to wording, stressing
anonymity
![Page 13: Appraising my Teaching Skills using MSF](https://reader033.vdocuments.mx/reader033/viewer/2022061201/547942d9b4af9fe2158b47a2/html5/thumbnails/13.jpg)
RESULTS
![Page 14: Appraising my Teaching Skills using MSF](https://reader033.vdocuments.mx/reader033/viewer/2022061201/547942d9b4af9fe2158b47a2/html5/thumbnails/14.jpg)
Question Scores (Fractions of 100%)
Stimul
atin
g
Teac
her Cen
tred
Probl
em S
olving
Good
Feed
back
Knowle
dge
Expl
ains
Con
cept
s
Appro
acha
ble
Ridicul
e
Relax
ed a
tmos
pher
e
Respe
cts
Intim
idat
ing
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
1234
![Page 15: Appraising my Teaching Skills using MSF](https://reader033.vdocuments.mx/reader033/viewer/2022061201/547942d9b4af9fe2158b47a2/html5/thumbnails/15.jpg)
Scoring
For negative aspects the score is reversed Therefore: Higher score is a good thing
Response Positive
Negative
Strongly agree 4 0
Agree 3 1
Uncertain 2 2
Disagree 1 3
Strongly Disagree 0 4
![Page 16: Appraising my Teaching Skills using MSF](https://reader033.vdocuments.mx/reader033/viewer/2022061201/547942d9b4af9fe2158b47a2/html5/thumbnails/16.jpg)
Free Responses (1)
What are the strengths of my ward-based teaching? When a situation arises you take time to discuss the
case (23/44*)
Enthusiastic and approachable… make[s] the learner think the problem through in a logical manner (30/40)
…instead of just giving an answer, you get us to think through the problem and help us work out the solution ourselves (24/44)
*Commenter’s overall score for quantitative elements shown in parentheses
![Page 17: Appraising my Teaching Skills using MSF](https://reader033.vdocuments.mx/reader033/viewer/2022061201/547942d9b4af9fe2158b47a2/html5/thumbnails/17.jpg)
Free Responses (2)
What do you like LEAST about my style of ward-based teaching? (Only 5/11 replied) Sometimes assume too much which can be
uncomfortable (20/40)
Sometimes can go into a lot of depth for too long (26/44)
… on some occasions it would be easier to get an answer, especially when it gets busy, but I would not learn as well… (24/44)
Sometimes it becomes a little didactic (25/44)
![Page 18: Appraising my Teaching Skills using MSF](https://reader033.vdocuments.mx/reader033/viewer/2022061201/547942d9b4af9fe2158b47a2/html5/thumbnails/18.jpg)
Free Responses (3)
How does the pressure of on-the-spot quizzing affect your learning (+vely or –vely)? (In the US, “Pimping” generally viewed positively*)
8/11 were generally positive, 2 didn’t answer Only negative response:
Too pressurizing and can be intimidating and lead to not wanting to ask for teaching (29/44)
*Wear D et al, 2005. “Pimping: perspectives of 4th year medical students”. Teaching and Learning in Medicine. 17(2) 184-191
![Page 19: Appraising my Teaching Skills using MSF](https://reader033.vdocuments.mx/reader033/viewer/2022061201/547942d9b4af9fe2158b47a2/html5/thumbnails/19.jpg)
Free Responses (4)
Every teacher can improve his skills – how would you suggest I improve my ward-based teaching? Be a bit more aware of when people have got lost in your
explanations (20/40)
You are a good teacher and have a lot of knowledge, but could improve by talking more succinctly and not rambling (26/44)
There is very little to improve on, your teaching is excellent (25/44)
I think sometimes you loose [sic] the balance and focus too much on teaching rather than seeing the patients… and thus even though your teaching is good, you are not setting a good example for juniors (28/40)
![Page 20: Appraising my Teaching Skills using MSF](https://reader033.vdocuments.mx/reader033/viewer/2022061201/547942d9b4af9fe2158b47a2/html5/thumbnails/20.jpg)
ANALYSIS
![Page 21: Appraising my Teaching Skills using MSF](https://reader033.vdocuments.mx/reader033/viewer/2022061201/547942d9b4af9fe2158b47a2/html5/thumbnails/21.jpg)
Robust?
Some basic statistical analysis of average total score: Median = average (59%, around a 2.5) 95% CI: ±19%
Some observations can be made: Lower scorers more likely to give a critique in the
weaknesses question Poor correlation between ridicule / intimidating / relaxed
questions.
The lowest scorer commented: “the relaxed manner of teaching is definitely a good approach” He/she also disagreed with the ‘relaxed atmosphere’
item (?)
![Page 22: Appraising my Teaching Skills using MSF](https://reader033.vdocuments.mx/reader033/viewer/2022061201/547942d9b4af9fe2158b47a2/html5/thumbnails/22.jpg)
Findings – My Teaching
Quantitative results Overall positive (59% compares with DREEM results) I can be intimidating
Yet the atmosphere is generally relaxed (=3rd highest scoring item)
3 comments specifically mentioned friendly / approachable
Themes emerging from qualitative results Rambling / long-windedness
Correlates with ‘teacher-centred’ Few negative comments about atmosphere
Only one comment directly mentioned ‘intimidating’ or synonyms.
Also note comment about assuming too much = ‘uncomfortable’ This rater also agreed with both the intimidate and ridicule questions
![Page 23: Appraising my Teaching Skills using MSF](https://reader033.vdocuments.mx/reader033/viewer/2022061201/547942d9b4af9fe2158b47a2/html5/thumbnails/23.jpg)
Findings – The Process
Getting good feedback is more difficult than it seems
Despite: Limiting scope Using validated questions Piloting Anonymous data entry
The results didn’t tell me much I didn’t already know Partially corroborated ‘intimidating’ problem Confirmed ‘long-winded’ problem
![Page 24: Appraising my Teaching Skills using MSF](https://reader033.vdocuments.mx/reader033/viewer/2022061201/547942d9b4af9fe2158b47a2/html5/thumbnails/24.jpg)
ASSIMILATION
Borg Cube by Martin Teufel - http://www.infosun.fim.uni-passau.de/br/lehrstuhl/Sommercamp/virtualworld/2005/galerie/
![Page 25: Appraising my Teaching Skills using MSF](https://reader033.vdocuments.mx/reader033/viewer/2022061201/547942d9b4af9fe2158b47a2/html5/thumbnails/25.jpg)
How I see myself
Image from http://www.theinsider.com/photos/1079058_TOM_CRUISE_FOR_TOP_GUN_2. Top Gun - Paramount Pictures.
![Page 26: Appraising my Teaching Skills using MSF](https://reader033.vdocuments.mx/reader033/viewer/2022061201/547942d9b4af9fe2158b47a2/html5/thumbnails/26.jpg)
How I see myself
![Page 27: Appraising my Teaching Skills using MSF](https://reader033.vdocuments.mx/reader033/viewer/2022061201/547942d9b4af9fe2158b47a2/html5/thumbnails/27.jpg)
YOU BOY!
What I need to avoid:
Val-de-ree! Val-de ha ha ha ha ha ha!
![Page 28: Appraising my Teaching Skills using MSF](https://reader033.vdocuments.mx/reader033/viewer/2022061201/547942d9b4af9fe2158b47a2/html5/thumbnails/28.jpg)
Further Actions
I may not be intimidating to most… But clearly a few may find me so How do I know who they are? What do I do about it?
Design my own feedback forms for my teaching sessions Add demographics Push for free-text responses
Don’t be afraid of constructive criticism Criticism is worst when I don’t know how to fix the
problem
![Page 29: Appraising my Teaching Skills using MSF](https://reader033.vdocuments.mx/reader033/viewer/2022061201/547942d9b4af9fe2158b47a2/html5/thumbnails/29.jpg)
Links:
Email: [email protected]
http://colinsmededblog.blogspot.com
Twitter: @drcolinmitchell
Questions?
http://obamicon.pastemagazine.com