Abstracts

Researching Medical Education

23rd November 2010

The Royal Institute of British Architects, 66 Portland Place, London

Index

Page Title Authors3 Use of video taping for direct observation of case based learning: a feasible

method for analyzing small group teachingR McEwan

4-5 Using Kirkpatrick’s levels to establish and test a conceptual model of reflective thinking and patient safety in a cohort of medical students as they advance through a medical curriculum

L Ambrose, K Stirling, D Murphy & JS Kerr

6 Bedside teaching – do junior doctors and medical students make a winning team?

A Nair & S Singh

7 The evaluation of an inter-professional multimedia musculoskeletal examination teaching resource: a qualitative study

C Griffin, C Chung, V Tzortziou-Brown & D Morrissey

8 Ethnicity and academic under achievement early in the medical course M Carroll

9 An exploration of the methodological opportunities and limitations of case studies for research on educating health professionals living with HIV in Kenya

C O’Keeffe

10 Co-operation in a Competitive World P Bryant

11 Corpus analysis of problem-based learning transcripts: A new method to look into PBL

A L Da Silva &R Dennick

12 Gender, culture and professional identity: a qualitative enquiry into the effects of the learning and professional environment on the construction of a surgical identity

L Etheridge

13 Unpicking issues within the osteoarthritis consultation in primary care using videotaped surgeries

Z Paskins, AB Hassell &PR Croft

14 Conceptions of learning and leadership – a qualitative study of senior faculty in a research-intensive university

A Berlin

15 From theory to data analysis: using mixed qualitative methods to explore a complex educational intervention

S Yardley

2

Use of video-taping for direct observation of case based learning: a feasible method for analysing small group teaching

R McEwen, L Varadhan, S Williams, A Hassell

R McEwen, Clinical teaching fellow, CEC building, City General Hospital NHS Trust, Stoke on Trent ST4 6QG

ContextAs part of the new curriculum being implemented in Keele Medical School a locally developed form of Case Based Learning (CBL) was implemented in the 3rd year. CBL was performed in small groups with an expert tutor; a generic case was discussed during the opening session. Closing session was performed a week later where students presented cases they had seen relating to the generic case. Traditionally small group sessions have been assessed by questionnaire or focus group discussion, with little direct observation of sessions published in the literature.

MethodologyAs part of the evaluation process a pilot study was undertaken to assess feasibility of video-taping these sessions. A selection of CBL sessions, including opening and closing, were video-taped over a two week period after gaining formal consent from tutors and students. Small DVD video recorders were left un-manned in the rooms. No group was video-taped more than once. One major challenge was the methodology for analysing the recordings. Video-tapes were assessed for group interactions (a tool was developed and internally validated for scoring) and adherence to locally developed CBL guidelines.

ResultsVideo-taping was generally well accepted, apart from one tutor and one student declining consent. Nine sessions including medicine, surgery and paediatrics were video-taped. One video-tape was lost due to technical fault after recording. There was wide variation in the quantity of group interactions but all sessions were predominantly tutor driven. Inter-student interaction was minimal. Although the expected content was broadly covered, the process was widely variable.

ConclusionsVideo-taping is feasible and a simple and efficient method of directly observing small group teaching, allowing evaluation of a variety of domains. These recordings could be used in staff development or CBL training for students and staff. There remains work to be done in developing the methodology for analysis of recordings.

3

Using Kirkpatrick’s levels to establish and test a conceptual model of reflective thinking and patient safety in a cohort of medical students as they advance through a medical curriculum.

L Ambrose, K Stirling, D Murphy, J S Ker

L Ambrose, Keele University, Clinical Education Centre, University Hospital of North Staffordshire, Newcastle Road, Stoke on Trent, Staffordshire, ST4 6QG

ContextUnderstanding how medical students learn about patient safety is a key area in medical education. A series of studies have been conducted using Kirkpatrick’s (1) levels to follow a cohort of students through the new curriculum in Dundee which has integrated patient safety activities from year 1. The initial patient safety programme in year 1 was met with a strong negative response by a sub set of students (2). This reaction was explored through qualitative data from focus groups. The analysis identified a conceptual model which was subsequently tested over time using appropriate methods aligned to Kirkpatrick’s levels to assess knowledge, attitudes and behaviours as the students ascended the curriculum.

MethodsStudy 1 - Aligned to Kirkpatrick’s level 1. The aim of this study was to understand the student response to a programme of activities using patient safety tools introduced in 2006 using qualitative data. Study 2- Aligned to Kirkpatrick’s level 2. This study tested the association between knowledge and attitudes of patient safety with reflective thinking. Students completed validated questionnaires which tested their reflective thinking and understanding and future intentions about patient safety (3, 4). Study 3 -Aligned to Kirkpatrick’s level 3. This study used the ward simulation exercise which students undertake in year 5 as the basis for measuring safe behaviour and error. Participants completed a questionnaire to assess reflective thinking. In studies 2 and 3 correlations coefficients were calculated to identify associations between knowledge, attitudes and behaviours with reflective thinking.

Study 3 (Year 5)

Study 2 (Year 3)

Study 1 (Year 1)

Figure 1. Diagram illustrating methodological approach

4

4

3

2

1

Kirkpatrick’s levels:4. Outcomes3. Behaviours2. Knowledge

/Attitudes1. Reactions

Results The three studies were completed between 2006 and 2010. Study 1 used qualitative data to capture students’ reactions at level 1. The analysis identified a conceptual model linking reflection and patient safety which was then subsequently tested in studies 2 and 3. In study 2 significant correlations were identified between students’ knowledge and attitudes and reflective thinking. In study 3 significant correlations were identified between safe behaviour, observed errors in simulated practice and reflective thinking.

Conclusion By using the approach described here, the initial qualitative data has been explored using methodologies appropriate to students’ knowledge and skills at different levels of the curriculum. This has enabled the conceptual model to be tested over time using Kirkpatrick’s framework.

References1. Kirkpatrick, D L. (1959). Evaluating Training Programs, 2nd ed. San Francisco:  Berrett Koehler2. Ambrose L,Pennington G , Ker J.S Junior medical students’ perspectives on involvement in a patient safety initiative. ASME Association for the Study of Medical Education, Annual Scientific Meeting. Aberdeen 2006.3. Kember D, Leung D, Jones A, Yuen Loke A, Mckay J, Sinclair K et al. Development of a Questionnaire to Measure the Level of Reflective Thinking. Assessment & Evaluation in Higher Education, Vol. 25, No. 4, 20004. Patey R, Flin R, Cuthbertson BH, MacDonald L, Mearns K, Cleland J, Williams D. Patient safety: helping medical students understand error in healthcare Qual Saf Health Care. 2007 Aug;16(4):256-9.

5

Bedside teaching – do junior doctors and medical students make a winning team?

A Nair, S Singh

S Singh, 10 Lancaster Avenue, West Norwood, London, SE27 9DZ.

ObjectivesBedside teaching by junior doctors plays an important and growing role in undergraduate teaching. In our study we had three aims. We set out to assess the role junior doctors play in preparation for finals and whether there were significant differences in the expectations and limitations of bedside teaching between teachers and students. Finally we assessed the value of having a structured bedside teaching programme of up to eight weeks, as is in place in our teaching hospital. MethodWe designed a single survey that both students and teachers answered anonymously using 5 point Likert scales. The survey was sent to 149 medical students and 43 Foundation Year 1 doctors who all took part in the structured bedside teaching programme at Basildon and Thurrock University Hospital. Students answered the questionnaire after completing their MBBS exams. The questions were designed using components of the Cleveland Clinical Teaching Effectiveness Instrument, separated into 10 different areas and were positively and negatively keyed to help reduce acquiescence bias.  Using a single survey and non-parametric statistical methodology for both groups allowed us to highlight any differences between the groups effectively.

Results & Discussion (Results pending August 2010) Given the increasing demand on junior doctors to teach medical students, our results will show whether final year students had confidence in their tutor. It is important that students and teachers have similar expectations of any mode of teaching. This study will highlight whether differences exist between bedside tutors and students. Addressing these discrepancies should help improve the quality of teaching and the learning experience. Bedside teaching in many hospitals is ad hoc and we asked whether a more structured approach was beneficial for teaching. We also explored other roles for such a programme including providing mentoring and support for students, work experience and developing teaching skills.

6

The Evaluation of an Inter-Professional Multimedia Musculoskeletal Examination Teaching Resource: A Qualitative Study

C Griffin, C Chung, V Tzortziou-Brown, D Morrissey

C Griffin, 106 White Hill, Chesham, Buckinghamshire, HP5 1AR

BackgroundInter-professional education (IPE) has been used more frequently over the last thirty years to encourage collaborative teamwork within healthcare.

ObjectivesThe aims of this study were primarily to assess the impact of multimedia instruction on musculoskeletal clinical examination (MCE) skill acquisition. Secondly, to evaluate students’ perceptions of the value of IPE and multi disciplinary team (MDT) work.

MethodA survey was administered to a purposively sampled group of post-graduate students, comprising of medical practitioners and physiotherapists (n=26). The sample was diverse in age, speciality, and MDT and IPE experience. The intervention was four sports injury assessment DVDs made by expert clinicians modelling inter-professional team working. Post-intervention semi-structured interviews, conducted with a sample from the survey group (n=10) until data saturation occurred, were audio-recorded and analysed by thematic content analysis (TCA).

ResultsThe survey response rate was 85%. Responses revealed 46% of the group described their competence in MCE as ‘developing skills’ and 50% described having ‘some experience’. Attitudes towards IPE and MDT work were highly positive. The interviews revealed five main themes: a real time approach with expert instruction; multiple perspectives; diversity within the cohort; changing behaviour by improving MCE technique and enhancing interactive skills.

ConclusionThis study is the first to provide evidence for the self-perceived learning of MCE skills through an inter-professional DVD simulation amongst post-graduate students. Self-perceived learnt skills included detailed examinations, special joint testing, and biomechanical assessments. Furthermore, the DVDs improved students’ appreciation of IPE, healthcare teamwork, and awareness of other disciplines. The use of DVD simulations to teach and assist with the delivery of MCE and IPE warrants further investigation.

Key wordsInter-professional education; musculoskeletal; multimedia; learning; skills

7

Ethnicity and academic under-achievement early in the medical course

M Carroll

M Carroll, Centre for Medical Education, Barts and The London, London E1 2AD

Medical students and graduates from ethnic minority groups under-achieve academically at every level. We sought to identify the underlying causes in one cohort of our Year 1 medical students. We combined qualitative methods (questionnaire and focus groups) and quantitative methods (statistical analysis of examination performance at MBBS Part 1 and possible variables, both demographic and academic). Students were categorised as either ‘white’ or ‘non-white’ on the basis of self-declared ethnicity.

Relative to white students, non-white students: were significantly younger; had a significantly lower proportion of graduate entrants; scored significantly lower marks on all 4 papers at MBBS Part 1; were much more likely to fail; had closely comparable prior educational attainment (from UCAS tariff scores) and potential (in 2 out of 3 UKCAT sub-domain scores); attended PBL tutorials to the same extent. Regression analysis confirmed that the only unique significant variables contributing to examination performance were ethnicity and UCAS scores; graduate status was a contributory variable to coursework performance. Qualitative analysis suggested that the family background of non-white students influenced their work ethic in multiple ways that might impair academic achievement; however the low response rates limited reliable conclusions.

Our research indicates that non-white students arrive at our medical school with closely comparable academic attainment and potential to those of non-white students, but that their experience in Year 1 somehow undermines their academic progress. Further work is needed to: combine more cohorts to improve statistical analysis and enable differentiation of ethnic sub-groups; analyse the influence of socio-economic background; and engage non-white students more effectively in the qualitative analysis.

8

An exploration of the methodological opportunities and limitations of case studies for research on educating health professionals living with HIV in Kenya

C O’Keeffe

C O’Keeffe, Faculty Development Education Lead, London Deanery, 32 Russell Square, London

Understandings of professionalism held among health care workers can perpetuate perceptions that health professionals are role models who have knowledge and resources to protect themselves from infection with HIV, even though they are often subjected to similar gender and cultural norms that predispose community members more generally to infection. There is evidence from Kenya to suggest that health workers are therefore subjected to increased levels of stigma and discrimination from local communities and from within the health system, often preventing them from accessing treatment, further exacerbating the already critical shortage of human resources for health. To address this problem support groups for health workers living with HIV have been established within Ministry of Health facilities.

A qualitative case study approach within a contructivist paradigm was used to explore the nature and purpose of these support groups, the perceived effect of the groups on the personal and professional lives of their members and on their experiences of stigma and discrimination. Data was collected using narrative interviews with 22 support group members and 13 key informants. The study provided new knowledge about how support groups can help to promote learning to address HIV related stigma and discrimination within health facilities.

This paper will focus on the methodological challenges related to researching support groups for health professionals living with HIV in Kenya. The issues include: cross cultural power relations between the researcher and study participants, and challenges associated with conducting interviews on sensitive issues. The value of case study research for exploring this topic, and the nature of knowledge produced will be examined. The limitations of this approach will also be considered and suggestions made for the design of future research on this issue.

9

Cooperation in a Competitive World

P Bryant

P Bryant, Lecturer UEA, Faculty of Health, Level 2 MED Building, University of East Anglia, Norwich, NR4 7TJ

ContextDoctors of tomorrow are expected to learn and work effectively within multi-professional teams. (General Medical Council 2009) Group work is emphasised as important in the undergraduate medical curriculum by GMC in order to promote better team performance in the working environment. UEA Medical School runs a Problem Based Learning (PBL) curriculum in which students spend many hours a week working together in small groups. Management of the group by a facilitator is important in achieving cooperation within the group (Azar 2008) but external factors may also be influential. In 2005, it became a requirement for each medical school, to place its students into quartiles A-D, (banding) for entry to post graduate medical specialty training posts. (Foundation Programme 2010 ) Many studies have demonstrated that cooperative learning is more effective at producing individual success than competitive learning (Salvin 2008) but banding may conflict with team cooperation and learning.

MethodologyThe project will be divided into 4 phases1. In depth interviews with medical students to gain insight into the learning environment.2. Development of a cooperative learning environment scale.3. Statistical analysis of student’s exam results in competitive and cooperative environments. 4. Consideration of development of group reward system to promote for cooperative working / learning if this appears more effective.

ResultsPhase 1. I will use qualitative data analysis for interviews and code data using grounded theory or the constant comparison method.

Main conclusions The conclusions will be used to inform undergraduate curriculum development at UEA, other PBl medical school and those using or introducing group work.

ReferencesAzar, Sammy A. 2008. Navigating Problem-based Learning: London : Churchill Livingstone Elsevier. Foundation Programme. Available at http://www.foundationprogramme.nhs.uk/pages/medical-students/faqs#answer14 ( Accessed 24. June 2010) General Medical Council, 2009, Tomorrow's Doctors 2009. London: General Medical Council. Slavin, Robert E. Johns Hopkins University, 1995. Research on Cooperative Learning and Achievement: What We Know, What We Need to Know. Adapted from

Slavin, 1992. Available at http://socialfamily535.pbworks.com/f/slavin1996%5B1%5D.pdf Accessed June 2010

10

Corpus Analysis of Problem-Based Learning Transcripts: A new method to look into PBL

A L Da Silva, R Dennick

A L Da Silva, Medical Education Unit, Medical School, University of Nottingham, NG7 2UH

Problem Based Learning (PBL) is one of the most researched areas of medical education with numerous papers and meta-analyses concerned with its overall effectiveness in comparison to traditional methods. Fewer studies, however, have looked at the processes occurring within the PBL group discussions. In particular, it is remarkable that relatively little attention has been paid to the fact that students who engage in PBL talk to each other for 3–5 hours per week, which may lead to hundreds of hours of interpersonal discourse over a complete PBL curriculum. Research has not yet capture or analyzed the cognitive activities that take place during this “talking”.

We have used corpus analysis, a renowned research method from linguistics, to research into the cognitive activities taking place during the PBL discussions. 60 hours of videos from both first and second year PBL sessions were transcribe and uploaded to corpus analysis software (Wmatrix2 University of Lancaster). Using this method we were able to identify variations in technical language, questions, reasoning and explanations a) within one group over time (longitudinally); b) between 1st and 2nd year groups (cross sectional); c) between different modules d)between different facilitators. We were also able to measure (using log-likelihood) of how significant were the differences between the discourses used in the sessions for each of the categories analyzed.

We propose that this methodology will become a powerful tool to help explore the cognitive and linguistic development of students as they engage in PBL discourse. In this presentation we propose to present this method, how we have adapted it to the research in PBL in medicine and further explore its application in research in medical education.

11

Gender, culture and professional identity: a qualitative enquiry into the effects of the learning and professional environment on the construction of a surgical identity

L Etheridge

L Etheridge, ACME, 4th floor Holborn Union Building, Highgate Hill, London, N19 5LW

ContextJunior doctors are now required to select the specialty they wish to train in during their initial Foundation training. Choices made at medical school and the type of Foundation posts worked often impact on success in achieving a training post. Therefore, young doctors may be forced to make choices early on, possibly based on incomplete or stereotyped information.

Overall, 28% of NHS consultants are now women. However, in 2008 only 8.4% of surgical consultants were women (Elston, 2009). So while the numbers of women are rising overall, there are clear differences in some specialties, especially surgery.

Research aimsMy aim is to further theorise the relationship between gender and the surgical profession by exploring how doctors construct a professional identity as a surgeon, what the influences on this are and how it affects their roles in the workplace.

MethodologyThe dominant conceptualisation of gender in the medical workplace is essentialist in nature. My ontological position, by contrast, is a sociocultural one. I start with the premise that doctors develop as gendered subjects through the discursive practices that structure their worlds (Pringle, 2003). Holland et al’s (1998) concept of figured worlds supposes that “identities are lived in and through activity and so must be conceptualised as they develop in social practice”.

I will undertake a case study of doctors training in a particular region, 5 men and 5 women. I will conduct narrative biographical interviews with participants. By collecting and analysing these personal stories I hope to gain an understanding of who the surgeons claim to be, their behaviour and how they make sense of their world.

ResultsThis study is part of my EdD. By November 2010 I will have conducted my interviews and plan to present my initial data and analysis for feedback and discussion.

12

Unpicking issues within the osteoarthritis consultation in primary care using videotaped surgeries Z Paskins, AB Hassell, PR Croft

Z Paskins, Clinical Lecturer and Honorary Specialist Registrar In Rheumatology Arthritis Research UK Primary Care Centre, Primary Care Sciences, Keele University Staffordshire, ST5 5BG

Osteoarthritis (OA) is a common cause of pain and disability, and comprehensive guidelines exist regarding its optimal treatment1. Current research suggests patients with osteoarthritis have significant unmet needs2; however, little is known about what currently happens in the consultation between GP and patient when joint pain is discussed. The Arthritis Research UK Primary Care Centre is currently undertaking NIHR funded research concerned with implementing OA guidelines, including a comprehensive training package for GPs. This begs the question: what are GPs’ educational needs in regard to the OA consultation?

This PhD study aims to explore in depth the OA consultation using video recordings of consecutive consultations in primary care. The patients and GPs involved in these consultations will then be invited for an interview where they will be shown the video clips of their own consultation. The interviews will provide contextual information for the videoed consultations as well as gathering further information about GPs attitudes regarding the consultation with patients with OA in general and patients’ views of the consultation and their interpretation of language used by the doctor.

This is an example of ethnographic research in a clinical setting to inform educational needs assessment; the challenges of analysis are common to ethnographic study in educational settings. The video and interview data will be subject to qualitative and quantitative analysis and an analytical framework will be presented.

By videoing consultations, we hope to gain a rich data set that will inform and be of interest to practising clinicians and educationalists alike.

References1.National Institute for Health and Clinical Excellence 2008, The care and management of osteoarthritis in adult [CG59]2.Arthritis Care 2004, OA Nation, Arthritis Care, London.

13

Conceptions of learning and leadership – a qualitative study of senior faculty in a research-intensive university

A Berlin

A Berlin, Division of Medical Education and Division Population Health, University College London, Medical School Building, London WC1E 6AU

The idea that the keys to public sector quality are organisational learning and improved leadership have gained currency recently. (PIU, 2001). There is a theoretical argument that educational institutions could benefit from considering their approach to leadership alongside their approach to learning. (Bolam et al 2005; Gibbs, 2007; McKimm, 2004).

As part of my EdD I undertook a qualitative study exploring this link. The core premise informing the methodology was that leadership and (to an important extent) learning are mediated through social processes in relationships and discourse, which are, in turn, informed by individuals’ conceptions.

I conducted eleven in-depth interviews with four overlapping groups: faculty in (a) the wider university and (b) the Medical School who were either (c) senior (vice dean or above) or (d) “aspiring” leaders (senior lecturers, professors identified as future leaders). I was privileged to be able to interview colleagues from all faculties including the four most senior faculty in the University. The interviews proved candid and all contained a number of striking elements: contradictions, in particular with regard to the leadership of research versus education; and moments when participants appeared to be “telling me things” (as a member of staff) rather than merely answering the question.

This project raises methodological, ethical, and practical dilemmas, which are of wider interest to Doctoral students and only partly resolved. Starting as a constructivist endeavour it became necessary, but awkward, to consider additional stances to incorporate themes of power and institutional culture. The high status and candour of some of participants enriched the data. I felt a moral obligation to share my findings but trepidation regarding presenting the underlying social science and handling the ethical complexities for my participants and myself as an insider researcher.

References. Performance and Innovation Unit [PIU] (2001) Strengthening leadership in the public sector. Cabinet Office HMSO Bolam, R. et al. (2005). Creating and Sustaining Effective Professional Learning Communities. http://www.dfes.gov.uk/research/data/uploadfiles/RR637.pdf McKimm, J. (2004) Developing Tomorrow’s Leaders in Health and Social Care Education Case studies in leadership in medical and health care education. HEA,

Subject Centre: Medicine, Dentistry and Veterinary Medicine Newcastle Gibbs,G. Knapper,C. & Picinnin, S.(2008) Departmental leadership for quality teaching - an international comparative study of effective practice. University of

Oxford. http://www.lfhe.ac.uk/research/projects/gibbsoxford.html/

14

From theory to data analysis: using mixed qualitative methods to explore a complex educational intervention

S Yardley

S Yardley, Keele Medical School, Keele University, Staffordshire, ST5 5BG

IntroductionLittle empirical work has related socio-cultural theories to empirical data from the initial years of medical degrees. Set within the context of a new curriculum this doctoral study sought to understand socio-cultural consequences resulting from early experience placements through qualitative interviews with students (n=23), faculty (n=13) and placement providers (n-20). This paper presents the theoretical framework, within which data was situated, and justification for the choice of methodological tools. The research received prospective ethical approval.

AimTo develop mixed qualitative methods alongside multiple theoretical perspectives to achieve an interpretative analysis that remained coherent with empirical data.

MethodologyA theoretical framework was developed through interpretative dialogue between empirical data and socio-cultural theories including Situated Learning (Lave and Wenger 1991) and Activity Theory (Engeström 2001). Scott’s concept of Mētis (1998) was used to develop a cohesive perspective on creating meaning through dynamic interactions. Mētis provides a framework for considering not only how people create meaning but also when and how they choose to use it, and value it, relative to formally recognised knowledge.

ResultsThe theories above omit guidelines for practical data management. Therefore, an iterative analysis was developed based on their theoretical roots in the philosophies of constructionism and interpretative interactionism whilst allowing data to also challenge the framework. Practical analytic tools were drawn from thematic (identification of content), narrative (identification of the story being told), interpretative (what does the meaning created by participants signify?) and discourse (what can be understood from the language used by participants?) approaches. Examples of complementary overarching data types will be discussed.

ConclusionsThis study demonstrates theoretical development though dialogue with empirical data. Using mixed qualitative methods alongside multiple theoretical perspectives allows deeper interpretation of socio-cultural consequences from early experience.

ReferencesEngeström, Y. Expansive learning at work: toward an activity theoretical reconceptualization. Journal of Education and Work 2001:14;133-56.Lave J, & Wenger E. Situated Learning: Legitimate peripheral participation. Cambridge, CUP, 1991.Scott, J., C. Seeing like a state: how certain schemes to improve the human condition have failed. New Haven, Yale University Press, 1998.

15