Visit report - Barts and the London School of Medicine and Dentistry (Queen Mary University of London) Malta MBBS programme

This visit is part of the new schools quality assurance annual cycle.

Our visits check that organisations are complying with the standards and requirements as set out in Promoting Excellence: Standards for medical education and training.

Summary Medical school Barts and the London School of Medicine and Dentistry

University Queen Mary University of London (QMUL)

Programmes MBBS Malta

Date of visit

6 March 2018 (visit)

24 May 2018 (OSCE observation)

5 July 2018 (wrap-up meeting – some commentary in the report reflects school updates at this time)

Key findings

Barts and the London School of Medicine and Dentistry (the school) accepted its first cohort of students to the Malta programme in August 2017. Of the original 45 students (year one) enrolled on the course, 40 remain. Since the last visit good progress has been made in preparing the GP tutors (on Malta) and student-selected component (SSC) and problem-based learning (PBL) tutors (on Gozo) for their roles. The tutors we met seemed well prepared, engaged and committed to education and training. The assessment we observed in May 2018 was well structured

Academic Year 2017 - 2018

2

and allowed students to demonstrate a range of skills and attributes which were blueprinted against the curriculum.

It was gratifying to learn that the majority of the concerns raised by the students during the visit in March 2018 did not relate specifically to the educational and training components of their programme. Indeed, the students appeared satisfied that the quality of the education and learning received was good.

The school has, however, encountered the following issues since the last visit:

• The school’s plan to house the students in their own medical school based at the Gozo General Hospital (GGH) site has been hampered by delays in the construction of the new medical school (firstly, by the discovery of Roman remains and more recently by the change in ownership from Vitals Global Healthcare (VGH) to Steward Healthcare). Completion of the new school is now delayed until early 2019 (with planned full occupation from March 2019).

• Alternative facilities have been found at a local sixth form college on Gozo, off the GGH site, and there are plans to expand the rooms available for use from August 2018 in anticipation of the next cohort of students arriving (potentially up to 60 students).

• In addition, planning permission delays have impacted on the start of the planned Anatomy Centre construction, adjacent to the GGH site. However, the school advised that they have been given permanent use of the ground floor of an existing building on the GGH campus which they believe will provide an even better facility than was originally planned. Refurbishment should allow occupancy at the beginning of the new academic year in September 2018. Currently anatomy is taught in purpose-designed space in the sixth form college.

• The students’ residences, which are 3.5 miles from the proposed new medical school building and GGH, were found to not meet UK Fire Regulations. During the modifications, the students were unable to use their

3

ovens and hobs. The school halved their rent and provided them with a daily allowance of €20-30 as a gesture of goodwill during this period.

• The school admitted that they under-estimated the importance of the Maltese language on the island and did not anticipate that patients would prefer to speak Maltese when consulting with doctors and in clinical settings. The Maltese Medical Council has recently introduced a rule that requires demonstrable Maltese language proficiency after 12 months in post. The consequences of this are being explored by the school with the Malta Foundation Programme.

The above issues have led to some discontent among the students.

Our visit identified an apparent disconnect between the strength of students’ feelings about their overall experience and the school’s perceptions of that experience. We are concerned that there is potential for a breakdown in trust between the school and its students in the Malta programme unless communication is improved. Some students told us that they do not feel that their voices are being heard when they raise concerns about the facilities infrastructure, new language requirements, and student support needs. The school advised us that they take these concerns very seriously and are actively addressing communication between the school and its students. The student-staff liaison committee is a key component in addressing these concerns and will be a focus for future information-gathering.

4

Update on open requirements and recommendations

Open requirements Update Report paragraph

1 The educational culture in the Mater Dei Hospital must align to the Barts and the London School of Medicine and Dentistry arrangements (The students will not be in secondary care placements until the academic year 2019-2020).

To be discussed at subsequent quality assurance activities. The school must continue to work closely with each local education provider (LEP) on a continuing basis to align educational cultures.

n/a

2 Quality management mechanisms at Barts and the London School of Medicine and Dentistry and quality control mechanisms with local education providers in Malta and Gozo require further work and detail. This should include:

- Identification of an educational leader within each local education provider, including at primary care providers;

- More formal links between Barts and the London School of Medicine and Dentistry and the local education providers;

- Formal appraisal and feedback mechanisms between organisations and between students and teachers.

This requirement is still in progress. The school has appointed a number of educational leaders (designated associate deans) but there are still more to be appointed.

The school plans to introduce appraisal systems but these have not yet been tested. A peer review process for teachers has already been implemented.

25, 42-51

3 More granular information on the numbers of students in clinical placements must be made available to provide a clear picture of the numbers of students from Barts and the London School of Medicine and Dentistry and the University of Malta and the numbers of clinical

The requirement for further detail has been met. In future we will be testing our understanding of the information provided and, over time, will continue to review the educational capacity of clinical placements.

Document submission

5

educators in each place at any one time. Currently this level of detail is unclear.

4 Paediatrics, mental health and obstetrics and gynaecology are areas requiring more detailed clinical educational capacity mapping as part of Barts and the London School of Medicine and Dentistry’s stress-testing and risk mitigation work.

See above - to be discussed at subsequent quality assurance activities.

n/a

Open recommendations Update Report paragraph

1 Barts and the London School of Medicine and Dentistry will need to show how they are introducing a learning culture that is consistent and clear across all learning environments.

This requirement has been partially met. The school’s faculty development actions so far seem satisfactory as the current tutors feel well trained and prepared for their role. We will review this area as part of our future quality assurance activity.

42-54, 26

2 Barts and the London School of Medicine and Dentistry will need to demonstrate their ambitions for inter-professional learning.

This requirement has been partially met.

During the visit, we heard that students have been participating in joint sessions with student nurses at Gozo General Hospital. We will review this area further at subsequent quality assurance visits.

6

3 We would like to see how opportunities for students to use the Mater Dei Hospitals skills centre for learning are being developed.

To be discussed at subsequent quality assurance activities.

n/a

4 Barts and the London School of Medicine and Dentistry will need

This requirement has been partially met. We heard about a

42-54

6

to demonstrate how they are ensuring clinical teachers meet necessary standards so that students receive a consistent experience with the London students.

series of training events that the GP tutors and SSC/PBL tutors have attended prior to taking up their new responsibilities. We will continue to review this area at future visits.

The medical school must continue to meet GMC standards governing the recognition and approval of trainers.

5 The primary care representatives at the Mosta Polyclinic had not yet received general practitioner tutor development. All GP educators need educator development before students arrive.

This requirement has been met. 42-54

6 Barts and the London School of Medicine and Dentistry will need to demonstrate how they are progressing secondary care educator development, and specifically Education Leads, including how they will work with local education providers to ensure tutors receive adequate training and development.

This requirement is in progress. Prior to the visit, we received a response to the requirements and actions highlighted in the March 2017 visit in which QMUL outlined its staff development programme for Malta educators. The school reassured us that all educators in Malta have access to the same training and support opportunities as colleagues in the UK.

42-54

7 The implementation of the Barts and the London School of Medicine and Dentistry curriculum will require further work, particularly with clinical teachers. The need for curriculum alignments between Barts and the London School of Medicine and Dentistry and the University

This requirement has been partially met. We heard that the school has slightly amended the order of the year one modules, but that in general the curriculum replicates the London version.

At the last visit we heard about the school’s plans to possibly

55-58

7

of Malta curriculum, particularly Years 3 to 5, will require you to demonstrate strong collaboration at operational levels.

collaborate with the University of Malta. We would encourage the school to continue in its efforts to work closely with the University of Malta particularly considering the need to align the culture of Mater Dei Hospital with Barts and the London School of Medicine and Dentistry.

We look forward to reviewing how the school intends to align the curriculum particularly for years 3 to 5.

Areas that are working well

We note areas where we have found that not only our standards are met, but they are well embedded in the organisation.

Number Theme Areas that are working well Report paragraph

1 4 (R4.1) We found that the GP tutors we met are engaged and committed to education and training. They are learning from the contact that they are having with the UK school, leading to a subsequent cross-fertilisation of approach with the University of Malta students that they teach.

46

2 4 (R4.1) We found that the problem-based learning (PBL) tutor and student-selected components (SSC) tutors that we met from Gozo General Hospital are engaged and well prepared for their role.

47-49

3 5 (R5.3) We were pleased to hear that anatomy teaching was a highlight of the students’ first six months.

56

4 5 (R5.3) The students commended the number of SSC options available.

58

8

5 5 (R5.6) Overall, it appears that the assessment processes and policies in London have been replicated in Malta.

65-76

Requirements

We set requirements where we have found that our standards are not being met. Each requirement is:

targeted

outlines which part of the standard is not being met

mapped to evidence gathered during the visit.

We will monitor each organisation’s response and will expect evidence that progress is being made.

Number Theme Requirements Report paragraph

1 1, 3, 5 (R1.1, R1.2, R1.5, R1.19, R3.2, R3.13, R5.4)

The school must demonstrate a culture that both seeks, and responds to, feedback from students and educators (and indirectly, from patients via the LEP arrangements); this, especially around compliance with standards of patient safety and care, and on education and training. Whilst never binary, these can be viewed as patient/service-facing and student-facing. The school should improve communication with current and future cohorts of students to ensure that they receive timely and accurate information about the concerns they raise, both educational and pastoral.

Patient-facing issues:

a) We noted a lack of awareness of the policy of raising concerns. The school must make sure learners know what to do if they have concerns about quality of care, and they should encourage students to engage with these processes.

1-5, 7-15, 27-33, 37-39, 59-63

9

b) The students have continued concerns about the prevalence of the Maltese language in clinical settings and about the future implications for their education. The school should be explicit in its expectations in this area and be responsive to any local rulings and legislative changes.

Student-facing issues: a) The students have continued

concerns about progress with regards to their facilities and learning environments (buildings). The school should be as transparent as possible (we understand the constraints of being an intermediary in a complex chain of command) with students when explaining the future timeline for completion of the medical school and other facilities, and seek their feedback regarding how to improve their learning environments and study spaces in the interim period. We recommend that the school involves the students more in decision-making with regards to the non-curricular aspects of their time on Gozo.

b) The school must ensure that the students have access to resources to support their health and wellbeing, and to pastoral support, including confidential counselling services and occupational health services.

2 3 (R3.16) The school must ensure that students understand the approach to careers support, and have access to such support in a timely way that commands the student body’s trust.

40

3 5 (R5.3) While the school is clear that histology teaching is delivered appropriately and is

56

10

consistent with the London programme, the students’ perceptions are different. The school should analyse feedback from students on this aspect of teaching and respond.

4 3, 4 (R3.13, R4.1)

The school must provide clarity with regard to plans for, and implementation of, appraisal systems for both students and educators.

39, 51

5 5 (R5.6) The role of student support should be completely separate from that of assessment.

72

Recommendations

We set recommendations where we have found areas for improvement related to our standards. They highlight areas an organisation should address to improve, in line with best practice.

Number Theme Recommendation Report paragraph

1 1, 3 (1.19, R2.3)

The school should align the expectations of the current student cohort to the reality of what will be delivered and ensure that any future marketing for potential students is explicit.

4, 21

2 3 (R3.5) To prepare students for their application to the foundation programme, the school should share plans for their introduction to situational judgement tests.

36

3 5 (R5.6) We would recommend that the school indicates on the OSCE station instructions the number of stages or questions included in the station so that students are able to pace themselves and successfully complete each station within the required timeframe.

76

4 5 (R5.8) We recommend that OSCE examiners’ badges should include their name in case students want to provide feedback afterwards.

88

11

Findings The findings below reflect evidence gathered in advance of and during our visits, mapped to our standards.

Please note that not every requirement within Promoting Excellence is addressed. We report on ‘exceptions’, e.g. where things are working particularly well or where there is a risk that standards may not be met.

Theme 1: Learning environment and culture

Standards S1.1 The learning environment is safe for patients and supportive for learners and educators. The culture is caring, compassionate and provides a good standard of care and experience for patients, carers and families. S1.2 The learning environment and organisational culture value and support education and training so that learners are able to demonstrate what is expected in Good medical practice and to achieve the learning outcomes required by their curriculum.

Raising concerns (R1.1); Dealing with concerns (R1.2)

1 Before the visit in March 2018, we received details of the school’s policy on raising concerns in London, which is replicated in Malta. The school indicated that these arrangements were highlighted to the students as part of their induction week and that both clinical and non-clinical staff were made aware of the policy and processes during their training. The school also informed us that there may be a need for a Malta-specific pathway for more serious concerns and that this is being developed.

2 This was supported by the quality management team who told us that students had been instructed on how to raise concerns about clinical care. We also heard that the GP lead and clinicians in Malta were enthusiastic about improving standards and modernising and were therefore keen for students to raise any concerns they had.

3 During the visit, however, we found little evidence that learners and educators knew how to raise concerns about patient safety. The GP tutors we met were unaware of a policy regarding raising concerns, although they commented that they could access documents on the QMUL website if required. They told us that the students could approach the GP tutor if they wanted to raise concerns or they could contact the Head of Department. None of the students we met were aware of a policy regarding raising concerns relating to clinical care. Some students had identified some concerns with regards to clinical practice but seemed unsure how to raise these issues. The school must make sure learners know what to do if they have concerns about quality of care, and they should encourage students to engage with these processes.

12

4 Before the visit and during our initial meeting with the school’s senior management team, the school told us that a number of students had raised concerns about their experience in Malta but that many of the students’ concerns related to the student experience (e.g. travel, accommodation, facilities) rather than curriculum/educational concerns. Two formal complaints were submitted to the university on behalf of a number of students. The management told us that they had tried to address the students’ concerns despite feeling that some of the complaints were unreasonable. When we met with the students they presented the team with a document detailing a number of concerns about their experience. Their spokespersons, in the presence of most other students, told us that despite having raised the issues with the school and with QMUL, they didn’t feel that they’d received a satisfactory response. The school subsequently informed us that the complaints to the university were not upheld. The students also believe that the school’s marketing was misleading, particularly with regards to the unfinished medical school, and their route to registration. It will be important for the school to reconsider its marketing style for future applicants and take into account the views of the current students when doing so.

Seeking and responding to feedback (R1.5)

5 We found that the school demonstrates a culture that seeks feedback from learners and educators on education and training but we felt that further work is required to ensure that any feedback is acknowledged and leads to an appropriate response. Prior to the visit we heard that students use the Bristol Online Survey (BOS) to provide anonymous feedback about their educational experience (both classroom and in the clinical environment) and their teachers. We also heard that teaching leads receive analysis of the results within two weeks of the evaluation. The tutors we met confirmed that they asked the students for feedback at the end of the SSC and PBL sessions. We heard that all Medicine in Society GP tutors attended a feedback and debrief session in January 2018 but other than this, the tutors were not aware of any other formal plans for feedback and debriefing sessions. They said that they would welcome the opportunity to receive feedback and improve.

Multiprofessional teamwork and learning (R1.17)

6 We found that during the recent placements at Gozo General Hospital, the staff promoted a culture of learning and collaboration between professions. We met with a group of SSC tutors and a PBL tutor based at Gozo General Hospital who are responsible for making sure that the students’ SSCs and PBL sessions are worthwhile and educational. The tutors told us that during the recent SSCs, students had the opportunity to interact with nursing staff and nursing students and attend outpatients, ward rounds and tutorials.

13

Capacity, resources and facilities (R1.19)

7 There have been a number of setbacks in the construction of the new medical school building. Following the discovery of Roman remains, building work was suspended pending archaeological investigations. This was then followed by a delay caused by the takeover of VGH, the previous concessionaire, by Steward Healthcare. During the visit in March we heard that building work had resumed and the senior management team anticipated that the new anatomy building would be ready by September 2018 and that they should be able to occupy the main building by March 2019. The school anticipated that anatomy teaching would start in December 2018. The school had agreed arrangements for embalming cadavers with the University of Malta in the interim period. The school had held meetings with Steward Healthcare and was confident that the project would now be fast-tracked.

8 Despite efforts by the school to keep students informed of progress, those we met did not feel that the school had kept them sufficiently abreast of the timeline regarding the completion of their school building or the transfer of ownership from VGH to Steward Healthcare. They felt that they had been misled before starting the course that the building was merely awaiting ‘finishing touches’ and had not been told that the medical school was still yet to be built.

9 The students told us that they did not feel that the sixth form college was suitable for their needs. They highlighted the lack of adequate library and study space as a particular issue and also pointed to the restricted access to the PBL and discussion rooms. Although the school had created a computer room, the students felt that there was still insufficient communal study space. The school told us in March that they intended to expand the library over summer 2018 and that they would renovate another floor of the building in anticipation of the new cohort of students arriving. The school also told us that following complaints about a lack of quiet, study area in the sixth form centre, they had identified several additional new rooms for the students to use. At the time of the visit in March, the students were unaware of the school’s plans to create a better quiet study space specifically for their use. We recommend that the school updates the students on its plans to improve their study area as soon as possible and we will be interested to hear whether student satisfaction increases as a result of this improvement.

10 Despite the building delays, we found that the school had succeeded in ensuring that the resources and facilities available in the sixth form centre were sufficient to deliver safe and relevant learning opportunities for the students. For example: The school told us that additional PBL rooms and a permanent clinical skills space would be available until the move to the medical school. A clinical skills laboratory had been set up in one of the extra flats in the students’ hall of residence so that the students could access it at all times. An area had also been set up as a ward area in the sixth form college.

14

11 We heard that there is almost one PC to one student in Malta whereas one PC to eight students in London.

12 We visited some of the facilities in the sixth form college in March and observed the students in an anatomy session. We heard that wet dissection would be available from August 2018 and in future, it would be offered to both year one and year two students. The school also told us that each anatomy session was offered twice so that students had ample opportunity to attend. At the time of the visit in March 2018, the students had already completed two exams in anatomy and their results were comparable with students based at the London site.

13 The school told us that the students’ residences did not meet UK fire regulations and as result, the university had asked the school to make some modifications. During this period, students had been unable to use their ovens and hobs. As a gesture of goodwill, the school had halved the students’ rent and given them money to buy food.

14 The school told us that they had kept the students informed of the latest developments regarding their residences by email. They intended to hold a face-to-face meeting with them to update them on the progress with the medical school, as soon as they had received official confirmation of the likely occupation date.

15 With regards to clinical capacity at the local education providers, we heard that there are plans to increase the services at Gozo General Hospital, which will lead to greater capacity for the future intake of students in future years. We will be interested to hear more about how this is progressing at our future visits.

16 We heard from the CEO at Gozo General Hospital that she was pleased with the way that the students were met with enthusiasm from everyone at the hospital during their clinical placements. The students were separated into groups to prevent over-crowding and so that everyone had the opportunity to experience different areas, for example, theatre.

17 We were reassured to hear that the UoM students were not timetabled to attend GP clinics at the same time as the QMUL students, therefore in the initial placements that had taken place prior to March 2018, there had been no capacity issues and the GP tutors had not felt overwhelmed by too many students. The tutors are provided with a timetable so they know when to expect which students. We will continue to review this area during our future quality assurance activities.

15

Theme 2: Education governance and leadership

Quality manage/control systems and processes (R2.1)

18 The quality management team told us that the school has set up specific quality management systems and processes. Prior to the visit, the school submitted documentation outlining the school’s strategic vision.

19 From documentation reviewed before the visit in March, we noted that the school intends to visit all LEPs at least annually but that initially the visits will be much more frequent. We also heard that the associate deans will attend termly meetings to discuss educational provision and quality, similar to the school’s process in London. During the visit, we heard that the school’s quality management team have visited all the polyclinics where students are placed since September 2017 and that, as there are only seven polyclinics, they will repeat this schedule every year if there are students attached to a particular Medicine in Society tutor. However, we heard that re-visits will only take place if issues arise. The school informed us that this replicates the school’s process for visiting clinics in the UK. We would recommend that regular visits take place during the early stages of the programme, regardless of whether issues arise, and we would welcome the opportunity to review the outcome of the LEP visits as part of our future quality assurance activity.

20 The quality management team has not yet formally visited Gozo General Hospital although frequent informal visits have taken place particularly in preparation for year 2, particularly to the long-stay unit where the Medicine in Society students will be placed in year 2. Again, we would like to review the outcome of the visits, as well as receive further information on the planned re-development and expansion of the hospital.

Considering impact on learners of policies, systems, processes (R2.3)

21 The students we met were keen to explain that they wanted to become the best doctors possible and their QMUL Malta programme to be highly regarded. They were concerned that issues such as differential tariffs (between minimum requirements for

Standards S2.1 The educational governance system continuously improves the quality and outcomes of education and training by measuring performance against the standards, demonstrating accountability, and responding when standards are not being met. S2.2 The educational and clinical governance systems are integrated, allowing organisations to address concerns about patient safety, the standard of care, and the standard of education and training. S2.3 The educational governance system makes sure that education and training is fair and is based on principles of equality and diversity.

16

entry to the Malta and London programmes) and uncertainties around future scholarship funding might undermine the reputation of the Malta programme.

Systems and processes to monitor quality on placements (R2.6); Concerns about quality of education and training (R2.7); Sharing and reporting information about quality of education and training (R2.8)

22 We found that the school has systems and processes to monitor the quality of teaching, support, facilities and learning opportunities on placements. We heard that the BOS is used for Malta students in the same way as for students in London. Students give feedback on their placements via the BOS system; this feedback is then shared with the Head of the Assessment Unit and the Head of Quality.

23 Students at both London and Malta sites have been asked to complete the BOS survey. In Malta, the student response had been poor at 57% (at the time of the visit in March 2018), although still better than in London. We heard that the quality management team is working hard to encourage students to complete the BOS.

24 We heard that the results of the BOS feed into the staff-student liaison committee (SSLC), the minutes of which are posted on the quality management website. At the time of the visit, we heard that the SSLC was only operating at a basic level and was only tackling minor issues and was not linked to London’s SSLC.

Educators for medical students (R2.13); Recruitment, selection and appointment of learners and educators (R2.20)

25 Prior to the visit in March 2018, and in response to the concerns raised at the March 2017 visit, the school informed us that associate deans would be appointed for each local education provider. They also submitted job descriptions and person specifications for these roles. An associate dean for Gozo General Hospital and an associate dean for GP have since been appointed, with further appointments still to be made.

17

Theme 3: Supporting learners

Standard S3.1 Learners receive educational and pastoral support to be able to demonstrate what is expected in Good medical practice and achieve the learning outcomes required by their curriculum.

Good medical practice and ethical concerns (R3.1)

26 At our visit in 2017, we asked the school to consider how to align the cultural differences of the two sites, London and Malta, so that students based in Malta are not disadvantaged by not having access to experience within the NHS. During the visit in March 2018, some students raised concerns about their perceptions of the gap between the Maltese medical system and the NHS system. They feared that their development as doctors might be hampered by not working with clinicians who demonstrate what is expected in Good medical practice. The quality management team suggested that although the system in Malta is different to the NHS, the students can have an equivalent experience even if it is not identical. We also heard that the school is making attempts to align the culture of the two sites, by concentrating on faculty development. For example, the school plans to arrange peer review sessions, as well as direct observation of teaching in Malta, and will also request feedback from students. The school also intends to appraise all clinical tutors annually for their role. We look forward to hearing more about this during our future quality assurance activity.

Learner's health and wellbeing; educational and pastoral support (R3.2)

27 We are concerned that some students do not feel that they have access to suitable resources to support their health and wellbeing, and to educational and pastoral support. The students raised concerns about the quality of student support available. We heard that the students had only just been given a student handbook on pastoral support the day before the visit. The school advised that this resource is on the virtual learning environment (VLE) and available to all students from day one.

28 By the time of the visit in March 2018, one of the student cohort had tragically died and a number of students were feeling particularly upset. The students did not feel that the support and counselling that they had been offered was sufficient for their needs, and told us that at times they had needed to travel to Malta at their own expense for counselling sessions at the Richmond Foundation (a counselling service in Malta). This view was not shared by the school’s management team who felt that advice had been offered but not always taken up by the students. Following the specific incident of this tragic death, the school had arranged for counsellors to be available in Gozo on the first working day thereafter to support students (and staff). The school management team was aware that some of the students were unhappy with the rapidity of response in terms of accessing counselling support in other less

18

acute situations. The school has therefore managed to identify three counsellors who are able to provide a more regular drop-in service on Gozo itself.

29 We heard during our visit in March 2018 that the Head of Student Support who provides pastoral support had also been signing off the students’ assignments. While this dual role had ceased following a student complaint prior to the visit, the students expressed concern that those who were supposed to be offering them pastoral support also had a role to play in their progression on the course. We heard from the student support team that an academic tutor works alongside student support and deals with the students’ academic progress and refers any students in need of pastoral support to the student support team. The students did not seem to recognise this arrangement and it needs reinforcing to them as a process of support.

30 The students told us that when travelling to the GP polyclinic (once every fortnight), they had to leave very early in the morning and did not return to their residence until 7pm or 8pm. They felt that there was a lot of wasted time during the day and had raised this with the GP tutors who were in charge of the schedule. The GP tutors were open to changing the timetable for the day, but the students felt that the school was reluctant to resolve this. We heard from the school management, however, that the Medicine in Society days would now finish earlier so that students could take an earlier ferry back to Gozo, and that this change had come about as a direct result of feedback given in the BOS. We recommend that the school communicates any changes to the students more effectively so that the students can see that the school is listening to their complaints and responding positively.

31 The students raised concerns about access to healthcare for themselves. They reported some issues with arranging necessary vaccines and some had not been able to resolve this until after their SSC placement. The school advised that there is a worldwide shortage of Hepatitis B vaccine, necessitating prioritisation of access. The school advised that this had been explained to the students who had been advised to avoid exposure-prone procedures and contact with body fluids.

32 We heard that the student support staff in Malta are supported by the student support office in London and can call them at any time for advice. The Head of Student Support attends the student support board in London as Malta rep.

33 The Head of Student Support told us that she gave the students a lecture at the start of the course with instructions on how to make contact with her.

Information on reasonable adjustments (R3.4); Supporting transition (R3.5)

34 The school told us that they had not yet come across any challenges with regards to reasonable adjustments on Malta or Gozo but that they would consult with London if they did. In terms of the new building, appropriate facilities have been considered and the building will be very modern in its design.

19

35 Prior to the visit, the school told us that communication skills teaching in MBBS Year 1 includes guiding the visually impaired and communicating with hearing-impaired individuals. Trainers from London, including blind trainers, two audiologists and a hearing-impaired GP went to Malta to deliver the teaching to the new Maltese faculty.

36 The school informed us of its plans to support students through application to the foundation programme. We noted that practice in situational judgement tests (SJTs) had not yet started, however, and we would want to see the school’s plans for SJT support at future visits as this will be an important step in preparing students for their application to the foundation programme.

Feedback on performance, development and progress (R3.13)

37 Prior to the visit, we received documentation that stated that the faculty would be providing both formative and summative feedback to students during the modules and placements and after examinations. We also heard that the intention was for students to meet with allocated mentors formally at least three times each year. During the visit in March 2018, we heard that there had been some delays in setting up the mentorship scheme but that all mentors had now been trained and that mentors and students had all met as a group once in 2018.

38 The students told us that they were not informed of the name of their mentor until several months into the course. Although they all had a named mentor by the time of the visit in March 2018, they stated that no individual meetings had taken place and there was uncertainty among the cohort regarding whether they should be making contact with their mentor or whether their mentor would arrange the first meeting. The school advised that group meetings had taken place but that individual meetings would be held in April each year. Similarly, at the time of the visit, the students said that they had only just found out the name of their personal/academic tutor. The school advised that these meetings had occurred in December 2017, and expressed their surprise at the students’ assertion.

39 Before the visit, the school informed us that the mentors will give the students a yearly appraisal. Although the appraisal currently does not contribute to students’ academic progression or ranking, the school has plans to introduce a more rigorous student appraisal system in London which will in time be rolled out for students in Malta. We will review this as part of our rolling quality assurance activity.

Career support and advice (R3.16)

40 We heard from the student support team that the students currently have access to London’s careers advice facilities but need to access this via Skype. This service will continue until the end of the 2018-2019 academic year, after which additional services will be in place on Malta. The students we met were unhappy with the

20

facilities on offer as they did not think that accessing the service via Skype was appropriate. However, the school advised that this is the same for London students.

41 We heard from some students that they were concerned about their career prospects. They told us that QMUL had previously promised to provide them with United States Medical Licensing Examination (USMLE) guidance or online support, but had failed to provide any information. The school informed us that students had been told about a complex mapping process currently underway and would be updated further in the next academic year.

21

Theme 4: Supporting Educators Standards S4.1 Educators are selected, inducted, trained and appraised to reflect their education and training responsibilities.

S4.2 Educators receive the support, resources and time to meet their education and training responsibilities.

Induction, training, appraisal for educators (R4.1)

42 At the time of the visit in March 2018, we found that the educators appointed so far had had access to appropriate training for their role. The school’s intention is for all Malta-based staff to have access to the same training opportunities as their London-based colleagues. Prior to the visit, the school submitted information to us regarding its staff development training for educators on Malta.

43 We heard that educational supervisor training had already taken place in August 2017 at Gozo General Hospital and that this training was mapped to the professional development framework for educators. Additional educational supervisor training was planned for Karin Grech Hospital in April 2018. This training would be for clinical educators who would be teaching students in their secondary care placements. We heard that that the school intends to provide educational supervisor refresher courses and to recommend on-line courses which will contribute to a three yearly appraisal for educational leadership. We will review the success of this training at our next visit.

44 Prior to the visit, the school also informed us that the GP tutors had attended a series of workshops and training sessions orientated to the teaching methods used at QMUL and the school’s curriculum from July 2017. We heard that eight GP tutors had attended a workshop on reflective writing alongside their allocated students at the beginning of term. We also heard that there would be a new project to extend the training to each of the other primary care providers commencing in May 2018.

45 Before the visit, the school informed us that the GP training programme is planned on a rolling basis, three times per year, to guarantee new GP tutors are trained and existing tutors are refreshed. We heard that going forward there will be module-specific training for experienced tutors taking on teaching in Years 3, 4 and 5 with the on-going provision of the new tutor training on an annual basis as needed for new community-based tutors. The next tutor training sessions were due to take place in April and June 2018.

46 The GP tutors we met reported that they were given appropriate guidance and training on their role. They confirmed that they attended two introductory induction sessions before they started teaching the students. At the time of the visit, they told us that they felt well prepared and commented that they had improved over the last

22

six months as they had become more familiar with their role. We also heard of an excellent cross-fertilisation approach with tutors learning new methods when teaching the QMUL students and transferring this to their University of Malta students (year four), particularly relating to a more patient-centric style (‘human approach’) when interacting with patients.

47 Before the visit, we also heard that the school had been developing training to ensure that the SSC tutors were fully equipped to carry out their role (as many were used to only working with senior students or junior doctors). The school told us that 17 SSC project supervisors had attended a three-hour training workshop delivered by London-based faculty in January 2018. The course particularly focused on the delivery of an appropriate induction for all students, guidance about the placements, the student’s role, and the level of appropriate supervision to match the student’s competence and experience.

48 We heard that the school also delivered PBL training in July 2017. Eleven clinicians from Gozo General Hospital attended a curriculum briefing and interactional two-part PBL training sessions. Similarly, clinical skills and OSCE examiner training was due to be delivered in April 2018 by the Head of Assessment, the Head of Clinical Skills and members of the London clinical skills team.

49 The documentation submitted before the visit was supported by the SSC tutors based on Gozo that we met who all confirmed that they had received a full programme of training prior to the SSCs. The tutors told us that they had attended training courses prior to the students’ arrival and that they modified their teaching to match the QMUL curriculum. They also commented that they were used to elective students spending time in the hospital, and since these students had similar objectives to SSCs, this had allowed them to adapt quickly to SSCs.

50 GP tutors have access to their students’ e-portfolio.

51 It is clear that appraisal arrangements are still being planned and are not yet in place. Before the visit, we heard from the senior management team that the associate dean in each LEP will be responsible for the clinical educational appraisal arrangements of the educational supervisors and that they will be responsible for the appraisal of individual clinical supervisors. The GP tutors we met were unaware of any plans for appraisal. We would be interested to see how quickly these appraisal arrangements can be established.

Accessible resources for educators (R4.3)

52 The GP tutors confirmed that they have sufficient time to meet their educational and training responsibilities. They are released from clinic to be able to teach.

23

Educators' concerns or difficulties (R4.4)

53 We found that the tutors were aware of how to access support when dealing with concerns or difficulties they face as part of their educational responsibilities. They told us that if they had concerns about any of the students, they would record this on the SSC form or PBL form or they would bring it up with the coordinators locally or with student support if their concerns were very serious.

54 The GP tutors also confirmed that they receive very good support from QMUL and can request assistance or advice if required, for example, if they are concerned about student progress.

24

Theme 5: Developing and implementing curricula and assessments

Undergraduate curricular design (R5.3)

55 We heard that the school is working hard to ensure that the training experience across both London and Malta sites is similar and that both cohorts of students are treated the same. The school told us they have made very few modifications to the London-based curriculum for use in Malta apart from re-ordering the modules. Lecture slides are shared across the two sites and learning objectives delivered in each session are the same.

56 During the visit in March, the students told us that the anatomy lectures were excellent. On the other hand, the students reported a lack of confidence in the quality of teaching in histology. Earlier in the course, they had access to a number of lectures given by a QMUL lecturer but since that time, they were dissatisfied with the supplementary teaching that had been provided. The school told us that the arrangements are the same for both sites and that teaching is educationally adequate to meet the learning outcomes.

57 The students suggested that lecturer slides should always be provided online before each lecture, but that this was not currently the case. In general, they felt that there was a lack of disability and dyslexia support available. The school was able to signpost a record of contacts and reports with the school’s dyslexia support person in Malta.

58 In the documentation we received prior to the visit, the school outlined in detail the SSCs that the students undertake in year 1, which includes two two-week long projects and one ongoing project that allow students to study areas of particular interest to them. Clinicians at Gozo General Hospital have developed 32 new projects, ranging from emergency medicine and intensive care to pathology laboratory work and hyperbaric medicine. The students had just finished their first two-week SSC at the time of the visit. We were pleased to hear that the students were happy with the number of SSC titles available. They also commended the ratio of students to staff.

Standard S5.1 Medical school curricula and assessments are developed and implemented so that medical students are able to achieve the learning outcomes required by graduates.

S5.2 Postgraduate curricula and assessments are implemented so that doctors in training are able to demonstrate what is expected in Good medical practice and to achieve the learning outcomes required by their curriculum.

25

Undergraduate clinical placements (R5.4)

59 We heard that when the students attend the GP polyclinics, each Medicine in Society topic is linked to the students’ academic work. The tutors discuss the objectives of the day with the students and adapt their activity for the day according to what will be most useful for the students, for example, they may attend house visits, or attend clinics.

60 We heard that students attend PBL sessions twice a week for two hours. PBL lectures are synchronised with the UK-based sessions.

61 The tutors we met did not report any major difficulties with access to English-speaking patients. They reported that most patients understand English even if they do not speak it very well. The GP tutors told us that they try to pre-plan patients to make sure that the students can interact mostly with English-speaking patients. The tutors did not perceive language as in impediment to the students’ learning; they told us that English is spoken throughout Malta and that school exams and books are mostly in English.

62 In contrast, the students reported inconsistencies in their experience while on placement. At times, they said, tutors were happy to translate or give a summary of the discussion but at other times, the tutors did not summarise what was being said, which the students felt impacted on their overall learning opportunities. On average, they felt that 75% to 85% of consultations were in Maltese.

63 The senior management team admitted that they had under-estimated the extent to which the Maltese language was used in clinical settings and GP consultations in particular. We heard that the Maltese Medical Council has announced a 12 month rule which may affect students who wish to take up a foundation programme in Malta. The school is exploring options such as introducing a formal medical Maltese course, an SSC in Maltese or an evening course. The school told us that they have canvassed the students to find out what kind of course they would like to take. However, the students we met did not seem aware of the school’s plans to offer courses in medical Maltese nor did they demonstrate any strong interest in learning Maltese. They felt that they had been misled regarding the need for proficiency in Maltese to study in Malta.

Assessing GMC outcomes for graduates (R5.5)

64 We were pleased to hear during the March visit that exam results to date had been similar across both London and Malta sites.

Fair, reliable and valid assessments (R5.6)

65 During the visit in March 2018, the students told us that they had completed assessments but that at times they only heard about the structure of the exam at

26

very late notice or were given incorrect information about the structure of the assessment. Subsequently, the school told us that the Head of Assessment visited Malta and delivered the assessment introductory lecture during induction week. An extra examination lecture with example questions was delivered before both the first and second exam. Full assessment and progression documentation is available in the assessment area on the student VLE.

66 We observed the school’s first OSCE in Malta in May 2018. We found the OSCE to be well organised, with a high level of staff involvement and commitment. All students were very professional in their conduct and appearance. The OSCEs ran well and there were no language issues.

67 We heard that the same OSCE was run for both London and Malta students. The OSCE was blueprinted against the curriculum and both Malta and London staff had been involved in the development of the OSCE.

68 The Malta and London OSCEs took place over the course of one day with one circuit in Malta and seven in London. Students rotated through nine stations, each six minutes in length with a prior one minute reading period. It was encouraging to hear that the assessment was being run at the same time as in London and that if any changes occurred to stations in London, these were immediately fed back to staff in Malta so that the changes could be replicated. The experience for both London and Malta students was as equal as it could have been.

69 The school prepared the students for their assessment during their Medicine in Society placements. In addition, we heard that London-based staff had travelled to Malta to deliver a clinical skills session to all students, which was similar to the session conducted in London. This also included a formative OSCE. As a result, students appeared adequately prepared for the OSCE.

70 We heard that the school uses the Borderline Group Regression Method of scoring and that the students’ scores from both London and Malta campuses are combined for the calculation to standard set.

71 We felt that the OSCE was well organised logistically with generally appropriate facilities for a clinical assessment. A number of staff members were on hand to direct students, examiners and actors, as well as taking note of any problems that occurred. Reserve examiners and patients were also available, and we heard that these individuals were used.

72 We were disappointed to note that Head of Student Support was in charge of managing the OSCE circuits. Although we recognise that the Head of Student Support only had an organisational role in the OSCE and played no part in the assessment of students, as mentioned earlier in the report, we feel that the role of student support should be completely separate from that of assessment processes. Students may find

27

it difficult requesting support from a member of staff who was intimately involved in their assessment. The school subsequently told us that in order to remove a perception of involvement in assessment, alternative arrangements will be made in the future.

73 During our OSCE observation, we noted that student briefings took place in the morning and afternoon before each assessment circuit. The briefing reminded students of the structure of the assessment, how to rotate around the stations and the school’s expectations about confidentiality. During the OSCE, ‘room controllers’ were available during each circuit if students required assistance or were unsure which station was next.

74 We noted during our OSCE observation that when students had completed the task in one station, they were clearly able to hear students and patients in adjacent stations. Noise levels are a common problem with OSCE assessments and difficult to manage, but the school may benefit from considering this when designing the examination to minimise noise distraction between stations as much as possible.

75 We observed the actors/patient volunteers throughout the clinical assessment, and all appeared to be well briefed and consistent, with a good understanding of their role and the station. We were particularly impressed with the ‘hard of hearing’ actor.

76 Overall, we felt that the stations tested a broad range of knowledge and skills, allowing students to demonstrate their ability to integrate basic and clinical science. We were pleased to see good emphasis on ‘show me’ instead of ‘describe’. We noted, however, that each station task did not always fill the allocated time and also observed several instances where students were unable to complete the examination within time. We also felt that the students were not always given sufficient information in the explanation for each station task. We would recommend that the school indicates on the station instructions the number of stages or questions included in the station (and an indication of the proportion of marks allocated to them) so that students are able to pace themselves and successfully complete each station within the required timeframe.

Examiners and assessors (R5.8)

77 The GP tutors we met during our visit in March 2018 confirmed that they are involved in assessing the students. Students are expected to write a reflective piece after each session and the tutors give them feedback every few weeks.

78 During the visit in March, we also heard that the school had arranged a programme of OSCE training for examiners which would involve London examiners coming to Malta and Malta examiners spending time in London. The school aimed to replicate the same exam experience on both campuses.

28

79 In May, we met with the assessment lead for the MBBS for both London and Malta programmes. We heard that the school has plans to recruit an assessment lead for Malta in the coming year.

80 The assessment lead was knowledgeable and there was obvious use of good assessment practice both in setting the OSCE, e.g. blueprinting against the curriculum, and in analysing the quality of the assessment after the review. For example, we heard that one of the post-analysis reviews involved reviewing the examiners’ mean score and comparing their mean against the entire cohort.

81 None of the tutors we met in March had been asked to participate in OSCEs. In May, when we attended the OSCE, we heard that the examiners were all based on Malta and had self-selected for the role. At the time of the OSCE we heard that only one of the examiners had been involved in teaching the current cohort of students. However, the school subsequently advised us that all examiners had taught students on clinical placement SSCs.

82 We heard that examiners had undertaken a 90-minute training session, which included a lecture and a video, during which they were taught how to distinguish between a good, mediocre and failing candidate. This allowed them to all work towards the same standard.

83 We were pleased to see that the school provided OSCE examiners with tablet computers for scoring but in the event of any possible technology failure, the school had prepared for this by providing stations with back-up paper marking sheets, and told examiners that marks would be transferred at a later stage.

84 We heard that the school intends to provide individual feedback to the students for the first time this year, since the free text box on the tablet scoresheet is compulsory. We observed examiners completing the free text box throughout many assessments.

85 The clinical assessment examiner briefing we observed included relevant information such as the structure of the assessment, and how to use the technology. The briefing appeared to serve as a reminder of key points included in the initial examiner training which had been conducted in advance of the assessment. The examiner briefing reminded examiners to observe the performance of the student at a particular task, to score according to the marking schedule, to contribute to the good conduct of the exam, to read and be familiar with the contents of their station and to stop assessing students when the timer finished. We also noted that the examiner briefing included reminders about how inappropriate behaviour should be managed.

86 The school also highlighted in the briefing that examiners were not to deviate from the script or communicate with students outside the parameters of stations. While we felt that the majority of examiners did communicate effectively with the students when asking questions, on occasion we heard some examiners prompting, re-wording

29

the questions or providing feedback during the examination. This may simply be because it was the first time these examiners had been involved in this OSCE in Malta and we have no doubt that they will grow with experience.

87 We were pleased to see the presence of an experienced peer review examiner and an external examiner at the OSCE. Several new examiners volunteered to be peer reviewed, which demonstrated their commitment to learning and improvement.

88 We were pleased to see that examiner feedback was encouraged after the assessment. We recommend that examiners’ badges should include their name in case students want to provide feedback afterwards too.

Reasonable adjustments in the assessment and delivery of curricula (R5.12)

89 During our OSCE observations we heard that additional time in OSCE stations was awarded for students with reasonable adjustments, but that examiners were not aware of the additional time allocated. As such, we were satisfied with the policy on reasonable adjustments for the OSCE.

30

Team leader Gillian Needham

Visitors on 6.3.18

John Dormer, Carol Gray, Katie Johnston

GMC staff Martin Hart, Jane MacPherson

Visitors on 24.5.18

John Dormer

GMC staff Jane MacPherson