Review of Year 1 HAE II course

• Course occurs in the winter term of Year 1

• Course Director – Virginia Lyons, PhD

• Course has 67 curricular hours

• Course was last reviewed in Oct 2007

Subcommittee review team: Aniko Fejes-Toth, David Mullins, Will Boyce

Course Objectives – Content Review

There are 16 course objectives that fulfill Geisel competencies as follows:• 10 address specific knowledge in the preclinical

domain, and seem appropriate• 1 addresses clinical care skills• 2 address communication skills• 6 address components of professionalism• 1 addresses personal improvementCurrently no course objectives are mapped to Geisel competency 6 (health care systems), which is appropriate for this year 1 course.

Course Objectives – Content Review

Course objectives are:• provided in the syllabus• written in the correct

format• appropriate for a year 1

course

Objectives: Step I Brochure• The Human Anatomy and Embryology (HAE) course content

was mapped to the 2013-14 NBME Step 1 Content Description and General Information Brochure

• Generally, the course covered NBME topics associated with embryology and anatomy and function of the nervous, musculoskeletal, respiratory , cardiovascular, gastrointestinal, renal/urinary, and reproductive systems (all anatomic categories on the Step 1 Content Outline)

• The course does covers a variety of diseases and injuries– For example, content related to the musculoskeletal system

included herniated disks (back), rotator cuff tears (shoulder), and carpal tunnel syndrome (wrist)

– Specific diseases and injuries were discussed for all major systems

Comparison to Discipline-based Organization

• In 1997, the AACBNS (Association of Anatomy, Cell Biology and Neurobiology Chairpersons) produced a document about curriculum content in the anatomical sciences in the US/Canada; this appears to be the most recent document of this type (there is a 2007 version for schools in Great Britain and Ireland)

• The content of the HAE course closely correlates with the recommendations in this document, covering all of the major learning objectives for human gross and developmental anatomy

Session Objectives• Session objectives are provided to students at the

beginning of each chapter section of the iBook• Session objectives are written in the correct format• Occasionally revisions had been made to the session

objectives that were not updated in Ilios (so the session objectives in Ilios did not match perfectly to those in the iBook)

Unplanned redundancy• The curriculum database was used to assess redundancy in the

curriculum regarding various topics in the course • For broad concepts (e.g. “altered anatomical structure,” “gross

morphology”), there was no overlap with other courses• For more specific concepts (e.g., “embryology”), limited

redundancy (5 non HAE session objectives) was complementary (for example, in SBM201, a session objective is to “Describe the relationship between embryology and anatomy in the pathophysiology of congenital lung disease”)

• For specific topics, redundancy existed that was planned or a logical pedagogic progression (e.g. material introduced in HAE that was expanded in SBM courses), consistent with HAE serving as the introduction to most anatomic structures and systems

Summary regarding Objectives• Course objectives are provided in the syllabus and

are written in the correct format using verbs with measurable outcomes

• Course objectives correlate well with material in the Step I brochure and the standards of the national professional organization

• Session objectives are provided to students and written in the correct format

• Significant unplanned redundancy was not found

Course Learning Opportunities• Large group interactive session 21 hrs. (31%)

these are flipped classroom sessions where students work with their colleagues to solve problems

• Small group conferences 4 hrs. (6%)16 students (divided into groups of 4) work together to solve clinical case problems, then discuss with the entire group and facilitator

• Laboratory 39 hrs. (58%)students work typically in groups of 4 doing anatomical dissection, osteology and radiology

• Clinical Correlation 3 hrs. (5%)clinical faculty discuss the otoscopic exam, craniofacial defects and demonstrate laryngoscopy

Optional reviews before quizzes/final exam (6.5 hrs.)

Summary regarding Pedagogy• Course uses a flipped-classroom model, thus the

percentage of lectures is low (5%); the 3 lectures in the course are clinical correlations presented by clinical faculty

• Students have numerous opportunities during class time to practice applying their knowledge

• Course does not currently include opportunities for students to define their own learning objectives (part of the LCME definition of active learning)

• Optional reviews are provided to the entire class; struggling students are also offered faculty-led group tutoring

Assessment• Written Quizzes (50% of course grade)– Typically cover one regional unit (e.g. thorax)– 20 questions on each quiz (MCQ, matching, fill-in-blank)

• Final Exam (40% of course grade)– written portion: 50 questions (MCQ, matching)– practical portion: 50 questions (fill-in-blank)

• Readiness tests (5% of course grade)- assess preparation for flipped classroom sessions

• Team performance (5% of course grade)- credit awarded for attending/participating in laboratory

and small group sessions

Assessment - Quizzes• What formats do they use for questions? – MCQ 67.5%, matching questions 20%, image recognition

12.5%– The questions correlate extremely well with session

objectives.– ~70% of the questions are “recall” type; however, the

nature of the material necessitates this.– ~50% of the questions employ patient-related stems.– Some questions require application of knowledge– No negative formats were used for quizzes

Assessment – Final Exam

Written portion• 44% of the questions were MCQs• 36% were matching questions• 20% “mixed questions”, matching structures with

embryonic origin• The questions correlated well with session objectives.

Practical portion• Consists of 50 identification questions • Students identify structures on cadavers, bones,

radiologic images and cadaver cross-sections

Assessment – Final Exam• The questions were roughly proportional to the two

main topics covered in this term (i.e. lower extremity and head/neck)

• ~80% of the questions on the written portion of the exam are “recall”; ~36% applied to a clinical situation.

• No negative formats were used for the final exam.• All questions on the practical portion of the exam

require students to identify structures; 16% require students to apply their knowledge to identify structures in images (e.g. xrays, CT scans, etc.)

Assessment• The readiness tests are novel and assess the results

of active learning• Assessment of team performance (5% of course

grade) is a good idea; encourages team-work.

Summary regarding Assessment• Excellent correlation between questions and session

objectives• Appropriate balance of topics covered• Format of the questions is well balanced, MCQ,

matching & fill-in-the-blank.• Due to the nature of the material, the majority of the

questions are “recall” but “application” type questions are also present. Even in “recall” questions stems often are clinical/patient-related.

• Overall excellent job in assessment.

Measures of

Quality – AAMC

GQ

BASIC SCIENCESGeisel

mean 2008Geisel

mean 2009Geisel

mean 2010Geisel

mean 2011Geisel

mean 2012All schools

means 2012

Behavioral Science 3.1 3.3 3.2 3.3 3.3 3.1Biochemistry 2.7 2.8 2.7 2.5 2.6 2.6Biostatistics/Epidemiology 2.7 2.8 2.9 3.2 3.2 2.8Genetics 2.9 2.8 2.9 2.8 2.8 2.8Gross anatomy/Embryology 3.4 3.5 3.6 3.5 3.6 3.4Histology 2.8 3.0 2.8 2.9 3.1 2.9Immunology 2.8 3.0 2.9 3.0 3.1 3.1Microbiology 3.1 3.4 3.1 3.2 3.3 3.1Neuroscience 3.2 3.2 3.2 3.0 3.0 3.2On Doctoring 3.3 3.3 3.5 3.4 3.5 3.4Pathology 3.1 3.1 3.2 3.1 3.4 3.3Pathophysiology of Disease 3.6 3.7 3.5 3.5 3.5 3.5Pharmacology 3.1 3.5 3.4 3.1 3.1 3.0Physiology 3.5 3.5 3.6 3.6 3.5 3.4

Measures of Quality – AAMC GQ“Indicate how well you think that instruction in anatomy/embryology prepared you for clinical clerkships and electives.” [1=poor; 2=fair; 3=good; 4=excellent]

Measures of Quality – Step I

2009* 2010* 2011* 2012* Means 09-12

TRADITIONAL CORE DISCIPLINES

Biochemistry 0.50 0.30 0.40 0.30 0.38

Biostatistics/Epidemiology 0.65 0.90 0.73 0.43 0.68

Genetics 0.40 0.30 0.48 0.28 0.37

Gross anatomy/Embryology 0.50 0.35 0.53 0.33 0.43

Histology/Cell Biology 0.53 0.30 0.40 0.37 0.40

Microbiology/Immunology 0.63 0.42 0.41 0.31 0.44

Pathology 0.55 0.35 0.42 0.26 0.40

Pharmacology 0.63 0.15 0.39 0.22 0.35

Physiology 0.67 0.32 0.47 0.38 0.46

*values reported for core disciplines are SD above the US/Can mean for Geisel mean scores

Measures of

Quality – AAMC

GQ

Year 1 courses Overall Satisfaction AY 2013-2014

Human Anatomy and Embryology II 4.57 Human Anatomy and Embryology I 4.33 Physiology-Renal 4.32 Biostatistics and Epidemiology 4.28 Metabolic Basis of Disease 4.23 Biochemical and Genetic Basis of Medicine 3.97 Neuroscience 3.90 Physiology-Endocrine 3.81 CTO 3.79 Basic Science of Microbial Disease 3.78 Virology 3.71 General Pathology 3.46 Physiology-Cardiovascular 3.45 Physiology-Respiration 3.38 Immunology 2.35

Measures of Quality – Course Reviews

scale [1=poor; 2=fair; 3=good; 4=very good; 5=excellent]

scale [1=poor; 2=fair; 3=good; 4=very good; 5=excellent]

Measures of Quality – Course Reviews

HAE II 2011 (23%)*

HAE II 2012 (97%)*

HAE II 2013 (88%)*

Overall satisfaction of course 4.75 4.54 4.57

Overall usefulness of lectures/large group 4.58 3.62 not surveyed

Overall usefulness of small groups 3.85 4.20 not surveyed

Overall usefulness of laboratories 4.40 4.48 not surveyed

Congruence of assessment questions to material emphasized in course 4.45 4.61 4.48

Clarity of learning objectives not surveyed not surveyed 4.52

*student participation rate on course evaluation

Measures of Quality – Course Reviews

Strengths: Faculty• “The faculty for anatomy are outstanding. They are

extremely accessible and do a great job of organizing the material.”

• “The anatomy faculty is top notch and are some of the most caring, intelligent teachers I have ever had.”

Measures of Quality – Course ReviewsStrengths: Organization/Pedagogy of the Course• “The anatomy course is by far one of the best

organized and delivered courses.”• “The entire course was incredibly well organized.

There were so many resources provided to the students, including the special resources on Blackboard, which were very helpful.”

• “I loved the "flipped classroom" sessions--always useful and engaging.”

• “The small group discussions were excellent in helping me work through difficult questions/concepts.”

Measures of Quality – Course ReviewsStrengths: Course Materials• “The course materials were excellent. Clearly written, in

a logical presentation, and concise.”• “I liked the anatomy website, especially radiology and

visible human slides.”• “The dissection videos Dr. Catlin made were really

helpful in the course.”• “The information presented in the ibooks was very well

organized, very clear, and made learning the material as easy as possible.”

• “I loved the interactive activities in the ibook and online.”

Measures of Quality – Course ReviewsSuggestions for Improvement: Embryology• “I felt it would be beneficial to learn embryology after

the cadaver lab and large group sessions for the specific portion of the body we are studying. I think trying to learn the embryology of a structure before really knowing where it is or what it does was a bit confusing at times and led me to not retain the embryology as well.”

• “It would also be nice to put embryology at the end so that we can draw connections to structures that we have previously learned to their embryologic origins. Learning embryology first was confusing because I did not know what or where the adult structures were.”

Measures of Quality – Course Reviews

Suggestions for Improvement: Pedagogy• “I would like to see more small group case discussion

sessions. We had a considerable amount in the first term and only one or two maybe in this second term.”

• “ More small group sessions would be beneficial”• “I would have liked to switch lab groups for the

second term. I think it would have been a nice opportunity to work with different people.”

Measures of Quality – Course Reviews

Suggestions for Improvement: iBooks• “Images in the iBook that were not in the gallery

were too small and there is no way to zoom in- even in the gallery some of the words in the images could not be read.”

• “Consistency of iBook formatting so that all images can be opened in gallery format (largest viewing size available).”

• “iBook videos could be improved…I think the videos would be better if they were more dynamic. ”

• AAMC GQ data places the course in the “good-to-excellent” range, and students score above the mean on Step I in anatomy/embryology

• Students rate the course in the “very good to excellent” range; it is the highest rated course for Y1

• Strengths in the course include dedicated faculty, clear organization, numerous course resources and engaging pedagogies

• Improvements could be made in the placement of embryology sessions, number of small group sessions and the format of the iBook

Summary regarding Measures of Quality

Summary of Recommendations• There are no significant problems with the course

objectives; a few session objectives in Ilios need to be updated to match the objectives currently distributed in the course

• The percentage of lectures in the course (5%) is well under LCME guidelines, thus does not need to be adjusted

• The course director should explore opportunities to include activities in the course that meet the current LCME definition of active learning

Summary of Recommendations• Assessment questions are well-written and correlate

with objectives/material emphasized in the course; some improvement could be made in the number of questions that assess application of knowledge

• The course is well-taught and highly regarded. Minor improvements could be made in the embryology sessions and the newly created course iBooks.

Action Plan• The course director will update any session objectives in Ilios

and/or the new AAMC curriculum database to make sure they match the session objectives distributed in the course

• We will pilot a PBD activity this year (problem based dissection) that meets the LCME definition of active learning

• The course director will review questions on the quizzes and exams and incorporate more application questions where appropriate

• Embryology sessions for particular organ systems will be reordered in the schedule so that students learn the adult anatomy first (this is also being done in the fall term of the course)

• We are revising the formatting of the course iBooks to incorporate student suggestions