using quality system regulations and fda design control ... · using quality system regulations and...
TRANSCRIPT
AC 2010-409: USING QUALITY SYSTEM REGULATIONS AND FDA DESIGNCONTROL GUIDANCE AS A BASIS FOR CAPSTONE SENIOR DESIGN
Robert Gettens, Western New England College
Michael Rust, Western New Engalnd CollegeAssistant Professor of Biomedical Engineering
Diane Testa, Western New England College
Judy Cezeaux, Western New England College
© American Society for Engineering Education, 2010
Page 15.1335.1
Using Quality System Regulations and FDA Design Control
Guidance as a Basis for Capstone Senior Design
Abstract
Medical device development in the industrial setting follows the tenets of Quality System
Regulations (QSR) and the design control guidance of the U.S. Food and Drug Administration
(FDA). Many biomedical engineers learn the language and practices of QSR and design controls
on the job. Experiential learning in these areas gives biomedical engineering graduates a
valuable skill set coveted by medical device companies. This skill set will position biomedical
engineers apart from other engineering disciplines and will help more completely define the
biomedical engineer.
The Biomedical Engineering Department at Western New England College has developed an
approach to the capstone senior design course which integrates QSR and design controls into the
curriculum. This integration uses an experiential method in which students follow the guidelines
for design control and QSR, closely mimicking best practices seen in the medical device
industry.
The idea to incorporate QSR and FDA design control guidance was generated largely through
the Department’s industrial advisory board. Members of our board from the medical device
industry see a knowledge gap in QSR and design control in recent hires from the general pool of
engineering graduates. The incorporation of these elements into our capstone design course, not
just in theory, but in practice, seeks to alleviate this gap.
Introduction
According to the 2009 AIMBE biomedical engineering placement survey, 49% of
bachelor-level graduates obtained employment in industry.1 The U.S. Department of Labor
projects an employment growth rate of 72% for biomedical engineers in the decade 2008-2018.
This growth rate is much faster than for other engineering disciplines.2 Reasons for this
projected rapid increase include the demand for more technically sophisticated medical devices
due to an aging population, and concern for the development of more cost effective medical
procedures.2 This increased demand coupled with an existing trend of engineers going to the
medical device industry necessitates a change in the academic setting to better prepare and train
these engineers for careers in biomedical device and related industries. The objective of this
Page 15.1335.2
paper is to present an experientially-based pedagogical method using the senior capstone design
course to train engineers directly in the procedures of the Quality System Regulation (QSR), thus
better preparing graduates for careers in the biomedical device workplace.
A pilot survey of faculty, students and industry sources concerning engineering design courses
across disciplines demonstrated an emerging theme of learning and development of professional
skills in these courses.3 Indeed in recent years the importance of preparing biomedical engineers
professionally through the use of the capstone design course has been stressed by a number of
programs.4-6
Pedagogical techniques being used in biomedical engineering curricula to introduce
students to “real-world problem-solving”, which was presented by Ropella, Kelso and Enderle,
include the use of computer simulation, internships and cooperative education, guest speakers,
guest instructors, field trips, bioethics instruction and problem-centered instruction.5 At
Bucknell, a four course sequence over the Junior and Senior Years was implemented in order to
introduce students to such skills as regulatory issues, teamwork, environmental impacts, formal
decision making, computer-aided design, machining, rapid prototyping, cell culture and
statistical analysis.4 Importantly these skills are taught and practiced prior to embarking on the
senior capstone design project.4 At the University of Virginia professional skills such as job
searching, interviewing, written and oral communication, ethics, negotiation skills, leadership,
intellectual property and entrepreneurship have been integrated into the senior capstone design
course.6 Our capstone design course offers an experiential method that builds upon these
professional skills.
For engineers to be effective in the medical device industry they must be familiar with and be
able to adhere to Food and Drug Administration (FDA) regulations as outlined in Title 21 of the
U.S. Code of Federal Regulations. Section 820 of Title 21 governs QSR. The design controls put
forth in Subsection 820.30 of the QSR are of particular importance to engineers involved in the
design process. A summary of 21CFR820.30 from a user perspective is outlined in the FDA
design control guidance document.7
The importance of design over research projects is firmly established for senior capstone
design courses, particularly as directed by guidelines of the ABET, Inc.8 Therefore, since
accredited biomedical engineering programs must offer design-based projects and design in the
biomedical device industry must follow the design controls put forth by 21CFR820.30, it is
logical that academic programs should attempt to incorporate these regulations into the capstone
design course to some extent. In previous biomedical engineering education conferences hints
Page 15.1335.3
of merging these two concepts were presented. At the 2009 BME-IDEA Biennial meeting the
incorporation of 21CFR820.30 in the Case Western Summer Design Experience was presented.9
A discussion of the need for and current resistance to incorporating design controls into the
capstone design course was discussed by Jay Goldberg in the IEEE Engineering in Medicine and
Biology Magazine.10
Prior to employing this method of delivering the capstone project we followed a more
traditional academic structure. At that time, the course structure was a two semester sequence of
senior capstone design. A fall written and oral proposal was followed by spring project
execution and final oral defense and written report. The emphasis of the projects was
engineering design even though an academic structure was in place.
The impetus behind our endeavor to integrate 21CFR820.30 into our senior capstone course
came from our industrial advisory board. Members of the board, and specifically those from the
biomedical industry, indicated to our department that the new hire engineers they were
employing had only a cursory knowledge of FDA regulations, the quality function and design
control. We were advised to better incorporate 21CFR820.30 into our senior capstone course. It
was pointed out that knowledge of the FDA design control process could be one of the major
skill sets separating biomedical engineers from other engineers. This would make the
undergraduate biomedical engineer an attractive asset for a medical device employer.
This paper outlines a method to incorporate 21CFR820.30 into a capstone design course. It
should be noted that the method attempts only to simulate working in the biomedical device
industry. The method does not and could not replace the massive workforce and procedural
documentation required to obtain FDA approval for a biomedical device.
General Course Structure
The general course structure used in this work incorporates many of the tenets put forth in Jay
Goldberg’s book on biomedical engineering capstone design courses.11
Similar to many
programs, the senior capstone design project is delivered in a series of two courses. A 3-credit
fall course covers the initial phases of the design process. A 4-credit spring course builds on the
fall course and incorporates the majority of the prototype fabrication process and device testing.
During both semesters students meet with faculty advisors for weekly status update reports.
These updates last roughly one hour. Meetings with clinical and industrial advisors are also
Page 15.1335.4
encouraged. The fall course includes a weekly lecture followed by a working laboratory section
later in the week. The lecture typically introduces the topic to be covered in the working section.
Lecture topics cover areas of professionalism focused around the FDA design control guidance.
Written deliverable documents based on working sessions are scheduled to document the design
process as well as guide the students toward successful completion of their project. A summary
of the presented lectures, working sessions and project deliverables (due dates are for the draft
forms) is shown (Table 1).
Table 1: General course design for the fall section of the capstone design course. Lecture
is for 1 hour. Lab activities range from 3-4 hours. All deliverable due dates are for draft
documents to guide student project planning.
Page 15.1335.5
Ideas from several other programs were incorporated in this work. An example is the two week
introductory design experience used at Bucknell University and presented at the 2009 BME-
IDEA Biennial conference.12
Rather than offer the activity at the start of the semester, as
Bucknell did, we offered it midway through the course (Table 1: week 7). Initial feedback from
students indicated that this timing was ideal, since at that point in the course they were familiar
enough with the design process to effectively engage the exercise.
Incorporation of Design Control
Design Reviews
The course structure outlined above was built around the FDA design control guidance.
Design controls were built into the design process using the traditional waterfall model presented
in the FDA Design Control Guidance (Fig. 1).7 For the fall course, design reviews are held at the
first three phases of the design process, that is, after “user needs” solicitation, creation of design
input and finally after the design process. The final design review constitutes a design freeze,
and is held with a large community of clinical and industrial experts outside of the institution as
well as engineering faculty members. Ideally this would be the case for all of the design
reviews, but has not been implemented due to practical considerations. In the spring semester
design reviews focus on the design output and ideally design verification and validation.
Figure 1: Traditional waterfall process reproduced from the FDA design control guidance
(a federal document).7 The fall capstone design course focuses on the first three phases
of the design process (orange oval), while the spring semester focuses on the final two
phases of the design process (green oval).
Page 15.1335.6
Design History Files
In addition to focusing design reviews on the FDA design control model, the capstone program
mimics the project documentation required for the development of a biomedical device. It
should be stressed that the capstone project only mimics the documentation required for a
biomedical device. The major portion of this documentation is the maintenance of a design
history file (DHF) (Fig. 2). In an industrial setting the file would more appropriately be called a
project folder since students log additional information in the folders than would typically be
required for an FDA audited DHF. Examples of this additional information are inclusion of
project planning and financial aspects of the project. It would not be practical to include an
entire design history file here due to size limitations. An example table of contents is included
(Fig. 3) to give a feel for the types of documents included in a design history file using our
method.
Figure 2: Examples of typical student design history files. These files are maintained
by each student and assessed at the end of each term. The opened file shows a completed
change control form for one deliverable.
Page 15.1335.7
Figure 3: An example table of contents from a design history file. Due to size limitations
posting an entire file is not practical. This table of contents gives an idea of the types of
documents included in a design history file using our method.
The majority of capstone projects in our program are medical device projects and most
students also move on to the medical device industry. A similar program, however, could be
tailored for pharmaceutical design history files if such projects become available.
Change Control
Students are instructed that the DHF is a living document thus changes to deliverables are
expected and welcomed. These changes are maintained using a formal change control process
which mimics that seen in industry. As aspects of deliverables change during the course of the
project the most recent version of the deliverable is placed at the top of that deliverable section
(Fig. 2). On top of the deliverable document a change control form is placed. The change
control form indicates the project, deliverable, revision number, contents, reason for the change
and approvals of the project leader (student), technical director and a quality reviewer (Fig. 4).
This process mimics the quality system practiced in the device industry. Typically, the technical
director is the project advisor and the quality reviewer is another biomedical engineering faculty
member. In the industrial setting the quality function would follow a separate hierarchy of
supervision. For the purposes of introducing students to design controls, using a second faculty
member as a reviewer was deemed an appropriate model. Note that when all signatures are
Page 15.1335.8
obtained students receive an approval stamp from the department chair for that deliverable. This
does not, of course, preclude follow-on changes to the deliverable.
Figure 3: Example of a change control document for a project deliverable. The student is
the project leader while the technical director and quality reviewer are faculty members.
Student Assessment and ABET
Students are assessed by the faculty on professionalism, maintenance of the DHF and
performance on design reviews. The professionalism portion of the assessment is based on
maintenance of a laboratory notebook, project leadership and preparedness for meetings. The
DHF is assessed based on a grading matrix and rubric for each deliverable (Table 2). Page 15.1335.9
Performance on design reviews are similarly assessed using a grading rubric focusing on 1) the
aesthetics of the performance and 2) the technical content of the review.
Table 2: Grading matrix used for the DHF portion of the capstone design course.
Since the DHF is a living document a certain amount of liberty must be given in the assessment
of the deliverables. Faculty must be able to assess the grey area of design, eloquently described
by Gassert et al. in their paper concerning research vs. design in capstone courses.8 This
Page 15.1335.10
freedom is particularly needed in the fall semester when certain deliverables based on individual
projects may be largely incomplete. Examples of note are the design verification and validation
plan as well as global considerations of the design. It should also be stressed that projects
evolve at different rates, and this must be taken into consideration. All of these factors are of
particular concern when incorporating a design control process such as that described in this
paper.
The incorporation of QSR and design controls into the capstone design course is only in its
second year with fourteen students having been through the program. Therefore, the data needed
for a critical assessment on the impact for graduates in the industrial setting is not yet available.
Also we did not yet receive IRB approval to use quantitative information on student performance
for research purposes so we are not able to report those data. Initial reports do indicate that the
process does indeed better prepare students for the language and requirements of design control
and QSR. Additionally, we received very positive feedback from our Industrial Advisory Board
on the incorporation of this program. John Kirwan, President of Incite Innovation, LLC gave the
following incite in response to the program, "As a biomedical industry veteran, I frequently
evaluate skill sets of potential new hires. Having a solid education in the engineering
fundamentals coupled with a firm grasp of design controls and quality systems regulation
provides recent graduates with the definite advantage of being able to join a R&D group and hit
the ground running."
While many of the ABET assessment criteria could be assessed in the capstone design courses
our program chooses to specifically assess criteria 3c, 3e and 3h. The criteria definitions are 3c:
an ability to apply to design a system, component, or process to meet desired needs within
realistic constraints such as economic, environmental, societal, political, ethical, health and
safety, manufacturability, and sustainability, 3e: an ability to identify, formulate, and solve
engineering problems and 3h: the broad education necessary to understand the impact of
engineering solutions in a global, economic, environmental, and societal context. A summary
of these criteria, the delivery strategy and assessment methods are shown in Table 3.
Page 15.1335.11
Table 3: Summary of the ABET criteria assessed by the two semester senior capstone
design courses using design control as a basis for instruction.
Lessons Learned
The program described in this paper has been implemented for one and one half years
incorporating two fall semesters and one spring semester. In addition to standard student
surveys, formal after action reviews were held at the end of each fall semester. Several valuable
lessons were learned from these sessions:
≠ Students felt that the change control process should be streamlined.
≠ Students felt that the entire faculty, not just the capstone course director, should be better
educated on 21CFR820-30 and the implemented program. It should be pointed out that
Page 15.1335.12
we have full buy-in on this program from our faculty. Our faculty is learning the
process along with the students and becoming more and more knowledgeable with QSR
as we move forward.
≠ The requirements needed to elicit user needs should be communicated early in the
semester so that ample time is given to complete the necessary paperwork. This
pertains specifically to institutional review board (IRB) approval to conduct user
surveys.
≠ Students felt that underclassmen, freshman-junior level should be invited to and take part
in design reviews as outside observers.
≠ Design projects stemming from research have difficulty fitting into the design model
presented. These projects should be avoided or the structure should be altered to make
allowances.
One area of difficulty for students was taking the lead as the project manager. Students are, of
course, used to a unilateral approach in the faculty-student relationship, in which information is
given by the faculty member to the student. It may be challenging for students to break this
cycle and begin generating knowledge on their own, but this effort is ultimately necessary for
their development. It was also found that Gantt charts were an underutilized resource. Students
suggested that as part of weekly project meetings they should update and bring project Gantt
charts. It was felt that this would help guide them in leading projects and more efficiently use
the Gantt chart tool.
Conclusions
Knowledge of the requirements to develop a medical device, specifically QSR and design
control is one key facet that sets biomedical engineers apart from the other engineering
disciplines. Practicing the tenets of design control, rather than simply having those tenets
dictated, better prepares biomedical engineers for the medical device workplace. The program
described here is an easy to implement system that mimics the design control process in a
medical device company. The method provides a means for students to practice being design
engineers in the “real-world”. The attainment of this skill set will be a key asset for the
biomedical engineering community, setting us apart from our engineering colleagues and making
our students employment exceedingly desirable by the medical device community.
Page 15.1335.13
Bibliography
1. Thurston, P., "Academic Council Graduation and Job Placement Rates Survey," American Institute for
Medical and Biological Engineering, November 2009.
2. Bureau of Labor Statistics, "Engineers", U.S. Department of Labor, www.bls.gov/oco/ocos027.htm
(accessed 1/7/2010).
3. Howe, S., Lasser, R., Su, K., Pedicini, S., "Content in capstone design courses: Pilot survey results from
faculty, students, and industry," Proceedings for the 2009 ASEE Annual Conference and Exposition, Austin,
TX, June 14-17, 2009.
4. Ebenstein, D., Tranquillo, J., Cavanagh, D., "Developing student design and professional skills in an
undergraduate biomedical engineering curriculum," Proceedins for the 114th Annual ASEE Conference and
Exposition, Honolulu, HI, June 24-27, 2007.
5. Ropella, K.M., Kelso, D.M., Enderle, J.D.,"Preparing biomedical engineers for real-world problem-
solving," Proceedings for the 2001 ASEE Annual Conference and Exposition, Albuquerque, NM, June 24-27,
2001.
6. Allen, T., Peirce-Cottler, S., "Career development and professionalism within a biomedical engineering
capstone course," Proceedings for the 2008 ASEE Annual Conference and Exposition, Pittsburg, PA, June 22-
24, 2008.
7. FDA: Center for Devices and Radiological Health, "Design Control Guidance for Medical Device
Manufacturers" 2007.
8. Gassert, J., Enderle, J.D., Lerner, A., Richerson, S., Katona, P., "Design versus research; ABET
requirements for design and why research cannot substitute for design," Proceedings for the 113th Annual
ASEE Conference and Exposition, Chicago, IL, June 18-21, 2006.
9. Tyler D., "Innovations in Preparing for the Undergraduate Design Experience," 2009 BME-IDEA Biennial
Meeting, Pittsburgh, PA October 7, 2009.
10. Goldberg, J.R., "Incorporating design controls into capstone design courses," IEEE Eng Med Biol Mag,
27:105-106, 2008.
11. Goldberg, J.R., "Capstone Design Courses: Producing Industry-Ready Biomedical Engineers," Morgan
& Claypool Publishers, 2007.
12. Tranquillo, J., "Innovations in Preparing for the Undergraduate Design Experience," 2009 BME-IDEA
Biennial Meeting, Pittsburgh, PA October 7, 2009.
Page 15.1335.14