This article was downloaded by: [Temple University Libraries]On: 13 November 2014, At: 00:52Publisher: Taylor & FrancisInforma Ltd Registered in England and Wales Registered Number: 1072954Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH,UK

Accountability in Research:Policies and Quality AssurancePublication details, including instructions forauthors and subscription information:http://www.tandfonline.com/loi/gacr20

Teaching and LearningResponsible Research Conduct:Influences of Prior Experienceson Acceptance of New IdeasRICHARD MCGEE a , JULKA ALMQUIST b , JILL L.KELLER c & STEVEN J. JACOBSEN da Northwestern University, Feinberg School ofMedicine , Chicago, IL, USAb Department of Planning, Policy, and Design, Schoolof Social Ecology , University of California , Irvine,CA, USAc College of Medicine, University of Arizona , Tucson,AZ, USAd Southern California Permanente Medical Group ,Pasadena, CA, USAPublished online: 24 Jan 2008.

To cite this article: RICHARD MCGEE , JULKA ALMQUIST , JILL L. KELLER & STEVEN J.JACOBSEN (2008) Teaching and Learning Responsible Research Conduct: Influences ofPrior Experiences on Acceptance of New Ideas, Accountability in Research: Policiesand Quality Assurance, 15:1, 30-62

To link to this article: http://dx.doi.org/10.1080/08989620701783758

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all theinformation (the “Content”) contained in the publications on our platform.However, Taylor & Francis, our agents, and our licensors make no

representations or warranties whatsoever as to the accuracy, completeness,or suitability for any purpose of the Content. Any opinions and viewsexpressed in this publication are the opinions and views of the authors, andare not the views of or endorsed by Taylor & Francis. The accuracy of theContent should not be relied upon and should be independently verified withprimary sources of information. Taylor and Francis shall not be liable for anylosses, actions, claims, proceedings, demands, costs, expenses, damages,and other liabilities whatsoever or howsoever caused arising directly orindirectly in connection with, in relation to or arising out of the use of theContent.

This article may be used for research, teaching, and private study purposes.Any substantial or systematic reproduction, redistribution, reselling, loan,sub-licensing, systematic supply, or distribution in any form to anyone isexpressly forbidden. Terms & Conditions of access and use can be found athttp://www.tandfonline.com/page/terms-and-conditions

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

00:

52 1

3 N

ovem

ber

2014

Accountability in Research, 15:30–62, 2008Copyright © Taylor & Francis Group, LLCISSN: 0898-9621 print / 1545-5815 onlineDOI: 10.1080/08989620701783758

30

GACR0898-96211545-5815Accountability in Research, Vol. 15, No. 1, Nov 2008: pp. 0–0Accountability in ResearchTEACHING AND LEARNING RESPONSIBLE RESEARCH CONDUCT: INFLUENCES OF PRIOR EXPERIENCES

ON ACCEPTANCE OF NEW IDEAS

Prior Experience and Learning Responsible ConductR. McGee et al. RICHARD MCGEE1, JULKA ALMQUIST2, JILL L. KELLER3, and STEVEN J. JACOBSEN4

1Northwestern University, Feinberg School of Medicine, Chicago, IL, USA2Department of Planning, Policy, and Design, School of Social Ecology,

University of California, Irvine, CA, USA3College of Medicine, University of Arizona, Tucson, AZ, USA

4Southern California Permanente Medical Group, Pasadena, CA, USA

Despite requirements for Responsible Conduct of Research (RCR) training,little is known about how much this training actually influences the think-ing and behaviors of participants. Interview-based qualitative researchmethods were used to study the reactions of Ph.D. students and postdoctoralfellows to what was taught in an RCR course. For trainees with limited priorRCR experience, or who agreed with what was taught, it was relatively easyto influence their attitudes and how they thought they would use the newinformation in the future. However, if their prior experiences or existingknowledge conflicted with what was taught they resisted and often rejectednew ideas that were presented. Interviews also revealed the tremendouslycomplex process trainees must undergo trying to resolve or integrate all of thedifferent perspectives they receive on RCR from other sources. These resultsrevealed the importance of viewing RCR training from the perspective oflearning theory and how prior knowledge influences what people learn. Theresults also support the need for periodic rather than one-time RCR trainingto counter the often conflicting views and practices young scientists experi-ence in real-life research settings.

Keywords: learning research conduct, prior knowledge of research ethics,research ethics, research norms, teaching research conduct

Address correspondence to Richard McGee, Ph.D., Associate Dean, Faculty Affairs,Northwestern University, Feinberg School of Medicine, 303 E Chicago Avenue, Ward4–161, Chicago, IL 60611, USA. E-mail: r-mcgee@northwestern.edu

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

00:

52 1

3 N

ovem

ber

2014

Prior Experience and Learning Responsible Conduct 31

Introduction

Since the early 1990s, the National Institutes of Health (NIH) hasrequired formal training in responsible conduct of research(RCR) for predoctoral and postdoctoral trainees supported byinstitutional and individual National Research Service Awards.This requirement was largely driven by a few high profile cases ofresearch misconduct in the 1980s and sustained by the sense thatresponsible conduct of research is so important that knowledgeand acquisition of skills in RCR should not be left to chance. TheNIH provides a suggested list of course topics but it is the respon-sibility of the institution to organize and develop curriculum forthese courses (NIH, 1994). Extensive efforts to identify teachingmaterials and methods went into the creation of programs andmany different approaches have emerged, including classroom-based courses, weekend workshops, and web-based training(Office of Research Integrity Education).

Many have called for increased and ongoing instruction inRCR and bioethics (Mastroianni and Kahn, 1998, 1999; Eisen andBerry, 2002) but despite requirements for training in RCR, rela-tively little is known about the impact of the training on partici-pants. It is generally thought that courses can increase knowledgeof policies, options for dealing with suspected misconduct,normative behaviors within the research community, etc. The fewsurvey studies that have been conducted, however, have foundlittle evidence that courses measurably affect student attitudes,including their apparent willingness to engage in practices thatdisregard what they have been taught in classes (Plemmons et al.,2006; Brown and Kalichman, 1998; Kalichman and Friedman,1992; Bebeau and Thoma, 1994). It is unclear if this inability ofclasses to change student thinking and possible future behaviorsis a failure of course design, delivery, or other interfering influ-ences on learning.

One limitation of most interventions designed to influenceRCR is that they seldom take into account one of the core princi-ples of learning as a process. That is, learning always takes placefrom the perspective of what an individual knows or believes to betrue already. The prior experiences with and perceived knowl-edge about any concept creates the context from which a“student” hears new information and compares it to their existing

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

00:

52 1

3 N

ovem

ber

2014

32 R. McGee et al.

ideas. If the new information is consistent with what they alreadyknow, there is generally little resistance to accepting it. If what isbeing taught conflicts with existing beliefs, however, then theteaching must provide a compelling argument and a moreengaged learning process for the individual to change theirbeliefs (Piaget, 1978; Vygotsky, 1978). Since research trainees arecontinually immersed in experiences that shape their under-standing and beliefs with respect to RCR issues (Eastwood et al.,1996; Anderson, 1994; Anderson et al., 1994), it can be assumedthat formal RCR training occurs within a constantly changingcontext. As a result, this study was undertaken to develop anunderstanding of the range of trainee responses to an RCRcourse and the thinking behind them. Specifically, it wasdesigned to address several questions: How does a trainee’s priorknowledge influence how they respond to what is taught in anRCR course? Why is it apparently difficult to change trainees’thinking on some topics of RCR? Do trainees believe what theylearn in an RCR course will change what they do in the future?To this end, semistructured interviews were used because inter-views reveal a greater range of perspectives and patterns in think-ing by allowing participants to describe their situation rather thanselect from predetermined options of a survey. Moreover, inter-views can elicit more revealing responses than open-ended surveyquestions because follow-up questions can be used to clarify andunderstand the initial answers (Seidman, 1998). The goal in qual-itative studies such as this is to understand the breadth and vari-ety of response and not draw statistical inference. By design, thehope is to draw a study sample that represents the breadth ofresponses, without necessarily representing the relative propor-tions of the source population. Therefore, one needs to be cau-tious in interpreting relative frequencies, beyond representingvariability.

Methods

Mayo Clinic College of Medicine Responsible Conduct of Research Course

The RCR course at Mayo Clinic College of Medicine generally fol-lowed the core areas of instruction recommended by the NIH.The topics discussed in the course are shown in Table 1. The

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

00:

52 1

3 N

ovem

ber

2014

Prior Experience and Learning Responsible Conduct 33

course was taught by multiple faculty members who are experts inthe specific topics. Mayo Clinic required the RCR course for allpostdoctoral fellows on NIH training grants or fellowships, andall Ph.D. and M.D./Ph.D. graduate students. It was also requiredof MD residents and fellows enrolled in the Masters or Certificatein Clinical Research. The Ph.D. postdocs came with a range ofexperiences, from those just out of graduate school to those withseveral years of research beyond their degrees. MD postdocs wereoften at the beginning stage of their research training and gradu-ate students usually took the course after their first year of gradu-ate school. Total enrollment in the class ranges between about125 and 175. It was 127 for 2003, the primary study year.

The course met weekly and each session was 90–120 minutesin length. The majority of the content was taught through lec-tures, but case studies and class discussions were also included.There were no tests; pass/fail grading was based on attendance.To pass the course, registrants had to attend every lecture orwatch a video tape of the lecture, but they were not allowed toview more than four of the 11 session on videotape to pass. Thisrequirement was based on the premise that more was gained byin-class participation.

For the two topics studied in depth, Authorship and Conflictof Interest (COI), instructors used different teaching approaches.The Authorship class presented the guidelines of the Interna-tional Committee of Medical Journal Editors (ICMJE) (ICMJE,2006) and then went on to discuss the degree to which these

TABLE 1 Mayo Clinic Responsible Conduct of Research course topics

Introduction – Framing the issues – included pre-course surveyScientific fraud and its consequencesData storage and ownershipCopyright, fair use, and the electronic eraThe use of animals in researchRecombinant DNA technology and ethical implicationsAuthorshipConflict of interestThe ethical inclusion of human beings in medical researchThe IRB and its role in protecting human subjectsPre-class survey revisited – What have we learned?

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

00:

52 1

3 N

ovem

ber

2014

34 R. McGee et al.

guidelines were agreed to by the biomedical community andattempts to enforce them. The instructor also discussed honorificor gift authorships and their consequences at length. Livelydebate always ensued with this class as the guidelines provide astringent view of what it takes to achieve recognition as an author.

ICMJE Recommended Criteria for Authorship (16):Authorship credit should be based on 1) substantial contributions to conception

and design, or acquisition of data, or analysis and interpretation of data; 2) draft-ing the article or revising it critically for important intellectual content; and 3) finalapproval of the version to be published. Authors should meet conditions 1, 2, and 3.

The COI instructor’s goals were to heighten the awareness oftrainees to the many different types of COI one could encounterbeyond the obvious financial ones which most had heard about.The second major goal was to help trainees begin to see how aresearcher is often managing potential conflicting interestsrather than being able to completely eliminate them. Case studiesand examples of potential conflicts of interest were presented tothe class which stimulated discussion of these issues.

Study Design

This study was reviewed and approved by the Mayo Clinic Institu-tional Review Board and verbal consent was obtained from eachparticipant. When the course was taught in the summer of 2002,focus groups were used as a pilot study. The primary researchstudy was conducted using one-on-one interviews for the coursetaught in 2003. No major changes were made in the coursebetween years. The purpose of the pilot study was to: 1) obtaininitial insights into the range of trainee knowledge of RCR issuesprior to the course; 2) observe how participant’s prior experi-ences affected their reactions to the course; 3) develop and testquestions to be used in individual interviews for the primarystudy; and 4) find tentative themes for analysis of individual inter-views in the primary study.

To initiate the pilot study, all graduate students and postdoc-toral fellows who registered for the course in summer, 2002, weresent an email invitation to participate in pre-course focus groups.Two focus groups were formed from volunteers, graduate

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

00:

52 1

3 N

ovem

ber

2014

Prior Experience and Learning Responsible Conduct 35

students (n=7) and postdocs (n=4), based on the presumptionthat postdocs would have more experience with RCR topics thangraduate students. Open-ended questions about each of the top-ics to be covered in the course were presented to each focusgroup to determine the breadth of their understanding andthinking about RCR topics prior to the course.

The second part of the pilot study examined how studentsreacted to each RCR class. To recruit participants for this part ofthe pilot, another email invitation was sent to all class registrantsjust before the course started. They were invited to participatein eight, one hour focus groups right after each class session.These groups were also divided into graduate students (n=7)and postdocs (n=2) and participants were offered snacks and asmall remuneration of $20 per focus group attended. For eachafter-class focus group the same seven questions were asked(Table 2) to determine what the participants had learned inclass and how they reacted to what was presented or discussed inclass. The focus group discussions were audio recorded andtranscribed. The number of individuals who volunteered to bein the weekly focus groups was small, probably due to the extratime commitment beyond the class. Those who participated,however, represented a wide spectrum of past experiences andviewpoints which allowed the focus groups to serve theirintended purposes.

TABLE 2 Pilot study post-class focus group questions

Post-class focus group questions

• Tell me what you knew about today’s topic before you came to class today.• What would you say were some of the key messages you took from today’s class?• Give me some examples of things you heard that you already knew about and

some things that were new to you.• Was there anything you heard today which you don’t agree with or doesn’t fit

with your past experiences?• Have you ever directly or indirectly been aware of any behaviors among

scientists at any stage of development that you would see as questionable or improper based on what you heard today?

• How do you think you future views or behaviors might have been affected by today’s discussion?

• If you were to give feedback to the instructor or design this session yourself, what issues would you add or subtract? Why?

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

00:

52 1

3 N

ovem

ber

2014

36 R. McGee et al.

During the primary research study, semi-structured individualinterviews were used to investigate two core topics in RCR, Author-ship and COI. These two topics were chosen because the pilot studyrevealed little prior knowledge and much confusion about COI,and strong opinions and frank disagreement on principles andpractices of authorship. E-mail invitations to participate in the studywere sent to those who were registered for the course in 2003. Inter-views, lasting 30–60 minutes, were conducted with each participantbefore the RCR course began and then following the sessions onAuthorship and COI. The pre-course interview addressed past expe-riences with RCR and included specific questions related to author-ship and COI. After the authorship and COI classes, the samequestions were again asked along with additional questions thataddressed key messages that students heard and their reactions tothem. Participants were also asked how they thought they might usewhat was taught in the course in the future. All of the questionsused are listed in Table 3. The interviewer (JA) read each of thequestions to each participant and then probed with follow-up ques-tions to clarify and expand upon their initial responses.

A total of 30 students and postdoctoral fellows agreed to par-ticipate in and followed through with the interviews. Most inter-views took place within two or three days of the authorship andCOI lectures and no longer than one week after each class. Allinterviews were audio recorded and transcribed. A much higherfraction of course participants volunteered for the study than theprevious year, probably due to the smaller commitment of timeand a higher, IRB-approved, remuneration ($40 per interview)for their time and effort. Participants were accepted into thestudy in the order they volunteered. A higher fraction of womenthan men volunteered for and completed the study.

Analysis of Interview Data

Interview transcripts were analyzed by JA and RM using compara-tive analysis (Glaser and Strauss, 1967) aided by the qualitativeanalysis program, QSR NVivo® (QSR International). The analysissoftware and coding scheme allowed us to study: the comparisonof participant knowledge, attitudes, and projected behaviorsbefore and after each class; the degree to which they accepted orrejected new ideas that aligned or conflicted with what they

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

00:

52 1

3 N

ovem

ber

2014

Prior Experience and Learning Responsible Conduct 37

TABLE 3 Interview questions asked in pre-course interviews and interviews following classes on Authorship and Conflict of Interest

Background information (only asked in Pre-course Interview):

• What is your position here at Mayo? How long have you been here?• Tell me about your education and training.• Have you had any formal training in RCR?• Have you ever had any formal conversations with mentors about RCR?• Have you ever had any personal either positive or negative experiences with

RCR? Have you had any experiences with authorship? Conflict of interest?• What is your perception of how frequently there are problems or violations of RCR?

Authorship:

• How much and what kind of contribution should be necessary to justify authorship.• For lab techs?• For statisticians?• Undergraduates?• Clinical fellows?

• How should order and inclusiveness of authorship be determined – should it rest with PI or should it be a collaborative decision making process.

• What would you do if you and your PI disagreed about the order of authorship? How would you handle the situation?

• Is it possible to deal with conflicts when there is such a big difference in terms of power and authority? Would you confront a PI?

• What would you do if someone omitted you from a paper and you felt that your effort clearly warranted authorship?

• How do you feel about courtesy authorship?• Is there any one group of people that needs authorship more than

others – should need determine authorship?• Should journals require that all authors signify the type and amount of

contribution they made? If not why?• Should the biomedical community establish a set of rules governing

authorship? Why or why not?• (postdocs) Reflect back on how you leaned about the rules regarding authorship?

Did someone sit down to talk to you? Did it happen through experience?• (grad students) Where have you learned or where do you anticipate

learning about rules regarding authorship?• In the future when you have your own lab or just in general how do you

anticipate dealing with issues of authorship?Conflict of Interest:

• When we use the expression conflict of interest what does it mean to you? What are the kinds of situations in your mind that could be seen as a conflict of interest?

(Continued)

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

00:

52 1

3 N

ovem

ber

2014

38 R. McGee et al.

thought should be the norm; participant reactions to andattempts to resolve issues for which there was no clearly agreedupon norm; and their views of rules that might simplify yetrestrict the range of options for authorship and COI expecta-tions. In order to examine the effects of prior experiences onresponses to the course, individuals were asked as part of theinterview whether or not they had prior formal or informal train-ing in RCR. The participants were then grouped and comparedby their prior experience and training with authorship and COIprinciples, irrespective of their degree (Ph.D. vs. M.D.) or level oftraining (graduate student vs. postdoc). Comparison groups werecreated in this way because it became quickly apparent from thepilot study and early interviews that prior experience varied asmuch or more within a trainee type (i.e., Ph.D. vs. M.D.) and

TABLE 3 (Continued)

Background information (only asked in Pre-course Interview):

• Do you have any experiences or situations where you were either involved in or observed conflict of interest?

• Some people feel that they can separate their judgment from potential financial gain.• Do you think that most people could do this?• In your mind what level of financial gain raises the risk of conflict of interest

to an unacceptable level?• To what degree do you feel that participating in the review of research grants

puts you at risk of taking advantage of privileged information?• Have you ever received any training on conflict of interest? By whom or in

what way?• Are you familiar with Mayo’s policies regarding conflict of interest? (If yes)

Tell me what you know about them. (If no) Has anyone ever told you where to find out about them?

• How and by whom do you feel that guidelines on conflict of interest should be established? By this we mean at discipline specific level, by individual institu-tions or at a national level? Why? Do you think the biomedical community would be better served by rules at the national level?

Additional questions asked during interviews after classes on Authorship and Conflict of Interest

• What were the key messages you heard in the class on (Authorship or Conflict of Interest)?

• Do you think you will do anything differently in the future based on what you heard in class?

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

00:

52 1

3 N

ovem

ber

2014

Prior Experience and Learning Responsible Conduct 39

stage of training (Ph.D. student vs. postdoc) as between them;type or stage of training could not be used as a surrogate foramount of prior experience with RCR topics. That is, a beginninggraduate student could have had more experience and priortraining with authorship, and more strongly held views, than amuch more senior M.D. or even Ph.D. postdoc. Overall, however,most individuals had little prior experience with COI.

Results

Pilot Study Focus Groups

The pre-course focus groups revealed a great range of prior experi-ence with, knowledge of, and opinions about all of the topics to becovered in the course. It was a vivid portrayal of the heterogeneityamong research trainees with respect to these topics and the priorknowledge and opinions each person brought to their learning inthe course. The focus groups that followed each class sessionrevealed a great deal about how students and postdocs reacted towhat they heard in the classes. All of the participants were able toprovide key messages from the lecture and easily identify informa-tion they were not aware of before the class. There was much discus-sion in both focus groups about “gray areas,” such as whereguidelines were not definitive, but individuals with more experiencegave more sophisticated responses and were able to relate past expe-riences to the topics covered in class. However, it was not uncom-mon for individuals to disagree with each other and/or what wastaught in class; participants frequently were not persuaded to changetheir views by class presentations or discussions.

One of the questions most frequently raised about courses inRCR is whether or not they can actually influence future behav-iors. Since it is impossible to measure future behaviors in a studysuch as this, participants in the focus groups were asked whetherthey thought anything they heard in each session would influencewhat they did in the future, and, if so, how. Both graduate stu-dents and postdocs gave many examples of how they thought theclasses would influence how they would run a lab and conduct sci-ence in the future.

Frequent points of discussion in all focus group sessions werethe conflicts between what was taught in class and what they saw

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

00:

52 1

3 N

ovem

ber

2014

40 R. McGee et al.

and experienced in real life. This was especially true in any areaof high impact on professional advancement or where “rules”were not codified or agreed to by a high fraction of the researchcommunity (e.g., authorship, level of compensation allowedbefore presumption of COI, data sharing, etc.). It was quite com-mon for participants to express the view that it was nice in theorybut at the end of the day, they would do whatever their researchmentor said. It was clear that each person had listened to the pre-sentations and discussions in each class from the context of whatthey already knew, thought they knew, and how they thoughtthings should work. In essence, they were being asked to learnwhat was taught in class against a background of knowledge theyhad acquired through previous means, both formal and informal.The stronger they felt they knew how things worked, the moreresistant they were to any idea that challenged those perceptions.The results of the analysis of the focus group transcripts wereused to develop the questions for individual interviews as well asto develop the initial coding system for the transcripts from theprimary study. This ability to observe and record differencesamong individuals is one of the strengths of focus group methods.

Primary Study: Individual Interviews

STUDY PARTICIPANTS

The demographic composition of the 30 individuals who com-pleted the primary study is shown in Table 4. As noted in theMethods section, participants were enrolled in the study when

TABLE 4 Demographics of study participants compared to the entire class

Entire Class (%)

Study Participants (%)

Total 127 30Male 67 (53%) 8 (26%)Female 60 (47%) 22 (74%)M.D. postdoctoral fellows 66 (52%) 111 (37%)Ph.D. postdoctoral fellows 25 (20%) 6 (20%)M.D., Ph.D. postdoctoral fellows 5 (4%) 1 (3%)Ph.D. graduate students 31 (24%) 131 (43%)

1One Ph.D. graduate student already held the M.D. degree.

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

00:

52 1

3 N

ovem

ber

2014

Prior Experience and Learning Responsible Conduct 41

they responded to the e-mail soliciting participation. The studyparticipants included a noticeably higher proportion of womenand graduate students than the course as a whole. No attempt wasmade to obtain a gender and degree distribution that reflectedthe class distribution because the pilot study had indicated thatprior experience could not be predicted by either of these twovariables, and the goal of the primary study was to obtain thebreadth of responses, not to provide quantitative inference.

RANGE OF PRIOR EXPERIENCE WITH RCR AMONG TRAINEES

All of the Ph.D. and M.D./Ph.D. postdocs had some prior infor-mal and/or formal training in RCR, including authorship andconflict of interest, whereas both graduate students and M.D.postdocs had varying degrees of prior training in these areas(Table 5). More than half of the graduate students and M.D. post-docs had no prior training (14 of 24).

As with the focus groups, the interviews revealed how eachindividual’s knowledge and/or opinions about authorship andconflict of interest were influenced by prior training and experi-ence in RCR much more than their stage of training. Conse-quently, this range of prior experience with RCR provided theopportunity to study and compare responses to the RCR courseas a function of prior knowledge of the field. The description ofthe results that follows is organized into several themes. Thesethemes include: the extent to which learning and reactions towhat is taught is influenced by prior knowledge that does or doesnot conflict with what is being taught; learning when a definitiveexpected behavior is not provided or agreed upon by the biomed-ical research community; the views of participants about rulesand guidelines that could provide more definitive expected

TABLE 5 Prior training in Responsible Conduct of Research among study participants

Prior FormalTraining

Prior InformalTraining

No PriorTraining

Graduate students 3 of 13 4 of 13 8 of 13M.D. postdocs 5 of 11 4 of 11 6 of 11Ph.D. and M.D./Ph.D. postdocs 5 of 7 7 of 7 0 of 7Total 13 of 31 15 of 31 14 of 31

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

00:

52 1

3 N

ovem

ber

2014

42 R. McGee et al.

behavior; the degree to which participants felt adequately pre-pared by the course on the topics studied; and student percep-tions about how the course would influence their futurebehaviors related to authorship and COI. For all quotations, “I”refers to interviewer and “P” refers to participant in the study.

LEARNING NEW INFORMATION THAT DOES NOT CONFLICT WITH EXISTING KNOWLEDGE

When individuals had limited prior knowledge of the topicsbeing taught, they heard the information conveyed by theinstructor and any ensuing discussion with minimal context bywhich to judge the information. Likewise, if they were familiarwith a topic, and what was being taught did not conflict with whatthey knew or believed, there was minimal resistance to adoptingwhat was taught. A good example of a situation for which few hadany knowledge was the instructor describing a PI spending exces-sive time away from the lab promoting his or her self-interests andnot meeting their responsibilities to mentor trainees. In theextreme this becomes a frank conflict of interest (i.e., conflict ofeffort). None of the participants mentioned this as an example ofa conflict of interest before the class but about a quarter of themspontaneously mentioned it after the class. For example:

I: Can you give me one that’s not financial?P: One of the examples he gave in class that I thought was pretty good was for

in particular, like an investigator, making sure that they don’t have a conflictof time interest, so I guess it would be like if you’re supposed to be mentor-ing students for a certain amount of time each week, not like taking on toomany extra obligations that prevent you from doing that. (Ph.D. student withno prior formal or informal training)

Another potential conflict of interest never mentioned ininterviews before class was professional COI related to careeradvancement rather than financial gain. After the class, manypeople brought it up in different ways, such as:

I: What kind of situation do you think of when someone isn’t conscious of con-flict of interest?

P: Any type of conflict that would arise from trying to advance your career. It’svery difficult working to be promoted at work and it would be easy to dothings that were not correct. (Ph.D. postdoc with formal and informal prior RCRtraining)

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

00:

52 1

3 N

ovem

ber

2014

Prior Experience and Learning Responsible Conduct 43

One of the best examples of an idea that did not conflictwith prior knowledge was the suggestion by the authorshipinstructor that authorship inclusion and order discussions takeplace very early in a research study. This practice could mini-mize conflicts that arise when authorship issues are not dis-cussed until papers are being written. When asked how theythought inclusion and order of authorship should be deter-mined before the class, all but four individuals felt it should becollaborative, with those four believing it should reside with thePI of the lab. Several of those who favored the collaborativeapproach mentioned, however, that the ultimate decisionresided with the PI. Only a few mentioned the idea of discus-sions early and several related experiences where authorshipconflicts arose as a paper was being written. After the class,more than half of the participants volunteered the idea of earlyauthorship decisions when asked either how authorship shouldbe established or how they planned to determine authorship inthe future.

I: In the future when you have your own lab or when you’re a PI, how do youanticipate dealing with issues of authorship?

P: I think that bringing it up beforehand when you’re outlining the experi-ments you’re going to do, bringing it up then is a very good idea because it’skind of saying you’re in charge of this, and you’re going to be the firstauthor and doing it that way, and then as things progress or things change,then change authorship and just let the people know ahead of time sothey’re not surprised. . . .

I: You think the class influenced this idea?P: Yes, I mean I never thought about it beforehand that you should talk about

it, but it makes sense. (Ph.D. student with prior informal RCR training)

I: How should order and inclusiveness of authorship be determined? Should itrest with the PI or should it be a collaborative decision making process?

P: I think the last time I answered this collaborative, and I think I agree withthat even more so now. We talked in class about this being done before youeven start, kind of sitting down and being prospective. I guess it would justbe discussing it over more or less as a team. (M.D. postdoc with no prior RCRtraining)

The idea of deciding authorship early in the process of aresearch project could fit into the beliefs of almost everyone, irre-spective of whether they felt the actual decision on authorshipwas collaborative or hierarchical. Many participants adopted the

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

00:

52 1

3 N

ovem

ber

2014

44 R. McGee et al.

principle taught by the instructor, recognized some of its limita-tions, and easily incorporated it into what they would try to do inthe future. From a learning perspective, these examples describethe simplest task for instruction in RCR–-providing knowledgeand suggestions for behaviors where learning is not complicatedby prior knowledge that conflicts with what is being taught. Espe-cially for COI, class participants acquired a great deal of newknowledge about potential real or apparent conflicts of interestthey are likely to face throughout their careers.

LEARNING NEW INFORMATION THAT CONFLICTS WITH EXISTING KNOWLEDGE

Probably one of the least agreed-upon topics within the realm ofRCR is: “What level and type of contribution is required for some-one to be included as an author in a publication?”. Early inresearch training, even as an undergraduate, students areexposed to a myriad of philosophical opinions, practical chal-lenges of defining contribution, historical precedents of honor-ific authorships, professional needs to appear as an author, andefforts to attempt to establish guidelines or rules on authorshipby some journals. Thus, a high fraction of individuals in this studyhad existing opinions about the level and types of contributionthat was sufficient to warrant including someone as an author ona publication. The basis for these opinions varied widely, fromformal instruction and conversations with mentors, to personalexperience and passive observation of who did and did not endup as authors in papers from research groups in which they hadworked.

The criteria individuals gave for inclusion as an authorwere separable into two broad themes which we characterizedas Low Threshold and High Threshold. Low Threshold per-spectives saw authorship as justified for any concrete contribu-tion to a paper, even a single experiment, table, or figure ofdata, with no additional input to the paper. High Thresholdresponses maintained that authorship required more in-depthcontribution, usually intellectual contributions such as studydesign input, technical improvements, writing, etc. Before theauthorship class, 10 of 30 described their belief in a lowthreshold, 15 described a high threshold opinion, and 5described something a bit above the low threshold, such as a

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

00:

52 1

3 N

ovem

ber

2014

Prior Experience and Learning Responsible Conduct 45

demonstrated understanding of the research project as a whole.The criteria for inclusion as an author presented by the instruc-tor was the high threshold view of the ICMJE (2006) (see Meth-ods). After the authorship class, six people had shifted from lowto high threshold, four held on to their low threshold view, onehad shifted from high to low, and three held on to a lowerthreshold for undergraduates. Although some had adopted thenew view presented by the instructor, others had rejected it, andothers held on to views where justification of authorship was rel-ative to some other factors such as stage of training rather thanan absolute standard. These results provide an example of thebroad range and often unpredictable nature of trainees’ reac-tions to complex subjects discussed in RCR courses. The quotesthat follow give examples of the thinking in which they wereengaged as they compared what they heard to what theybelieved before the class.

I: How much and what kind of contribution do you think is necessary to justifyauthorship?

P: Should I clear it with Dr. (instructor) or should I tell you what I think?I: I want to know what you think.P: I disagree with him a little bit in that his emphasis was on intellectual contri-

bution and the fact that he had four or five main points in the various stagesof conceiving and designing the experiment, analyzing the data, working onthe manuscript, the drafts, and those sorts of things. . .Where I disagree wasanalyzing the whole area of the technical contribution and the feeling thattechnical contributions are somehow separate from the intellectual contri-butions and therefore don’t warrant authorship. I feel fairly strongly thatthat’s not appropriate.

I: Do you mean someone who’s giving a technical contribution should begiven an authorship?

P: I think they should be because I think the great ideas mean nothing if youhave no data. If you have no data you have nothing to analyze, and if youhave no analyzed data, you have nothing to write about, and so to say thatthe people who collected that data should not be listed as authors, to me it’sa failure of our current authorship system. (M.D. postdoctoral fellow with noprevious formal or informal RCR training)

I: What about a lab tech. What kind of a contribution does a lab tech need tomake to justify authorship?

P: I know in my first interview I said that it was clear cut that if she was doingexperiments, he or she should be allowed definite authorship. I still believethat any lab technician that is definitely a vital part of the project. . .if theyhave done some of the experiments that actually go in as figures, I still truly

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

00:

52 1

3 N

ovem

ber

2014

46 R. McGee et al.

believe that they should be an author on that paper. . . I think they shouldhave to be given an authorship because they are going to be the ones thathave to explain exactly how the experiment was done . . . (Ph.D. student with noprior formal or informal RCR training.)

I: How much and what kind of contribution do you think is necessary to justifyauthorship?

P: I still think that if you put in a figure or table that would be enough. He saidthat you need to have some sort of intellectual contribution, but sometimes Ithink that the people who do the most work may not put in the biggest intel-lectual contribution, but they did all the leg work for it so maybe they deserveauthorship as well. . .If they weren’t there, it probably would not have beendone in some instances. (Ph.D. student with prior informal RCR training)

Another area of common disagreement is in courtesy orhonorific authorships. The instructor very clearly and stronglyportrayed these as inappropriate and actually harmful to science.Despite this clear stance, many individuals did not shift theirthinking to come in line with the instructor:

I: What if someone included you in a paper? They knew that you really wanteda really good faculty position after your fellowship. It was a good paper, agood journal and you proofread or something so they inserted your nameon the paper because they knew it would be good for your career. . . Howwould you feel about that?

P: So, if in my eyes, what I did was really worthless, so it is kind of an honorarything, a nice thing to do. And it would help me?

I: Yes, it would look good on a CV.P: But I know about the work?I: Yes.P: I wouldn’t call or write them a letter that said take my name off. If I knew

about the work and agree with the work, actually did something and it was ina way, their way to say thank you, I guess I would leave it alone. (M.D. postdocwith no prior RCR training)

I: How do you feel about courtesy authorship?P: I have mixed feelings even after the course. I still feel that there should be some

room for it. It’s a no-no everywhere, but I disagree. . .I know it goes against whatI have heard, but you know, for example right now I have been very successfulover the last few weeks in my research, and someone helped me with a culture,and he’s not very involved in my study but you know. . .we share. I help him andhe’s helping me. (M.D./Ph.D. postdoc with prior informal RCR training)

I: How do you feel about courtesy authorship or honorary authorship?P: I’m for it.I: You are?

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

00:

52 1

3 N

ovem

ber

2014

Prior Experience and Learning Responsible Conduct 47

P: I think in some situations. I’ve worked with people in the last few years whoare known in the field a lot more than I am because I’m starting and actuallyI was thinking about writing up a paper, and in this situation, I thinksometimes if you can put their name on a paper, even if it has a small contri-bution and then it goes to a journal where they’ve seen their name before,sometimes that can be the difference, whether it diminishes your work ornot. . .it gives you a better standpoint. (Ph.D. student with prior informal RCRtraining)

Several different rationales were provided for disagreeingwith the instructor, including self-interest (better chance for thepaper to be accepted), recognition of prior contributions to thefield, or not wanting to go against long-standing protocol. Thecase built by the instructor did not appear to be compellingenough to convince those who held an opposing view that honor-ific authorships hurt science in the long run; for those whoresisted, they saw it as a relatively harmless or potentially benefi-cial practice that was not a big problem.

LEARNING WHEN A QUESTION DOES NOT HAVE A DEFINITIVE ANSWER

One question asked of all study participants before and after theclass on Conflict of Interest was: “For you, what level of financialgain would raise the risk of a conflict of interest to an unaccept-able level?” Before the class, responses ranged wildly with littleindication that many had ever thought about the issue of anacceptable financial threshold. During the class, the instructortalked about this concept, how it has been established differentlyby different institutions, and that $10,000/year was the level setfor Mayo Clinic. The instructor also introduced the idea that theamount of financial gain that could influence the judgment of anindividual could be relative to their salary level. For example,$1,000 could be a significant influence for someone earning$20,000/year whereas it might have minimal significance tosomeone earning $200,000/year. On this topic, class participantswere left without a definitive answer but with the sense that thethreshold could not be higher than a significant fraction of whatsomeone was earning, and to be sure to find out the amount onecould accept anywhere they worked. Study participants reacteddifferently given this level of ambiguity but a very high fractionsaw it as tied more closely to a fraction of one’s salary than anabsolute amount:

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

00:

52 1

3 N

ovem

ber

2014

48 R. McGee et al.

M.D. postdoc with prior formal and informal RCR training-–before and after the classBefore the classI: In your mind, what level of financial gain would raise the risk of conflict of

interest to unacceptable level?P: It is hard to say, I don’t know. You know, it kind of depends on who you are.I: . . .some people are strict like saying I won’t even take a pen, and some peo-

ple are saying millions of dollars. . .Where do you draw the line?P: I guess $10 or less. I have to say $10 because I did not have a watch at that

conference and one of them was giving away a watch, and I couldn’t figureout what time any of the programs were starting, I kept missing the begin-ning of things, so I took the watch. I was missing out on my conference.

After the classI: For you, what level of financial gain would basically raise the risk of a conflict

of interest to an unacceptable level?P: Probably $100.I: So $10,000 was way off the mark! (laughter)P: I would probably recuse myself for $100. A $100 is a significant amount of

money to me. $10,000 would be astronomically high.I: Do you think that will change as you start making more money?P: I can’t imagine a situation. . .in class, I voted for $500, which was below the

rest of the choices. But I can’t imagine where $500 wouldn’t be a significantamount of money.

Ph.D. postdoc with prior informal RCR training–before and after the classBefore ClassI: In your mind, what level of financial gain raises the risk of conflict of interest

to an unacceptable level?P: I don’t know. I guess enough that you could support yourself with, I mean

like $100 a year isn’t anything but $20,000, $50,000 a year is a lot.I: That would be enough to sway somebody’s judgment?P: Yes.

After the classI: Do you think people can separate their judgment from potential financial

gain?P: I think it depends on the person. Probably not. Probably most everybody

can use more money, so if somebody is offering you $60,000 to do a study,how could you say no?

I: Do you think you could separate your judgment from potential financialgain?

P: I think so because now I know and I would feel guilty. If I were in his situa-tion I would have the company give the money to my researching account.Then I would use the money for my lab, not myself. Maybe I would take$10,000 and have $50,000 go to my research account.

I: For you, what level of financial gain would raise the risk of a conflict of inter-est to an unacceptable level?

P: Well, they said $10,000. That seems like a lot to me. Ten thousand dollarsis a lot of money to me! I don’t know. I think the comment came up in

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

00:

52 1

3 N

ovem

ber

2014

Prior Experience and Learning Responsible Conduct 49

class where it is really relative. If you are making a ton of money,$10,000 is nothing. If you are not making a lot of money, $10,000 is a lot.If I was in that situation where somebody was going to give me $60,000 todo a study, maybe if I needed summer support, find a 9 month or10 month track position, I would take a month of summer support fromthat.

Another example of an individual response after the COI classI: In your mind, what level of financial gain would raise the risk of conflict of

interest to an unacceptable level?P: This was kind of interesting when they talked about it in class, but for peo-

ple making different amounts of money, it seems to be quite a varyingscale probably. I guess that at the point I am. . .it wouldn’t take very much.It would take maybe $1000, but I think later, when I’m making moremoney, that level would probably get raised a little bit at least. But I thinkeven if something is as simple as $1000 even if you’re making $100,000 peryear, you still might have things going on a little bit where you’re nottotally subjective with what you’re doing. (Ph.D. student with no prior RCRtraining)

In the absence of a definitive statement or rule withrespect to the level of extra compensation that might leadsomeone’s judgment to be compromised, individuals usedtheir own perspectives and tried to come up with an answer.Many formed the opinion that the threshold was relative toone’s other income despite the example that Mayo Clinic setthe threshold at $10,000 irrespective of income. Some adopteda much lower threshold based on their own perceptions andexperiences.

HOW TRAINEES VIEW RULES AND GUIDELINES TO GOVERN AUTHORSHIP AND CONFLICT OF INTEREST

For both authorship and conflict of interest, class participantswere presented with many situations where there were no clearlyagreed-upon rules to tell them what they should or were expectedto do. To determine how they would react to being provided withrules or guidelines to follow, the interviews specifically asked themwhether or not they felt all journals should require the corre-sponding author to indicate the nature of the contribution madeby each author, whether or not they would favor rules or guide-lines to follow, and if so, how those guidelines would be estab-lished. With respect to a requirement to indicate the nature of thecontribution of each author in order to justify authorship at the

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

00:

52 1

3 N

ovem

ber

2014

50 R. McGee et al.

time of manuscript submission, over half of the participants feltthis would be a good idea before class. After the class, less thanone third thought this should be a standard practice. Seven partic-ipants shifted away from agreeing with this proposal whereas onlyone shifted toward it. Despite the potential for this requirement toreduce unjustified authorships, which most felt were wrong, mostdid not endorse the journal practice either due to philosophicalor practical objections by the time the authorship class was over.

I: Do you think that journals should require that all authors justify the typeand amount of contribution they make?

P: No, not necessarily, because there is a discretionary thing about science thatyou would hope that scientists have enough of an ethical practice that theywouldn’t have to really justify what everybody did. Some people make signif-icant contributions that don’t necessarily sound very big on paper. . .Thereis a collaborative group of people that think if you are good enough to belisted on that paper, then it should be on there. (Ph.D. student with no priorformal or informal RCR training)

I: Do you think that journals should require that all authors justify the typeand amount of contribution they make.

P: No I don’t.I: Why?P: I guess for me that is a point of regulation that has gotten too far. There are

people who abuse the system but I think laying out the guidelines as they[have] been laid out and accepted. I think you leave that to people to makemoral or ethical or decisions among themselves. I just don’t think that tryingto quantify it would be very successful. (Ph.D. student with past formal and infor-mal RCR training)

The study participants did not leave the class with a consis-tent view about a journal’s practice of requiring a declaration ofcontribution to decrease the frequency of honorary authorshipsor standardize the contribution necessary to justify inclusion asan author. Without a conscious intent to do so, the class onauthorship appeared to decrease the proportion of study partici-pants receptive to this practice, making them more resistant toany move by journals to adopt it, whether one agrees with it ornot. However, once again, trainee responses to what was taughtwas highly variable and unpredictable.

When asked whether or not they were in favor of establishingrules to cover authorship inclusion across the biomedical com-munity, a little more than half of the participants felt this would

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

00:

52 1

3 N

ovem

ber

2014

Prior Experience and Learning Responsible Conduct 51

be a good idea before the class, with only four people changingtheir views after the class (two in each direction). Severalexpressed skepticism that such rules could be established andagreed to, but many favored them if they could be established. Inthis instance, few participants changed their thinking about theserules.

By contrast, virtually everyone in the study was in favor ofrules or at least guidelines to clarify conflict of interest issues.Before the class on COI, over half favored guidelines at the insti-tutional level rather than the national level. After the class,more than three quarters favored institutional rules overnational rules. The class presentation did not purposefully favorinstitutional guidelines, but many individuals came away withthe conclusion this was where COI guidelines should reside.This outcome exemplifies how RCR training interventions caninfluence the thinking of participants even beyond consciousmessages being taught.

TRAINEES LEFT THE COURSE FEELING PREPARED TO DEAL WITH WHAT THEY WILL ENCOUNTER IN THE FUTURE

One very important question with respect to learning within thecontext of RCR courses was whether or not participants felt theyknew what was expected of them and were prepared to handlefuture situations they might encounter. Perhaps surprisingly, allbut one of the study participants expressed confidence in theirpreparation to handle authorship situations in the future. Manyindicated that they thought they would learn and experiencemore as they went along, and realized that there were few hardand fast rules. With respect to COI, a topic to which many hadnever given much thought before the class, all but threeexpressed confidence in being able to deal with COI in thefuture. Many believed they would learn more as they went alongand had a good sense of where to get help if needed. From alearning perspective, they almost uniformly felt they knewenough for now.

CAN A COURSE INFLUENCE FUTURE BEHAVIORS?Probably the biggest question raised with respect to RCR trainingin general is whether or not it can have any lasting or measurableimpact on future behaviors. From a learning perspective, this

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

00:

52 1

3 N

ovem

ber

2014

52 R. McGee et al.

must be viewed from the context of what their behaviors wouldhave been before the intervention, what change the interventioninitiates, and subsequent learning/experiences which alter orcounteract the impact of the interventions. Very complex longitu-dinal studies would be required to determine actual behaviorchanges, but our study could determine if students thought thecourse would influence their future behaviors. Many examples ofhow individuals thought their behaviors might change as a resultof the class sessions can be categorized and within the followingthemes:

1. Course provided new concepts that elicited projections of future behaviors

I: Where and how do you anticipate learning more about conflict ofinterest?

P: Probably by experience. Whenever you might be in a situation where youare uncomfortable and realize there is conflict of interest. I guess I will prob-ably think about it more now that we have been talking about it. If I’m notsure, then I will, hopefully, be able to look it up somewhere and figure outwhether it was. So I guess through experience. (Ph.D. postdoc with prior infor-mal RCR training)

2. Course expanded knowledge of concepts and stimulated thinking ofhow to handle situations

I: How do you think you’ll handle it (peer review) when you’re reviewinggrants?

P: The lecturer did say if the ideas that are presented in that grant give you newideas, then that’s not stealing their stuff, so I think I am going to use that asa defense mechanism and just volunteer to look at as many grants as I can.(Ph.D. student with past informal RCR training)

3. Awareness of behaviors considered acceptable in some cultures but notin others

I: What were the key messages from the session on conflict of interest?P: One is it is a very complicated issue and all boils down to your con-

science, like it’s more of a conscience call because there are some issueswhen you think that it’s just alright but actually its not alright, and so itall depends on your background and how you’re raised, your values thatyou have.

I: Anything else?P: I just realized too that some of the things I’ve been doing is like a classic

example of conflict of interest, like when I was practicing back home, they

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

00:

52 1

3 N

ovem

ber

2014

Prior Experience and Learning Responsible Conduct 53

would ask me to lecture, they would give me an honorarium. Sometimes it istoo much, sometimes too little and there are also add-ons. I think I wouldadmit that sometimes the reason I give lectures for this company is because Ilike the perks, and that is really bad. . .

I: So before you hadn’t realized those things might be conflict of interest?P: No, just like everybody does it. Well, when everybody does it, it’s normal

now. Now that you know better, well, it’s not.I: So in the future if you are offered something like that, would you still do it,

would you think twice about it?P: I would think twice. There’s a reason why you learn these things so you

can improve in so many ways, even your judgment about lecturing forsome companies. (M.D. postdoc with no prior RCR training))

4. New strategies learned

I: How should order and inclusiveness of authorship be determined?Should it rest with the PI or should it a collaborative decision-makingprocess?

P: I think it should not be left to the PI. At first I thought it should be becausethe project belongs to him and he is the one who is responsible for thedesigning and leading. But I think the person who really contributes themost should be. If there is any conflict about the first authorship maybe theyshould discuss it with the PI. . .

I: What if there was a disagreement and you had to go to the PI and you dis-agreed with your PI about the order of authorship. How would you handlethat situation?

P: First I would need to talk with him about it. If it does not work, maybeI would go to the chair of the department.

I: You wouldn’t have a problem going to someone outside of the lab?P: Before the class, actually, I had (trouble). But after that, because they sug-

gested that that is the right way to solve this kind of problem. So I think nowmaybe it should be like this. (Ph.D. Student with no prior formal or informal RCRtraining)

I: Do you think the biomedical community should establish a set of rulesgoverning authorship?

P: I think if they have guidelines, not rules, like yesterday he brought up alot of questions of who should and who should not be on the paper, andI think it was something good for me, so the next time we sit and talkabout a project, I would talk about these things up front. I’m probablygoing to switch labs soon so I am just going to start talking to people. Iwill definitely ask about how these people determine authorship. . .it isone lesson I learned, that authorship is something you talk about andwill ask about it at the next place I go. (Ph.D. postdoc with prior formal andinformal RCR training)

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

00:

52 1

3 N

ovem

ber

2014

54 R. McGee et al.

5. New expectations or principles not understood previously and how tocarry them out

I: In the future, when you have your own lab, how do you anticipate dealingwith issues of authorship?

P: if I need to include people in my study and to get them tobecome authors you have to meet and then set up the order of theauthorship and somehow define the contribution that each one wouldmake so that there won’t be anymore problems as the project is goingalong. . . .

I: You’d handle things very early?P: Very early yes.I: What if, despite doing that, problems do arise?P: That’s expected, and I’ve heard that a lot of problems arise. . . . We will have

to talk about it in the group, be professional about it, and if you need helpand you go to the chairman ask the chairman (to) help.

I: Okay. Has this class affected your thinking on that?P: Very much. I am very glad I got this class.I: You explained that at the beginning a lot of these things just weren’t really

considerations for you until now.P: I surprised myself, because there is a lot of things I didn’t know, I take for

granted.I: Has it been that way with a lot of the lectures so far or has it just been in par-

ticular the authorship one?P: In particular this authorship. And even in our lab I hear some fellows com-

plaining about the reason about how come their names weren’t included orwhy they didn’t become the first author. It was then that this lecture reallyhelped me understand why. (M.D. postdoc with no prior formal or informal RCRtraining)

6. Reflective considerations of how new information fits with existingbeliefs

I: Do you think the lecture affected your thinking or approach about this?About how you would deal with issues of authorship?

P: Well in two ways. One is it did bring up things I had never thought ofbefore. There was information provided about a couple of journal arti-cles that actually addressed the issue. It was helpful to know that thereare actually written guidelines somewhere that you can reference. In thewhole area of technical vs. the intellectual contributions, it didn’t changemy point of view, but I spent quite a bit of time afterwards thinking aboutit, and so I think it helped me clarify in my own mind why I felt the way Idid and what my rationale was so that I think it is a little easier for me todefend my position that it was before I took the course. (M.D. postdoc withno prior formal or informal RCR training)

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

00:

52 1

3 N

ovem

ber

2014

Prior Experience and Learning Responsible Conduct 55

The participants in the study demonstrated the complex pro-cess by which they were taking what they heard in the classes,attempting to integrate the new knowledge into their previousunderstanding of the topics, and often being able to state whatthey thought they would do in the future. The comments of oneexperienced Ph.D. postdoc captured especially well the reitera-tive, cumulative process of learning that is taking place through-out research training:

I: In the future, when you have your own lab, or you’re a PI, how do you antic-ipate dealing with issues of authorship?

P: I think that its important that when you start a research project, to saythat I think that we can get x number of papers out of this and here arethe papers, and if you have multiple people working on a project. So sayif you have two grad students you can say, I want you to take this part ofthe project so that you’re first author on this one paper, and the otherperson take this part of the project so you can be first author on theother paper, and then if it wasn’t a collaborative effort, then say its threepeople to keep it simple, then I’d be last because I’m the PI who came upwith the idea. I think basically it needs to be clearly defined, and I alsothink that when you start a project, you need to have publications inmind, because if you don’t, I think that gets confusing as to who is goingto do what. . . Then these new people come in, you need to say okay, thisis what we have set up, and this is how things are going to go if you endup working on one of these projects where the publication is alreadystarted, you won’t be first author. You will be second author and so on.I think it really needs to be clearly defined, just to prevent people fromfeeling bad.

I: Did you feel that way. I can’t remember in your first interview if you felt thatway before or were you influenced at all by him saying that in class?

P: I think that he probably influenced me a little bit, but I am also basing iton experience, on one of the projects that I’m working on now. Newpeople keep coming in and it just gets more and more awkward, the morepeople that work on a project, because the PI doesn’t say, all right, I seethat you are busy, and he says this after the fact when a person arrives, I seethat you’re busy and you have interests in other areas, so bringing in some-body else to work on this project. That made me feel badly, wheneverI can’t stop my boss from hiring other people, but he thought that he wasdoing something good by freeing up some time so that I could work ontwo projects, which he did, but the whole transition between me and thenew person, something just didn’t go well. It felt uncomfortable for mebecause it’s like here’s this person, she’s going to be doing this part ofyour project.

I: Do you feel prepared to deal with issues of authorship or do you feel unclearand want more guidance?

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

00:

52 1

3 N

ovem

ber

2014

56 R. McGee et al.

P: I think I’m prepared. I think it’s just an uncomfortable topic and I hope thatwhen I’m a PI that I do well with it, that I can deal with it well, and I canmake people feel comfortable about authorship. But I will say that I don’talways feel comfortable bringing it up, like now. Oh, I see this future publi-cation. I think we should talk about authorship. I don’t feel comfortablewith that. But also it is something that my boss does not seem to be one whodeals well with conflict, so I kind of hope it works out well. He’s just not thetype of person that will define authorship, so I should probably say some-thing. (Ph.D. Postdoc with much experience)

Discussion

Few would argue against the importance of responsible and ethi-cal behaviors among scientists but discussions of how to achievethis goal often become contentious. Some might argue “You can’tteach ethics” while others contend it has to be learned throughexperience from research mentors. It is unlikely that formal orinformal training in RCR will prevent truly aberrant behavior,such as falsification of data, but young scientists must somehowlearn the normative behaviors expected of them by the scientificcommunity. The purpose of this study was to look at the degreeto which a formal RCR course can shape trainees’ thinking aboutcritical elements of these normative behaviors.

It is important to note that the interview-based research meth-ods of this study are quite different from the more commonly usedsurvey methods in studies on RCR education. As shown by thequotes from the interviews, they are particularly good for reveal-ing the thinking of individuals, understanding how their thinkingis or is not influenced by the course, and the competing influ-ences the students are attempting to resolve. The range ofresponses is revealed nicely with interviews, and they can be excep-tionally powerful for the identification of important issues toprobe using larger samples and survey tools. Interview methodsare not good, however, for ensuring representative sampling of apopulation, such as all of the participants in the class, due to smallsample size and unknown self-selection biases among those whoagree to participate. Thus, frequency of responses provides someidea of prevalence but no statistical inference to a broader popula-tion is possible from frequency numbers.

Previous studies have documented that trainees acquireknowledge of the principles and practices of RCR through

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

00:

52 1

3 N

ovem

ber

2014

Prior Experience and Learning Responsible Conduct 57

courses but most observed little influence on their ways of think-ing about potentially irresponsible or unethical behaviors (Officeof Research Integrity Education, 1998; Mastroianni and Kahn,1998, 1999; Eisen and Berry, 2002). The results of the presentstudy are consistent with the knowledge gains seen with surveymethods but explore more deeply the potential reasons behindthe difficulty of shifting thinking on questionable practices.

There is great variability in knowledge of and experiencewith RCR among young Ph.D. students and postdoctoral fellows.The implications of this variability are very important with respectto what individuals learn and incorporate into their thinkingfrom a course or other experiences. As in any learning situation,the learning occurs within the context of what an individualknows or thinks they know. If what they hear is new or does notchallenge their thinking in other ways, it is relatively easy to incor-porate new knowledge into their frames of reference. If what theyhear challenges or is in opposition to the prior knowledge theyhave acquired, the “learning” is more difficult as it requires themto let go of a belief or change their thinking. Examples of both ofthese situations were found in the study.

Trainees who were naïve with respect to conflicts of interestbeyond financial COI were receptive to and found interesting theadditional ways in which COI can be defined and can occur. Mostof the other types of COI didn’t challenge their thinking; how-ever, they also had little experience to appreciate the nuancesinvolved. The articulation of different types of COI gave themdefinitions and ideas. By opening their eyes to these other possi-bilities one would hope they would be better prepared to see andrespond to them in the future but that remains to be seen. Bycontrast, most trainees had experience with, or at least couldanticipate, authoring papers and already had ideas about whatone should contribute in order to justify authorship. They wereall acutely aware of how publications are make-or-break for theircareers, so the stakes for not getting their names on papers arevery high. For some people, the high bar for authorship sug-gested by the instructor conflicted with either their personalviews of fairness or their prior experiences with authorship inclu-sion decisions which were more liberal than the rules given inclass. Some expressed concern that they would have to achievethis high bar whereas others would not. Thus, some resisted the

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

00:

52 1

3 N

ovem

ber

2014

58 R. McGee et al.

higher standard of authorship or refused to accept it. The factthat discussions in the class revealed that other students also dis-agreed with the proposed rules probably made it more sociallyacceptable to reject them, and the ambiguity generated by differ-ent labs making different authorship decisions further justifiedtheir rejection. This is an example of just one of many ways inwhich theoretical normative behaviors are presented in RCRtraining that are not consistently observed in real life. Theabsence of enforced rules that are agreed to by the entire scien-tific community makes achieving adherence to a purported stan-dard very difficult.

One of the core questions we wanted to address was whetheror not there was evidence that the course would change futurebehaviors. Obviously no actual change in behavior could be mea-sured, but a first step toward a behavior change would be theintention to act in a certain way. When asked if they thought whatthey heard in the course would change what they did in thefuture, many examples were given but by far the most frequentcomment was to discuss authorship on papers early or before theresearch was started. They saw this as a way to avoid potential con-flicts over authorship at the time of writing. In real life, this isoften easier said than done given the evolving nature of research,but it exemplifies their desires to avoid conflicts, be fair, andadopt realistic practices. On the other hand, few gave examplesof things they would do differently that represented significantchanges in their thinking about complex issues like who shouldbe an author. Consequently, it still appears that a single course ofthe type studied here will have only limited ability to changebehavior away from a previously planned course of action.

One often-asked question is “Should RCR training be doneformally in courses or left to research mentors?”. Even if onetried, it could never be accomplished solely in classes as traineeswill be continually “learning” from their experiences. Also, it isnot realistic for every mentor to spend extensive one-on-one timewith every trainee, covering all of the important topics, adjustingtheir interactions for the prior knowledge and experience levelsof each. Even if they did take the time, what they would teachtheir trainees will vary since mentors have different prior experi-ences and views. Formal training provides great economy of timeand scale, and this and previous reports have amply

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

00:

52 1

3 N

ovem

ber

2014

Prior Experience and Learning Responsible Conduct 59

demonstrated its value to trainees. Based on our results, an argu-ment could be easily made for the value of trainees engaging informal exposure to RCR training more than once during theirtraining. As their experiences broaden, the complexity of thequestions and conflicts they see and have to deal with increases.The challenge for teachers is to create interventions that gobeyond the repetition of the same topics in the same way overand over again.

Our observations also have significant implications for thosecharged with teaching RCR. No matter what topic is being taught,teachers should be aware of the range of prior experiences indi-viduals have had, the degree to which there is an agreed-uponstandard or practice that can be taught, common alternativeviews, and the degree to which what is being taught may chal-lenge strongly held views. To shift the thinking of knowledgeableindividuals, the teacher will have to confront the student’s priorknowledge and provide a learning process by which individualsare guided to a new way of thinking. Simply stating one is rightand the other is wrong is unlikely to have much influence. Provid-ing theoretical frameworks or available research on complexissues is one way to frame normative behaviors and influence thecomplex decision making asked of students. For example, the dis-cussion of financial COI was largely focused on the level of finan-cial gain institutions set as being acceptable. Many studentssensed the amounts were too high, that even smaller rewardscould influence judgment. In fact, research has shown that thedrug prescribing behaviors of physicians are influenced by evensmall compensations and rewards from the pharmaceutical indus-try (Wazana, 2000). Discussions including research like this, con-flicting views on RCR principles, the consequences of notadopting proposed behaviors, and practical strategies for dealingwith the conflicting views all are likely to increase the chancesthat students will adopt new ideas. These principles of effectiveteaching are by no means unique to RCR but are fundamental toeffective teaching and learning (Bain, 2004). One of the partici-pants in the study identified the effectiveness of engaged learningwhen he contrasted this course to a previous experience he had:

I: You mentioned this, and everyone I’ve interviewed has mentioned that he saidto talk about things up front, and that seems to be very important. . . .

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

00:

52 1

3 N

ovem

ber

2014

60 R. McGee et al.

P: Yea, I was listening to some of the viewpoints of people speaking, whojust didn’t really understand where I was coming from at all. . .One timeI had a really good lecture on this in the five or six times that I’ve donethis, where they actually passed out little snippets and we got intogroups of four or five, and each person had a different role, and on thepiece of paper it said what they had done for this project, and eachperson had a different contribution. Within your group you had toargue who should be first author, who should be second author, andit was really good because the first year students who had neverdone this before, see what role they played and how they should getcredit for.

The importance of this example is that it shows how the exerciseforced the members of the group to make and support their deci-sion making process. They had to role play the process needed tomake a relatively complex choice and maintain the researchgroup’s cohesiveness.

Limitations of the Study

This study purposely looked at topics that would be classifiedmore under the category of normative behaviors in RCR thanmoral behavior linked to clearly defined research misconduct.Thus, the results do not speak directly to attempts by RCR train-ing to influence behavior defined as research misconduct. Also,the results say little about what teaching methods would be mosteffective at influencing thinking and behavior on any given topic.Most of the study participants were in relatively early stages oftraining so it is not clear if the apparent malleability of thinkingfor many participants would be seen in a group of advanced post-doctoral trainees and/or faculty. The principles of learning andbarriers to it, however, do apply throughout life.

The fraction of women who volunteered to participate inthe study substantially exceeded their representation in thecourse. Although the study was not designed to compareresponses to RCR training by gender, the proportion ofresponses (e.g., the high adoption of the proposal to discussauthorship early) might have been influenced by the gender mixof the study participants. Future studies comparing responses ofmen and women to RCR training using interview methods couldbe quite interesting.

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

00:

52 1

3 N

ovem

ber

2014

Prior Experience and Learning Responsible Conduct 61

Conclusions

The results of this study demonstrate how prior experiences caninfluence how an individual reacts to and is or is not influenced bya formal course in RCR. When an individual hears something dur-ing RCR training that does not conflict with their prior knowledgeor experiences they are likely to accept it. If they are presented withan idea that conflicts with what they believe to be true they mayreject it unless presented with a compelling argument to changetheir thinking, or evidence that challenges underlying assumptionsbehind their view. Instructors of RCR need to keep these principlesin mind, especially if they are discussing topics or practices of con-duct which are not universally agreed-upon. The data show thattrainees do internalize what they hear and can develop new inten-tions of how to handle situations related to RCR in the future.Whether or not the course actually changed their behavior andthey carry out those intentions will likely depend on the degree towhich they encounter conflicting views or interests that lead themto change those intended courses of actions in the future.

Acknowledgments

This research was supported by the Research on Research Integrity Program, anORI/NIH collaboration, grant R01 NS-044527.

References

Anderson, M. (1994). The graduate student experience and subscription to thenorms of science, Research in Higher Education, 35: 273–299.

Anderson, M., Louis K. S., and Earle J. (1994). Disciplinary and departmentaleffects on observation of faculty and graduate student misconduct, Journal ofHigher Education, 65: 331–350.

Bain, K. (2004). What the Best College Teachers Do. Cambridge, MA: HarvardUniversity Press.

Brown, S., Kalichman, M. W. (1998). Effects of training in the responsible con-duct of research: A survey of graduate students in experimental science,Science and Engineering Ethics, 4: 487–498.

Bebeau, M., and Thoma, S. J. (1994). The impact of a dental ethics curriculumon moral reasoning, Journal of Dental Education, 58: 684–692.

Eastwood, S., Derish P., Leash E., and Ordway S., (1996). Ethical issues in bio-medical research: Perceptions and practices of postdoctoral research fellowsresponding to a survey, Science and Engineering Ethics, 2: 89–113.

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

00:

52 1

3 N

ovem

ber

2014

62 R. McGee et al.

Eisen, A., and Berry, R. M. (2002). The absent professor: Why we don’t teachresearch ethics and what to do about it, American Journal of Bioethics, 2: 38–49.

Glaser, B., and Strauss, A. (1967). The Discovery of Grounded Theory. Chicago, IL:Aldine.

Kalichman, M. W., and Friedman, P. J. (1992). A pilot study of biomedical train-ees’ perceptions concerning research ethics, Academic Medicine, 67: 769–775.

Mastroianni, A. C., Kahn, J. P. (1998). The importance of expanding currenttraining in the responsible conduct of research, Academic Medicine, 73:1249–1254.

Mastroianni, A. C., and Kahn, J. P. (1999). Encouraging accountability inresearch: A pilot assessment of training efforts, Accountability in Research, 7:85–100.

Office of Research Integrity Education. Available at: <http://ori.dhhs.gov/education/products/>

Piaget, J. (1978) Success and Understanding. Cambridge: Harvard University Press.Plemmons, D. K., Brody, S. A., and Kalichman, M. W. (2006). Student percep-

tions of the effectiveness of education in the responsible conduct of research.Science and Engineering Ethics, 12: 571–582.

Reminder and update: Requirement for instruction in the responsible conductof research in national research service award institutional training grants.(1994). NIH Guide for Grants and Contracts. 23(23). Available at: <http://grants.nih.gov/grants/guide/notice-files/not94-200.html> Accessed December10, 2007.

Seidman, I. (1998). Interviewing as Qualitative Research: A Guide for Researchers inEducation and Social Sciences. New York: Teachers College Press.

Vygotsky, L. S. (1978) Mind in Society: The Development of the Higher Order Psycholog-ical Processes. Cambridge, MA: Harvard University Press.

Uniform requirements for manuscripts submitted to biomedical journals:Writing and editing for biomedical publication. (2006). International Associa-tion of Medical Journal Editors, Available at: http://www.icmje.org/ AccessedDecember 10, 2007.

Wazana, A. (2000). Physicians and the pharmaceutical industry: Is a gift ever justa gift? JAMA, 373–380.

Dow

nloa

ded

by [

Tem

ple

Uni

vers

ity L

ibra

ries

] at

00:

52 1

3 N

ovem

ber

2014