the whistleblower effect :
DESCRIPTION
The Whistleblower Effect :. A Quantitative Analysis Demonstrating that Reporting Students for Academic Dishonesty Negatively Impacts Faculty Evaluations Mihran Aroian. Presentation Outline. Student Judicial Services Overview Faculty-in-Residence program Faculty Perceptions - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: The Whistleblower Effect :](https://reader036.vdocuments.mx/reader036/viewer/2022062310/56816268550346895dd2d576/html5/thumbnails/1.jpg)
The Whistleblower Effect:
A Quantitative Analysis Demonstrating that Reporting Students for Academic Dishonesty Negatively Impacts Faculty Evaluations
Mihran Aroian
![Page 2: The Whistleblower Effect :](https://reader036.vdocuments.mx/reader036/viewer/2022062310/56816268550346895dd2d576/html5/thumbnails/2.jpg)
Presentation Outline- Student Judicial Services Overview- Faculty-in-Residence program- Faculty Perceptions- Why Conduct this Research- Results- Recommendations- Questions
![Page 3: The Whistleblower Effect :](https://reader036.vdocuments.mx/reader036/viewer/2022062310/56816268550346895dd2d576/html5/thumbnails/3.jpg)
SJS Overview
![Page 4: The Whistleblower Effect :](https://reader036.vdocuments.mx/reader036/viewer/2022062310/56816268550346895dd2d576/html5/thumbnails/4.jpg)
Student Judicial Services
As a part of The University of Texas at Austin’s Office of the Dean of Students, Student Judicial
Services (SJS) fosters moral development on campus through the resolution of academic integrity and conduct-related matters as well as educational
outreach to students, faculty, and staff.
In addition to our educational component, SJS investigates alleged violations of the Institutional
Rules (both academic and non-academic) and implement the disciplinary process with a focus on
student learning and development.
![Page 5: The Whistleblower Effect :](https://reader036.vdocuments.mx/reader036/viewer/2022062310/56816268550346895dd2d576/html5/thumbnails/5.jpg)
Primary SJS Roles• Outreach/Education• Faculty Consultations• Case Management
• 1,600-1,800 annual referrals (AI and conduct)
• Institutional Rule Revision
![Page 6: The Whistleblower Effect :](https://reader036.vdocuments.mx/reader036/viewer/2022062310/56816268550346895dd2d576/html5/thumbnails/6.jpg)
Some Statistics 3,300 faculty 2,500 TA’s 50,000 student body 140 academic departments 2011-2013 academic years
320 faculty/TA’s reported cases to SJS 1,155 AI cases over 2 years 50% of all submissions came from (can you
guess how many faculty)? 15 out of 320
![Page 7: The Whistleblower Effect :](https://reader036.vdocuments.mx/reader036/viewer/2022062310/56816268550346895dd2d576/html5/thumbnails/7.jpg)
SJS Data Administrative disposition versus
Faculty Disposition – 55/45 Top 3 AI violations
Cheating, Plagiarism, Collusion Classification
Freshman (10); Sophomore (20%); Junior (20%); Senior (35%); Grad/Professional (15%)
GPA – even distribution Gender (M/F) – 65/35
![Page 8: The Whistleblower Effect :](https://reader036.vdocuments.mx/reader036/viewer/2022062310/56816268550346895dd2d576/html5/thumbnails/8.jpg)
Establishing a Faculty-in-Residence
Program at UT Austin
![Page 9: The Whistleblower Effect :](https://reader036.vdocuments.mx/reader036/viewer/2022062310/56816268550346895dd2d576/html5/thumbnails/9.jpg)
Academic Integrity As Steve Covey says “seek first to
understand, then to be understood” Improving AI is simple
“just as plain as the nose on your face” For example:
What is the shape and color of a Stop Sign? What is the shape and color of a Yield Sign?
![Page 10: The Whistleblower Effect :](https://reader036.vdocuments.mx/reader036/viewer/2022062310/56816268550346895dd2d576/html5/thumbnails/10.jpg)
![Page 11: The Whistleblower Effect :](https://reader036.vdocuments.mx/reader036/viewer/2022062310/56816268550346895dd2d576/html5/thumbnails/11.jpg)
![Page 12: The Whistleblower Effect :](https://reader036.vdocuments.mx/reader036/viewer/2022062310/56816268550346895dd2d576/html5/thumbnails/12.jpg)
Benefits to SJS Dedicated outreach time not available to typical
conduct staff Cultivation of relationships outside of “crisis”
events typical in conduct offices Increased exposure through meetings with
stakeholders previously unidentified by SJS Ally outside of the office to speak with unaffiliated
individuals Addition of a “faculty perspective” to conversations
within the conduct office Discussions at the Macro-level (student conduct
as a field) rather than at the Micro-level (a student’s conduct)
![Page 13: The Whistleblower Effect :](https://reader036.vdocuments.mx/reader036/viewer/2022062310/56816268550346895dd2d576/html5/thumbnails/13.jpg)
Benefits for Me! Greater appreciation for SJS Understanding the complexity of student
conduct Part of the team Ability to cross boundaries Support SJS with other stakeholders Educate campus stakeholders regarding SJS Involvement with student groups Being part of the solution Satisfying my own needs as an educator
![Page 14: The Whistleblower Effect :](https://reader036.vdocuments.mx/reader036/viewer/2022062310/56816268550346895dd2d576/html5/thumbnails/14.jpg)
Faculty Perceptions Lack of understanding by faculty as to
what constitutes cheating. Not all faculty engaged in high AI
standards. Faculty unaware of common forms of
cheating. Faculty unwilling to expend effort to
meet with students and file paperwork. Faculty avoid student conflict/tension AI is an afterthought in the class
![Page 15: The Whistleblower Effect :](https://reader036.vdocuments.mx/reader036/viewer/2022062310/56816268550346895dd2d576/html5/thumbnails/15.jpg)
Faculty Perceptions Lack of consistent message to
students. High level of indifference by many
campus stakeholders. Some faculty do not care because they
are not rewarded for teaching. Faculty need assistance on identifying
cheating and how to deal with students who do cheat.
![Page 16: The Whistleblower Effect :](https://reader036.vdocuments.mx/reader036/viewer/2022062310/56816268550346895dd2d576/html5/thumbnails/16.jpg)
The Whistleblower Effect: A Quantitative Analysis Demonstrating that Reporting Students for Academic Dishonesty Negatively Impacts Faculty Evaluations
Authors: Mihran Aroian Raymond Brown
![Page 17: The Whistleblower Effect :](https://reader036.vdocuments.mx/reader036/viewer/2022062310/56816268550346895dd2d576/html5/thumbnails/17.jpg)
Scenario A Three students submit essentially the
same writing assignment. Student A provided paper to student B & C Student B & C take responsibility Student A believes he is innocent Students upset at instructor Students are found in violation Students are members of the same student
organization as are other students in the class
![Page 18: The Whistleblower Effect :](https://reader036.vdocuments.mx/reader036/viewer/2022062310/56816268550346895dd2d576/html5/thumbnails/18.jpg)
Scenario B A group presentation of six students
includes significant plagiarized material Initially, nobody takes responsibility One-on-one conversations Students start throwing each other under the
bus Case sent to SJS for investigation Investigation completed the following
semester Two of the six found in violation
![Page 19: The Whistleblower Effect :](https://reader036.vdocuments.mx/reader036/viewer/2022062310/56816268550346895dd2d576/html5/thumbnails/19.jpg)
Question Will the end-of-semester evaluations
for these instructors increase, decrease, or not be affected by this situation? Scenario A? Scenario B?
![Page 20: The Whistleblower Effect :](https://reader036.vdocuments.mx/reader036/viewer/2022062310/56816268550346895dd2d576/html5/thumbnails/20.jpg)
Why This Study? Interviewing stakeholders on campus
as the faculty-in-residence Faculty were hesitant to report students Faculty want to avoid conflict/confrontations Belief that reporting students would impede
professional advancement As someone who always reports, I was
shocked! Original hypothesis
Perception by faculty was a false perception and analysis would demonstrate no correlation
![Page 21: The Whistleblower Effect :](https://reader036.vdocuments.mx/reader036/viewer/2022062310/56816268550346895dd2d576/html5/thumbnails/21.jpg)
Introduction Examine the impact of sanctioning
students for acts of academic misconduct on student evaluations of teaching (SETs). use of student evaluations for faculty
advancement and promotions faculty may be conflicted between enforcing
academic integrity and maximizing their professional advancement with high SET scores.
![Page 22: The Whistleblower Effect :](https://reader036.vdocuments.mx/reader036/viewer/2022062310/56816268550346895dd2d576/html5/thumbnails/22.jpg)
Method A multi-level modeling design with
8,940 end-of-semester student evaluations
32 faculty members from 17 academic departments was employed to determine if there was a whistleblower effect.
![Page 23: The Whistleblower Effect :](https://reader036.vdocuments.mx/reader036/viewer/2022062310/56816268550346895dd2d576/html5/thumbnails/23.jpg)
Literature Search Extensive qualitative articles Lack of quantitative analysis Only quantitative studies focused on
grade inflation and higher SETs GPAs have been increasing while SAT
scores have been decreasing
![Page 24: The Whistleblower Effect :](https://reader036.vdocuments.mx/reader036/viewer/2022062310/56816268550346895dd2d576/html5/thumbnails/24.jpg)
Genesis of Research Design Initial evaluation - faculty against peer
instructors Compare faculty against all faculty
within department Compare faculty against all faculty
within college Final analysis – compare faculty
against themselves
![Page 25: The Whistleblower Effect :](https://reader036.vdocuments.mx/reader036/viewer/2022062310/56816268550346895dd2d576/html5/thumbnails/25.jpg)
Creation of Sample Compare faculty against themselves Analyze evaluations for course in a
semester when they reported cheating Analyze evaluations for same course in
a semester when they did not report cheating
Evaluation must be for the same class Looked for the closest semester in
temporal proximity - summers excluded Must have reported 3 or more cases
![Page 26: The Whistleblower Effect :](https://reader036.vdocuments.mx/reader036/viewer/2022062310/56816268550346895dd2d576/html5/thumbnails/26.jpg)
Challenges to Methodology Highest reporting instructors reported
cheating every semester therefore were not included
Cases had to be submitted to SJS prior to 10 days before the end of semester Course Instructor Survey’s are given up to 10
days before the end of the semester
![Page 27: The Whistleblower Effect :](https://reader036.vdocuments.mx/reader036/viewer/2022062310/56816268550346895dd2d576/html5/thumbnails/27.jpg)
The Course Instructor Survey Five point scale ranging from “strongly
disagree” to “strongly agree” Questions asked on survey
The course was well organized The instructor communicated information
effectively The instructor showed interest in the progress of
the student The tests and assignments were usually graded
and returned promptly The instructor made me feel free to ask
questions, disagree, and express my ideas
![Page 28: The Whistleblower Effect :](https://reader036.vdocuments.mx/reader036/viewer/2022062310/56816268550346895dd2d576/html5/thumbnails/28.jpg)
Survey Overall rating for the instructor and
course ranging from “very unsatisfactory” to “excellent” on a 5-point scale Overall, this instructor was Overall, this course was
Key criteria for performance evaluation is the overall instructor rating
![Page 29: The Whistleblower Effect :](https://reader036.vdocuments.mx/reader036/viewer/2022062310/56816268550346895dd2d576/html5/thumbnails/29.jpg)
Limitations of Analysis Results based on data collected at one
university Unable to link evaluations to students Unable to test highest reporting faculty
![Page 30: The Whistleblower Effect :](https://reader036.vdocuments.mx/reader036/viewer/2022062310/56816268550346895dd2d576/html5/thumbnails/30.jpg)
Results Statistically significant effect for
reporting students (p<.001) Estimated mean rating for classes in
which students were not reported was 3.95
Estimated mean rating for classes in which students were reported was 3.75
Hypothesis - instructors who report students tend to be more demanding of their students in general than those who do not report
![Page 31: The Whistleblower Effect :](https://reader036.vdocuments.mx/reader036/viewer/2022062310/56816268550346895dd2d576/html5/thumbnails/31.jpg)
Discussion Is it time for a policy review?
Instructors expected to enforce AI Colleges want to instill ethics – beyond college By not enforcing, instructors are implicitly
condoning cheating and undermining accurate student assessment
Evaluations play a critical role for advancement for both tenure-track and non tenure-track faculty Do you harm yourself when you enforce AI?
![Page 32: The Whistleblower Effect :](https://reader036.vdocuments.mx/reader036/viewer/2022062310/56816268550346895dd2d576/html5/thumbnails/32.jpg)
Student Responsibility UT participates in the National
Assessment of Student Conduct Adjudication Process (NASCAP) Project 21 higher education institutions participate Upon completion of adjudication, students
complete survey When asked the question “What was the
outcome of your case” 6.9% of UT students answered “my case was dismissed”
In reality, only 0.15% of the cases were dismissed
![Page 33: The Whistleblower Effect :](https://reader036.vdocuments.mx/reader036/viewer/2022062310/56816268550346895dd2d576/html5/thumbnails/33.jpg)
Recommendations Encourage all faculty to communicate
expectations of AI Provide regular communication to
faculty regarding: How students cheat How to deal with cheaters How to confront students Best practices How to reduce cheating
Require students to complete an online AI tutorial each year
![Page 34: The Whistleblower Effect :](https://reader036.vdocuments.mx/reader036/viewer/2022062310/56816268550346895dd2d576/html5/thumbnails/34.jpg)
Recommendations Encourage faculty to increase
frequency of message Each academic department should
have an AI point person Faculty should regularly update
assignments and exams
![Page 35: The Whistleblower Effect :](https://reader036.vdocuments.mx/reader036/viewer/2022062310/56816268550346895dd2d576/html5/thumbnails/35.jpg)
Recommendations Include a question about faculty AI
expectations and practices on evals AI should be part of new faculty
orientation Administrators should include
consideration for faculty who report cheating regarding merit review and promotions
![Page 37: The Whistleblower Effect :](https://reader036.vdocuments.mx/reader036/viewer/2022062310/56816268550346895dd2d576/html5/thumbnails/37.jpg)