Program Learning Outcomes at the Graduate LevelSusan Bibler Coutin, Associate Dean of the Graduate Division and

Professor of Criminology, Law and Society and AnthropologyMark Warschauer, Associate Dean and Professor, School of

EducationAnn Van Sant, Associate Professor of English Ruth Quinnan, Director of Academic Affairs, Graduate DivisionJanuary 2013

Why Graduate Program Learning Outcomes (PLOs)?o UCI underwent WASC Reaccreditation review in 2012:o http://www.accreditation.uci.edu/effectiveness.htmlo WASC has been focused on Learning Outcomes at the

undergraduate level. New focus on Graduate Outcomes and Assessment in the WASC review process.

o Campus Response: In May 2011 Graduate Council developed guidelines to

provide a framework for graduate programs as they develop PLOs and assessment mechanisms for programs to use for PLO development.

These guidelines recognize that each program requires flexibility in this process, as individual disciplinary objectives differ.

Campus Response (continued)

In Fall 2011 Graduate Council and Graduate Division initiated PLO Pilot with five academic units: Anthropology; Criminology, Law and Society; Education; English/ and Mathematics.

Diverse program workgroup chosen intentionally to address differing academic needs and directions in order to help guide the broader campus with this effort.

Workgroup involved students and provided participants with an opportunity to discuss how to approach PLO development related to their discipline .

Participants consulted with faculty in their respective programs; some variation in program-level involvement; some convened a faculty subcommittee while others worked with an existing subcommittee or brought materials to faculty meetings for discussion.

Challenges for Workgroup: Lack of models to follow: although programs

assess students in many ways, there were very few examples available. This is truly a new approach.

Undergraduate examples were not relevant as they are very course-focused. At the graduate level coursework does not map to desired outcomes and success.

Should PLOs measure students or programs? Ideally, they should measure both but focus on program success in helping students achieve program goals.

Assessment: How to assess in meaningful ways?

Results• Participants found the process to be

rewarding• Graduate student participation was also

very useful • All five programs successfully developed

PLOs and assessment mechanisms and plans

• The PLOs are intended to be “living documents” that will undergo revisions and adjustments

• The campus is committed to the sustainability of program learning outcomes and assessment, and these will ultimately be incorporated into the formal Program Review process.

Going Forward

o Graduate Council notified Deans in S2012 that all graduate programs (Ph.D. and Master’s) must develop PLOs and assessment goals over the next two academic years.

o Schools submitted plans for PLO development to Graduate Council in October 2012. All programs must have PLOs and assessment mechanisms by June 2014.

o During 2012-2013 the pilot programs chose 2 or 3 PLOs that they will be assessing . Not advisable or feasible to assess all PLOs at one time; be realistic with plans and goals.

o Assessment results will identify which program aspects to improve or change.

o Goal: Translate results into sustainable way to enhance student success. Develop a structure for ongoing assessment, informed and targeted resource allocations, and future improvements.

Pilot Participant Experiences• How have your colleagues been involved and

what attitudes did they have?• How have your students been involved?• How was it helpful to be part of a working group

and to collaborate with other departments?• How did you develop the PLOs?• What are your current plans to assess PLOs? 

Describe where you are in the process• Do you have tips and suggestions for others?• What challenges were encountered?

Questions/Discussion

How should we move forward? What resources are

needed/desired for this effort?• Helpful links: http://www.accreditation.uci.edu/Documents/AppendixA.3.-GraduateLearningOutcomes.pdf

• Website?• Other?