Linfield Curriculum

AAC&U General Education and AssessmentFebruary 2014Martha VanCleaveProfessor of Mathematics, Interim Director of Division of Continuing Education

Using ePortfolios for General Education Assessment

Linfield College1Linfield College Private, comprehensive, undergraduate institution in the Pacific Northwest

47 majors through three programs:McMinnville - traditional liberal arts and pre-professional programs (1700 students)

A Portland Campus - Linfield-Good Samaritan School of Nursing (450 students)

The Adult Degree Program - a pioneer in adult education primarily online (805, 410 FTE students).

Linfield College

The Linfield Curriculum(LC)Linfields general education curriculum was revised in 2007The Linfield Curriculum StructureFour major components:Inquiry Seminar(INQS) writing intensive first year course

Six Modes of InquiryDiversity Studies

Major Writing-Intensive (MWI).

Linfield College4The Six Modes of InquiryCreative Studies (CS)

Individuals, Systems, and Societies (IS)

Natural World (NW)

Quantitative Reasoning (QR)

Ultimate Questions (UQ)

Vital Past (VP)

Linfield College5Upper-Division Mode of InquiryAn upper-division course (300 or above)

In one of the six modes of inquiryCS, IS, NW, QR, UQ or VP

From outside the major department

Linfield College6Diversity Studies Global Pluralisms (GP)

U.S. Pluralisms (US)

Linfield College7Linfield Curriculum Course Requirements - LCsStudents are required to complete:Inquiry Seminar (INQS) 1 course during first or second yearModes of Inquiry 6 courses,1 from each areaDiversity Studies - 2 courses,1 from each areaUpper-Division Mode of Inquiry - 1 courseMajor Writing Intensive Course - 1 courseThe INQS is 4 semester credits, all other courses are a minimum of 3 creditsTotal of 11 courses, minimum of 34 credits

Learning OutcomesUnique for each Mode of Inquiry and Diversity Studies areaDeveloped by a separate group of faculty (working group) for each Mode of Inquiry and Diversity Studies areaConsistent with LEAP Essential Learning Outcomes/VALUE rubricsRequests for courses to fulfill requirements reviewed and approved by working groupLearning outcomes utilized when reviewing coursesLearning Outcomes must be included in syllabusSyllabus must describe how learning outcomes will be metLinfield Curriculum Assessment Plan (LCAT)adopted 2010LCATStudents must demonstrate meeting the learning objectives of each mode of inquiry and area of diversity studies through:Direct evidenceChoosing an exemplar or exemplarsPosting the work in an online repository

Indirect EvidenceSupporting the choice of these exemplars with a reflection on how they meet the learning outcomes

Linfield College11LCATFaculty Working group for each LC designationDevelop rubric for assessing evidence of student learning outcomesAssess random sample of student submissions

Linfield CollegeDirect evidence assessment of exemplarsIndirect evidence - assessment of accompanying supporting paragraphs

12LCATAssessment of LC submissions on a rotating schedule2 designations per yearFirst assessment summer 2011 - QR and VP2012 US and NW2013 GP and UQ2014 CS and ISCycle repeats

Linfield CollegeDirect evidence assessment of exemplarsIndirect evidence - assessment of accompanying supporting paragraphs

13First Round of Assessment QR and VP assessedQR and VP assessmentExemplars submitted in LMS

Limited ability to provide instructions for students

No tool for collecting demographic data

Submissions downloaded for evaluation

Record of evaluation compiled manually

Linfield College15Assessing QR

ExemplarySatisfactoryUnsatisfactoryNA1. Pose questions involving quantitative relationships . . .35 (39.8%)24 (27.3%)25 (28.4%)4 (4.5%)2. Analyze problems by discussing models 17 (19.3%)29 (33.0%)38 (43.2%)4 (4.5%)3. Understand uses and constraints 7 (8%)17 (19.3%)59 (67.0%)5 (5.7%)4. Communicate and critique quantitative arguments.10 (11.4%)20 (22.7%)52 (59.1%)6 (6.8%)Assessing QRCalibrate use of assessment rubricRandom selection of submissions scored by 2 readers, then consensusResults

Linfield CollegeTeam of 4 faculty read and scored 10 submissions as a group

Each submission scored individually by 2 faculty. Final score assigned by consensus

17Assessing QRObservations and recommendationsRated satisfactory or above#1 67.1%#2 52.3%#3 27.3%#4 34.1%

Difficulty submitting materials from online courses

Students need guidance

Revision of learning outcomes

Need benchmarks

Linfield CollegeQualitative observations, recommendations1. The percentages of submissions rated as Satisfactory or above for the four learning outcomes were: LO1: 67.1%; LO2: 52.3%; LO3: 27.3%; LO4: 34.1%. 2. Identifying constraints, assumptions, and critiques of quantitative arguments were all embedded in Learning Outcomes with other kinds of skills. We recommend that a students ability to critique a quantitative argument, and to discuss its assumptions and limitations, be made its own Learning Outcome.3. Students who wished to submit material from online courses had a difficult time doing so. It may be impossible in some cases for students to access old assignments or exams. There needs to be some mechanism by which online course materials may be utilized in assessment.4. Students need guidance on selecting exemplars and writing an appropriate narrative. The narrative should be used by the student to clarify how the uploaded material fulfills each of the four learning outcomes. It is not enough to simply write: This artifact shows how this class meets the QR learning outcomes. In addition, students should always include the questions or prompts along with their answers and responses.5. We recommend revising the QR Learning Outcomes for better clarity and grouping of similar skills into discrete outcomes. We propose:1) Frame real-world questions using mathematical representations2) Apply a model to deduce consequences or make predictions3) Verbally communicate a quantitative argument4) Verbally critique a quantitative argument6. The QR group should discuss and propose benchmarks for each learning outcome, including goals for percentages of Satisfactory or better and for Exemplary. It could be appropriate to expect a higher proportion of Exemplary submissions for students fulfilling the Upper Division portion of the Linfield Curriculum compared to students fulfilling the basic requirements of the different modes of inquiry. The proposed benchmarks should be conveyed to the Curriculum Committee, where they may proceed through the appropriate channels for adoption by the faculty.

18Lessons LearnedStudents need direction and structure

Faculty should make connections between coursework and LC learning outcomes

Linfield College19Implementing an ePortfolio systemSelecting the ePortfolio PlatformWorking groupFaculty Dean of Student AffairsRegistrarAssociate Dean of FacultyChief Technology OfficerDirector of Educational Technologies

Linfield CollegeCurriculum Committee, LC working groups, and interested departments

21Selecting the ePortfolio PlatformCriteriaTiered accessMatricesEvaluationAreas for student reflectionAbility to aggregate dataAesthetics and usabilitySupport from provider

Linfield CollegeTiered access including being able to restrict access to specific individuals/groups both external and internal Matrices we need the ability to have multiple matrices accessible for student useEvaluation especially the ability to easily access information in a usable formatAreas for student reflection essential to the Linfield Curriculum Assessment PlanAbility to aggregate data in a way that is useful for institutional research, the registrar, accreditation, individual departments, and others with a need for dataAesthetics and usability if students dont like its aesthetics they wont buy into its use, if faculty and other users dont find it easy to use they will be reluctant to implement22

TaskStream and LinfieldCreate a Directed Response Folio (DRF)Work done by team of facultySupported by TaskStream Mentoring Services

Linfield College23The DRF

Linfield College.

24The DRF formhttps://www.taskstream.com/Main/main_frame.aspForm for each areaCourse LocationCourse Subject CodeCourse NumberCourse NameCourse Delivery (option that best describes how this course was taught)Year Course Was TakenClass StandingTerm Course Was Taken

Linfield CollegeThe DRF formLC OutcomesPlease select all of the QR learning outcomes covered in this course. For courses taken at Linfield, this information can be found in your course syllabus. For transfer courses, check all that apply. Note that for courses taken at Linfield, all four learning outcomes should be selected. In addition, if this is a transfer course, please select "Transfer Course."Description of Assignment(s) for Submission Supporting Description of Quantitative Reasoning Exemplars

Linfield CollegeTaskStream and LinfieldEnroll faculty and students Create single sign-on using CatNET Username and Passwordhttps://www.linfield.edu/taskstream/index.php

Batch process enrollment via secure FTP site

Linfield College

Assessment Using TaskstreamDeveloping Rubrics with Taskstream

Beneficial Features of TaskStreamDRF provides structure for students

Form developed to provide explicit instructions

Online evaluation using the rubric

Data storage

Data reporting

Linfield CollegeForm includes:listing of the learning outcomes for each LCrequires student to select which outcomes were meetrequires student to describe the assignment, including specific instructions, that the exemplar fulfillsprompts student to reflect explicitly on how the submitted work meets the selected outcomes

Online evaluationscored using rubricability to reconcile electronicallyprovides comprehensive data on evaluation

Data storageWith the LMS, student work had to be downloaded and reviewed LMS changed, server unstableWith TaskStream data available long term

Data reportingAble to generate reports

30Assessing NW

ExemplarySatisfactoryUnsatisfactoryAve. Score1. Use scientific method. . .13 (17%)6 (8%)56 (75%)0.432. Think critically27 (36%)29 (39%)19 (25%)1.113. Historical development of ideas12 (16%)5 (7%)58 (77%)0.394. Appreciate how knowledge affects...12 (16%)26 (35%)37 (49%)0.67Assessing NWCalibrate use of assessment rubricRandom selection of submissions scored by readersResults

Linfield CollegeTeam of 4 faculty read and scored 10 submissions as a group

Each submission scored individually by 2 faculty. Final score assigned by consensus

32Assessing NWObservations and recommendationsRated satisfactory or above#1 25%#2 75%#3 23%#4 51%Only 6 submission (8%) received at least a satisfactory for all four learning outcomes13 submissions (17%) received no satisfactory scoresMost students submitted a single documentLack of understanding of what constitutes a goo exemplarRevision of learning outcomes

Linfield CollegeQualitative observations, recommendations1. The percentages of submissions rated as Satisfactory or above for the four learning outcomes were: LO1: 67.1%; LO2: 52.3%; LO3: 27.3%; LO4: 34.1%. 2. Identifying constraints, assumptions, and critiques of quantitative arguments were all embedded in Learning Outcomes with other kinds of skills. We recommend that a students ability to critique a quantitative argument, and to discuss its assumptions and limitations, be made its own Learning Outcome.3. Students who wished to submit material from online courses had a difficult time doing so. It may be impossible in some cases for students to access old assignments or exams. There needs to be some mechanism by which online course materials may be utilized in assessment.4. Students need guidance on selecting exemplars and writing an appropriate narrative. The narrative should be used by the student to clarify how the uploaded material fulfills each of the four learning outcomes. It is not enough to simply write: This artifact shows how this class meets the QR learning outcomes. In addition, students should always include the questions or prompts along with their answers and responses.5. We recommend revising the QR Learning Outcomes for better clarity and grouping of similar skills into discrete outcomes. We propose:1) Frame real-world questions using mathematical representations2) Apply a model to deduce consequences or make predictions3) Verbally communicate a quantitative argument4) Verbally critique a quantitative argument6. The QR group should discuss and propose benchmarks for each learning outcome, including goals for percentages of Satisfactory or better and for Exemplary. It could be appropriate to expect a higher proportion of Exemplary submissions for students fulfilling the Upper Division portion of the Linfield Curriculum compared to students fulfilling the basic requirements of the different modes of inquiry. The proposed benchmarks should be conveyed to the Curriculum Committee, where they may proceed through the appropriate channels for adoption by the faculty.

33New NW Learning OutcomesCourses with the NW designation are intended to help students understand the scientific method. In particular, students must demonstrate1) an understanding of the theoretical and/or experimental background of a particular topic or model, sufficient to form a hypothesis2) an ability to critically analyze results of scientific inquiry in light of assumptions,3) an understanding of how scientific results can be extended to more general situations in contemporary societyCourses with NW designation address all of the above learning outcomes.

Lessons LearnedMore attention needs to be paid to developing studentsunderstanding of what makes a good exemplar

Linfield College35Advantages of Using TaskstreamStreamlines mechanics of evaluation processAdditional reports available (summary statistics)Provides for more meaningful faculty discussionsBetween pairs of faculty evaluatorsAmong members of working group (4 faculty)With faculty teaching courses in each area through sharing of assessment results

ChallengesStudent understanding of the LC assessment Individual student data is not assessedProblematic to ensure students take the task seriouslyNeed to further develop reflective habit of mindFaculty developmentUse of ePortfolioConsistent articulation of relationship between LC learning outcomes and course content

Linfield College37ThanksLinfield CollegeCharles Dunn - Curriculum Committee Chair, director of LC Assessment, co-author of DRFSusan Agre-Kippenhan Vice President of Academic Affairs, Dean of Faculty

TaskStreamBen Coulter Manager, Systems ImplementationJose Romero Mentoring Services Lauren McCarty Associate Director of MarketingDara Wexler Education Solutions SpecialistTrudy Milburn - Education Solutions Specialist

Linfield College38ContactQuestions contact Martha VanCleave mvcleave@linfield.edu

Linfield College

39