core competency data report information literacy › oie › content › pdf › core...core...

Information Literacy

Core Competency Data Report

PEPPERDINE UNIVERSITY

OFFICE OF INSTITUTIONAL EFFECTIVENESS

2016 – 2018

Report prepared by the Office of Institutional Effectiveness Source: Livetext by Watermark, 2019

Core Competencies: Information Literacy

Background

WASC Information on the Five Core Competencies

The five core competencies – writing, oral communication, quantitative reasoning, critical thinking, and information literacy – are critical higher-order intellectual skills for students to develop in order to be successful at school, at work, and in their private and civic lives. Accordingly, WSCUC’s institutional review process calls upon institutions to describe how the curriculum addresses each of these competencies, explain their learning outcomes in relation to the core competencies, and demonstrate the extent to which these outcomes are achieved.

The 2013 Handbook of Accreditation, Criteria for Review 2.2a states:

“Baccalaureate programs engage students in an integrated course of study of sufficient breadth and depth to prepare them for work, citizenship, and life-long learning. These programs ensure the development of core competencies including, but not limited to, written and oral communication, quantitative reasoning, information literacy, and critical thinking.

Component 4 (Educational Quality) of the Institutional Review Process asks for institutions “to describe how the curriculum addresses each of the five core competencies, explain their learning outcomes in relation to those core competencies, and demonstrate, through evidence of student performance, the extent to which those outcomes are achieved.”

Participants in the Assessment of Information Literacy

Information literacy, as defined by AAC&U, is the ability to know when there is a need for information, to be able to identify, locate, evaluate, and effectively and responsibly use and share that information for the problem at hand.

Two of Pepperdine’s schools participated in the assessment of the Information Literacy Core Competency at Pepperdine University between fall 2016 and fall 2018: (1) Seaver College and (2) Graziadio Business School.

Eleven Undergraduate programs from Seaver College participated: (1) Art, (2) French, (3) Psychology, (4) Theater, (5) Art History, (6) Music, (7) Philosophy, (8) Social Work, (9) International Studies and Languages, (10) Nutritional Science, and (11) Business Administration. One hundred forty-seven individual undergraduate students contributed work for the assessment, in addition to the five groups of students from ART 390 who participated. This represents an appropriate sample size of Seaver College students.

One Graduate program and twenty-five students from Graziadio Business School Participated: (1) MBA: Economics.

2


Outcomes

Pepperdine Information Literacy Outcomes, based on the AAC&U Information Literacy Value Rubric Student Will:

1. Determine the Extent of Information Needed: Effectively defines the scope of the research question or thesis. Effectively determines key concepts. Types of information (sources) selected directly relate to concepts or answer research question.

2. Access the Needed Information: Accesses the information using effective well-designed search strategies and most appropriate information sources.

3. Evaluate Information and its Sources Critically: Chooses a variety of information sources appropriate to the scope and discipline of the research question. Selects sources after considering the importance (to the researched topic) of the multiple criteria used (such as relevance to the research question, currency, authority, audience, and bias or point of view).

4. Use Information Effectively to Accomplish a Specific Purpose: Communicates, organizes and synthesizes information from sources to fully achieve a specific purpose, with clarity and depth.

5. Access and Use Information Ethically and Legally: Students use correctly all of the following information use strategies (use of citations and references; choice of paraphrasing, summary, or quoting; using information in ways that are true to original context; distinguishing between common knowledge and ideas requiring attribution) and demonstrate a full understanding of the ethical and legal restrictions on the use of published, confidential, and/or proprietary information.

Methodology:

To assess the core competency of Information Literacy across the Institution, assessment leaders asked programs to submit scored rubrics and students’ work using the AAC&U VALUE rubric for Information Literacy. Whenever possible this data was submitted from capstone or senior level courses so that the data reflects student learning at the point of graduation. Scores were entered into LiveText and compiled by assessment staff in the Office of Institutional Effectiveness.

In keeping with best practices at our peer and aspirational schools, as well as following the AAC&U VALUE Rubric for Information Literacy, this project had two parts. The AAC&U rubric design required that we ask students to complete two assignments.

3


Assignment 1: A research journal or diary that culminates in a reflective essay about the research process.

Participants were required to write three to four brief research journal entries throughout the semester reflecting on process. We then asked students to turn those journal entries into a reflective essay that outlined the process that they followed with their research.

The purpose of the journal entries was to give the students solid material from which to draw when writing the reflective essay. The reflective essay was assessed using three dimensions of the information literacy rubric, which align with Pepperdine Information Literacy Outcomes #1 (Determine the Extent of Information Needed), #2 (Access the Needed Information), and #3 (Evaluate Information and its Sources Critically).

Assignment 2: The research paper.

The research paper was program specific. Faculty from each program decided on the student assignment and the research paper could take whatever form was appropriate for each specific discipline. The research paper was assessed using two dimensions of the information literacy rubric, which align with Pepperdine Information Literacy Outcomes #4 (Use Information Effectively to Accomplish a Specific Purpose) and #5 (Access and Use Information Ethically and Legally).

The VALUE Rubric for Information Literacy is recommended for assessing a collection of work, such as research papers, editorials, and speeches, rather than a single work. Evidence of a student’s research process can provide vital context for understanding a student’s information gathering process and further demonstrates a student’s information literacy proficiency.

This assessment covers the work of students from fall 2016 through fall 2018. Faculty instructors assessed an initial sample size of research papers and reflective essays for Seaver College and Graziadio Business School from classes offered in fall 2016 and fall 2017. To provide additional data for the Information Literacy Core Competency assessment, in Fall 2018 three librarians from Seaver College assessed research papers from three Seaver College courses offered in Fall 2017 and Spring 2018.

Each applicable dimension of the rubric was scored on a four-point scale, with 1 = benchmark, 2 and 3 = milestones, and 4 = capstone. Scores in the benchmark area primarily demonstrated a very basic understanding of information literacy and indicated a lack of competency in achieving one of Pepperdine’s five Information Literacy Learning Outcomes. Scores in the capstone area, however, indicated mastery of the material at the level of a graduate student or career professional. To achieve a ranking in the milestone or capstone categories, students needed to demonstrate a progressively complex application of information literacy skills, including defining the scope of a question, applying search strategies, choosing and applying criteria to evaluate information, organizing information effectively, and correctly using multiple information use strategies.

4


Analysis

Overall, the results of the assessment indicate that Pepperdine students are meeting the information literacy outcomes and achieving competency in information literacy. The expected benchmark for information literacy is for 75% of students in a course to reach the milestone (2) level or higher within each learning outcome assessed. Across each of the five dimensions of the VALUE rubric, the average score of the 68 Seaver College students assessed by faculty was at the milestone (3) level. The 25 Graziadio Graduate students scored slightly lower across four of the five dimensions (learning outcomes one through four) when compared to the average scores of Seaver students, but scored higher than Seaver students scored on learning outcome five. The average score of graduate students was at the milestone (3) level or at the upper end of the milestone (2) level. It is difficult to draw conclusions on the scores of graduate students due to the small sample size assessed, as well as the possibility of differing faculty expectations of information literacy competency at the graduate level. Finally, when the average scores of librarian evaluations of seventy-nine Seaver College students are considered, it is evident that all student groups in this assessment achieved nearly the same average scores on outcome four, scoring at the milestone (3) level.

It is also worth noting that the average scores of students assessed by librarians on learning outcome five was at least one-half point lower than the other groups’ average scores for this dimension. The average scores of the 79 students assessed by Librarians for outcome five were between the milestone (2) and milestone (3) level, while the average scores of Seaver College students and Graziadio students were at the milestone (3) level or higher. This scoring discrepancy likely reflects the expertise of the librarian evaluators; outcome five focuses on information use strategies, and as information professionals, Pepperdine librarians may be keener in spotting these nuances in research papers. However, the average librarians’ scores for learning outcome four were in line with both the average scores of faculty-assessed Seaver students and graduate students.

On average, Seaver students scored highest in learning outcomes one and five. The highest average score for graduate students was also in learning outcome five. Seaver students scored the lowest on outcome two, while graduate students scored the lowest on outcome three. While the lowest average scores for all three groups was above the milestone (2) level of accomplishment, which indicates the groups are meeting expected levels of competency, the outcomes that align with lower scores are those related to accessing needed information for Seaver students and critically evaluating information for Graziadio students.

Demographics (Gender and Ethnicity):

Of the 68 students participating in the 2016-2017 assessment, the fall 2016 sample included 10 male students and 22 female students, and the spring 2017 sample included 18 male students and 18 female students. The assessment results were disaggregated by gender for five courses: FRE 380, THEA 311, MUS 355, PHIL 480, and SW 300. Male students in FRE 380 (N = 4) scored significantly lower than female students (N = 5) on both outcomes assessed (outcomes 4 and 5). Likewise, in THEA 311 male students (N = 2) scored lower than female students across all dimensions of the rubric, except for learning outcome #2, in which scores were nearly equal. Differences among gender were less evident in MUS 355, PHIL 480, and SW 300. In MUS 355, male students (N = 5) scored slightly higher than female students (N = 2) on outcomes #1 and #2, while females scored higher on outcomes #3 and #5, and male and female students scored equally on outcome #4. In SW 300, male students (N = 3) and female students (N = 5)

5


scored nearly equally on outcomes #1 and #2, while male students scored higher than female students on outcome #3. In PHIL 480, the largest class assessed that was disaggregated by gender, male students (N = 9) scored higher on outcomes #1 and #3, and female students (N = 8) scored higher on outcome #4, with male and female students scoring nearly equally on outcomes #2 and #5. When these scores are analyzed holistically, the averaged results of this assessment suggest that males score higher on outcomes #1 and #2, females score higher on outcomes #4 and #5, and male and female students score equally on outcome #3. However, due to differing sample sizes and small ranges between scores, this analysis may not apply to the student population as a whole.

Summary of assessment scores by gender and course (f = female students scored higher; m = male students scored higher).

Course

Outcome #1: Determine the Extent of Information Needed

Outcome #2: Access the Needed Information

Outcome #3: Evaluate Information and its Sources Critically

Outcome #4: Use information effectively

Outcome #5: Access and Use Information Ethically and Legally

FRE 380 N/A N/A N/A F F

THEA 311 F M F F F

MUS 355 M M F Equal scores F

PHIL 480 M Equal Scores M F Equal scores

SW 300 Equal Scores Equal Scores M N/A N/A

Regarding ethnicity, the fall 2016 assessment included four non-resident Alien students, eight Hispanic or Latino students, two Asian students, three Black or African American students, thirteen white students, and two students of two or more races. The spring 2017 assessment included six non-resident Alien students, five Hispanic or Latino students, three Asian students, one Black or African American student, eighteen white students, and three students whose Race or Ethnicity was unknown.

Scores were disaggregated by ethnicity in two courses, FRE 380 and SW 300. In FRE 380, white students’ scores nearly equaled those of minority students on learning outcome four and white students scored higher than minority students on learning outcome five. The average scores of both groups were at the milestone (2) level, indicating both groups of students met the competency. It is difficult to draw a conclusion based on ethnicity in this course due to the comparatively small sample size of white students (N= 2). For SW 300, the scores were compared among three non-resident alien students, two minority students, and three white students. Overall, white students scored higher than minority and international students on learning outcomes one, two, and three. White students reached the

6


milestone (3) level, while minority and international students scored at the upper milestone (2) level. International students scored slightly higher than minority students on learning outcomes one and three, while minority students scored higher than international students did on learning outcome two. Due to the small sample sizes of these groups and relatively small range of scores, it is unclear whether there is a statistically significant difference in scores among ethnicity.

Results disaggregated by program:

• ART 390: ART 390 included four assessors and each student was assessed multiple times. The average scores on outcomes #4 and #5 were slightly lower than the average score of Seaver students and the average Librarian evaluation scores. This course met the expected benchmark for outcome #4, with 75% of students scoring at the milestone (2) level or higher. This course did not meet the expected benchmark for outcome #5, as 65% of students scored at the milestone (2) level or higher. This may be attributed to the nature of the assignment or the fact that this was a group project. However, a high percentage of students scored at the capstone level for outcomes #4 (15%) and #5 (25%).

• FRE 380: The average scores on outcomes #4 and #5 were lower than the average score of Seaver College students and the average Librarian evaluation scores. As mentioned above, male students scored significantly lower than female students on both outcomes, and white students scored slightly higher than minority students overall. Students met expected competency levels for both outcomes in this course, with 89% of students achieving at the milestone (2) level or higher for outcome #4, and 78% of students doing so for outcome #5.

• PSYC 310: Students enrolled in PSYC 310 scored higher on outcome #4 and outcome #5 when compared to both the average scores of Seaver Students and average Librarian evaluation scores. Students met the expected benchmarks for this course, and 100% of students scored at the milestone (2) level or higher for outcome #4 and outcome #5. An especially high percentage of students scored at the capstone level; 44% reached the capstone mark on outcome #4, and 94% of students reached the capstone level on outcome #5. This high level of capstone achievement may reveal an issue with applying the rubric, as discussed below.

• THEA 311: Across all five outcomes, students enrolled in THEA 311 scored lower than the average scores of Seaver students and average scores of the Librarian evaluation. Female students scored higher than male students in all dimensions except outcome #2, as mentioned above. While the average scores of students in THEA 311 were lower, the course meets the expected benchmarks for information literacy, with the exception of outcomes #3 and #4. In these two areas, students scored just slightly below the expected benchmark of 75%. For outcome #1, 86% of students scored at the milestone (2) level or higher. 100% of students met did so for outcome #2. 72% of students met the expected benchmark for outcome #3. 71% of students met the expected benchmark for outcome #4. 86% of students met the expected

7


benchmark for outcome #5.

• ARTH 490: Students enrolled in ARTH 490 scored slightly lower than the average score of Seaver College students on outcomes #1 and #2, but higher than the average score of Seaver students and average Librarian evaluation scores on outcomes #4 and #5. Students met the expected benchmarks for this course, and 100% of students scored at the milestone (2) level or higher for all five outcomes.

• MUS 355: Across all five outcomes, students enrolled in MUS 355 scored equally or higher than the average scores of Seaver College Students and average Librarian evaluation scores. Students met the expected benchmarks for this course, and 100% of students scored at the milestone (2) level or higher for all five outcomes. A very high percentage of students achieved at the capstone level for outcomes #3 (43%) and #5 (71%). This distribution of scores at the capstone level may suggest an issue with applying the rubric, as discussed below.

• PHIL 480: Students enrolled in PHIL 480 scored higher across all five dimensions when compared to the average score of Seaver College students and the average Librarian evaluation scores. Students met the expected benchmarks for this course, and 100% of students scored at the milestone (2) level or higher for all five outcomes. More than 40% of students reached the capstone level in each of the five dimensions; 76% of students scored at the capstone level for outcome #5.

• SW 300: Students enrolled in SW 300 scored lower than the average score of Seaver College Students, but 100% of students scored at the milestone (2) level or higher for all three dimensions of the reflective essay rubric.

• ECNM 602 (Graziadio Graduate School): Students enrolled in ECNM 602 met the expected benchmark for information literacy. 96% of students met the benchmark for outcome #1, 84% for outcome #2, 96% for outcome #3, 100% for outcome #4, and 100% for outcome #5. Since this is a graduate level class, a high percentage of students scored at the capstone level, especially in outcome #5 (44%).

• INTS 455 (Librarian evaluation): Students enrolled in INTS 455 scored lower than the average scores of Seaver College and average scores of librarian evaluation. 100% of students met the expected benchmark for outcome #4, and 89% of students met the expected benchmark for outcome #5.

• NUTR 499 (Librarian evaluation): Students enrolled in NUTR 499 scored higher than the average score of Seaver College and librarian evaluation for outcome #4, and lower than the average score of Seaver College, but higher than the average librarian score for outcome #5. Students

8


met the expected benchmarks for this course, 100% of students scored at the milestone (2) level or higher for outcomes #4 and #5.

• BA 497 (Librarian evaluation): Students enrolled in BA 497 scored equally with the average scores of Seaver students on outcome #4 and lower than Seaver students but equally with scores of Librarian evaluations on outcome #5. Students met the expected benchmarks for this course, with 99% of students meeting the expected benchmark for outcome #4, and 98% for outcome #5.

Percentage of students by course who scored at the milestone (2) level or higher for each outcome:

Course

Outcome #1: Determine

the Extent of Information

Needed

Outcome #2: Access the

Needed Information

Outcome #3: Evaluate

Information and its

Sources Critically

Outcome #4: Use

information effectively

Outcome #5: Access and

Use Information Ethically and

Legally

ART 390 N/A N/A N/A 75% 65%

FRE 380 N/A N/A N/A 89% 78%

PSYC 310 N/A N/A N/A 100% 100%

THEA 311 86% 100% 72% 71% 86%

ARTH 490 100% 100% 100% 100% 100%

MUS 355 100% 100% 100% 100% 100%

PHIL 480 100% 100% 100% 100% 100%

SW 300 100% 100% 100% N/A N/A

ECNM 602 96% 84% 96% 100% 100%

INTS 455 N/A N/A N/A 100% 89%

NUTR 499 N/A N/A N/A 100% 100%

BA 497 N/A N/A N/A 99% 98%

9


In summary, ten of the twelve classes in this assessment met the expected competency level for all five learning outcomes. All courses met the expected benchmark level of 75% for at least one of the five outcomes. While ART 390 and THEA 311 did not quite meet all of the expected benchmark levels, these two courses were not far from expectations. It is important to keep in mind the context of class size when reading these numbers. Seven of the twelve courses in this assessment had fewer than ten students, which can magnify the percentage of scores; small class sizes can skew percentages. To put this in perspective, only two students in THEA 311 did not meet the competency for outcomes four and five.

The assessment had a strong sample size of 172 students plus the five ART 390 groups. This analysis indicates the University is meeting outcomes for the Information Literacy Core Competency. This assessment and analysis also indicate there are no notable programmatic areas of concern for this core competency and that the University is meeting these outcomes satisfactorily, exceeding expectations in many cases.

One opportunity for future growth is to broaden program participation in the assessment of information literacy. While this assessment featured strong representation from most of Seaver College, there is a notable absence of participation by Humanities programs, such as English or History. In addition, the assessment lacked representation from the graduate schools.

Overall, the AAC&U Information Literacy Value Rubric aligns well with the Pepperdine Information Literacy Outcomes. An area of concern with the rubric in this assessment, however, is that too many students scored at the capstone level for multiple courses. The expected level of accomplishment for the capstone level is 2% to 5% of students per course; some courses, such as PSYC 310, MUS 355, and PHIL 480, however, saw students reaching the capstone level at a rate of 70% or higher. Since Pepperdine University offers “capstone” level classes, faculty who are assessing these classes may associate the level of course with the level of achievement on the rubric. One way of adjusting this distribution of scores may be to rename the “capstone” level of accomplishment on the rubric and well as to clarify the meaning of this level as it relates to the rubric. While this skew does not affect the overall competency rates of this assessment since these students would have scored at the milestone level regardless, it does raise questions, assessors’ application of the rubric, and the appropriate distribution of accomplishment levels. Assessment staff in OIE may wish to address these issues with faculty prior to the next assessment of information literacy.

10


AAC&U Information Literacy VALUE Rubric - Dimensions Divided by Assignment

Capstone Milestones Benchmark 4 3 2 1

Determine the Effectively defines the scope of the Defines the scope of the Defines the scope of the Has difficulty defining the extent of research question or thesis. Effectively research question or research question or scope of the research

information needed

determines key concepts. Types of information (sources) selected directly relate to concepts or answer research

thesis completely. Can determine key concepts. Types of information

thesis incompletely (parts are missing, remains too broad or

question or thesis. Has difficulty determining key concepts. Types of

question. (sources) selected relate too narrow, etc.). Can information (sources) Reflective to concepts or answer determine key selected do not relate to

Essay research question. concepts. Types of information (sources) selected partially relate to concepts or answer research question.

concepts or answer research question.

Access the needed Accesses information using effective, Accesses information Accesses information Accesses information information well-designed search strategies and using variety of search using simple search randomly, retrieves

Reflective most appropriate information sources. strategies and some relevant information

strategies retrieves information from

information that lacks relevance and quality.

Essay sources. Demonstrates ability to refine search.

limited and similar sources.

Evaluate Chooses a variety of information Chooses a variety of Chooses a variety of Chooses a few information and sources appropriate to the scope and information sources information sources. information sources.

its sources critically

discipline of the research question. Selects sources after considering the importance (to the researched topic) of the multiple criteria used (such as

appropriate to the scope and discipline of the research question. Selects sources using

Selects sources using basic criteria (such as relevance to the research question and

Selects sources using limited criteria (such as relevance to the research question).

Reflective relevance to the research question, multiple criteria (such as currency).

Essay currency, authority, audience, and bias or point of view).

relevance to the research question, currency, and authority).

Use information Communicates, organizes and Communicates, Communicates and Communicates effectively to synthesizes information from sources organizes and organizes information information from

accomplish a specific purpose

to fully achieve a specific purpose, with clarity and depth

synthesizes information from sources. Intended purpose is achieved.

from sources. The information is not yet synthesized, so the intended purpose is not

sources. The information is fragmented and/or used inappropriately (misquoted, taken out of

Research fully achieved. context, or incorrectly

Paper paraphrased, etc.), so the intended purpose is not achieved.

Access and use Students use correctly all of the Students use correctly Students use correctly Students use correctly information following information use strategies three of the following two of the following one of the following

ethically and legally Research

(use of citations and references; choice of paraphrasing, summary, or quoting; using information in ways that are true to original context; distinguishing

information use strategies (use of citations and references; choice of paraphrasing,

information use strategies (use of citations and references; choice of

information use strategies (use of citations and references; choice of paraphrasing,

Paper between common knowledge and ideas requiring attribution) and

summary, or quoting; using information in

paraphrasing, summary, or quoting; using

summary, or quoting; using information in ways

demonstrate a full understanding of ways that are true to information in ways that that are true to original the ethical and legal restrictions on the original context; are true to original context; distinguishing use of published, confidential and/or distinguishing between context; distinguishing between common proprietary information. common knowledge and between common knowledge and ideas

ideas requiring knowledge and ideas requiring attribution) and attribution) and requiring attribution) demonstrates a full demonstrates a full and demonstrates a full understanding of the understanding of the understanding of the ethical and legal ethical and legal ethical and legal restrictions on the use of restrictions on the use of restrictions on the use published, confidential published, confidential of published, and/or proprietary and/or proprietary confidential and/or information. information. proprietary information.

11


SEAVER COLLEGE

&

GRAZIADIO GRADUATE SCHOOL

&

LIBRARIAN EVALUATION

12


Combined Average rubric scores by Dimensions for Seaver & Graziadio, Fall 2016, Spring 2017, Fall 2017, Spring 2018

y4.0

3.0

2.0

1.0

0.0

3.1 2.9 3.0 3.0 3.1 3.0 2.7 2.9 2.5 2.4

2.9

Reflective Essa Research Paper

■ ■ ■

3.4

Determine the extent Access the needed Evaluate information Use information Access and use of information needed information and its sources effectively to information ethically

critically accomplish a specific and legally purpose

Seaver College ( N=68 students + 5 groups) Graziadio Graduate ( N=25 students) Librarian Evaluation (N=79 students)

Average Score ofSeaver College

( N = 68 students + 5 groups )

AverageScores of Graziadio Graduate ( N = 25

students )

AverageScores of Librarian

Evaluation ( N = 79

students)

Determine the extent of information needed 3.1 3.0 -

Access the needed information 2.9 2.7 -

Evaluate information and its sources critically 3.0 2.4 -

Use information effectively to accomplish a specific purpose

3.0 2.9 2.9

Access and use information ethically and legally 3.1 3.4 2.5

Findings: Seaver students scored slightly higher on all dimensions of the Reflective Essay rubric when compared to Graziadio graduate students. For Research Paper, Seaver students scored slightly higher than Graziadio graduate students and Librarian evaluation on “Use information effectively to accomplish a specific purpose” and Graziadio students had the highest score on “Access and use information ethically and legally” when compared to both Seaver and Librarian Evaluation. The N for Seaver students and Librarian were more than twice as large as Graziadio students.

13


SEAVER COLLEGE Fall 2016 & Spring 2017

14


Table 1. Contributing programs for Seaver College* Fall 2016, Spring 2017

Year Course Instructors/Assessors # of Students

Rubric Dimensions Assessed

ART 390 Gretchen Batcheller, Timothy Pownall, Joseph Piasentin, Bethany Wilson

5 Groups Research Paper

Fall 2016 FRE 380 Mina Soroosh 9 Research Paper

PSYC 310 Jessica Cail 16 Research Paper

THEA 311 Bradley Griffin 7 Reflective Essay & Research Paper

ARTH 490 Lauren Kilroy – Ewbank 4 Reflective Essay & Research Paper

Spring 2017

MUS 355 Gary Cobb 7 Reflective Essay & Research Paper

PHIL 480 Garrett Pendergraft 17 Reflective Essay & Research Paper

SW 300 Emily Scott-Lowe 8 Reflective Essay

*Note: Overall sample size for Seaver College: 68 students with additional 5 groups of students contributing programs

Not all groups submitted both the reflective essay and research paper

15


Table 2. Demographics: 68 Seaver Student Population Included in Assessment (Excl. ART 3901)

Year Gender FRE 380 PSYC 310 THEA 311 Total

Fall 2016 Male 4 4 2 10 Female 5 12 5 22 Total 9 16 7 32

Year Gender ARTH 490 MUS 355 PHIL 480 SW 300 Total

Spring 2017 Male 1 5 9 3 18 Female 3 2 8 5 18 Total 4 7 17 8 36

Table 3. Demographics: 68 Seaver Student Population Included in Assessment (Excl. ART 390*)

Ethnicity Fall 2016 Spring 2017

FRE 380

PSYC 310

THEA 311 Total ARTH

490 MUS 355

PHIL 480

SW 300 Total

Non-resident Alien - 2 2 4 1 1 1 3 6

Hispanic or Latino 6 2 - 8 - 1 3 1 5

Asian - 2 - 2 - 3 - - 3 Black or African American 1 2 - 3 - - - 1 1

White 2 7 4 13 3 1 11 3 18

Two or more Races - 1 1 2 - - - - -Race/Ethnicity Unknown - - - - - 1 2 - 3

Total 9 16 7 32 4 7 17 8 36

1 Art 390 is a group project , no demographics data

16


Group Project Fine Arts Program ART 390 – Fall 2016

Average Scores by Dimension

4.0

3.0

2.0

1.0

0.0

2.5 2.4

3.0 3.1 2.9

2.5

Research Paper

■ ■

Use information effectively to accomplish a Access and use information ethically and legally specific purpose

Average Scores of ART 390 ( N=5 students) Average Score of Seaver College ( N = 68 students + 5 groups ) Average Scores of Librarian Evaluation ( N = 79 students)

Average Scores of ART 390

( N=7 groups)




Evaluation ( N = 79 students)

Use information effectively to accomplish a specific purpose 2.5 3.0 2.9


Findings: ART 390 did not submit student Reflective Essays. Their assessment data was performed on their Research Paper using 2 dimensions on the rubric “Use information effectively to accomplish a specific purpose” and “Access and use information ethically and legally”. The students enrolled in ART 390 scored slightly lower on both dimensions when compared to Seaver students and Librarian Evaluation.

17

■ ■ ■ ■


Group Project Fine Arts Program ART 390 – Fall 2016

ART 390 Scores Overall2


Access and use information ethically and legally

3 (15%) 8 (40%) 4 (20%) 5 (25%)

5 (25%) 5 (25%) 3 (15%) 7 (35%)

Capstone (4 pts) Milestones (3 pts) Milestones (2 pts) Benchmark (1 pts)


Capstone (4 pts)

15%

Milestones (3 pts)

40%

Milestones (2 pts)

20%

Benchmark (1 pts)

25%

Access and use information ethically and legally 25% 25% 15% 35%

Level of Accomplishment Capstone Milestones Milestones Benchmark

Informat ion not Present

(4 pts) (3 pts) (2 pts) (1 pts) % of students expected

to score in each level 2-5% Between 35- 45% Between 35- 45% under 10-15% 0%

Findings by

Dimension


C ap s to ne s c or es exc eed t he es t a b l i s hed benc h ma r k . S t udent s s c or ed w i t h i n t he b enc hma r k a t t he M i l e s to n e s (3 ) l ev e l . S t udent s s c or ed b e l ow t he b enc hma r k i n t he M i l e s to ne s (2 ) l ev e l . S t udent s ’ s c or es g r ea t l y exc eed t he es t a b l i s hed benc hma r k a t t he B e n c h m ark l ev e l .


S c or es a t t he M i l e s to n e s (3 ) a nd M i l e s to ne s (2 ) l ev e l s ar e be l ow t he b enc hma r k . S t ud ent s ’ s c or es a t t he C ap s to n e a nd B e nc h m ark l ev e l s g r ea tl y exc eed t he es t a b l i s hed benc hma r k .

2 ART 390 includes four assessors. Each student is assessed multiple times.

18


French Program FRE 380 – Fall 2016


4.0

3.0

2.0

1.0

0.0

2.4 2.2

3.0 3.1 2.9 2.5

Research Paper

■ ■

Use information effectively to accomplish a specific Access and use information ethically and legally purpose

Average Scores of FRE 380 ( N=9 students) Average Score of Seaver College ( N = 68 students + 5 groups ) Average Scores of Librarian Evaluation ( N = 79 students)

Average Scores of FRE 380

( N = 9 students)




Evaluation ( N = 79

students) Use information effectively to accomplish a specific purpose 2.4 3.0 2.9


Findings: FRE 380 did not submit student Reflective Essays. Their assessment data was performed on their Research Paper using 2 dimensions on the rubric “Use information effectively to accomplish a specific purpose” and “Access and use information ethically and legally”. The students enrolled in FRE 380 scored lower on both dimensions when compared to Seaver students and Librarian Evaluation.

19

■ ■


FRE 380 – Fall 2016 Average Scores by Gender


Male ( N=4 students) Female ( N=5 students)

1.8 1.5

3.0 2.8

0.0

1.0

2.0

3.0

4.0 Research Paper

Male ( N = 4 students)

Female ( N = 5 students)

Use information effectively to accomplish a specific purpose 1.8 3.0

Access and use information ethically and legally 1.5 2.8

Findings: The male students in FRE 380 scored significantly lower in the two dimensions that were assessed when compared to the female students for Fall 2016. For “Use information effectively to accomplish a specific purpose,” male students averaged a 1.8 while the female students’ average was 3.0. And for the dimensions “Access and use information ethically and legally” male students was 1.5 and female students was 2.8.

20


FRE 380 – Fall 2016 Average Scores by Ethnicity


Minority ( N=7 students) White ( N=2 students)

2.4 2.1

2.5 2.5

0.0

1.0

2.0

3.0

4.0 Research Pa

per

■ ■

Minority ( N = 7 students)

White ( N = 2 students)



Findings: The minority students in FRE 380 scored slightly lower in the two dimensions that were assessed when compared to the white students for Fall 2016. For “Use information effectively to accomplish a specific purpose,” minority students averaged a 2.4 while the female students’ average was 2.5. And for the dimensions “Access and use information ethically and legally” minority students was 2.1 and white students was 2.5.

21

■ ■ ■ ■


FRE 380 – Fall 2016 FRE 380 Scores Overall



1 (11%) 3 (33%) 4 (45%)

1 (11%) 2 (22%) 4 (45%) 2 (22%)


1 (11%)

Capstone (4 pts)

Milestones (3 pts)

Milestones (2 pts)

Benchmark (1 pts)

Use information effectively to accomplish a specific purpose 11% 33% 45% 11%






Findings by

Dimension


S t udent s s c or ed w i t h i n t he es t a b l i s hed be nc hma r k a t t he M i l e s to ne s (2 ) a nd B e n ch m ar k l ev e l s . C ap s ton e l ev e l sc or es exc eed t he es t a b l i s hed benc hma r k . S t ud ent s ’ s c or es a r e s l ig ht ly be l ow t he be nc hma r k a t t he M i l e s to n e s (3 ) l ev e l .


S t udent s s c or ed w i t h i n t he b enc hma r k a t t he M i l e s to n e s (2 ) l ev e l . S c or es a t t he B e n ch m ar k a nd C ap s ton e l ev el s a r e a bov e t he es t a b l i s hed benc h ma r k . S c or es a t t he M i l e s to n e s (3 ) l ev e l a r e be l ow t he benc h ma r k .

22


Psychology Program PSYC 310 – Fall 2016


4.0

3.0

2.0

1.0

0.0

3.9 3.8

3.0 3.1 2.9 2.5

Research Paper

■

■


Average Scores of PSYC 310 ( N=16 students) Average Score of Seaver College ( N = 68 students + 5 groups ) Average Scores of Librarian Evaluation ( N = 79 students)

Average Scores of PSYC 310

( N = 16 students)




Evaluation ( N = 79



Findings: PSYC 310 did not submit student Reflective Essays. Their assessment data was performed on their Research Paper using 2 dimensions on the rubric “Use information effectively to accomplish a specific purpose” and “Access and use information ethically and legally”. The students enrolled in PSYC 310 scored higher on both dimensions when compared to Seaver students and Librarian Evaluation.

23

)

)

■ ■ ■ ■


PSYC 310 – Fall 2016 PSYC 310 Scores Overall

Use information effectively to accomplish a specific purpose 7 (44%) 8 (50%)

Access and use information ethically and legally 15 (94%)


1 (6%

1 (6%

Capstone (4 pts)

Milestones (3 pts)

Milestones (2 pts)

Benchmark (1 pts)

Use information effectively to accomplish a specific purpose 44% 50% 6% 0%






Findings by

Dimension


S t udent s ’ s c or es g r ea t l y exc eed t he es t a b l i s hed benc h ma r k a t t he C ap s to n e l ev e l . M i le s to n e s (3 ) l ev e l s c or es a l s o exc eed t he benc h ma r k . Due t o t he h i g h per c ent a g e of s c or es i n t he C ap s to n e a nd M i l e s to ne s (3 ) l ev e l s , s c or es a t t he M i l e s to ne s (2 ) l ev e l a r e f ar be l ow t he es t a b l i s hed benc hma r k , a nd n o s t ud ent s s c or ed a t t he B e n c hm ar k l ev e l .


S c or es a t t he C ap s to n e l ev e l g r ea tl y exc eed t he benc hma r k . S t udent s s c or ed f a r be l ow t he benc h ma r k a t t he M i l e s to n es (3 ) l ev e l . No s t ude nt s c or ed a t t he M i l e s to n e s (2 ) a nd t he B e nc h m ark l ev e l s .

24


Theatre Program THEA 311– Fall 2016


2.6 2.4 2.1 2.1

2.4

3.1 2.9 3.0 3.0 3.1 2.9 2.5

0.0

1.0

2.0

3.0

4.0 Reflective Essay

Research Paper

■

■


critically accomplish a specific and legally purpose

Average Scores of THEA 311 ( N=7 students) Average Score of Seaver College ( N = 68 students + 5 groups ) Average Scores of Librarian Evaluation ( N = 79 students)

Average Scores of THEA 311

( N = 7 students)



Average Scoresof Librarian Evaluation

( N = 79 students)

Determine the extent of information needed 2.6 3.1

-

Access the needed information 2.4 2.9 -Evaluate information and its sources critically 2.1 3.0

-



Findings: THEA 311 submitted student Reflective Essays and Research Papers. All 5 dimensions of the rubric were used for assessment. The students enrolled in THEA 311 scored lower on all 5 dimensions when compared to Seaver students and Librarian Evaluation.

25


THEA 311– Fall 2016 Average Scores by Gender

2.0 2.5

1.5 2.0 2.0

2.8 2.4 2.4 2.2

2.6

0.0

1.0

2.0

3.0


Research Paper

■ ■

Determine the Access the needed Evaluate information Use information Access and use extent of information and its sources effectively to information ethically

information needed critically accomplish a specific and legally purpose




Determine the extent of information needed 2.0 2.8 Access the needed information 2.5 2.4 Evaluate information and its sources critically 1.5 2.4 Use information effectively to accomplish a specific purpose 2.0 2.2 Access and use information ethically and legally 2.0 2.6

Findings: The female students in THEA 311 scored higher averages than males students in all dimensions except for “Access the needed information”

26

■ ■ ■ ■


THEA 311– Fall 2016 THEA 311 Scores Overall

and legally

Determine the extent of information needed 1 (14%) 3 (43%) 2 (29%) 1 (14%)

Access the needed information 3 (43%) 4 (57%)

Evaluate information and its sources critically 3 (43%) 2 (29%) 2 (29%)

Use information effectively to accomplish a specific purpose 1 (14%) 1 (14%) 3 (43%) 2 (29%)

Access and use information ethically 1 (14%) 2 (29%) 3 (43%) 1 (14%)


Capstone (4 pts)

Milestones (3 pts)

Milestones (2 pts)

Benchmark (1 pts)

Determine the extent of information needed 14% 43% 29% 14% Access the needed information 0% 43% 57% 0% Evaluate information and its sources critically 0% 43% 29% 29% Use information effectively to accomplish a specific purpose 14% 14% 43% 29% Access and use information ethically and legally 14% 29% 43% 14%



(4 pts) (3 pts) (2 pts) (1 pts) % of students expected to score

in each level 2-5% Between 35- 45% Between 35- 45% under 10-15% 0%

Findings by

Dimension

Determine the extent of information needed

S t u d e n t s s c o r e d a b o v e t h e e s t a b l i s h e d b e n c h m a r k a t t h e C a p s t o ne l e v e l . S c o r e s w e r e w i t h i n t h e e s t a b l i s h e d b e n c h m a r k a t t h e M i le s t on e s ( 3 ) a n d B en c h m a r k l e v e l s . S t u d e n t s s c o r e d b e l o w t h e b e n c h m a r k a t t h e M i le s t on e s ( 2) l e v e l .

Access the needed information

S t u d e n t s s c o r e d w i t h i n t h e b e n c h m a r k a t t h e M i le s to ne s ( 3 ) le v e l . M i le s t on e s ( 2 ) l e v e l s c o r e s ar e a b o v e t h e e s t a b l i s h e d b e n c h m a r k . N o s t u d e n t s s c o r e d a t t h e B e n c h m a r k a n d t h e C a p s t o ne l e v e l s .

Evaluate information and its sources critically

S t u d e n t s s c o r e d w i t h i n t h e b e n c h m a r k a t t h e M i le s to ne s ( 3 ) le v e l . M i le s t on e s ( 2 ) l e v e l s c o r e s ar e b e l o w t h e e s t a b l i s h e d b e n c h m a r k . S c o r e s a t t h e B e n c h m a r k l e v e l g r e a t l y e x c e e d e d t h e e s t a b l i s h e d b e n c h m a r k . N o s t u d e n t s s c o r e d a t t h e C a p s t o n e l e v e l .


S t u d e n t s s c o r e d a b o v e t h e b e n c h m a r k a t t h e C a p s t o n e a n d B e n c h m a r k l e v e l s . T h e s c o r e s a t Mi le s t on e s ( 2 ) l e v e l ar e w i t h i n t h e e s t a b l i s h e d b e n c h m a r k . S c o r e s a t t h e M il e s to ne s (3 ) a r e be l o w t h e b e n c h m a r k .


T h e s c o r e s a t t h e C a p s t o n e l e v e l e x c e e d t h e e s t a b l i s h e d b e n c h m a r k . S t u d e n t s s c o r e d w i t h i n t h e e s t a b l i s h e d b e n c h m a r k a t t h e B e n c h m a r k a n d t h e M il e st on e s ( 2 ) l e v e l s . T h e s c o r e s a t t he M i le s t on e s ( 3 ) l e v e l a r e b e l o w t h e b e n c h m a r k .

27


Art History Program ARTH 490 – Spring 2017


3.0 2.7

3.0 3.5 3.5

3.1 2.9 3.0 3.0 3.1 2.9 2.5

0.0

1.0

2.0

3.0

4.0

Reflective Essay Research Paper

• •


critically accomplish a specific and legally Average Scores of ARTH 490 ( N= 4 students) purpose Average Score of Seaver College ( N = 68 students + 5 groups ) Average Scores of Librarian Evaluation ( N = 79 students)

Average Scores of ARTH 490

( N = 4 students)




( N = 79 students)


-


-



Findings: ARTH 490 submitted student Reflective Essays and Research Papers. All 5 dimensions of the rubric were used for assessment. For Reflective Essay, the students enrolled in ARTH 490 scored slightly lower on “Determine the extent of information needed” and “Access the needed information” and scored equally on “Evaluate information and its sources critically” when compared to Seaver students. While they scored higher on all 2 dimensions of Research Paper when compared to Seaver students and Librarian Evaluation.

28

1 (33%)

2 (50%)

2 (50%)

3 (100%)

2 (67%)

1 (33%)

2 (50%)

2 (50%)

1 (33%)

1 (33%)







I

■ ■ ■ ■


ARTH 490 – Spring 2017 ARTH 490 Scores Overall

Capstone (4 pts)

Milestones (3 pts)

Milestones (2 pts)

Benchmark (1 pts)

Determine the extent of information needed 0% 100% 0% 0% Access the needed information 0% 67% 33% 0% Evaluate information and its sources critically 33% 33% 33% 0% Use information effectively to accomplish a specific purpose 50% 50% 0% 0%






Findings by

Dimension


A l l s t udent s s c or ed a t t he M i l e s to n e s (3 ) l ev e l , w h ic h f a r exc eeds t he es t a b l i s hed benc hma r k . No s t ude nt s s c or ed a t t he B e n c h m ar k , M i l e s ton e s (2 ) , or C ap s ton e l ev e l s .


S t udent s s c or ed w el l a bov e t he es t a b l i s hed benc h ma r k a t t he M i l e s to n e s (3 ) l ev e l . S t udent s s c or ed s l i g htl y be l ow t he b enc hma r k a t t he M i l e s to n e s (2 ) l ev e l . No s t udent s s c or ed a t t he B e n c hm ar k or C ap s to n e l ev e l s .


S t udent s s c or ed a b ov e t he es t a b l i s hed benc h ma r k a t t he C ap s to n e l ev e l . S t udent s s c or ed s l ig ht l y l ow er t ha n t he be nc hma r k i n t he M i l e s to n e s (3 ) a nd M i l e s to n e s (2 ) l ev e l s . No s t udent s s c or ed a t t h e B e n c h m ark l ev e l .


S t udent s ’ s c or es g r ea t l y exc eed t he benc h ma r k a t t he C ap s to ne l ev e l . M i l e s ton e s (3 ) s c or es a l s o exc eed t h e es t a b l i s hed benc hma r k . No s t u dent s s c or ed i n t h e M i l e s to ne s (2 ) or B e nc h m ark l ev e l s .


S t udent s ’ s c or es g r ea t l y exc eed t he benc h ma r k a t t he C ap s to ne l ev e l . M i l e s ton e s (3 ) s c or es a l s o exc eed t h e es t a b l i s hed benc hma r k . No s t u dent s s c or ed i n t h e M i l e s to ne s (2 ) or B e nc h m ark l ev e l s .

29


Music Program MUS 355 – Spring 2017


3.1 3.1 3.4

3.0 3.7

3.1 2.9 3.0 3.0 3.1 2.9 2.5

0.0

1.0

2.0

3.0


Research Paper

■

■


critically accomplish a specific and legally purpose Average Scores of MUS 355 ( N=7 students)

Average Score of Seaver College ( N = 68 students + 5 groups ) Average Scores of Librarian Evaluation ( N = 79 students)

Average Scores of MUS 355

( N = 7 students)




( N = 79 students)


-


-



Findings: MUS 355 submitted student Reflective Essays and Research Papers. All 5 dimensions of the rubric were used for assessment. The students enrolled in MUS 355 scored equally on “Determine the extent of information needed” and “Use information effectively to accomplish a specific purpose” when compared to Seaver students. While they scored higher on the rest 3 dimensions when compared to Seaver students and Librarian Evaluation.

30


MUS 355 – Spring 2017

Average Scores by Gender

3.2 3.2 3.4

3.0 3.6

3.0 3.0 3.5

3.0

4.0

0.0

1.0

2.0

3.0

4.0 Research Paper Reflective Essay

■ ■

Determine the Access the needed Evaluate information Use information Access and use extent of information and its sources effectively to information ethically

information needed critically accomplish a specific and legally purpose


Male ( N = 5 students )


Determine the extent of information needed 3.2 3.0 Access the needed information 3.2 3.0 Evaluate information and its sources critically 3.4 3.5 Use information effectively to accomplish a specific purpose 3.0 3.0 Access and use information ethically and legally 3.6 4.0

Findings: The male students enrolled in MUS 355 scored slightly higher on “Determine the extent of information needed” and “Access the needed information” and scored equally on “Use information effectively to accomplish a specific purpose”, while they scored lower on “Evaluate information and its sources critically” and “Access and use information ethically and legally” when compared to female students.

31

I I I I

I I I

I I I

I I I

I I I I I I

■ ■ ■ ■


MUS 355 – Spring 2017

MUS 355 Scores Overall






1 (14%)

1 (14%)

3 (43%)

6 (86%)

2 (29%) 3 (43%) 2 (29%)

5 (71%)

6 (86%)

4 (57%)

2 (29%)


Capstone (4 pts)

Milestones (3 pts)

Milestones (2 pts)

Benchmark (1 pts)








S t u d e n t s s c o r e d a b o v e t h e e s t a b l i s h e d b e n c h m a r k a t t h e C a p s t o ne l e v e l . S t u d e n t s ’ s c o r e s a t t h e M i l e s to ne s (3 ) l e v e l f ar e x c e e d t h e b e n c h m a r k . N o s t u d e n t s s c o r e d i n t h e B e n c h m a r k o r Mi le s to ne s ( 2 ) l e v e l s .

Access the needed S t udent s s c or ed a b ov e t he es t a b l i s hed benc h ma r k a t t he C ap s to n e

Findings by

Dimension

information l ev e l . S t udent s ’ s c or es a t t he M i l e s to n e s (3 ) l ev e l f a r exc eed t he benc hma r k . No s t udent s s c or ed i n t he B e n c h m ar k or M i le s to n e s (2 ) l ev e l s .


S t u d e n t s ’ s c o r e s g r e a t l y e x c e e d t h e e s t a b l i s h e d b e n c h m a r k a t t h e C a p s t o ne l e v e l . Mi le s t on e s ( 3 ) s c o r e s a l s o e x c e e d t h e b e n c h m a r k . N o s t u d e n t s s c o r e d i n t h e B e n c h m a r k o r M il e s to ne s (2 ) l e v e l s .

Use information M i le s t on e ( 3 ) l e v e l s c o r e s ar e w i t h i n t h e e s t a b l i s h e d b e n c h m a r k .

effectively to S t u d e n t s ’ s c o r e s e x c e e d t h e e s t a b l i s h e d b e n c h m a r k i n t h e C a p s t o ne

accomplish a specific l e v e l . S t u d e n t s c o r e s a r e b e l o w t h e b e n c h m a r k i n t h e M i l e st on e ( 2 ) l e v e l . N o s t u d e n t s s c o r e d a t B e n c h m a r k l e v e l .

purpose Access and use information ethically and legally

S t u d e n t s ’ s c o r e s g r e a t l y e x c e e d t h e b e n c h m a r k a t t h e C a p s t o ne l e v e l . S t u d e n t s s c o r e d b e l o w t h e e s t a b l i s h e d b e n c h m a r k i n t h e M i le st o ne s (3 ) l e v e l . N o s t u d e n t s s c o r e d i n t h e M i l e s t on e s ( 2 ) o r B en c h m a rk l e v e l s .

32


Philosophy Program PHIL 480 – Spring 2017


3.5 3.2 3.2 3.2

3.8

3.1 2.9 3.0 3.0 3.1 2.9 2.5

0.0

1.0

2.0

3.0

4.0 Reflective Essay Research Paper

• •


critically accomplish a specific and legally

Average Scores of PHIL 480 ( N=17 students) purpose

Average Score of Seaver College ( N = 68 students + 5 groups ) Average Scores of Librarian Evaluation ( N = 79 students)

Average Scores of PHIL 480

( N = 17 students)




( N = 79 students)


-


-



Findings: PHIL 480 submitted student Reflective Essays and Research Papers. All 5 dimensions of the rubric were used for assessment. The students enrolled in PHIL 480 scored higher on all 5 dimensions when compared to Seaver students and Librarian Evaluation.

33


PHIL 480– Spring 2017 Average Scores by Gender

3.7 3.2 3.3

3.0

3.8 3.3 3.3

3.0 3.5

3.8

0.0

1.0

2.0

3.0

4.0 y

Reflective Essa Research Paper

■ ■

Determine the extent Access the needed Evaluate information Use information Access and use of information needed information and its sources critically effectively to information ethically

accomplish a specific and legally purpose





Access the needed information 3.2 3.3

Evaluate information and its sources critically 3.3 3.0



Findings: The male students enrolled in PHIL 480 scored slightly higher on “Determine the extent of information needed” and “Evaluate information and its sources critically” and scored equally on “Access and use information ethically and legally”, while they scored lower on “Access the needed information” and “Use information effectively to accomplish a specific purpose” when compared to female students.

34

-

■ ■ ■ ■


PHIL 480 – Spring 2017 PHIL 480 Scores Overall

Determine the extent of information needed 9 (53%) 7 (41%) 1 (6%)

Access the needed information 8 (47%) 5 (29%) 4 (24%)


Use information effectively to accomplish a specific purpose 7 (41%) 7 (41%) 3 (18%)

Access and use information ethically and legally 13 (76%) 4 (24%)


Capstone (4 pts)

Milestones (3 pts)

Milestones (2 pts)

Benchmark (1 pts)





(4 pts) (3 pts) (2 pts) (1 pts) % of students expected to

score in each level 2-5% Between 35- 45% Between 35- 45% under 10-15% 0%

Findings by

Dimension


M i l e s to ne s (3 ) l ev e l s c or es ar e w i t hi n t he es t a b l i s hed benc hma r k . S t udent s c or es g r ea t l y exc eed t he benc hma r k a t t he C ap s to n e l ev e l . M i l e s to ne s (2 ) l ev e l s c or es ar e f a r be l ow t he benc hma r k . N o s t udent s s c or ed a t t he B e n c hm ar k l ev e l .


S t udent s c or es g r ea t l y exc eed t he benc hma r k a t t he C ap s to n e l ev e l . S t udent s c or es i n t he M i l e s to n e s (2 ) a nd M i l e s to n e s (3 ) l ev e l s a r e be l ow t he b enc hma r k s . No s t u dent s s c or ed a t B e n c h m ar k lev e l .


M i l e s to ne s (3 ) l ev e l s c or es ar e w i t hi n t he es t a b l i s hed benc hma r k . M i l e s to ne s (2 ) l ev e l s c or es ar e be l ow t he be nc hma r k . S t ud ent s c or es g r eat l y exc eed t he benc h ma r k a t t he C ap s to n e l ev e l . No s t udent s s c or ed a t t he B e n c hm ar k l ev e l .


M i l e s to ne s (3 ) l ev e l s c or es ar e w i t hi n t he es t a b l i s hed benc hma r k . M i l e s to ne s (2 ) l ev e l s c or es ar e be l ow t he be nc hma r k . S t ud ent s c or es g r eat l y exc eed t he benc h ma r k a t t he C ap s to n e l ev e l . No s t udent s s c or ed a t t he B e n c hm ar k l ev e l .


S t udent s c or es g r ea t l y exc eed t he benc hma r k a t t he C ap s to n e l ev e l . M i l e s to ne s (3 ) l ev e l s c or es ar e be l ow t he be nc hma r k . No s t udent s s c or ed a t t he B e n c h m ar k or M i l e s ton e s (2 ) l ev e ls .

35

■ ■


Social Work Minor SW 300 – Spring 2017


2.8 2.6 2.9 3.1 2.9 3.0

0.0

1.0

2.0

3.0


Determine the extent of Access the needed information Evaluate information and its information needed sources critically

Average Scores of SW 300 ( N=8) Average Score of Seaver College ( N = 68 students + 5 groups )

Average Scores of SW 300 ( N = 8

students)



Determine the extent of information needed 2.8 3.1 Access the needed information 2.6 2.9 Evaluate information and its sources critically 2.9 3.0

Findings: SW 300 did not submit student Research Papers. Their assessment data was performed on their Reflective Essay using 3 dimensions of the rubric “Determine the extent of information needed”, “Access the needed information” and “Evaluate information and its sources critically”. The students enrolled in SW 300 scored lower on all 3 dimensions when compared to Seaver students.

36

■ ■


SW 300– Spring 2017

Average Scores by Gender

2.7 2.7

3.3 2.8 2.6 2.6

0.0

1.0

2.0

3.0


Determine the extent of information Access the needed information Evaluate information and its sources needed critically




Determine the extent of information needed 2.7 2.8 Access the needed information 2.7 2.6 Evaluate information and its sources critically 3.3 2.6

Findings: The male students enrolled in SW 300 scored slightly higher on rubric “Access the needed information” and “Evaluate information and its sources critically”, while they scored slightly lower on “Determine the extent of information needed” when compared to female students.

37


SW 300– Spring 2017

Average Scores by Ethnicity

4.0

3.0

2.0

1.0

0.0

2.5 2.5 2.5 2.7 2.3

2.7 3.0 3.0

3.3

Reflective Essay

• •

Determine the extent of Access the needed information Evaluate information and its information needed sources critically

Minority (N=2 students) International (N=3 students) White (N=3 students)

Minority ( N = 2 students)

International ( N = 3 students)

White ( N = 3 students)

Determine the extent of information needed 2.5 2.7 3.0 Access the needed information 2.5 2.3 3.0 Evaluate information and its sources critically 2.5 2.7 3.3

Findings: The white students enrolled in SW 300 have the highest score on all 3 dimensions. International students scored slightly higher on “Determine the extent of information needed” and “Evaluate information and its sources critically” when compared to the minority students.

38

■ ■ ■ ■


SW 300 – Spring 2017

SW 300 Scores Overall




6 (75%) 2 (25%)

5 (62%) 3 (38%)


Capstone (4 pts)

Milestones (3 pts)

Milestones (2 pts)

Benchmark (1 pts)

Determine the extent of information needed 0% 75% 25% 0% Access the needed information 0% 62% 38% 0% Evaluate information and its sources critically 13% 62% 25% 0%





Findings by

Dimension


S t udent s c or es f a r exc ee d t he benc hma r k i n t he M i l e s to ne s (3 ) l ev e l . M i l e s to ne s (2 ) l ev e l s c or es a r e be l ow t he b enc hma r k . No s t udent s s c or ed a t t he B e nc h m ark or C ap s ton e l ev e ls .


M i l e s to ne s (2 ) l ev e l s c or es ar e w i t hi n t he be nc hma r k . S t udent s c or es e xc eed t he benc h ma r k i n t he M i l e s to n e s (3 ) l ev e l . No s t ude nt s s c or ed a t t he B e n c h m ar k or C ap s to ne l ev e l s .


S t udent s c or es e xc eed t he benc h ma r k i n t he M i l e s to n e s (3 ) a nd C ap s to n e l ev e ls . M i l e s to ne s (2 ) l ev e l s c or es ar e be l ow t he b enc hma r k . No s t udent s s c or ed a t t he B e nc h m ark l ev e l .

39


GRAZIADIO GRADUATE SCHOOL

Fall 2016

40


Table 1. Contributing programs for Graziadio Graduate School



Fall 2016 ECNM 602 Jared Ashworth 25 Reflective Essay & Research Paper

41


Economics Program ECNM 602 – Fall 2016


4.0

3.0 3.0

2.7 3.0

2.4 2.7

2.4

Reflective Essay

Research Paper

■ ■

2.9 2.0

1.0

0.0

2.9 2.9

Determine the extent Access the needed Evaluate information Use information of information needed information and its sources effectively to

critically accomplish a specific purpose

Average Scores of ECNM 602 ( N = 25 students) Average Score of Graziadio Graduate ( N = 25 students) Average Scores of Librarian Evaluation ( N = 79 students)

3.4 3.4

2.5

Access and use information ethically

and legally

Average Scores of ECNM 602

( N = 25 students)

Average Score ofGraziadio Graduate ( N = 25

students)


( N = 79 students)


-


-



Findings: ECNM 602 submitted student Reflective Essays and Research Papers. All 5 dimensions of the rubric were used for assessment. The students enrolled in ECNM 602 scored higher on “Access and use information ethically and legally” when compared to Librarian Evaluation and scored equally on the rest 4 dimensions when compared to Graziadio students and Librarian Evaluation.

42

I I I

I I I

I I I

I I I

I I I I I I I I

■ ■ ■ ■


ECNM 602 – Fall 2016 ECNM 602 Scores Overall

Determine the extent of information needed 5 (20%) 17 (68%) 2 (8%) 1 (4%)

Access the needed information 6 (24%) 9 (36%) 6 (24%) 4 (16%)

Evaluate information and its sources critically 1 (4%) 8 (32%) 15 (60%) 1 (4%)

Use information effectively to accomplish a specific purpose 4 (16%) 14 (56%) 7 (28%)

Access and use information ethically and legally 11 (44%) 12 (48%) 2 (8%) Capstone (4 pts) Milestones (3 pts) Milestones (2 pts) Benchmark (1 pts)

Capstone (4 pts)

Milestones (3 pts)

Milestones (2 pts)

Benchmark (1 pts)

Determine the extent of information needed 20% 68% 8% 4%

Access the needed information 24% 36% 24% 16%

Evaluate information and its sources critically 4% 32% 60% 4%


16% 56% 28% 0%




(4 pts) (3 pts) (2 pts) (1 pts) % of students expected to score

in each level 2-5% Between 35- 45% Between 35- 45% under 10-15% 0%

Findings by Dimension


S t udent s c or es e xc eed t he es t a b l i s hed benc hma r k a t t he M i l e s to n e s (3 ) a nd C ap s to n e l ev e ls . B en c h m ar k a nd M i l e s to ne s (2 ) l eve l s c or es a r e f a r be l ow t he es t a b l i s hed benc hma r k .


M i l e s to ne s (3 ) l ev e l s c or es ar e w i t hi n t he be nc hma r k . C ap s to n e a nd B e nc h m ark l ev e l s c or es exc eed t he es t a b l i s hed benc hma r k . M i l e s to ne s (2 ) l ev e l s c or es ar e be l ow t he be nc hma r k .


C ap s to ne l ev e l s c or es ar e w i t hi n t he es t a b l i s hed benc h ma r k . S t udent s c or es g r ea t l y exc eed t he benc hma r k a t t he M i l e s t o n e s (2 ) l ev e l . B en c hm ar k a nd M i l e s to ne s (3 ) s c or es a r e be l ow t he benc hma r k .


S t udent s c or es e xc eed t he es t a b l i s hed benc hma r k a t t he M i l e s to n e s (3 ) a nd C ap s to n e l ev e ls . M i l e s to ne (2 ) l ev e l s c or es ar e be l ow t he benc hma r k . No s t udent s s c or ed a t t he B e n c h m ar k l ev e l .


C ap s to ne l ev e l s c or es g r ea tl y exc eed t he es t a b l i s hed benc h ma r k . M i l e s to ne s (3 ) l ev e l s c or es ar e s l ig ht ly gr ea t er t ha n t he be nc hma r k . M i l e s to ne s (2 ) l ev e l s c or es ar e f a r be l ow t he es t a b l i s hed benc hma r k . No s t udent s s c or ed a t B e n ch m ar k l ev el .

43


LIBRARIAN EVALUATION Fall 2017 & Spring 2018

44


Table 1. Contributing programs for Graziadio Graduate School*



Fall 2017 INTS 455 Jaimie Colvin, Jeremy Whitt,

Mary Elizabeth Parang 9 Research Paper

Fall 2017 NUTR 499 Jaimie Colvin, Jeremy Whitt 5 Research Paper

Spring 2018 BA 497 Jaimie Colvin, Jeremy Whitt, Marc Vinyard

65 Research Paper

*Note: The librarians in Payson library retrieved students' work in their senior year to provide additional data for the Information Literacy core competency assessment. This assessment examined 2 dimensions of the value rubric on “Use information effectively to accomplish a specific purpose” and “Access and use information ethically and legally”. In total 65 students artifacts were assessed by 2 to 3 assessors.

45


International Studies and Languages Program INTS 455 – Fall 2017


4.0

3.0

2.0

1.0

0.0

2.5 2.2

3.0 3.1 2.9 2.5

Research Paper

■ ■


Average Scores of INTS 455 ( N = 9 students) Average Score of Seaver College ( N = 68 students + 5 groups ) Average Scores of Librarian Evaluation ( N = 79 students)

Average Scores of INTS 455

( N = 9 students)




Evaluation ( N = 79



Findings: INTS 455 did not submit student Reflective Essays. Their assessment data was performed on their Research Paper using 2 dimensions on the rubric “Use information effectively to accomplish a specific purpose” and “Access and use information ethically and legally”. The students enrolled in INTS 455 scored lower on both dimensions when compared to Seaver students and Librarian Evaluation.

46

■ ■ ■ ■


INTS 455 – Fall 2017 INTS 455 Scores Overall

Use information effectively to accomplish a specific purpose 6% 39% 56%



Capstone (4 pts)

Milestones (3 pts)

Milestones (2 pts)

Benchmark (1 pts)


6% 39% 56% 0%






Findings by

Dimension


M i l e s to ne s (3 ) l ev e l s c or es ar e w i t hi n t he es t a b l i s hed benc hma r k . M i l e s to n e (2 ) s c or es ar e a bov e t he benc hma r k . C ap s to ne l ev e l s c or es ar e s l i g htl y h ig her t ha n t he benc hma r k . No s t udent s s c or ed a t t he B e n c h m ar k l ev e l .


B e nc h m ark l ev e l s c or es a r e w it h i n t he es t a b l i s hed benc hma r k . M i l e s to n e s (2 ) s c or es exc eed t h e be nc hma r k . M i l e s to ne s (3 ) l ev e l s c or es ar e be l ow t he es t a b l i s hed benc hma r k . C ap s to n e l ev e l s c or es a r e s l ig ht l y h ig her t ha n t he be nc hma r k .

47


Natural Science Program NUTR 499 – Fall 2017


Chart Titlesearch Paper

4.0

3.0

2.0

1.0

0.0

3.4

2.8 3.0 3.1 2.9 2.5

Re

■

■


Average Scores of NUTR 499 ( N = 5 students) Average Score of Seaver College ( N = 68 students + 5 groups ) Average Scores of Librarian Evaluation ( N = 79 students)

Average Scores of NUTR 499

( N = 5 students)




Evaluation ( N = 79

students) Use information effectively to accomplish a specific purpose

3.4 3.0 2.9


Findings: NUTR 499 did not submit student Reflective Essays. Their assessment data was performed on their Research Paper using 2 dimensions on the rubric “Use information effectively to accomplish a specific purpose” and “Access and use information ethically and legally”. The students enrolled in NUTR 499 scored higher on “Use information effectively to accomplish a specific purpose” when compared to Seaver students and Librarian Evaluation, while scored lower than Seaver students and higher than Librarian Evaluation on “Access and use information ethically and legally”.

48

■ ■ ■ ■


NUTR 499 – Fall 2017 NUTR 499 Scores Overall


Access and use information ethically and legally 80%


20%

Capstone (4 pts)

Milestones (3 pts)

Milestones (2 pts)

Benchmark (1 pts)


60% 20% 20% 0%






Findings by

Dimension


C ap s to ne l ev e l s c or es g r ea tl y exc eed t he es t a b l i s hed benc hma r k . Du e t o t he h i g h per c ent a g e of s c or es i n t h e C ap s to ne l ev e l , s c or es i n t he M i l e s to n e s (2 ) a nd M i l e s to n es (3 ) l ev e l s a r e be l ow t he benc hma r k . N o s t ude nt s s c or ed a t t he B e n c h m ar k l ev e l .


M i l e s to ne s (3 ) l ev e l s c or es gr ea tl y exc eed t h e es t a b l i s hed benc hma r k . M i l e s to n e s (2 ) l ev e l s c or es ar e be l ow t he benc hma r k . No s t udent s s c or ed a t t he B e n c h m ar k or C ap s to ne l ev e l s .

49


Business Administration Program BA 497 – Spring 2018


Chart TitleResearch Paper

4.0

3.0

2.0

1.0

0.0

3.0 2.5

3.0 3.1 2.9 2.5

• •


Average Scores of BA 497 ( N = 65 students) Average Score of Seaver College ( N = 68 students + 5 groups ) Average Scores of Librarian Evaluation ( N = 79 students)

Average Scores of BA 497 ( N = 65

students)




Evaluation ( N = 79



Findings: BA 497 did not submit student Reflective Essays. Their assessment data was performed on their Research Paper using 2 dimensions on the rubric “Use information effectively to accomplish a specific purpose” and “Access and use information ethically and legally”. The students enrolled in BA 497 scored equally with Seaver students on “Use information effectively to accomplish a specific purpose” and higher than Librarian Evaluation. While they scored equally with Librarian Evaluation and lower than Seaver Students on “Access and use information ethically and legally”.

50


BA 497 – Spring 2018 BA 497 Scores Overall


1%

4% 41% 53% 2%

■ ■ ■ ■



Capstone (4 pts)

Milestones (3 pts)

Milestones (2 pts)

Benchmark (1 pts)


8% 81% 10% 1%






Findings by

Dimension


M i l e s to ne s (3 ) s c or es gr ea t l y exc eed t he es t a b l i s hed benc hma r k . M i l e s to n e s (2 ) a nd B e n ch m ar k l ev e l s c or es ar e f a r be l ow t he es t a b l i s hed benc hma r k . C ap s to n e l ev e l s c or es a r e a bov e t he b enc h ma r k .


M i l e s to n e s (3 ) a nd C ap s to ne l ev e l s c or es a r e wi t h i n t he es t a b l i s hed benc hma r k . M i l e s to n e s (2 ) l ev e l s c or es a r e a bov e t he es t a b l i s hed benc h ma r k . B e nc h m ark l ev e l s c or es a r e be l ow t he es t a b l i s hed benc h ma r k .

51

core competency data report information literacy › oie › content › pdf › core...core...

Documents