student learning outcomes for student services modesto junior college january 2005 sue...
TRANSCRIPT
Student Learning Outcomes Student Learning Outcomes for Student Servicesfor Student ServicesModesto Junior College
January 2005
Sue Granger-DicksonCounselor
Bakersfield Community CollegeTerrence Willett
Director of ResearchGavilan Community College
Learning Outcomes for Learning Outcomes for TodayToday
Summarize history of accountability and how new standards came to be
Define learning outcomes and explain the underlying philosophy
Describe the assessment cycle and its role in the college
Write appropriate student learning outcomes for your program
Explain at least two methods to assess at least one student learning outcome including needed resources and potential barriers
Identify at least 3 resources to help you succeed in the learning outcomes challenge
Your expectations and Your expectations and needsneeds
HistoryHistory 1960’s– In the past, year-end reports demonstrated proper
management but not whether the programs directly aided student learning
– Colleges measured enrollments, participation rates, budget expenditures
1990’s– More recently have expanded to measure success,
retention, persistence, graduation, and transfer– Focus has also been on instructional delivery versus
student learning 2000’s
– Student Learning Outcomes Present state of MJC
Standard IIBStandard IIB The institution recruits and admits diverse students who
are able to benefit from its programs, consistent with its mission. Student support services address the identified needs of students and create a supportive learning environment. The entire student pathway through the institutional experience is characterized by a concern for student access, progress, learning, and success. The
institution systematically assesses student support services using student learning outcomes, faculty and staff input, and other appropriate measures in order to improve the effectiveness of these services.
What are Learning What are Learning Outcomes?Outcomes? “Learning outcomes” not a new concept
– Used in California in K-12, special ed, voc programs– Used in other college systems (i.e. Maryland,
England)– Barr and Tagg called for learning outcomes in 1995
Focuses more on students’ learning rather than just how we instruct or support them; outcome based
Emphasis on higher more complex level of learning
Clearly states educational intentions for students
See example SLO in Appendix E in handout
Sounds like behavioral Sounds like behavioral objectives!objectives!
Learning outcomes differ from behavioral objectives by – Being broadly stated– Having instructors assess gains in skills rather
than creating detailed list of specific topics and abilities to be mastered (Harden 2002)
Some authors do not perceive that the difference between learning outcomes and behavioral objectives is important (Prideaux 2000)
Important to distinguish the two to gain acceptance
6 Principles – O’Banion6 Principles – O’BanionThe Learning College…The Learning College…
…creates substantive change in individual learners …engages learners in the learning process as full
partners who must assume primary responsibility for their own choices
…creates and offers as many options for learning as possible
…assists learners to form and participate in collaborative learning activities
…defines the roles of learning facilitators in response to the needs of the learners
…succeeds only when improved and expanded learning can be documented for learners
Levels of AnalysisLevels of Analysis
InstitutionalDepartment or ProgramClass or ServiceAssignment or Student Interaction
New PerspectiveNew Perspective “We must treat the college as a learning
laboratory” – John Baker, VP SS Gavilan Dealing with whole student-their
uncertainty, response to that uncertainty, and their wisdom and courage-all matters of being (Harden 2002)
Shift from instruction as “core of the college” towards learning as central focus
True marriage between academic and life skills learning
Institutional ConcernsInstitutional Concerns
Students have a fundamental right to services and if these services are not or cannot be related to learning outcomes, their existence could be unfairly jeopardized (Collins 2002)
Uncritical application of business models and concepts to education
Fear all this measuring will be a waste of effort and not enhance student achievement – Luna, Gavilan College Academic Senate
Do students really care about learning? - Borden
Assessment at Bakersfield Assessment at Bakersfield CollegeCollege
2001-Outside audit-impetus to become a learning college
2002-Program & Budget review transformed into IEC
2002-Faculty sent for trainingJuly, 2002-New accreditation
standards
BC Assessment BC Assessment continued…continued…
2003-Campus wide audit– Development of mission statement– Description of how they IMPROVE
student learning– Opening Day workshop-each faculty
member write one SLO
BC Assessment BC Assessment continued…continued…
2003-Campus-wide cross disciplinary instruction workshops on writing SLO’s
Academic Senates adopts assessment as one of its goals– Philosophy statement developed– Creation of assessment plan at AAHE– Interviews of 41 concerned faculty
members
BC Assessment BC Assessment continued…continued…
2004-Student services SLO Workshops
General Education Committee adopts draft SLO’s
Critical Thinking Committee adopts draft SLO’s
Assessment coordinator selected with reassigned time
Gavilan’s ApproachGavilan’s Approach
Discussion began in Student Services Had SLO presentations at convocation and
meetings of department and program leads as well as attending external workshops
Institutional learning outcomes selected Course level learning outcomes written Currently developing program level
outcomes Next steps include publishing outcomes in
catalog and implementing assessment of outcomes
Gavilan Institutional Gavilan Institutional Learning Outcomes Learning Outcomes
(from Palomar College)(from Palomar College)CommunicationCognitionInformation CompetencySocial InteractionsAesthetic ResponsivenessPersonal Development and
Responsibility
Gavilan Institutional Gavilan Institutional Learning OutcomesLearning Outcomes
Communication– Listening– Reading– Writing
Gavilan Instutional Gavilan Instutional Learning OutcomesLearning Outcomes
Cognition– Analysis & Synthesis– Problem Solving– Creative Thinking– Quantitative Reasoning– Transfer of Knowledge & Skills
to a New Context
Gavilan Instutional Gavilan Instutional Learning OutcomesLearning Outcomes
Information Competency– Research– Technological
Gavilan Instutional Gavilan Instutional Learning OutcomesLearning Outcomes
Social Interaction– Teamwork– Effective Citizenship
Gavilan Instutional Gavilan Instutional Learning OutcomesLearning Outcomes
Aesthetic Responsiveness– Differentiate between people who
can sing and people who can’t e.g. Brittney Spears vs. Norah Jones
– Distinguish between art that offends you, art that is cheesy, art you can’t understand, and art that is too expensive
Gavilan Instutional Gavilan Instutional Learning OutcomesLearning Outcomes
Personal Development and Responsibility– Students will develop individual responsibility, personal
integrity, and respect for diverse people and cultures– Self-management: Students will demonstrate habits of
intellectual exploration, personal responsibility and physical well being
– Ethics and Values: Students will demonstrate an understanding of ethical issues that will enhance their capacity for making sound judgments and decisions
– Respect for Diverse People and Cultures: Students will respect and work with diverse people including those with different cultural and linguistic backgrounds and different abilities
Service →Learning Outcome ↓
Communication
Student understanding and explaining goals/ plans; use advice from counselor
Student discussing and understanding requirements with staff
Student explaining what is desired: registering, transcripts, adding, etc.
CognitionUnderstand GE patterns; Develop ed plan
Calculating income, need, and unit loads
Calculating units and fees
Information Competency
Using web to gather program information
Comprehension of forms and instructions
Comprehension of forms and instructions
Social Interaction
Working with college personnel
Working with college personnel/ mentoring
Working with college personnel/ mentoring
Aesthetic Responsiveness
Form design, web page layout
Form design, web page layout
Form design, catalog, schedule web pages layout
Personal Development
And Responsibility
Understand concept of balancing school, work, and family
Provide accurate information; meet deadlines
Provide accurate information; meet deadlines
Counseling Financial Aid Admissions and Records
Opportunities for Student Opportunities for Student ServicesServices
Support students in taking more responsibility for their own learning = framework for action (Barr and Tagg 1995)
Acknowledges that Student Services personnel are also educators
Your Turn! Your Turn!
Think of an instance you directly observed a student learning while interacting with your program or service outside of a classroom experience
Write a mission statement for your program or service
Writing Learning Writing Learning OutcomesOutcomes
Should be based on our mission and values
In an environment that nurtures creativity and intellectual curiosity, Gavilan College serves the community by providing a high quality learning experience which prepares students for transfer, technical and public service careers, life-long learning, and participation in a diverse global society
Writing Learning Writing Learning OutcomesOutcomesOutcomes must be observable so
you can measure or record themCourse level learning outcomes
sound like behavioral objectives (or really are the same) but LO tie into institutional learning outcomes
Keep in mind that outcomes are tools to guide evidence collection for assessment
Assessment CycleAssessment Cycle
Create or Modify Goals and Outcomes
Implement Programs or Strategies
Evaluate
Draft BC Assessment PlanDraft BC Assessment Plan
Class and Course Assessment Senate Assessment Committee & Curriculum Committee
Program Assessment Program Review
Institutional Assessment Institutional Effectiveness Stu
dent
Service
s &
Learn
ing
Support
Service
s
Assess
ment
Closing the LoopClosing the Loop
Develop, modify, or review a curriculum, course/program.
Develop Student Learning Outcomes
Design & Measure Student Learning as a result of the Curriculum, Course, or Program
Collect, discuss, and analyze data.
Determine refinements based on outcomes data.
Closing the Assessment Loop
Dimensions of EvidenceDimensions of Evidence
Quantitative or qualitative– Not everything that can be counted counts and not
everything that counts can be counted -Einstein Direct or indirect Norm- or criterion-referenced Should be representative and relevant Need several pieces of evidence to point to a
conclusion– e.g. Student complains of fever and aches, their
temperature is 102º F, tonsils are not inflamed, eyes are red and irritated, posture appears weak. Notice mix of types of evidence that all point to same conclusion…flu!
Common Assessment Common Assessment MethodsMethods Tests
– Locally developed or Standardized Performances
– Recital, Presentation, or Demonstration Cumulative
– Portfolios, Capstone Projects Surveys
– Attitudes and perceptions of students, staff, employers Database Tracked Academic Behavior
– Grades, Graduation, Lab Usage, Persistence Embedded Assessment
– Using grading process to measure ILO Narrative
– Staff and student journals, interviews, focus groups
Example Method
Randomly assign
students to service
Randomly assign
students to receive
information about services
Correlate use with
performance or skills
measures
SurveysCase study and
journals
Strength of Evidence
Can claim causality
Weaker causality claim
Causality cannot be claimed, useful in conjunction with other information
Causality cannot be claimed, useful in conjunction with other information
Causality cannot be claimed but complex and difficult to measure effects can be noted
Ethical Consideration
Denies access to a service that may or may not be effective for some students
All students have access but some receive less information
No restriction of access or information
Use student time to complete survey
Confidentiality
Embedded Assessment Embedded Assessment ExampleExampleCriteria Joe Jane Liam Celeste Avg
Arithmetic with fractions
3 4 5 4 4
Calculating basic areas
3 4 3 5 3.75
Order of operations
3 4 5 5 4.25
Right triangles
3 4 4 4 3.75
Total 12 16 17 18
Student Grade
C B B A
Down for grades, across for outcomes assessment
–after Nichols
Assessment TipsAssessment Tips Collect data from a representative sample
rather than everyone in population Collect only a few well chosen pieces of
data Have assessment stem from activities you
already do if possible Team up with others to share assessment
work Design activities and outcomes with
assessment in mind Ask for advice from your colleagues -
especially vocational faculty who relate to learning outcomes as core competencies
Writing Learning Writing Learning OutcomesOutcomes Good verbs (Blooms’ Taxonomy):
– Compile Arrange Classify– Analyze Identify Operate– Design Solve Write– Apply Differentiate Calculate– Demonstrate Formulate Compose– Explain Predict Assess– Compare Estimate Critique
Bad verbs– Know Understand Appreciate Learn
Good verbs are clear and observable, bad verbs are vague states of mind
Example Learning Example Learning Outcomes at Outcomes at
program/service levelprogram/service level Disabled Student Services
– The student will be able to explain his/her individual academic strengths and weaknesses
Maryland Health Education – Students will demonstrate an understanding
of health promotion and disease prevention concepts to establish a foundation for leading healthy, productive lives.
Admissions and Records– Students can “work the system”– Demonstrate patience while waiting in line
More Student Service More Student Service ExamplesExamples
From Ventura College SLO Workshop– DSPS = Students will demonstrate self-
advocacy skills with instructors and staff– EOPS = Student will identify and describe
resources available on the campus. – Counseling = Student will be able to state
informed academic goal(s). – Financial Aid = Faculty and staff will be able to
describe basic aspects of financial aid available to students and how students can apply for financial aid.
Gavilan Counseling SLO’sGavilan Counseling SLO’s
Students will identify college resources, procedures, and policies that support their academic success. (Cognition)– Students will complete the college orientation and score
at least an 80% on the orientation quiz. Students will use technology to gather, process,
and articulate career options and choices. (Info Comp)– Students referred to the Eureka system (online/center)
will complete the assessments and discuss outcomes with counselor. (track completion on SARS)
– Students enrolled in Guidance 1 will utilize the internet for research and declare a major at the end of the course.
Instructional Program Level Instructional Program Level SLO ExamplesSLO Examples
Sinclair College– Math = Apply mathematical models
to real world problems– Amer. Gov. = Communicate the
significance of facts, concepts, and ideas in spoken and written English, which is clear, precise, and logical
– ECE = Demonstrate professionalism in the child care setting
More Instructional PSLOMore Instructional PSLO
Parkland Community College– English = Students will create college-level
written text for multiple purposes and diverse audiences that demonstrate depth of critical thought and that observe the conventions of standard written English.
– Psychology = Our students will apply major theoretical perspectives and methodological approaches used in the discipline of psychology to particular social phenomena or autobiographical circumstances.
Your Turn! Your Turn!
Write at least 2 learning outcomes for your program and at least 2 assessment strategies for each outcome
Reflect on Your SLO’s:Reflect on Your SLO’s:
Will the student understand? Is this truly an overarching outcome? Can you blend? Does this take into consideration the
other areas your program interfaces with?
If you had only 6 outcomes for your program, is this one of them?
In reality, can this be done?
Questions to ContemplateQuestions to Contemplate
Why should I use student learning outcomes for evaluating my program?
What are some strategies for writing effective learning outcomes?
What are some differences between qualitative and a quantitative measurements?
How can anecdotal reports or case studies be used in program assessment?
What are the pros and cons of using sampling methods for assessment?
Remember:Remember:Everyone is a
learner when it comes to
assessment
Creating Venues for Creating Venues for DialogueDialogue
Invite outside speakersUse department/committee
meetingsHold lunch meetingsSend people for training to create
championsHold cross disciplinary workshopsConduct campus-wide workshops
AcknowledgementsAcknowledgements Gavilan Student Services Division
especially:– John Baker, Margery Regalado, Jane Maringer
Gavilan Instructional Division especially:– Marty Johnson– Marlene Dwyer & her Curriculum Committee– Department Chairs– Academic Senate, Enrique Luna
Research and Planning Group especially:– Harriett Robles, Brad Philips, Fred Trapp, Bob
Gabriner, Jerry Rudman WASC
– Darlene Pacheco, Barbara Beno
Thank YouThank You