OCTOBER 2011
Considerations forAssessing Program
Objectives
Malcolm LeMayDirector of OperationsCollege of Business
Considerations and approaches Focus on accreditation requirements and
expectations Minimize burden on faculty while ensuring
participation Close the loop from assessment to improvement Ensure value-added activities and focus on
tracking/reporting relevant info Use technology to reduce confusion, ensure
consistency and improve process
Assessment of Learning – Lessons Learned
In other words…
1. What will our students learn in our program? What are our expectations?
2. How and when will they learn it?3. How will we know they have learned it or
not?4. What will we do if they have not learned
it?
Process for Assessment of Student Learning
1. Define goals and objectives2. Align curricula with goals3. Identify instruments and measures to
assess learning4. Collect, analyze and disseminate
assessment information5. Use assessment information for
continuous improvement
Concept of Accreditation Maturity Model
Capability Maturity Model:1. Initial - ad hoc2. Repeatable practices3. Practices defined4. Managed and measured5. Optimizing
Developed interpretation of AMM levels for all AACSB standards
Training during faculty retreat 18 months prior to Accreditation visit
Conducted self-assessment on AMM levelsResults
Increased faculty awareness and buy-in Process gaps identified Improved understanding of expectations
Accreditation Maturity Model
Applying AMM for AssessmentStandard 16 – Learning Goals Adapting expectations to the school’s mission and cultural
circumstances, the school specifies learning goals and demonstrates achievement of learning goals.
Learning goals address: General knowledge and skills (i.e., communications, problem-
solving, ethical reasoning) Management or discipline specific knowledge and skills
(entrepreneurial perspective, professional behavior)
Basis for judgment: Demonstrate that students meet the learning goals If learning goals are not being met, demonstrate efforts to eliminate
the discrepancy
Applying AMM for Assessment
Level 1 – General objectives exist and are measured through traditional course evaluation (initial)
Level 2 – Program objectives exist, assessment of learning relies on indirect measures (repeatable)
Level 3 – Program objectives integrated with curriculum development; relies on direct assessment (defined)
Level 4 – Assessment process generates quantitative and qualitative data, improvements are planned and implemented (managed)
Level 5 – Assessment is an ongoing process aimed at continuous improvement of programs, curricula and course delivery. Processes are periodically reviewed and improved (optimizing)
OSU Vision/Mission
COB Vision/MissionAssessment
Approaches:1.Direct assessment of
course outcomes mapped to program objectives
2.Direct assessment of program objectives through course embedded measures (e.g., exams, projects, papers)
3.Direct assessment through stand-alone performance (MBA oral exams)
4.Indirect assessment of program objectives (student surveys)
Program Goal
#1
Program Objectiv
e 1.1Progra
m Objectiv
e 1.2
Program Goal
#2
PO 2.1
PO 2.2 Results:Exceeded – xx%Met – xx%Below – xx%
Responses/Actions:Changes to curriculum, pedagogy, coverage, prerequisites
Impact on program offerings:New courses, changes to sequencing, adjustments to program objectives and course outcomes
Faculty Involvement in Assurance of Learning Process
Courses Courses
Mapping Core Courses to Program Objectives
Mapping Course Outcomes to Objectives
Example (Business Administration)Program Goal #1: Students will identify,
assess and shape entrepreneurial opportunities in a variety of contexts. Program Objective 1.1: Recognize entrepreneurial
opportunities for new business ventures and evaluate their potential for business success. Course Outcome (BA-260.6): Apply venture
opportunity screening techniques to an actual start-up idea, and subsequently develop and prepare a feasibility plan. Course Outcome Assessment: Measured by
evaluating team projects requiring a presentation analyzing the feasibility of a new business concept.
Measurement methodsDirect and Course
Embedded: Individual/team presentations Oral exams Mock interview reviews Group collaboration assignments Simulation results Essay exam questions Course entrance exams MBA business plan competition
Indirect Exit surveys Recruiter feedback
Use of TechnologySimplify collection, analysis
and reportingEnsure consistencyImprove overall awareness of
learning and student successEase burden on facultyReduce reliance on Excel
spreadsheets
Goal 2 - Professional communication and behavioral skills
2.1 Communicate effectively and professionally in business situations through physical or virtual presence, writing, speaking, listening, and electronic media.
2.2 Demonstrate the ability to lead by using team building skills and facilitating collaborative behaviors in the accomplishment of group goals and objectives.
2.3 Demonstrate good work habits, time management and self discipline.
Assessment Findings Written communication skills
assessed below desired level Oral communication skills for
professional situations assessed below desired level
Limited opportunities to expose students to virtual collaboration.
Students lack professional behavioral skills
Students’ leadership skills assessed as less than strong
Desire greater participation in experiential learning
Actions Taken Added BA 353 Professional Development
course to the business core as a “Writing Intensive Course” to provide students with the opportunity to learn business communications earlier in the curriculum.
BA 353 added assessment of oral and presentation skills through mock interview sessions.
BA 370 added course projects using SharePoint to emphasize virtual collaboration skills.
BA 352 revised to emphasize practical skills in leadership and team building. The course is now taught exclusively to business students.
Implemented a professional development program which has improved the awareness levels of internship opportunities through greater publicity efforts, created a system for monitoring the number of internships undertaken by students during the year, and provided a year round program of professional development .
Target level 4 for all programsOngoing
Involve faculty in program mapping and analysis Use technology to reduce support requirements,
encourage consistency and ease demands on faculty
Improve reporting and follow-up to “close the loop”
Keep in mind “big picture” - continuous improvement
Aspire for “accreditation-ready”
Moving Forward with Program Assessment
Back up slides
1. Knowledge: Recall data or information
2. Comprehension: Understand the meaning, translation, interpolation, and interpretation of instructions and problems.
3. Application: Use a concept in a new situation or unprompted use of an abstraction.
4. Analysis: Separates material or concepts into component parts so that its organizational structure may be understood.
5. Synthesis: Builds a structure or pattern from diverse elements. Put parts together to form a whole, with emphasis on creating a new meaning or structure.
6. Evaluation: Make judgments about the value of ideas or materials
Bloom’s Taxonomy of Learning Domains
