assessment: developing learning outcomes
DESCRIPTION
Assessment: Developing Learning Outcomes. Teaching Learning Assessment Cycle. Establish Learning Goals Provide Learning Opportunities Assess Student Learning Use the Results. Establish Learning Goals Meaningful Learning goals should lead to the improvement of teaching and learning . - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/1.jpg)
Assessment:Developing
Learning Outcomes
![Page 2: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/2.jpg)
Teaching Learning Assessment Cycle
• Establish Learning Goals
• Provide Learning Opportunities
• Assess Student Learning
• Use the Results
![Page 3: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/3.jpg)
Establish Learning GoalsMeaningful Learning goals should lead to the improvement of teaching and learning
• Institution
• Program
• Course
![Page 4: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/4.jpg)
Institution: UCCC’s Mission Statement:SUNY Ulster is a vibrant community of learners distinguished by academic excellence, collaboration, innovation, service and responsible use of resources. As a public, comprehensive two year institution dedicated to providing affordable accessible education, we work in an ever changing environment to:
-Prepare students to transfer to four institutions-Prepare students for success in college and in the workforce-Provide enrichment and life long learning opportunities-Augment learning through the integration and application of emerging technologies-Prepare students to live and work in a global society-Play an active role in economic development, and-Enhance the quality of life for residents in Ulster County
![Page 5: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/5.jpg)
Benefits of Learning Outcomes
• Helps students understand the nature of the program or course and what skills or knowledge will be expected of them.
• Provides a guide to students so they are assured they have not missed any major content.
• Gives structure to the instructor in lecture development – what to teach
![Page 6: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/6.jpg)
Gives structure to the instructor in exam or assignment development – link to goals. Insures the student learning outcomes are linked to the college mission. Enables the public and four year schools to understand what the program or course intends to accomplish. Most importantly – leads to improvement of teaching and curriculum development.
![Page 7: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/7.jpg)
Developing Learning Goals:
• Meaningful learning goals will lead to improvement of teaching and learning.
• There should be an interconnectedness of goals in programs and courses.
![Page 8: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/8.jpg)
Developing Learning Outcomes
• Learning outcomes should be developed collaboratively from research and reflection.
• Collaborate:• This ensures the integration of course
outcomes to program outcomes. As many faculty as possible should be involved.
![Page 9: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/9.jpg)
Research• Internal Sources:–Mission statement–Current Syllabi–Previous program reviews and surveys – External Sources:–Associations–Advisory Councils–Other colleges Program descriptions (websites)
![Page 10: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/10.jpg)
• External Sources:
Advisory CouncilsOther colleges Program descriptions (websites)
Associations
![Page 11: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/11.jpg)
Reflection:
• Key word: WHY.
• Why do you want your students to know what you put down as a learning outcome.
![Page 12: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/12.jpg)
Middle States Recommended Discussion Topics:
• What is this course or discipline all about?
• What does the department aim to achieve with this program?
• What do we value about our discipline?
• What are the most important things students learn in this discipline?
• Why are those things important?
• What does our disciplinary association think students should learn?
![Page 13: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/13.jpg)
What do we want students to get out of this program?
How does this program relate to the college mission?
How does it relate to other programs?
What do our students do after graduation?
Why does the college offer this program?
Why is it important student study it?
How will this experience prepare them for post graduation?
What do we expect of students in this program?
![Page 14: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/14.jpg)
How will students be different after completing the program?
What will the student be able to perform after the completing the program?
What skills will the students acquire after the completing the program?
What indicators will be used to measure the student’s performance?
![Page 15: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/15.jpg)
Knowledge
Thinking
Attitudes
![Page 16: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/16.jpg)
Knowledge• Duplication of procedure, learning definitions
summarizing characteristics, understanding components.
• Memorization
• Not as stressed today as in previous decades
![Page 17: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/17.jpg)
Examples of Knowledge Goals
• Explain the three goals of a macro economy and how they are measured.• Summarize distinctive characteristics
of a various novelists• Describe the …scientific model
![Page 18: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/18.jpg)
Thinking
• Application of knowledge in a new context• Analysis or the ability to understand the relationships
amongst the components of a complex concept• Evaluation or making informed judgments on new
material through research and problem solving• Synthesis or the ability to theorize, generalize, reflect
generate new hypothesis or ne ideas.• Critical Thinking or seeking truth and ability to
question
![Page 19: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/19.jpg)
Examples of Thinking Goals• Explain a reaction• Explain the impact• Analyze errors• Compare and contrast• Explain why• Judge the effectiveness • Evaluate the validity
![Page 20: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/20.jpg)
Research and identify
Choose the appropriate procedure
Write a poem using..
Theorize design
Recognize
Design and conduct….
Identify a problem…..
![Page 21: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/21.jpg)
Attitudes • Dispositions and values• Appreciation of the subject• Enjoyment of learning• Metacognition• Formation of opinions and values • Development of a sense of ethics.
![Page 22: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/22.jpg)
Example of Attitude Outcomes
• Be intellectually curious• Choose an ethical course of action• Appreciate the merits of …• Identify the strengths and weakness of others’ work• Work independently• Work in groups• Develop leadership skills
![Page 23: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/23.jpg)
Writing Learning Outcomes
• Avoid broad nebulous terms:–Students will:
–Learn –Know–Understand–Become aware–Appreciate–Think critically
![Page 24: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/24.jpg)
Action words help students understand what we want them to learn and it makes assessment easier
• Describe the solar the key characteristics of each planet in the solar system
• Make appropriate references and deductions from biological information
• Develop graphic spreadsheet to support a decision
• Design a chemical experiment to test a hypothesis
![Page 25: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/25.jpg)
Systematically analyze and solve problems…
Defend one’s views
Present original interpretations…
Critically evaluate the effectiveness…
Apply basic problem solving skills…
Conduct research
![Page 26: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/26.jpg)
Assessment of the Learning Outcomes
![Page 27: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/27.jpg)
Two Categories of Measures
• Direct measures provide evidence of whether or not the student has command of the subject matter
• Indirect Measures provide data that is related to the act of learning. It implies the student has learned, i.e. how the student perceives whether or not he/she has learned. Questionnaires and surveys are indirect.
![Page 28: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/28.jpg)
Direct vs Indirect
• Direct evidence should
answer the questions what did the student learn? What did he/she not learn? But it does not answer the question why did the student learn it? Why didn’t the student learn it?
• Indirect evidence focuses on the learning process. Comments about homework or lectures or group work.
![Page 29: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/29.jpg)
Choosing Evaluation Instruments:
QUESTIONS TO ASK:
• 1. Is the evidence provided by the method linked to learning outcomes?
• 2. Is a standardized instrument appropriate for the learning goals of
the institution
• 3. Is the evaluation method comprehensive enough? • 4. Are important learning outcomes measured by multiple means?
![Page 30: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/30.jpg)
5.Do the test questions test what is desired to be learned?
6. Are the questions clear?
7. Do the questions elicit information useful for making improvements?
8. Does everyone interpret the response the same way?
9. Do the results make sense? Are the results corroborated by evidence?
![Page 31: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/31.jpg)
Tools and techniques:
![Page 32: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/32.jpg)
TestsStandardized vs Home grown
Quote from Middle states:Although a test created locally may not have the same statistical validity and reliability as a standardized instrument, it’s relevance to the specific learning goals in question may make it a more appropriate and more effective instrument.
![Page 33: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/33.jpg)
Test blueprint is an outline that matches test items to the intended learning outcomes.
![Page 34: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/34.jpg)
Rubrics
• A set of criteria used to assess papers, projects, performances, field experiences.
• Four Types: •Checklist•Rating Scale•Descriptive•Holistic
![Page 35: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/35.jpg)
Checklist Rubrics
• A list of yes or no or Did or Didn’t do
• Used primarily in lab or clinical settings
![Page 36: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/36.jpg)
Rating Scale Rubric
• This is a checklist with a rating scale added to show the degree to which the things you’re looking of are present
• Uses terms like Outstanding, Very Good Adequate Or
• Strongly Agree, Somewhat Agree etc..
![Page 37: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/37.jpg)
Descriptive Rubrics
• These give a description of what constitutes deficient adequate or exemplary
• Or unacceptable, Fair Proficient
![Page 38: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/38.jpg)
Holistic Rubrics
• For the more massive assessments, rating scale and descriptive rubrics may not be feasible i.e reading 1000 essays for placement.
• Holistic rubrics have short narrative descriptions of the characteristics for outstanding work, acceptable work, and unacceptable work
![Page 39: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/39.jpg)
Other Assessment Tools
• Self reflection
Engage in metacognition. Ask students to reflect on what they have learned
This gives insight into the learning process and it helps students integrate what they have learned
![Page 40: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/40.jpg)
Ratings and comments from internship supervisors
Placement Rates
Ad Hoc Surveys
Focus Groups
Portfolios
![Page 41: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/41.jpg)
What is Middle States Looking For?
• They will ask the fundamental questions: • • Is the Institution fulfilling it’s mission?• • Are institutional and program level goals clear to the public, students,
faculty and staff?• • Are Institutional programs and resources organized and coordinated to
achieve these goals?• • Is the Institution using assessment results to improve student learning and
advance the institution.•
![Page 42: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/42.jpg)
Characteristics of Assessment Processes that meet Middle States Expectations
• Useful Lead to improvement• Cost Effective No need to start from scratch use what is there• Reasonably Accurate Is evidence clear and visable -both direct and indirect• Planned Assessment must be interrelated to strategic plan and academic plan• Organized, systemized and sustained Implement an ongoing system and review it.
![Page 43: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/43.jpg)
What Should Be Documented?
• Learning outcomes in programs and courses . • An assessment Plan
• Assessment results
• Uses of results to show what is being changed towards improvement
![Page 44: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/44.jpg)
Assessment: Using the Results
To
Improve Teaching and Learning
![Page 45: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/45.jpg)
Three steps:
Choose a standard benchmark
Set a standard or benchmark for exemplary, adequate and inadequate
Setting targets for the collective class or program
![Page 46: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/46.jpg)
Choosing a Benchmark
![Page 47: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/47.jpg)
What is a Benchmark
• The process of comparing the results of your test, essay, paper or other assignment against something.
![Page 48: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/48.jpg)
Local Standards:
Department members agree to what is a good grade on a specific exam. Having faculty involved in courses and exams leads to more attention paid to assessment. External Standards:
This is the use of disciplinary association standards. Usually there are tests that are required from the association. Certifications have required exams with established benchmarks i.e. NCLEX
![Page 49: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/49.jpg)
Internal peer benchmarkThe bell curve – or looking at how Psychology
majors do on the economics test vs business majors. Or looking at all the students in the program. Which ones succeed and why?
External peer benchmarkWe compare Jane Doe’s 65 to other instructors’ or colleges’ 65.
![Page 50: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/50.jpg)
Setting Standards:
• Choose the bench mark most appropriate for your course or program and set a specific appropriate performance standard.
• Ask what is an example of exemplary work?• How does this differ from adequate and
inadequate?
![Page 51: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/51.jpg)
Setting Targets for Students’ Collective Performance
![Page 52: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/52.jpg)
What do you want the class or students in a program as a whole achieve?
• Express targets as a percentage. Ex: 90 percent of students earn at least a minimally adequate rating
• Or 95 percent of the students will earn a 65 % on all exams.
![Page 53: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/53.jpg)
Summarizing the Results
![Page 54: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/54.jpg)
Five ways to summarize results:
Tallies
Percentages
Aggregates: Scores
Averages
Qualitative Summaries
![Page 55: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/55.jpg)
Using The Results
![Page 56: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/56.jpg)
Identify areas of strengths and weaknesses
Compare results with what you wanted to hear
Create strategy for improving the learning experience
Base changes on real data
![Page 57: Assessment: Developing Learning Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/5681628e550346895dd2fb7a/html5/thumbnails/57.jpg)
Empower faculty to make those changes
Articulate exactly who is responsible for the changes
Share the results and actions with the whole institution