Learning Outcomes,Assessment Plans and Annual

ReportsJune 3, 2011Nancy P. Goldschmidt, PhD, Associate Vice Provost, Academic Affairs

Thomas Boudrot, EdD, Director, Teaching & Learning CenterSusan Adams, MA, Faculty Development Specialist, Teaching & Learning

CenterSarah Melton, MEd, Faculty Development Specialist, Teaching & Learning

Center

2

Meeting Purpose

1. Increase awareness about new OHSU policies– Program-level learning outcomes (second wave due January 15, 2012)

– Assessment Plans (based on approval of outcomes)

– First Annual Assessment Report (first wave due November 2012; all others due November 2013)

– Academic program review (every 5 years)

2. Review parameters for how to implement – Assessment Plan (and Report )Template

– Implementation Timeline

3. Apply standards to outcome statements developed by OHSU faculty.

4. Identify how to develop learning outcomes DIY or TLC

3

What is a student learning outcome?

• Is a student/learner-centered statement • Focuses on a result of a completed process of learning

(e.g., class, module, course, core courses, program)• Reflects overt learned behavior that can be observed

and assessedAsk yourself: What should the ideal graduate of your program know, understand and be able to do when they leave OHSU and enter the workforce (or whatever the next transition)?

Then, how are you going to know they have achieved standards? What is the evidence you need? How will you gather it?

4

OHSU Policy onProgram-Level Student Learning

Outcomes & Assessment

5

What is OHSU’s policy?

Faculty are responsible for designing the curriculum for their respective degree and certificate programs, including the didactic courses, clinical components, laboratory research components, internships, and other experiences designed for students to achieve the stated learning outcomes.

Course instructors are encouraged to identify and publish expected learning objectives and competences of a specific course or module (describe how performance will be assessed in the course syllabus and how the SLOs and their assessment relate to the degree-level SLOs.)

Faculty Responsibility

6

What is OHSU’s policy?

Each school is responsible for conducting regular and systematic assessment of student performance and demonstrating that students who complete their programs have achieved the stated degree or certificate outcomes/competences.

Each school will maintain a record of the meaning and use of results of assessments on curricular development, faculty development efforts, faculty recruitment, and events recognizing student achievement.

School and Program Responsibility

7

What is OHSU’s policy?University Responsibility

The University ensures that measurable student learning outcomes are identified for each approved degree and certificate program and published in an accessible format for current and prospective students, faculty, staff, external evaluators, and employers.

At a minimum, program-level learning outcome statements should be posted prominently in several places on the institution’s website and updated regularly.

8

Why did OHSU adopt a policy requiring student learning outcomes?

1. Institutional quality standards set by Northwest Commission on Colleges and Universities focuses on learning and QA and QI

2. State, institutional and school-level processes for approving proposals for new academic programs

3. OUS 5-year Review of New Academic Programs

4. Some specialized accrediting bodies mandate specific learning competences or areas of study to meet professional standards.

5. Recommendations from Lancet: Health professionals for a new century: transforming education to strengthen health systems in an interdependent world Lancet 2010; 376: 1923–58

9

What are the benefits?

• Inform students, or prospective students or other stakeholders, of what to expect (added value)

• Provide faculty a basis for:

• assessing individual student and group learning • wording questions to assess intended learning at

appropriate levels• giving feedback to students about their performance

• Articulate what qualifications mean to employers, society and universities

• Increase transparency, career mobility and comparability at a national or international level

10

How is a student learning outcome different from a program aim or goal?

• The broad aims, purposes or goals of a degree or certificate program may already be articulated in your program descriptions.

• Compared to aims, learning outcomes (or objectives) are more concrete and specific statements of what learners are expected to do, think or value as a result of the process of learning and teaching.– What are essential things students need to learn in the process? – What changes should come about in their knowledge, attitudes and

skills? – What will graduates be prepared to take on in their career/job/

profession?

11

Standards for Writing Student Learning Outcomes

12

SLO Writing Standards

1. Reflect your intentions for the level of learning or performance of all learners

2. Express as an action VERB that indicates the relation of the outcome to domains or types of learning (use the Verb dictionary as a reference)

3. Articulate in measurable terms to align later with assessments4. Cross reference professional body requirements as applicable5. Draft in the future tense

13

Conversation openers to brainstorm program-level SLOs

• The most important values which inform this program are...

• The academic content of this program concentrates on...

• The most important intellectual skills developed in the program are...

• The most useful practical skills, techniques and capabilities developed

are…

• Competency will be developed in...

• The most important ways in which a student will learn are...

• On completing the program we want students to know and understand...

• On completing the program we want students to be able to....

14

Due Dates

15

Process and Timeline

Activity Complete Responsibility

Complete Assessment Plan (1-3) TBD school Program faculty complete

Submit AP 1-3 for school-level review TBD school School structure/process

Submit AP (1-3) for institution review 15-Jan -12 Assessment Coordinator

Review, rate and provide feedback 01-Mar-12 Assessment Council

Submit AP (1-5) to Academic Affairs 01-Jun -12 Assessment Coordinator

Review Assessment Plans (1-5) 01-Aug -12 Assessment Council

Complete/submit Assessment Report (AP 1-7)

1-Nov -12 Program faculty to Assessment Coordinator

Review Assessment Reports (AP 1-7) 1-Mar -12 Assessment Council

Implementation Wave Two

Ongoing revision AP 1-3 Ongoing Program faculty resubmit

Review revised submissions Ongoing Assessment Council

Ongoing AP (1-5) revisions to meet assessment standards

Ongoing 6-months after learning outcomes approved

2012 -13 Assessment Report Due 01-Nov -13 6-month clock begins ticking after plan approved

16

OHSU Examples

17

At the end of the Masters in Biostatistics program, all of our graduates will be able to…

1. Apply intermediate to advanced biostatistical theory and techniques to design, plan and manage data collection efforts

2. Design and carry out independent analysis of own research project.

3. Conduct data analysis independently for research projects designed by student

4. Display appropriate team skills and attitudes that foster collaborative research.

5. Translate broad goals into specifications and procedures for statistical analysis and interpretation of results in basic, clinical, translational and public health research studies

6. Select appropriate statistical analysis software for assessment, decision-making and information-sharing (e.g., Stata, SAS, R or other special programs).

7. Communicate statistical methods and findings clearly and unambiguouslyto specialists and non-specialist audiences

18

At the end of the DMD program, all of our graduates will be able to…

1. Demonstrate an understanding of the concepts of professional ethics, health care principles and a familiarity with the ADA Principles of Ethics and Code of Professional Conduct.

2. Integrate and synthesize information to advance knowledge and skills through critical evaluation of biomedical literature and the application of new science.

3. Apply interpersonal and communication skills to effectively care for diverse populations of patients.

4. Diagnose diseases and abnormalities of the oral cavity and head/neck region. Obtain and assess information relative to systemic health of patients, initiating consultations and referrals to other health care professionals as appropriate.

5. Develop a comprehensive treatment and/or referral plan, based on current oral disease risk analyses’ and standard-of-care strategies.

6. Educate patients, parents and/or caregivers with individualized instructions for improving nutrition and maintaining good oral health, monitoring and re-evaluating as necessary

19

DMD SLOs Continued…

7. Recognize risk factors for medical emergencies and demonstrate the ability to manage dental and medical emergencies.

8. Provide treatment and/or referrals as appropriate with the goal of achieving physiologic form, oral function and health.

9. Utilize pharmacological therapies and behavioral techniques to prevent or manage pain and anxiety.

10.Apply principles of risk management, quality improvement, infection control and radiation safety to patient care.

11. Demonstrate the ability to self-assess competency and evaluate the outcomes of patient-centered dental health care.

12.Demonstrate the ability to function in community based disease prevention and health promotion activities.

13. Identify the business principles and programs used in the administration of a dental practice while complying with all regulations, policies and protocols that relate to health, safety and the law.

20

At the end of the Behavioral Neuroscience program,all of our PhD graduates will be able to:

1. Apply appropriate theories and concepts to design and conduct independent, innovative research.

2. Seek out and identify funding opportunities and write competitive grant applications.

3. Demonstrate an understanding of the processes for funding and evaluation of research.

4. Understand and apply responsible conduct of research guidelines relating to issues including, but not limited to, scientific integrity, plagiarism, confidentiality, copyright, ethics, consequences of misconduct, data handling, falsification, fabrication, authorship, and use of animal and human subjects.

5. Justify one's own research to the funding bodies and promote the public understanding of one's research field.

6. Seek out and secure opportunities to teach undergraduate/graduate students.

7. Demonstrate original, independent, and critical thinking, and the ability to develop theoretical concepts.

8. Produce material that is suitable for publication.

9. Demonstrate knowledge of recent advances within one's field and related fields.

10.Effectively support the learning of others when involved in teaching, mentoring, or demonstrating activities.

21

At the end of the doctoral program in Nursing, all of our PhD graduates will be able to:

1. Conduct research that generates, tests, refines, or extends practice relevant theory and knowledge for nursing;

2. Critically evaluate and synthesize research findings for building nursing knowledge and use that knowledge for research, practice, advocacy, or policy development; and

3. Promote continued advancement of the discipline of nursing through leadership, research, and practice; and

4. Collaborate with other disciplines in health-related research that is responsible to the needs and concerns of society.

5. Provide the tools necessary for the development of an independent program of research.

6. Compete successfully for grant funding to support research goals.

7. To prepare data based manuscripts. To present research findings at local and national conferences.

22

At the end of the doctoral program in Biomedical Engineering, all of our PhD graduates will be able to:

1. Participate actively and take initiatives in research activities

2. Demonstrate analytical skills

3. Demonstrate scholarly oral and written communication skills

4. Prepare integrative papers that makes original contributions to the field

5. Create theoretical frameworks based on relevant literature

6. Identify significant and original problems that will impact human health

7. Understand qualitative designs and approaches to qualitative data analysis

8. Choose methodologies appropriate to question(s)

9. Apply data analytic techniques correctly

10.Write a paper/thesis that is clear, well organized, and

grammatically correct

23

At the end of the Program X,all of our PhD graduates will be able to:

1. Participates actively and takes initiatives in research activities.

2. States research questions(s) clearly, succinctly and chooses appropriate methodology.

3. Produces material that is suitable for publication.

4. Is knowledgeable of key literature in the field.

5. Designs study appropriate to question(s).

6. Draws appropriate conclusion from findings.

7. Identifies significant and original problems.

8. Demonstrates original, independent, and critical thinking, and ability to develop theoretical concepts.

9. Demonstrates a broad understanding of the context in which the research takes place and creates a theoretical framework based on relevant literature.

10.Demonstrates an understanding of relevant research methodologies and techniques and their appropriate application within one’s research field.

11. Demonstrates scholarly written communication skills including clear, well-organized writing and in a style appropriate to purpose.

12.Demonstrates an ability to construct coherent arguments and articulate ideasclearly to a range of audiences through a variety of techniques.

24

Program X SLOs Continued…13.Demonstrates a thorough understanding of Genetic fundamentals and the essential

literature in their specific research area.14.Demonstrates an ability to analyze critically and evaluate one’s findings and those

of others.15.Demonstrates an ability to summarize, document, report and reflect on progress.16. Is creative and innovative in one’s approach to research.17.Presents one’s research, skills, personal attributes and experiences through

effective seminars, interviews, and CVs in order to compete favorably for jobs in academia, state federal, and private or industrial sectors.

18.Demonstrates oral communication skills, including clear and organized scientific presentations.

19.Ability to continuously accumulate knowledge of recent advances in one’s specific research field and in related areas.

20.Ability to take Information from a variety of disciplines related to one’s own research and to use this information to generate new ideas and approaches for solving problems.

21.Ability to view one’s research goals and results in terms of a “bigger picture”.22.Ability to “distance one’s self” from one’s experimental results and hence take a

more objective view of the quality of the result pros and the appropriateness of the interpretations.

23.Ability to apply genetic principles to a specific research problem and to appreciate the advantages and limitations of the “genetic” approach.

25

Student Learning Outcomes & Assessment

Input fromConstituencies

Input fromConstituencies

Evaluate ObjectivesEvaluate ObjectivesEvaluate Objectives/ Assess Outcomes

Evaluate Objectives/ Assess Outcomes

Formal InstructionStudent Activities

Formal InstructionStudent Activities

Establish Indicators for Outcomes to Lead

to Achievementof Objectives

Establish Indicators for Outcomes to Lead

to Achievementof Objectives

Determine HowOutcomes will be

Assessed

Determine HowOutcomes will be

Assessed

Determine HowOutcomes will be

Achieved

Determine HowOutcomes will be

Achieved

Determine OutcomesRequired to Achieve

Objectives

Determine OutcomesRequired to Achieve

Objectives

Determine educational objectives

Determine educational objectives

26

STUDENT LEARNING OUTCOMES (PARTS 1-3)

ASSESSMENT PLAN (PART 4)

ANNUAL ASSESSMENT REPORT (PARTS 5-7)

Template

27

TEMPLATE PARTS 1, 2 AND 3 DUE JANUARY 15, 2012

Student Learning Outcomes

28

Part 1. Summary Program Description Assessment Template

Part 1: Summary Date completed:

Program Name: XXXXXX

CIP Code: XX.XXXX (Find Academic Program Inventory in Fact Book )

Program/Department Chair: XXXXX email:

Person completing report:

Student access to information: www. (web link on (1) Academic Affairs web page (2) school/program homepage, and (2) student handbook

Program length: XXXX Admit by Student Cohort: YES NO

Authorized Award(s): XXXXXXXX

Number of graded credit hours (minimum): XXX credit units

Minimum standard of performance or acceptable OHSU GPA X.X

Describe

29

Part 2: Purpose/Aim/Goal

Prompt: This program is distinctive because it develops...

Assessment Template

30

Part 3. Learning Outcome StatementsAssessment Template

Standard lead-in phrase

At the end of the <program >, all of our <level of study> graduates will be able to:

Then list outcomes

Base on Bloom’s taxonomy

of learning domains.

Guidelines1. Limit to five to ten

statements about overt behavior that can be observed and measured

2. Be broad and integrated3. Reflect essential areas of

learning, threshold concepts or ways of thinking, knowing and practicing that result from a degree or certificate program

31

TEMPLATE PARTS 4, 5, AND 6Assessment Plan

32

Assessment Plan

Part 4 Student AssessmentsSLO 1 Describe how students are assessed

SLO 2 Describe how students are assessed

SLO 3 Etc.

SLO 4

SLO 5

SLO 6

SLO 7

SLO 8

SLO 9

SLO 10

33

Assessment Plan (Parts 4 and 5)

Part 5 Standards, Criteria and Rubrics SLO 1 How do you know that a student’s overt performance meets,

exceeds or fails to meet the explicit standards?

SLO 2

SLO 3

SLO 4

SLO 5

SLO 6

SLO 7

SLO 8

SLO 9

Identify how individual student performance on the direct assessments identified above are evaluated/judged to meet standards?

What are criteria/rubrics used to determine whether performance standards are met?

Do you compare the average first-time pass rate on professional licensure exams with the national average first-time pass rate?

34

TEMPLATE PARTS 6, 7 AND 8Annual Assessment Report

35

Assessment Report (Parts 6, 7 and 8)

Part 6 Results/Evidence/FindingsSLO 1

SLO 2

SLO 3

SLO 4

SLO 5

SLO 6

SLO 7

SLO 8

SLO 9

Identify what you have learned this year as well as compared to prior years? Are any trends emerging that need attention? Do you have an explanation for the level of performance if it is lower than expected? Higher than expected? An anomaly?

36

Part 7 Feedback “Closing the Loop”Assessment Report

Part 7 Feedback “Closing the Loop”SLO 1

SLO 2

SLO 3

SLO 4

SLO 5

SLO 6

SLO 7

SLO 8

SLO 9

Identify what you are going to do, or have done, as a result of your findings. Did a faculty group review the findings? Are you going to propose changes to the program? Course sequencing? Working with student affairs staff to plan something?

37

Part 8 Developmental EvaluationAssessment Report

Part 8 Developmental Evaluation.

.

.

.

.

.

.

.

.

Based on the changes you have identified and made, what has emerged? Was it what you expected? How is it interesting? What is your explanation? Would you describe the program as making improvement? If you are looking at the component parts of the program, how is the whole different from, more than, or the same as the sum of its parts?

38

1: DIY 2: GET TLC

Options for •Writing learning outcome statements•Designing Assessment Plans

39

Steps to write SLOs

1. Select the domain that best reflects the desired learning outcome for each SLO.

2. Draw upon the list of verbs categorized by domain and academic level (relative demand, complexity, learner autonomy, and depth of study) to write learning outcomes that are measureable and specific.

3. Complete the SLO statement, adding the detail (noun) needed to communicate effectively and transparently within the discipline

40

Time Saving Hints

• Are some learning outcomes shared with other programs because they are related to core program? –Write together with representatives from all programs to avoid

reinventing the wheel.–Could all PhD programs use a similar SLO for research competency?

• Are some SLOs statements unique to the program?– Have a smaller targeted faculty group focus on these.

• It is helpful to have SLOs at the course level to facilitate the process of mapping the curriculum to program assessments and courses. They are also useful for developing assessments based on the level of learning faculty are seeking from students. When these are available, it is a grand idea to include in the course syllabus.

41

Do SLOs reflect the intended program outcomes?

• Are the SLOs applicable regardless of the mode of delivery of educational content— traditional classroom courses, Internet courses, hybrid courses, part-time study and learning accommodations?

• Do they articulate a clear sense of progression through the levels or stages of study, learner autonomy, and level of understanding. Are the statements for one level of study different from a more advanced level of study?

• How do these elements relate to students moving successfully through the program?

• Are the SLOs memorable and meaningful to students, faculty, employers and society?

• Can each be “mapped” to the required courses, electives and other learning experiences to identify any gaps in the curriculum or unnecessary redundancies? This may suggest how to revise the SLO program-level statements, individual courses, or the design of the curriculum.

42

How do I get more information?

Question Contact

How do I find the forms I need? www.ohsu.edu/xd/education/student-services/academic-programs-and-assessment/assessment-evaluation/slo-assessment.cfm What if I am having trouble downloading or

finding the form?Ryan Bushek

How do I schedule a coaching session with TLC partners?

Susan Adams, Sarah Melton, Tom Boudrot

What is the process for submitting learning outcomes and assessment plans?

Assessment Partner

Where do I get interpretations of OHSU policy? Nancy Goldschmidt

How do I find out my scores on learning outcomes or assessment plan?

Nancy Goldschmidt

Can I get more time to meet these requirements?

Assessment Partner

43

Assessment Partners

Program Assessment Partner

Dental programs Phyllis Beemsterboer

MD program Jennifer Boyd

SOM PhD and Masters Allison Fryer

Nursing programs Tanya Ostrogorsky

Nursing programs Peggy Wros

Pharmacy David Bearden

Other programs Nancy Goldschmidt

44

Verb Dictionary for Outcome Statements

45

Agreed-upon frameworkBloom’s Taxonomy of Learning Domains

1. Knowledge/Cognitive process domain:

What the learner should know.

2. Performance/skill-based/psychomotor domain:

What the learner should be able to do.

3. Affective domain: What attitudes, feelings or values the learner should have to be successful in a career or profession.

46

Cognitive-based Learning Domain

Knowledge Comprehension Analysis Applicati on Synthesis Evaluati on

Recall facts, principles, and procedures

Literal understanding of facts and principles

Identify unstated assumptions, logical fallacies, distinguish facts from inferences

Use learning to solve problems or new contexts

Integrate learning from areas to solve problems and create something new

Judge and assess based on expressed criteria, ideas, methods

Define Convert Deconstruct Demonstrate Adapt AppraiseDescribe Defend Diagram Dramatize Categorize AssessIdentify Distinguish Differentiate Illustrate Combine Compare/

Contrast

Label Estimate Distinguish Modify Design ConcludeList Explain Illustrate Operate Explain CritiqueMatch Extend Infer Practice Generate ExplainName Generalize Prioritize Prepare Organize JustifyOutline Infer Relate Produce Plan InterpretRecall Predict Select Relate Prescribe RateReproduce Review Separate Show Reconstruct RecommendSelect Summarize Sort Solve Revise SupportState Translate Transform Use Specify Test

47

Skill-based Learning Domain

Imitate Manipulate Precise Articulate Naturalize

Align Align Accurately Confidence Automatically

Balance Balance Independently Coordination Effortlessly

Follow Follow Proficiently Harmony Naturally

Grasp Grasp With balance Integration Professionally

Hold Hold With control Proportion Routinely

Place Place Smoothness Spontaneously

Repeat Repeat Speed With ease

Rest (on) Rest (on) Stability With perfection

Step (here) Step (here) Timing With poise

48

Affective Learning Domain

Receive Respond Value Organize Characterize

Attend Applaud Act Abstract Avoid

Be aware Comply Argue Balance Display

Control Discuss Convince Compare Exhibit

Discern Follow Debate Decide Internalize

Hear Obey Display Define Manage

Listen Participate Express Formulate Require

Look Play Help Select Resist

Notice Practice Organize Systemize Resolve

Share Volunteer Prefer Theorize Revise

49

KNOWLEDGE/COGNITIVE DOMAIN

Further refinement and sophisticated development of cognitive/knowledge outcomes

50

Writing Learning Objectives in the Knowledge/Cognitive Domain

A statement of a learning objective contains a verb (an action) and an object (usually a noun): The verb generally refers to [actions associated with] the

intended cognitive process.

The object generally describes the knowledge students are expected to acquire or construct.

The cognitive process dimension represents a continuum of increasing cognitive complexity—from remember to create.

The knowledge dimension represents a range from concrete (factual) to abstract (metacognitive).

Source: Lorin W. Anderson and David R. Krathwohl, Eds. (2001), A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. New York: Addison Wesley Longman, Inc.

51

The Knowledge Dimension

A Factual Knowledge – The basic elements students must know to be acquainted with a discipline or solve problems in it.

Terminology; specific details and elements

B Conceptual Knowledge – The interrelationships among the basic elements within a larger structure that enable them to function together.

Classifications and categories; principles and generalizations; theories, models, and structuresC Procedural Knowledge— How to do something, methods of inquiry, and

criteria for using skills, algorithms, techniques, and methodsSubject-specific skills and algorithms; subject-specific techniques and methods; criteria for determining when to use appropriate proceduresD Metacognitive Knowledge— Knowledge of cognition in general as well as

awareness and knowledge of one’s own cognitionStrategic knowledge; appropriate contextual and conditional knowledge; self-knowledge

Major Types and Subtypes

52

Taxonomy TableCognitive Process and Knowledge Dimension Combinations

COGNITIVE PROCESS DIMENSION

KNOWLEDGE DIMENSION

1.REMEMBE

R

2.UNDERSTAN

D3.

APPLY4.

ANALYZE5.

EVALUATE6.

CREATE

A.FACTUAL

B. CONCEPTUAL

C. PROCEDURAL

D.META-COGNITIVE

Source: Lorin W. Anderson and David R. Krathwohl, Eds. (2001), A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. New York: Addison Wesley Longman, Inc.

These two dimensions combine into 24 verbs to describe your intentions for

student learning

53

Taxonomy TableCognitive Process and Knowledge Dimension Combinations

COGNITIVE PROCESS DIMENSION

KNOWLEDGE DIMENSION

1.REMEMBE

R

2.UNDERSTAN

D3.

APPLY4.

ANALYZE5.

EVALUATE6.

CREATE

A.FACTUAL List Summarize Respond Select Check for Generate

B. CONCEPTUAL

Recognize Classify Provide Differentiate Determine Assemble

C. PROCEDURAL

Recall Clarify Carry out Integrate Judge Design

D.META-COGNITIVE

Identify Predict Use Deconstruct Reflect Create

Source: Lorin W. Anderson and David R. Krathwohl, Eds. (2001), A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. New York: Addison Wesley Longman, Inc.

UNDERSTAND/REMEMBER Verbs Cognitive Domain Dictionary

“Understanding and Remembering” VerbsAssociate Explain PredictCompare Express RecognizeCompute Extend RestateConvert Extrapolate RewriteDefend Generalize Summarize

Describe Give examples TellDiscuss Identify Translate

Distinguish InferEstimate Paraphrase

55

Cognitive Domain Dictionary

“Understand” VERBS YOU CAN SELECTAssociate Explain PredictCompare Express RecognizeCompute Extend RestateConvert Extrapolate RewriteDefend Generalize SummarizeDescribe Give examples TellDiscuss Identify TranslateDistinguish InferEstimate Paraphrase

56

“Apply” VERBS YOU CAN SELECTCognitive Domain Dictionary

Add Discover ModifyApply Divide OperateCalculate Employ PracticeChange Examine PrepareClassify Graph ProduceComplete Illustrate ShowCompute Interpret SubtractConstruct Interpolate TranslateDemonstrate Manipulate Use

57

“ANALYZE” verbs you can select

Analyze Detect InferAppraise Develop InspectBreakdown Diagram InventoryCategorize Differentiate OutlineCompare Discriminate Point outCombine Distinguish RelateContrast Examine SelectDebate Experiment TestDesign Illustrate Utilize

Cognitive Domain Dictionary

58

“EVALUATE” Verbs You Can SelectCognitive Domain Dictionary

Appraise Determine Rate

Argue Estimate Score

Assess Evaluate Select

Choose Grade Support

Compare Interpret Test

Conclude Judge Value

Contrast Justify

Criticize Measure

Critique Rank

59

“SYNTHESIZE” verbs you can selectCognitive Domain Dictionary

Arrange Create OrderAssemble Design PlanCategorize Devise PrepareCollect Explain PrescribeCombine Formulate ReconstructCompile Generate ReviseComply Integrate SetupCompose Manage SpecifyConstruct Modify Summarize