cirtl spring 2016 college classroom meeting 9: transparency

1 collegeclassroom.ucsd.edu

If these students are

thinking anything other

than “I’m bored” what

is it?

CIRTL – The College Classroom Meeting 9:

Implementing Evidence-Based Teaching Methods

March 24, 2016

Unless otherwise noted, content is licensed under

a Creative Commons Attribution- 3.0 License.

Tom Holme

Department of Chemistry, Iowa State University

[email protected]

Peter Newbury

Center for Engaged Teaching, UC San Diego

[email protected]

collegeclassroom.ucsd.edu

Thinking about our own experience

collegeclassroom.ucsd.edu 3

One way that teaching practice stays “stuck” in old paradigms is

that we often use our experience as students to inform our teaching

practice. Let’s start out today thinking about what we encountered

as students.

Thinking about our own experience


In how many classes that you have taken can you remember a teacher providing explicit guidance about how activities would help you learn how to think about the topic?

A) none

B) 1 or 2 classes

C) Between 3 and 6 classes

D) More than 6 classes

One way that teaching practice stays “stuck” in old paradigms is

that we often use our experience as students to inform our teaching

practice. Let’s start out today thinking about what we encountered

as students.

The role of expectations


- It takes some significant effort for college students to

figure out what the expectations are for a course.

- A syllabus is often crammed with so much information

that they don’t really know how to gain anything other than

the most rudimentary facts from it.

- How do we understand expectations?

- How do we mold expectations?

Expectation differences


- there is data to suggest that students have different

expectations of their courses than the professors have of

them.

Expectations research


In both physics and chemistry researchers have investigated

the differences in expectations between students and their

professors.

The chemistry project was carried out by Stacey Bretz and

her group[1] and they developed a tool called CHEMX.

The Instrument: CHEMX

Cluster Representative Statement

Effort

(α=0.85)

“I read the text in detail and work

through many of the examples given

there.”

Concepts

(α=0.73)

“When I solve most exam or homework

problems, I explicitly think about the

concepts that underlie the problems.”

Math-link

(α=0.82)

“In this course, I do not expect to

understand equations in an intuitive

sense; they just have to be taken as

givens.”

Reality-link

(α=0.86)

“It is unnecessary for me to have to

relate chemistry to the real world.”

The Instrument: CHEMX

Cluster Representative Statement

Outcome

(α=0.73)

“Only a very few specially qualified people

are capable of really understanding

chemistry.”

Laboratory

(α=0.85)

“I really don’t expect to understand how

laboratory instruments work – they are just

tools that help me complete the lab.”

Visualization

(α=0.85)

“Solving a chemistry problem may require me

to be able to draw molecules in more than one

way.”

Results: Faculty Expectations

0

10

20

30

40

50

60

70

80

90

100

0 20 40 60 80 100

Per

cen

t F

avo

rab

le R

esp

on

se

Percent Unfavorable Response

Analytical

Biochemistry

Chemistry Education

Inorganic

Organic

Physical

0

10

20

30

40

50

60

70

80

90

100

0 20 40 60 80 100

Per

cen

t F

avo

rab

le R

esp

on

se

Percent Unfavorable Response

OP Pre GC1

OP Post GC1

OP Post GC2

OP Post OC2

OP Junior

Faculty

Results: Student Expectations Medium, Open-admission Public University (n=133)

Observations

12

- Student expectations for what they will do in science

classes don’t really match the expectations of professors.

- This is particularly true for introductory courses.

- Taking those introductory courses seems to make the

gap even wider in some (perhaps many) cases!

(We showed chemistry, but physics found similar things

[2])

Why do we have to study this?


Heading into breakout rooms:

- In your discipline, how many years do undergraduates

take classes before they are mostly in rooms with other

majors? (for example: introductory physics doesn’t have a

majority of physics majors in it.)

- In your experience (as a student or as part of the

teaching team) how well, if at all, do these early (service)

courses help students make connections between their

major interests and the material covered in the course?

Embracing the “service” course


In some cases, such as the

one-semester chemistry

courses that many

engineering students take,

it is possible to design

relevance into the course.

Features we incorporated


- To make this book useful to faculty who are often

devising a new course that will use it, we concentrated on

several things.

- First, we start and end each chapter with a connection

between the chemistry and an engineering application.

- Second, we include problems that are more like an

engineering scale.

- Third, we include problems that require students to

think about how the problem is solved, not the final answer.

Domain specific vs. domain general


- One area of cognitive science that is important in things

like problem solving is the distinction between domain

general and domain specific skills.

- Many of the things we’ve been emphasizing in this

course are domain general methods.

- This doesn’t mean they cannot be applied in domain

specific ways, however.

Strategy: Be explicit about cognitive

gains that students can make


Many of the things we’ve emphasize that can help students

learn are more successful if students understand why we

are doing them.

What are the best strategies for doing this?

Transparency in learning project


- one approach is the transparency in learning project that

we had you do some reading on.

The goal is to make the learning expectations explicit for students

in as many ways as possible.

Three key components.

(1) Transparent learning outcomes

(2) Transparent assignments linked to learning outcomes

(3) Transparent classroom activities

Transparent Learning Outcomes


Problem statement:

- If the only place they appear is on a syllabus, chances are your learning outcomes are not part of what your students actively consider in your course.

Way to resolve:

- Before each class starts, include how the material to be covered that day relates to the course learning outcomes.

- If possible have this about content and skills, not just content.

Example Discussion Breakout Rooms


Each Individual by themselves first.

- Think of a pretty generalizable skill that students might

encounter in an introductory course in your discipline.

- You’ll have a white board, so you can put in a textbox and

put the learning outcome up.

Discussion:

- Briefly describe to your roommates how you could

motivate that learning outcome at the start of a class.

Transparent assignments


- It takes more time, but if you can add learning outcomes

to your assignments, students are more likely to see the

relevance.

- It’s worth noting that newer electronic homework

systems are capable of letting you make these connections

to learning outcomes.

Example from the web.


On the fly -


- It may be a challenge to have this level of organization

when a class starts.

- Even if you can’t do a grid at the start, it’s still

important to tell students the connections for each

assignment.

Template


UNIVERSITY OF NEVADA, LAS VEGAS

© 2013 Mary-Ann Winkelmes http://www.unlv.edu/provost/teachingandlearning Principal Investigator [email protected]

Transparent Assignment Template

© 2013 Mary-Ann Winkelmes

This template can be used as a guide for developing, explaining, and discussing class activities and out-of-class assignments.

Making these aspects of each course activity or assignment explicitly clear to students has demonstrably enhanced students’ learning in a national study.1

Assignment Name Due date:

Purpose: Define the learning objectives, in language and terms that help students recognize how this assignment will

benefit their learning. Ideally, indicate how these are connected with institutional learning outcomes, and how the

specific knowledge and skills involved in this assignment will be important in students’ lives beyond the contexts of this assignment, this course, and this college.

Skills: The purpose of this assignment is to help you practice the following skills that are

essential to your success in this course / in school / in this field / in professional life beyond school:

Terms from Bloom’s Taxonomy of Educational Objectives may help you explain these skills in language

students will understand. Listed from cognitively simple to most complex, these skills are:

o understanding basic disciplinary knowledge and methods/tools

o applying basic disciplinary knowledge/tools to problem-solving in a similar but unfamiliar context

o analyzing

o synthesizing

o judging/evaluating and selecting best solutions

o creating/inventing a new interpretation, product, theory

Knowledge: This assignment will also help you to become familiar with the following

important content knowledge in this discipline:

1. 2.

Task: Define what activities the student should do/perfom. “Question cues” from this chart might be helpful:

http://www.asainstitute.org/conference2013/handouts/20-Bloom-Question-Cues-Chart.pdf. List any steps or

guidelines, or a recommended sequence for the students’ efforts. Specify any extraneous mistakes to be avoided.

Criteria for Success: Define the characteristics of the finished product. Provide specific examples of what these characteristics look like in

practice. With students, collaboratively analyze an example of good work before the students begin working. Offer a

critiqued example of excellent work with specific indicators of what makes the work successful. Explain how excellent

work differs from adequate work. It is often useful to provide a checklist of characteristics of successful work to help

the student know if s/he is doing high quality work while s/he is working on the assignment. This enables students to

evaluate the quality of their own efforts while they are working, and to judge the success of their completed work.

Students can also use your checklist to provide feedback on peers’ coursework. Indicate whether this task/product

will be graded and/or how it factors into the student’s overall grade for the course. Later, asking students to reflect

and comment on their completed, graded work empowers them to focus on changes to their learning strategies that

might improve their future work.

1 Winkelmes, Mary-Ann. “Transparency in Teaching: Faculty Share Data and Improve Students’ Learning.” Liberal Education 99,2 (Spring 2013);

Winkelmes et al, “A Teaching Intervention that Increases Underserved College Students’ Success…” Peer Review (Winter 2016).

*

* The author developed an earlier version of this template at the University of Illinois, Urbana-Champaign.

Beyond just transparency


- As assignments become more transparent, you probably

also need to think in terms of how to have them scaffold

student learning towards deeper understanding.

- This can be done using Bloom’s Taxonomy and asking

questions in more careful ways.

Question

template


Bloom’s Critical Thinking Cue Questions Cue Questions Based on Blooms’ Taxonomy of Critical Thinking

Adapted by C. Allen (January 2013) from Public Consulting Group’s Center for Resource Management,

in partnership with the Council of Chief State School Officers, August 2007

LOWER-ORDER THINKING SKILLS (BASIC THINKING)

3. APPLYING (Using learned knowledge in new

situations or to solve a real life problem)

• How would you use …? • What examples can you find to …? • How would you solve _______ using what you have learned …? • How would you organize _______ to show …? • How would you show your understanding of …? • What approach would you use to …? • How would you apply what you learned to develop …? • What other way would you plan to …? • What would result if …? • How can you make use of the facts to …? • What elements would you choose to change …? • What facts would you select to show …? • What questions would you ask in an interview with…?

2. UNDERSTANDING (Comprehension;

Explaining the meaning of information)

• How would you classify the type of …? • How would you compare …? contrast …? • How would you rephrase the meaning …? • What facts or ideas show …? • What is the main idea of …? • Which statements support …? • How can you explain what is meant …? • What can you say about …? • Which is the best answer …? • How would you summarize …?

1. REMEMBERING INFORMATION (Knowledge; recalling facts and information)

• What is …? • How is …? • Where is …? • When did _______ happen? • How did ______ happen? • How would you explain …? • How would you describe …? • What do you recall …? • How would you show …? • Who (what) were the main …? • What are three …? • What is the definition of...?

HIGHER-ORDER THINKING SKILLS (ABSTRACT THINKING)

6. CREATING (Putting ideas together to form a new and

different whole)

• What changes would you make to solve …? • How would you improve …? • What would happen if …? • How can you elaborate on the reason …? • What alternative can you propose …? • How can you invent …? • How would you adapt ________ to create a different …? • How could you change (modify) the plot (plan) …? • What could be done to minimize (maximize) …? • What way would you design …? • What could be combined to improve (change) …? • How would you test or formulate a theory for …? • What would you predict as the outcome of ...? • How can a model be constructed that would change …? • What is an original way for the …?

5. EVALUATING (Making judgments about the merits of

ideas, materials, or phenomena based on criteria)

• Why do you agree with the actions? The outcomes? • What is your opinion of …? (Must explain why) • How would you prove …? disprove …? • How can you assess the value or importance of …? • What would you recommend …? • How would you rate or evaluate the …? • What choice would you have made …? • How would you prioritize …? • What details would you use to support the view …? • Why was it better than …?

4. ANALYZING (Breaking down a whole into component

parts; Examining critically)

• What are the parts or features of …? • How is _______ related to …? • Why do you think …? • What is the theme …? • What motive is there …? • What conclusions can you draw …? • How would you classify …? • How can you identify the different parts …? • What evidence can you find …? • What is the relationship between …? • How can you make a distinction between …? • What is the function of …? • What ideas justify …?

Question

template


Bloom’s Critical Thinking Cue Questions Cue Questions Based on Blooms’ Taxonomy of Critical Thinking

Adapted by C. Allen (January 2013) from Public Consulting Group’s Center for Resource Management,

in partnership with the Council of Chief State School Officers, August 2007

LOWER-ORDER THINKING SKILLS (BASIC THINKING)

3. APPLYING (Using learned knowledge in new

situations or to solve a real life problem)

• How would you use …? • What examples can you find to …? • How would you solve _______ using what you have learned …? • How would you organize _______ to show …? • How would you show your understanding of …? • What approach would you use to …? • How would you apply what you learned to develop …? • What other way would you plan to …? • What would result if …? • How can you make use of the facts to …? • What elements would you choose to change …? • What facts would you select to show …? • What questions would you ask in an interview with…?

2. UNDERSTANDING (Comprehension;

Explaining the meaning of information)

• How would you classify the type of …? • How would you compare …? contrast …? • How would you rephrase the meaning …? • What facts or ideas show …? • What is the main idea of …? • Which statements support …? • How can you explain what is meant …? • What can you say about …? • Which is the best answer …? • How would you summarize …?

1. REMEMBERING INFORMATION (Knowledge; recalling facts and information)

• What is …? • How is …? • Where is …? • When did _______ happen? • How did ______ happen? • How would you explain …? • How would you describe …? • What do you recall …? • How would you show …? • Who (what) were the main …? • What are three …? • What is the definition of...?

HIGHER-ORDER THINKING SKILLS (ABSTRACT THINKING)

6. CREATING (Putting ideas together to form a new and

different whole)

• What changes would you make to solve …? • How would you improve …? • What would happen if …? • How can you elaborate on the reason …? • What alternative can you propose …? • How can you invent …? • How would you adapt ________ to create a different …? • How could you change (modify) the plot (plan) …? • What could be done to minimize (maximize) …? • What way would you design …? • What could be combined to improve (change) …? • How would you test or formulate a theory for …? • What would you predict as the outcome of ...? • How can a model be constructed that would change …? • What is an original way for the …?

5. EVALUATING (Making judgments about the merits of

ideas, materials, or phenomena based on criteria)

• Why do you agree with the actions? The outcomes? • What is your opinion of …? (Must explain why) • How would you prove …? disprove …? • How can you assess the value or importance of …? • What would you recommend …? • How would you rate or evaluate the …? • What choice would you have made …? • How would you prioritize …? • What details would you use to support the view …? • Why was it better than …?

4. ANALYZING (Breaking down a whole into component

parts; Examining critically)

• What are the parts or features of …? • How is _______ related to …? • Why do you think …? • What is the theme …? • What motive is there …? • What conclusions can you draw …? • How would you classify …? • How can you identify the different parts …? • What evidence can you find …? • What is the relationship between …? • How can you make a distinction between …? • What is the function of …? • What ideas justify …?

If you go back to your

breakout rooms and

look at your learning

outcome there. What

is the highest level

area on this template

that you can (fairly

quickly) think of a

homework question

related to the learning

outcome?

Next week


The topic next week is The First Day of Class

Watch the blog and your email for tasks to complete before

we meet.

References


[1] Grove, N.P.; Bretz, S. (2007) J. Chem. Educ., 84(9),

1416-1424

[2] Redish, E. F.; Saul, J. M. (1998) Am. J. Phys., 66, 212-

224