Bridging Gaps: integrating research expertise with curricular development aimed to synchronize upper division course goals with our large introductory classes

Dedra DemareeOregon State University

Background: Physics PhD research emphasis in Physics

Education Research (PER): Focused on writing to learn issues Thesis: TOWARD UNDERSTANDING WRITING TO

LEARN IN PHYSICS: INVESTIGATING STUDENT WRITING

Hired to lead introductory course reform at OSU Current intro courses are ~250 students per section Under-staffed, can not easily reduce class sizes Algebra-based, calc-based, and non-science classes My primary focus is implementing and assessing

course changes

Do We Know if Writing is Helpful in Physics Courses?

No one questions the benefits of educating people to write but why take the time to do it in the curriculum? and why explicitly in physics?

There is no clear evidence in the literature to show the effectiveness of writing to learn! Most writing studies are entirely qualitative and

not controlled

Why MIGHT Writing be Helpful?

everyone raised their hands when polled if they think writing helps learning (at American Association of Physics Teachers

conference) Students interviewed state “writing helps

learning” Students do better with a positive epistemology Writing involves logical argumentation structure Writing helps structure conceptual

understanding? Maybe writing = active engagement?

Why Might Students NOT Learn? Struggling with content and writing is overwhelming Issues with activating and managing their

knowledge resources (research exists to support this)

Ideas can’t be organized if they aren’t already present in some form Are writing activities striking the right balance?

Students may not be reflective when writing Can we generate writing assignments that force

reflection?

What Does Writing “Research” Tell Us? How writing could help learning: Ideas are transformed while writing Rhetorical goals are refined while writing Literature provides minimal evidence to support these

‘Knowledge-telling”: novices tell what they know - experts plan, write, and revise Novices -> cosmetic changes experts -> goal-oriented revisions

How do our students approach writing and revision? How can we quantitatively study this?

Develop methods for tracking and coding writing to allow for controlled studies of the effects of writing in the curriculum

Initial studies and results: Collaborated with English Department to do

controlled test of effect of writing and writing instruction on physics content knowledge Students who wrote did better on post-lab quizzes

compared to students who did traditional activities But no difference on lecture quizzes and exams

Writing instruction impacted physics quality in essays No difference was measured outside their writing

Difficult and time consuming to quantify writing quality – need to find better ways to study this!!

Tracking Writing in Physics by Inquiry: Force revisions within the assignments: Weekly essay with homework 1st draft 250 words – 2nd draft < 125 words!

Trying to force “major restructuring in your head, deciding what’s important and what’s not” (Scott Franklin, RIT)

Create tracking program for capturing details: Obtain text file of their essay and a separate log file

tracking writing events Track pauses, additions, deletes, locations, & times

Developed by Dr. Lei Bao and members of the Ohio State U. Physics Education Research Group

Entering the Text (What Students See):Notice the program tracks and displays a running word count.

It also saves the student’s name, email address, their section, and which assignment is being submitted

The Log File: Saves a snapshot of the text each time a student

pauses, backspaces, deletes, or moves the cursor Indexes each event, gives the time, what type of

activity the student is doing, the text snapshot, and the cursor location Tags include: Typing, Backsp, Naviga, Delete, Pa{s}

(pause and length in seconds), <CU> (cursor location) Example:

1310:57:027 AM: Typing A circuit is all <CU>1410:57:033 AM: Backsp A circuit <CU>1510:57:036 AM: Typing A circuit is the flow <CU>1610:58:043 AM: Pa{67} 1710:58:053 AM: Backsp <CU>

Quantitative Information we can get From the Data:

Do students mostly write new content, or do they go back and revise while or after they write?

When students revise do they cut in bulk and rewrite, or do they modify existing text?

How often students write vs. edit or pause How much do students work after the word limit? Do observed behaviors match self-reports from

interviews? Can look at a lot of writing at once, but

Can’t automate information on the quality of the revisions!!

Student with low final exam but good essay content – 2nd draft:

first rephrased needed content needed then cut extraneous text. Then did detailed editing pass through the entire essay, then one last check

Has clumps of edits around specific text (she reported

struggling with some ideas)

Relatively sophisticated revisions!

Location and Sizes of Revisions in Order they Occurred

0

0.2

0.4

0.6

0.8

1

1.2

0 50 100

Sequence of revision events

Loca

tion

of re

visi

on in

ess

ay

AdditionsDeletionsEdits

Results and conclusions: See evidence of novice vs. expert behavior…

High Exam Scoring students had more revision events, higher essay grades, more edits

in the middle of their essays, a higher percent of phrase-level edits (between 2-6 words in length)

We see no evidence that writing behavior changes with practice

We find no clear predictors based on tracked behavior for which essays will be good

Developed a valuable new tool and begun to characterize student behaviors

Need to apply this test to controlled writing studies

Course reform ideas: Students out of traditional introductory physics:

Minimal conceptual understanding “Plug-and-chug” problem solving skills Worse attitudes than when they registered

Interactive-engagement is more effective Sophisticated epistemologies are encouraged when

students are metacognitive We gain more with focus on Higher-order learning goals Traditional lecture halls do NOT encourage students to

build their knowledge! How to improve this in a large-lecture classroom??

Need an interactive environment!

Our plan for Calc-based sequence:

Currently have: 3 h lecture, 3 h lab, 1 h optional recitation 250 people per lecture, 30 per lab with 1 TA

Change to: 2 h lecture, 4 h activity-based learning in 2 h blocks,

possibly keep 1 h optional extra help time Possibly have lecture on M and F – keep all students

at same pace in activity sections 210 people per lecture, 70 per activity section with 1

senior instructor/TA and 1-2 TAs/undergrads

Why lecture at all? (besides staffing)Activities won’t be effective if students aren’t ready

Prepare students for activity-based hours Introduce definitions, Motivate students

Both can be done with readings Create common language use, Show examples

Both can be done in activity-based hours

Wrap-up after activity-based hours Summarize important points, Look at capstone

issues, Go over things people struggled with All can be done in activity-based hours

Case Study: Iowa State University Two lecture halls: one with fixed seats and one with swivel chairs – both nice and new Use “Peer Instruction”

Swivel chairs made a measurable difference in learning gains: Group discussions were

physically easier swivel lecture hall had higher

percentage of correct responses after talking to neighbors

swivel lecture hall did 6% points higher on the final exam

Proposal for Weniger 151:

Swivel chairs for ease of discussion Chairs and aisles organized to

promote group work Aisles for instructor access to all groups

Clump chairs in sections to minimize the number of needed aisles and maximize the number of seats

Boards (ideally smartboards) along the edges for groups to present ideas to the entire class

Multiple projectors up front so people can see from every angle

Camera to project demonstrations onto an overhead so everyone can see details

Weniger 151 layout details: Students face forward in staggered chairs for lecture Students can rotate to work in groups of 3-4 people Each section has 4 rows – can form 2 rows of groups

with people paired back to back Instructor has access to each group Minimally reduces the number of seats from 266 to just

over 200

Model for new physics classroom: Design a modern activity-

based classroom Design to fit our course

goals/activities Use modern technology to

increase options Test and assess new

curricular ideas in this space

Inspired by SCALE-UP and echoing goals and activities in Paradigms Start here because we

know this works

Can we do better than this? Lessons from PKAL (Project Kaleidoscope):

SHOW VIDEO (made by a KSU Anthropology class) Students…

Want to build community Use informal learning spaces Work more on online activities Rely on multi-tasking

Education community… Thinks about green concerns Highly values activity-based learning Knows the importance of assessment Emphasizes the use of technology

Building Flexibility for the future

Possibility of a design/work area

Cabinets that can be easily moved later

Flexible lighting, power, and media

Whiteboards on wheels

A window into and out of the room

A space that invites different types of activities Floor that allows for ease of making new configurations Technology that promotes collaborative work

Paradigms program ~10 years of reforming upper division physics Award-winning with Consistent NSF funding

Corinne Manogue just won the AAPT undergraduate teaching award

Team-based reform efforts unanimously approved by whole department

Brings active-engagement into advanced courses Integrated lecture/lab/discussions Group work Extensive use of small and large whiteboards

Our Approach Build on Paradigms expertise and borrow and

adapt materials developed by other schools Find goals that fit the needs of the students in

their majors Problem solving, group work… (ABET)

Find goals that fit the needs of physics majors as they segue to upper division Earlier activity-based experience, more

sophisticated problem solving, fit current need of more data analysis skills

Build our goals into the materials

Goals for New Curriculum: Model “real” scientific behavior Develop scientific skills - Have students:

Reflect on how they know what they know Actively reconcile their knowledge Understand the applicability of their models

Integrate simulations with experiments to explicitly address models and simplifications

Have students design and analyze their own experiments – teach them to build knowledge Teach data analysis in the labs – build this in to fit

current lack in overall program Understand estimations and approximations

Integrate goals into exams and homework assignments

Testing some ideas in energy course

Want students prepared for lecture Integrate pre-class reading assignments and

quizzes (following JITT model) Use existing technology – blackboard is powerful

Want to develop discourse skills – apply concepts to have “real” debate about issues Use class time to scaffold up to sophisticated

discussions Use online tools for collaborative writing – “Wiki” Group info gathering and posting then online

discussions

Ph212 homework Problem Solving Guide

Understand and restate the problem Read. Read the problem carefully. What are the key words? What information is given? What might you need to know in order to solve this? Explicitly

state (in your own words) what is the problem asking including clarifying the problem statement. For example, if the problem states when will the two cars collide, you can state when will the two cars have the same coordinates for x and t.

Visualize. Visualize the situation described with a mental picture. What are the important features of the situation? What physically might happen? Think about what physics might be involved? (Repeat steps 1.a and 1.b as needed until you’re ready for step 1.c )

Simplify. Think of what assumptions you can make: can you ignore the size of the objects and consider them particles? Can you ignore friction? (Usually if the information about some properties of objects or interactions is missing from a problem statement, this means it is not important and you can ignore it.) In your homework you must explicitly state how this simplifies the problem – for example if you are ignoring friction in a collision it means you will be using momentum conservation for the system.

Picture and translate. Translate the text of the problem into a picture – record all given quantities in the picture and identify symbolically (name!) the relevant variables and unknowns. Choose and show the coordinate axis(es). Explain your picture with words if that makes it more clear. (Sometimes this step can be skipped and you can combine it with step 2.b – but only if you are very confident with the other steps.)

Devise and explain the plan Determine what concepts/laws apply. Think what physics concepts are involved and which will be more helpful to solve the problem. For example, think

whether the problem involves concepts of energy or force. Explain why you made the choice of this (these) particular physics concept(s). You may want to refer to 1 c. here, as in the example given there.

Represent physically. Represent the situation with the appropriate type of physical representation. This can be a free-body diagram, an energy bar chart, a ray diagram…. (If you skipped step 1.d, you must record all the given quantities and symbols for relevant variables and unknowns here.)

Represent mathematically. Use the physical representation to construct a mathematical representation. Make sure that this representation is consistent with previous ones. You might need to use additional definitions of physical quantities or laws combined with these equations to solve the problem.

Carry out the plan Solve. Use mathematical relationships from part 2.c to solve for the unknown quantity (quantities). Make sure that you use consistent units. If you do not

have enough equations to solve for what you need, go back and check all above steps to make sure you haven’t overlooked some piece of physics given or implied by the situation.

Symbolic and numeric solutions. A complete solution should have the equations given in terms of the symbols, and only then should you plug in numbers to get a numerical answer

Look back – explain what you did, was your answer as expected? Evaluate the result. Have you answered all parts of the question? Is the number reasonable? Are the units appropriate? Does the result make sense in

limiting cases? Include a written explanation for why your result makes sense and what it tells you about the physics of the situation (what happens?) If solution does not make sense… go back and re-visit your interpretation of the problem and the assumptions you made – did you overlook something?

Was something that you thought could be ignored too large to ignore? Check your math, did you make a mistake?

Ph212 Problem Solving detail Understand and restate the problem

Read. Visualize. Simplify. Picture and translate.

Devise and explain the plan Determine what concepts/laws apply. Represent physically. Represent mathematically.

Carry out the plan Solve. Symbolic and numeric solutions.

Look back – explain what you did, was your answer as expected? Evaluate the result. If solution does not make sense…

(Adapted from ISLE and U. Minn)

Ph212 homework Grading RubricPoints: 0 1 2 3

1 a. Statement of what

the problem is asking

No problem statement is written

The problem statement is re-stated word for word

The problem is stated in the students own words but provides no more definition than the original statement

The problem is stated in the students own words, with the problem more directly defined than in the original question

1 c. Simplify and state

assumptions

No information is given about assumptions

Trivial or incorrect assumptions are listed

Correct assumptions are listed with no information about how they simplify the problem, or an important assumption is missing

Correct assumptions are listed along with a correct statement about how they simplify the problem

2 a. Statement

explaining which concepts/laws apply

No such statement is written

Incorrect concepts/laws are provided

There is a statement that explains which concepts/laws apply but does not explain why, or does not give the correct reasoning as to why

There is a statement clearly explaining which concepts/laws apply, as well as why they apply – this may refer to your response from 1 c.

2 b. Physical

representation

No physical representation is given

An incorrect physical representation is given, or one that is correct, but does not include any labels or defined quantities

A correct physical representation is given, but is not clearly labeled, does not include all quantities, or a clear representation is given but it contains a mistake

A clearly labeled, correct physical representation is given, with all quantities and symbols defined

2 c. Mathematical representation

No mathematical representation is given

The mathematical representation given is incorrect

An incomplete mathematical representation is given

A complete mathematical representation is given

3 a/b. Solution

No solution is given

Only a partial solution or an incorrect solution is given

Only the symbolic or numeric solution is given, or there is some mistake such as incorrect units

A complete solution is given both symbolically and numerically with correct units

4 a. Evaluation of the

result

No evaluation is given

Very little information is given to evaluate the result

A partial explanation is given for why the result makes sense (or does not make sense if the incorrect answer was reached), and what it tells us about the physics of the situation

A clear and complete explanation is given for why the result makes sense (or does not make sense if the incorrect answer was reached), and what it tells us about the physics of the situation

Homework Grading Rubric detail

4 a. Evaluation of the result 0: No evaluation is given 1: Very little information is given to evaluate the result 2: A partial explanation is given for why the result

makes sense (or does not make sense if the incorrect answer was reached), and what it tells us about the physics of the situation

3: A clear and complete explanation is given for why the result makes sense (or does not make sense if the incorrect answer was reached), and what it tells us about the physics of the situation

Guiding Questions for Physics Writing (Paradigms – Junior year) 1. State the problem. What is the problem that you are trying to solve, and what – if any – assumptions or idealizations are being made about the physical situation.

2. Outline the general strategy. What physics concepts are relevant? Which general physical equations will be useful in solving this problem? Explain how the physical quantities are related to one another? Connect the dots between any quantities in any ways that you can.

3. Explain your terminology. What is the role of each of the symbols in these equations? For constants, just list their names and values if used in numerical calculations. For variables, briefly describe what they represent.

4. Set-up your equations. How did you apply the information in your problem to the general equations? How did your example fit into and change the general equation. Think about how you went about putting in the information from the example you cared about, and any raw data taken, into the general equations.

5. Explain any data taking procedures used in collecting information needed to solve to solve the physical problem. Remember to include all pertinent information, including how to setup any apparatus used and detailed instructions on how data was acquired.

6. Organize your data. List any raw data taken. Use graphs and charts to show concisely the relevant quantities in relationship to one another.

7. Analyze your data. Explain how the data fits into the theory governing the problem you are solving. Comment on any unusual or anomalous data, providing an explanation of how it may have come about being recorded.

8. What were the mathematical manipulations used in the process of solving the problem? Show the steps of algebra used to solve any tricky parts of the problem, write a short sentence for each explaining why they are true, and include any areas of difficulty that may have lead to dead ends.

9. Reflect on your final answer. What is it that this answer tells you about the physical quantities involved, and how they are related to each other? Is this a limiting case, or are there limiting cases to this answer for which it is valid? Were there any better ways to solve the problem that you could consider? How did your solution compare and tie into work that others have done in this field of work? What was the most important, significant finding made in solving the problem?

Guiding Questions detail1. State the problem.2. Outline the general strategy. 3. Explain your terminology4. Set-up your equations5. Explain any data taking procedures used in collecting

information needed to solve the physical problem. 6. Organize your data.7. Analyze your data. 8. What were the mathematical manipulations used in

the process of solving the problem? 9. Reflect on your final answer.

Rubric detail: Content Criterion: Did the writer convey an understanding

of what the final results tell about the physics? Very Good: Writer clearly explained what the final results

tell about the physics of the problem and described what is physically interesting or unique about the solution to the problem.

Fair: An attempt is made to relate the mathematical manipulations to the physical concepts, but the physical situation is weakly related to these results.

Poor: The writer made no attempt at describing how their final solution related to the physical concepts.

Improving Efficiency Problems with current homework system:

High grading load for paid undergrad workers Papers get lost Returning papers is a pain Recording grade takes time and yields errors

Moving to online homework: Much of the work is graded automatically Records are kept automatically Writing-aspects can be built into the existing problems

and graded online more efficiently Gain additional features such as tutorials

Curricular Assessment Plans/Ideas:

Concept tests and exemplar problems on exams Attitude/epistemology surveys Free response surveys Specific assessments based on course goals

(example assessments from Purdue): looking at conceptual thinking in problem solving

Interviews using talk-aloud protocol Are important course ideas/skills are being used by students

looking at TA training and attitudes toward inquiry-based learning see if the TA attitudes toward teaching and learning are

impacted by teaching the course

Abstract Oregon State University has an innovative award wining upper-division

physics curriculum, but fairly traditional lower division large introductory courses. Mainly due to staffing constraints, little had been done to improve these courses despite the department’s dedication to team-based curricular development and active-engagement classrooms. More resources were needed to bridge these ideas into the intro courses, leading to my hire charged with leading the introductory course reform efforts. My expertise is in developing quantitative measures for studying the effectiveness of writing to learn (within the context of physics). I will report on ways that writing can be integrated into large introductory courses in a way that scaffolds students toward goals in our upper division courses, without adding a heavy burden on grading. As part of our curricular reform we are also renovating new classroom space: both large lecture hall space and a smaller active-engagement classroom. As part of this planning I recently attended the national Project Kaleidoscope meeting titled “Roundtable on the Future Undergraduate STEM Learning Environment.” I will report on lessons learned at this meeting and our vision for integrating our curricular reform with the classroom remodels.