standing on our heads how teaching engineering design looks different from a tbl perspective peter...

Standing on Our HeadsHow Teaching Engineering

Design Looks Different from a TBL Perspective

Peter OstafichukAntony Hodgson

University of British Columbia

Objectives

• To share our approach to TBL and our TBL experiences with you

• To outline two areas where we have faced challenges so:– you are aware of these potential problems– we can discuss/share ways to address these issues

“Standing on Our Heads”?

Value of InstructorInteraction

Knowledge

Skills

Judgment

Conventional TBL

Part 1: TBL in MECH 223

Part 1: TBL in MECH 223

• Some background on our course• Some of our approaches in TBL

– Team Formation– Assigned Readings– Learning Taxonomy – Module Structure– Group Dynamics Workshops

• Some feedback from students

Mech 2• New second year curriculum at UBC• 120 students• Fully integrated and team-taught

MECH 223 (4 weeks)

MECH 223 (3 weeks)

MECH 222

(7 weeks)

MECH 220 (4 weeks)

MECH 221 (10 weeks)

Te

rm 1

(S

ep

t-D

ec)

Te

rm 2

(Ja

n-A

pr)

MECH 223 (4 weeks)

MECH 223 (3 weeks)

MECH 222 (7 weeks)

MECH 220 (4 weeks)

MECH 221 (10 weeks)

MECH 223: Introduction to DesignMONDAY TUESDAY WEDNESDAY THURSDAY FRIDAY

8 00

8 30

9 00

9 30

10 00

10 30

11 00

11 30

12 00

12 30

13 00

13 30

14 00

14 30

15 00

15 30

MECH 223

Tutorial Tutorial Tutorial

Special EventTeam Meeting Team Meeting

Class

Computer Lab

Tech. Comm.

Design Lab

ClassClass Tech. Comm.

Tutorial

Construction Lab

Class

Class

ClassClass

Class

Class

Class

Class

Class

Team Formation• 20 teams of 5-6 formed by instructors• Heterogeneity maximized by distributing

– GPA and program streams– Myers-Briggs personality types– Male/Female balance– Machining, software, and communication abilities– Access to vehicle, laptop, and hand tools– Geographic area

• 5 teams together form a division; we encourage collaboration within divisions

Assigned Readings

• Readings of 30-60 pages per module• Drawn from various sources (10 different texts)• Each reading section is clearly identified as:

– Required: minimum expected reading; heavy emphasis on RAP quiz

– Recommended: additional reading/examples to help clarify concepts; light emphasis on RAP quiz

– Beneficial: supplementary readings to help with projects and professional development; minimal emphasis on RAP quiz

Learning Taxonomy

Category Behaviour Activity Evaluation

KnowledgeRecall,

recognize, understand

Readings, quizzes, tutorials

Multiple-choice,

short-answer

SkillsUse/apply knowledge

Tutorial and in-class

exercises

Short-answer,

simple tasks

JudgementSolve open-

ended problem

Discussions, assignments,

projects

“Engineering essay”

questions

Module Structure

Readings

RAP

Course Project

In-class activities

Out-of-class exercise

Debriefing

In-class

Out of class

Tutorial exercisesKnowledge

Skills

Judgment Repeated 6 times

Group Dynamics Workshops

• Two workshops (~3 hrs ea.) to raise awareness and give tools teams can use

• Workshop 1 – start of first half of course– Introduction to Myers-Briggs (MBTI)– Demonstrations, strengths/weaknesses of types– Problem solving model for teams using MBTI types

• Workshop 2 – start of second half of course– Feedback/reflection from first project– Simulated group activity with independent observer– Techniques for addressing group conflict

Student Feedback

• We use online surveys in our course to complement the formal evaluations

• Survey details:– Optional– Anonymous– Includes questions on TBL– Response rate 40-65%– Quantitative & qualitative

Preference for TBL vs Conventional

0%

10%

20%

30%

40%

50%

StrongTBL

Mild TBL

Neutral MildConven'l

StrongConven'l

Average

2005

2006

2007

Effectiveness of TBL

0 2 4 6 8 10

LearningContent

DevelopingTeam Skills

EnjoyingCourse

Average 2005 2006 2007

Student Feedback

• If readings are too long, some students stop doing them altogether

• There is a need for lectures (“mini-lectures”) – Students are not prepared to jump into exercises

based on readings alone– Many students want lectures to build confidence

with material

• With too many team activities, some students lost motivation to engage with team

Part 2: Our ‘Bleeding Edge’

Part 2: Our ‘Bleeding Edge’

Two major issues:• Class discussion after extended assignments• Peer feedback

Pattern:• Introduce issue (5 min)• Choose best approach (5 min)• Report choice and debrief (5 min)

Extended Assignment Example

Tutorials:

RAP CrashCourse

Readings

DesignChallenge

Constraint ID

ComponentFamiliarization

Prep forChallenge

TeamWork

PrinterDisassembly

TBL Theory• Engagement maximized

by making clear decision– Three S’s:

Same problem, Single answer, Simultaneous report

• Minimal writing– Team writing usually done

in isolation; not engaging

Design Challenge• Purpose: to apply skills learned

and engineering judgment to realistic design problem (lifting chair for MS sufferers)

FixedPivotsRollers

FullyLowered

FullyRaised Mechanism students

focus on

Design Challenge

• Design mechanism• Spec & source components

– use laptops in class

• Minimize price

• Time allowed: – 2.5 class hours– 2 tutorial hours– all over ~6 calendar days

Commitment Time• Students hand in 2-3 page

writeup:– Price ($)– Clear sketches– Parts list (supplier, price,

main reason for selection)– Rationale for main choices– Summary of sizing calcs

• Post-It Note on blackboard showing price – lowest on left

Defend Design• Encouraged to prepare 1

or 2 overheads• Team with lowest price

leads off• Critiques based on:

– Incompleteness– Faulty design

• Profs note good points; pass to TAs for marking– Few marks

What Went Well• Variety of activities

maintained interest• “Waves” built• Simultaneous reporting• Respectful listening to

peers

Issues• Idle students

– Watching others work– Low parallelism– Disproportionate contribution

• One speaker at a time– Energy level drops– Hard to check other teams’

work

• Quality varies year to year– Same elements– Different ‘buy-in’, effort

Proposed Changes

Decide which is best strategy:1. Divide class for reporting (groups of 30)

2. After simultaneous reporting, require exchange of reports before debate

3. Use ‘Google Docs’ for interactive debate

4. Increase mark value for assignment

5. Your suggestion?

Discussion

Peer Evaluation

• Mandatory weekly peer evaluations • Conducted using iPeer online evaluation tool (

http://ipeer.apsc.ubc.ca)• Quantitative and qualitative

– Distribute points between team members (100 pts per student on average)

– Each student must include written comment about each other student

• Team marks multiplied by average iPeer scores to determine students’ grades

Peer Evaluation ScheduleDescription Week Evaluation

Part 1

Project #1

1 -

2 Practice iPeer

3 iPeer 1

4 iPeer 2

Break5 iPeer 3

6-12 -

Part 2

Project #2

13 -

14 iPeer 4

15 iPeer 5

Exams 16 iPeer 6

Peer Evaluation: Details

• One week to complete each evaluation • After each evaluation, students receive:

– their average score – randomly-ordered anonymous comments from

team mates

• Penalty for late evaluations– -20% for up to 3 days late– 0 for over 3 days late

• Team marks (55% of course) multiplied by average evaluation score

Peer Evaluation: What We Like

• System gives students ongoing feedback– Chance to change behaviour early– Incentive to continually contribute to team

• Numerous evaluations for each student – Better indication of performance through term– Lots of data on student (confidence in ratings ↑)– Can identify trends

• Automated system– Easy for us to set up– Easy for students to use

Peer Evaluation: Problems

For a small subset of the teams, three common problems have come to our attention:

1. Peer evaluations used to punish team mates after disagreements (or reward friends)

2. Some students/teams afraid of “rocking the boat” (give equal scores to everyone, even when contributions were not equal)

3. Students hesitant to use peer evaluation to give effective feedback (fear of retribution)

Proposed Changes

Decide which is best strategy:1. Only count final evaluation

2. Do not release peer evaluations until the end of term

3. Do not release numerical peer evaluation scores (release comments)

4. Reduce peer evaluation impact on final grade

5. Your suggestion?

Discussion

Thank you!

Questions?

Other Problems• RAP questions: difficult to strike balance

• IF-AT team test – cheating

• Resistance to reading

MECH 223 Course Modules1: Design Process

– Project management– Generating ideas– Evaluating ideas

2: Performance Eval.– Estimation– Prototyping– Design tools

3: Mechanisms– Mechanical components– Material selection– Forming and shaping

4: Refinement– Specifications– Uncertainty analysis– Optimization

5: Implementation– Design for manufacturing– Ergonomics– Mitigating risk

6: Broader Context– Innovation – Intellectual property– Societal role of engineers

Outline

• Use of TBL in MECH 223– Knowledge-skills-judgment learning taxonomy– Our approach to TBL– Student feedback

• Challenges we have faced– Managing an effective class debriefing/discussion – Use of peer evaluation

• Discussion/QA