engaging faculty and students in talking about teaching and learning (informed by assessment data)

Engaging Faculty and Students in Talking about Teaching and Learning(Informed by Assessment Data) Karl A. SmithCivil Engineering - University of [email protected]://www.ce.umn.edu/~smith

Designing and Implementing High Performance Learning Teams

American Accounting AssociationAugust 2005

Assessment DataKnowledge Probe Classroom Assessment (minute paper)Mid-Term Review Student Management TeamSGID & Peer Review

Knowledge ProbeExample from MOT 8221What would you like to know about the students in your courses?

Participant Information

MOT 8221, Spring 2005

Name __________

Work Experience (describe briefly): (use back if necessary)

Previous Coursework/Experience in Project Management, Knowledge Management, Engineering Systems, Industrial Engineering/Operations Research (IE/OR), Management Science, and Quality Management (Six Sigma/TQM):

For the following areas, please rank your level of understanding according to the following scale:

1 = Little or no coursework/experience in this area.

2 = (Between 1 & 3).

3 = Moderate coursework/experience in this area

4 = (Between 3 & 5).

5 = A great deal of coursework/experience in this area.

Project Management12345

Knowledge Management12345

PMI-PMBOK

12345

Engineering Systems12345

IE/OR

12345

Modeling/Simulation12345

Mgmt Science

12345

Six Sigma/ TQM

12345

Computing Experience:

For each of the following, rate your proficiency and list any computer software:

1 = Never have used it.

2 = Know a little about it.

3 = Have used it some.

4 = Am very comfortable using it.

Rating

Specific Packages

Spreadsheet

1234

Project Management

1234

Statistical

1234

Modeling/simulation

1234

Data base

1234

Programming language

1234

Expectations from the course (use back if necessary):

MOT 8221 Spring 2005

PMQ1PMIQ2KMQ3LeadershipQ4EngSysQ5

IE/ORQ6Mod/SimQ7CASQ8MgmtSciQ96 SigmaQ10

Chart7

05953

136012

114430

121063

65641

136201

77233

163111

109300

710131

1

2

3

4

5

Sheet1

MOT8221 S 2005 PIF

12345Avg

12345

PMQ1059533.322

PMIQ21360121.822

KMQ31144302.022

LeaderQ41210633.422

EngSysQ5656412.522

IE/ORQ61362011.622

Mod/SimQ7772332.522

CASQ81631111.522

MgmtSciQ91093001.722

6 SigmaQ107101312.122

1234Avg

1234

SpreadQ1027133.522

PMQ215973.022

StatQ3613211.922

Mod/SimQ478342.222

DBQ506973.022

ProgQ6363102.922

Sheet1

1

2

3

4

5

Sheet2

1

2

3

4

Sheet3

MOT 8221 Spring 2005

SpreadQ1PMQ2StatQ3Mod/SimQ4DBQ5ProgQ6

Chart8

02713

1597

61321

7834

0697

36310

1

2

3

4

Sheet1

MOT8221 S 2005 PIF

12345Avg

12345

PMQ1059533.322

PMIQ21360121.822

KMQ31144302.022

LeaderQ41210633.422

EngSysQ5656412.522

IE/ORQ61362011.622

Mod/SimQ7772332.522

CASQ81631111.522

MgmtSciQ91093001.722

6 SigmaQ107101312.122

1234Avg

1234

SpreadQ1027133.522

PMQ215973.022

StatQ3613211.922

Mod/SimQ478342.222

DBQ506973.022

ProgQ6363102.922

Sheet1

1

2

3

4

5

Sheet2

1

2

3

4

Sheet3

Participant Information Sheet

MOT 8221, Spring 2004

Name __________

Work Experience (describe briefly): (use back if necessary)

Previous Coursework/Experience in Project Management, Knowledge Management, Engineering Systems, Industrial Engineering/Operations Research (IE/OR), Management Science, and Quality Management (Six Sigma/TQM):

For the following areas, please rank your level of understanding according to the following scale:

1 = Little or no coursework/experience in this area.

2 = (Between 1 & 3).

3 = Moderate coursework/experience in this area

4 = (Between 3 & 5).

5 = A great deal of coursework/experience in this area.

Project Management12345

Knowledge Management12345

PMI-PMBOK

12345

Engineering Systems12345

IE/OR

12345

Modeling/Simulation12345

Mgmt Science

12345

Six Sigma/ TQM

12345

Computing Experience:

For each of the following, rate your proficiency and list any computer software:

1 = Never have used it.

2 = Know a little about it.

3 = Have used it some.

4 = Am very comfortable using it.

Rating

Specific Packages

Spreadsheet

1234

Project Management

1234

Statistical

1234

Modeling/simulation

1234

Data base

1234

Programming language

1234

Expectations from the course (use back if necessary):

PMQ1KMQ2PMIQ3EngSysQ4IE/ORQ5Mod/SimQ6MgmtSciQ76 SigmaQ8

Chart2

2510100

961020

204120

127341

204210

139041

129510

136341

1

2

3

4

5

Sheet1

MOT8221 S 2004 PIF

12345Avg

12345

PMQ125101003.0

KMQ29610202.2

PMIQ32041201.4

EngSysQ41273412.1

IE/ORQ52042101.4

Mod/SimQ61390411.9

MgmtSciQ71295101.8

6 SigmaQ81363412.0

1234Avg

1234

SpreadQ1024213.7

PMIQ2271162.8

StatQ3611732.3

Mod/SimQ4913232.0

DBQ557782.7

ProgQ6451082.8

Sheet1

1

2

3

4

5

Sheet2

1

2

3

4

Sheet3

SpreadsheetQ1PMQ2StatQ3Mod/SimQ4DBQ5ProgQ6

Chart4

02421

27116

61173

91323

5778

45108

1

2

3

4

Sheet1

MOT8221 S 2004 PIF

12345Avg

12345

PMQ125101003.0

KMQ29610202.2

PMIQ32041201.4

EngSysQ41273412.1

IE/ORQ52042101.4

Mod/SimQ61390411.9

MgmtSciQ71295101.8

6 SigmaQ81363412.0

1234Avg

1234

SpreadQ1024213.7

PMIQ2271162.8

StatQ3611732.3

Mod/SimQ4913232.0

DBQ557782.7

ProgQ6451082.8

Sheet1

1

2

3

4

5

Sheet2

1

2

3

4

Sheet3

Knowledge ProbeWhat would you like to know about the students in your courses?

Assessment DataKnowledge Probe Classroom Assessment (minute paper)Mid-Term Review Student Management TeamPeer Review

Minute PaperWhat was the most useful or meaningful thing you learned during this session?What question(s) remain uppermost in your mind as we end this session?What was the muddiest point in this session?Give an example or applicationExplain in your own words . . .

Angelo, T.A. & Cross, K.P. 1993. Classroom assessment techniques: A handbook for college teachers. San Francisco: Jossey Bass.

Session Summary(Minute Paper)

Reflect on the session:

1. Most interesting, valuable, useful thing you learned.2. Question/Topic/Issue you would like to have addressed.3. Comments, suggestions, etc

Pace: Too slow 1 . . . . 5 Too fastRelevance: Little 1 . . . 5 LotsFormat: Ugh 1 . . . 5 Ah

Q4 Pace: Too slow 1 . . . . 5 Too fastQ5 Relevance: Little 1 . . . 5 LotsQ6 Format: Ugh 1 . . . 5 AhMOT 8221 - Spring 2004 - Session 1

Chart1

051381

044145

116162

1

2

3

4

5

Sheet1

MOT8221 S 2004 Session 1 Reflection

12345Avg

12345

PaceQ40513813.2

RelevanceQ50441453.7

FormatQ61161623.7

Sheet1

1

2

3

4

5

Sheet2

Sheet3

Session 1 CommentsMost interesting fast paced tower build exercise (cited by 10 or more); Engineering approach exerciseTeam efforts, Love the team building; plus/delta goodInquiry exercise good at reinforcing the group dynamics; Practical application of constructive controversy; controversy exercise, also new teamsPM is more than softwareUseful information that can be brought back to me company to useNot addressed scope creep; recipe for PM; little more on external factors; internal vs. external PM; formal PM life cycleQuestions Is project management a science or an art? How most companies do PM?Suggestion A little less reading, pleaseFormat nice balance of theory, background and some practice; class very interactive and interesting; wild first class, ugh, firehose! Great first class

Q4 Pace: Too slow 1 . . . . 5 Too fastQ5 Relevance: Little 1 . . . 5 LotsMOT 8221 - Spring 2004 - Session 2

Chart1

112340

047134

1

2

3

4

5

Sheet1


LowHighAvg

12345

PaceQ41123403.0


Sheet1

00000

00000

1

2

3

4

5

Sheet2

Sheet3

Q4 Pace: Too slow 1 . . . . 5 Too fastQ5 Relevance: Little 1 . . . 5 LotsQ6 Format: Ugh 1 . . . 5 AhMOT 8221 - Spring 2005 - Session 1

Chart1

112050

105174

0031113

1

2

3

4

5

Sheet1


12345Avg

12345

PaceQ41120503.127


FormatQ600311134.427

Sheet1

1

2

3

4

5

Sheet2

Sheet3

Q4 Pace: Too slow 1 . . . . 5 Too fastQ5 Relevance: Little 1 . . . 5 LotsQ6 Format: Ugh 1 . . . 5 AhMn/DOT Essential Skills for Project Managers May 2, 2005

Chart1

011040

00186

00267

1

2

3

4

5

Sheet1

Essential Skills for Project Managers - May 2, 2005

12345Avg

12345

PaceQ40110403.215


FormatQ6002674.315

Sheet1

1

2

3

4

5

Sheet2

Sheet3

http://eval.umn.eduMid-Term Review

MOT 8221 Mid-Semester Review

Sheet1

MOT8221 S 2005 Mid Semester Review

Workload

Q12%

ML0

L14

AS1872

M312

MM00

NR312

25

12345Avg

UMFGVGE

ExpectQ10011594.3

SpeakQ50016184.7

InterestQ600211124.4

AvailabilityQ700111134.5

FeedbackQ100069104.2

EvalQ110041284.2

Q1 Your understanding of what is expected of you in this course

Q5 The instructor's ability to speak clearly and audibly

Q6 The instructor's success in getting you interested or involved

Q7 The instructor's availability to answer questions or provide help

Q10 Helpfulness of feedback on assignments or class work

Q11 Degree to which evaluation procedures (e.g., exams, quizzes) measure your knowledge and understanding

Sheet1

Q12

Sheet2

U

M

FG

VG

E

Sheet3

Student Management TeamA student management team will be used in this course to operationalize Total Quality Management principles. The attributes of student management teams are described below, and the operation of the team is based on shared responsibility:Students, in conjunction with their instructor, are responsible for the success of any course. As student managers, your special responsibility is to monitor this course through your own experience, to receive comments from other students, to work as a team with your instructor on a regular basis, and to make recommendations to the instructor about how this course can be improved. (Nuhfer, 1990-1995).

Attributes of Student Management Teams3 - 4 students plus teaching team.Students have a managerial role and assume responsibility for the success of the class.Students meet weekly; professor attends every other week. Meetings generally last about one hour.Meet away from classroom and professor's office.Maintain log or journal of suggestions, actions and progress.May focus on the professor or on the content.Utilize group dynamics approach of TQM.

Chapter 8: Student Management Teams: The Heretics Path to Teaching Success by Edward B. Nuhfer

Wm. Campbell & Karl Smith. New Paradigms for College Teaching. Interaction Books, 1997.

Students as Co-DesignersGraduate TAs participating as members of the teaching teamUndergraduate TAs (near peers) as members of the teaching team

Approaches to Cooperative Learning in CE 4101W & 4102WInformal Book EndsFormal Task Groups projects in class and outsideCooperative Base Groups (Cohort Groups)Student Management Team

Active Learning: Cooperation in the College Classroom

Informal Cooperative Learning GroupsFormal Cooperative Learning GroupsCooperative Base Groups

Book Ends on a Class Session

Cooperative Learning Task GroupsPerkins, David. 2003. King Arthur's RoundTable: How collaborative conversations createsmart organizations. NY: Wiley.

WebCT Peer Review & FeedbackStudents work in Base GroupsWebCT provides private message areas for each groupOpportunity to use the Model-Practice Feedback LoopFeedback to whole group rather than individualsMore informationMore models and feedback to help students

WebCT Discussion Area

Feedback Posting Sample

Detailed Feedback to the Group

Model-Practice-Feedback LoopCooper and Robinson [18] surveyed the literature in higher education and found that ...the model-practice-feedback loop is among the most powerful instructional strategies available to teachers at all levels. teacher modelingstudent practice with multiple opportunitiesdescriptive feedback on the quality of their performance

Successes & ChallengesIncorporating formal cooperative groups with the peer review process offered the students: access to more examples of writing access to comments on both their own papers and those of their group members

Students need more explicit connections between the writing for class and the writing they will be doing in the workplace.Summer 2004 we incorporated an interview assignment to help students make this connection

Assessment DataKnowledge Probe Classroom Assessment (minute paper)Mid-Term Review Student Management TeamSGID & Peer Review

The biggest and most long-lasting reforms of undergraduate education will come when individual faculty or small groups of instructors adopt the view of themselves as reformers within their immediate sphere of influence, the classes they teach every day.

K. Patricia Cross

It could well be that faculty members of the twenty-first century college or university will find it necessary to set aside their roles as teachers and instead become designers of learning experiences, processes, and environments. James Duderstadt, 1999

8:30-9:30?Take copies of Active Lrn, HTMI, New Paradigms, Barr & Tagg's From teaching to learing is the most often requested article from Change mangazine

Bill Camplbell and I started working on New Paradigms in 1993.