ASSESSING TEAMWORK IN ENTREPRENEURSHIP CLASSES

Sadan Kulturel-Konak Abdullah Konak

Penn State Berks

Motivation

Design a unified framework for assessing professional skills.

Teamwork is ubiquitous in entrepreneurship classes.

Assessing teamwork knowledge, skills, and abilities is challenging.

Challenges in Assessing Teamwork Skills

Slowly mature Developing tools and rubrics that can track

the student development Students’ concerns

Challenges in Peer Assessment

Labor intensive Time consuming Feedback Confidentiality

Peer Evaluation Tools Communications Project Assessment Tool

(COMPASS) (Thomas et al., 1999) Comprehensive Assessment of Team

Member Effectiveness (CATME) (Loughry et al., 2007)

Self and Peer Assessment Resource Kit (SPARKplus) (Willey & Gardner, 2009)

Peer Evaluation & Assessment Resource (PEAR)

Designed for assessing teamwork Knowledge, Skills, and Abilities

Provides feedback Preserves confidentially Based on the Model of Domain

Learning (Alexander et al. ,1994)

Model of Domain Learning

I should ask questions.

Leading , recall/process, probing , and rhetorical.

Cool !Attend to the next seminar.

A Sample Mapping

Expected Competency /Attitude (5-point Likert Rating scale) I K S A C PInterested in the use of internet communications technologies to communicate x x

Uses internet communications technologies skillfully x x x

Initiates the use of most appropriate communications technologies for specific tasks

x x

Express a personal level interest to learn and use new internet communications technologies

x x

Expresses opinions and ideas in a clear manner in electronic correspondences x x

Expresses opinions and ideas in a clear and concise manner with obvious connection to topic in electronic correspondences

x x

competency/attitude: “use communication technologies skillfully”

PEAR Conceptual Model

Creating a Rubric

Creating an Assessment Select peer and self evaluation rubricsEnter the course data Add teams

Instructors can select customized or standard rubrics

Conducting an Assessment

The instructor sends emails

Each student receives a unique, encrypted URL link.

Students rate themselves and their peers

Reports

Summary Reports Question-based Student-based MDL-based

Export to Excel Email feedback to

students

A Pilot Study

Entrepreneurial Mindset Class (22 students) Three team projects (different teams) The eleven-item rubric by Van Duzer and

McMartin (2004) and five point Likert-type scale Total of 2615 ratings

Peer (212) vs Self (58) Ratings

No QuestionsPeerRating

Self Rating

p-value

1 Communicated ideas clearly/effectively. 4.22 4.61 0.0042 Contributed useful ideas that helped the group succeed. 4.25 4.65 0.0073 Delivered work when promised/needed. 4.21 4.58 0.0324 Encouraged group to complete the project on a timely basis. 4.16 4.58 0.0115 Failed to do an equal share of the work (Reverse). 2.65 1.68 0.0076 Had difficulty negotiating issues with members of the group (Reverse). 2.71 2.12 0.0667 Helped group overcome differences to reach effective solutions. 4.16 4.52 0.0088 Kept an open mind was willing to consider other’s ideas. 4.49 4.65 0.3159 Often tried to excessively dominate group discussions (Reverse). 2.70 2.42 0.160

10 Took a leadership role in some aspects of the project. 3.94 4.40 0.00511 Was fully engaged in discussions during meetings. 4.25 4.67 0.003

Pitfalls

Very high ratings, not consistent with the instructor observations

Some questions were avoided Deliberate versus non-deliberate

evaluations

Sample Interest Questions Attended a speaker event about teamwork Watched a documentary or training video about teamwork Watched a video clip outside of class work about teamwork Attended a workshop about teamwork Asked questions to an expert (professor, consultant etc.) about effective teamwork Performed a web search to learn about effective teamwork Read a book about teamwork Rate your level of interest in attending a free workshop on teamwork. Rate your level of interest in reading literature about effective teamwork. While you are browsing a news website, you have spotted an article called “How to be

Effective in Teamwork.” Rate your likelihood of reading this article? Rate your level of willingness to take an elective course in order to improve your teamwork

skills? In your institution, a renowned teamwork guru will give a workshop on teamwork skills. If you

have to pay $10 for this workshop, rate your level of interest in attending this workshop. Read an online article about teamwork Read a newspaper/magazine article about teamwork

Why should interest be measured?

Variable (Cronbach's Reliability) GPA Group

ClassStanding

Have you ever had a job that requires

significant teamwork?

Have you ever taken a course or training

on teamwork?Goal Setting (.712) -.019 .016 .154** .090*

Performance Evaluation (.637) -.010 .048 .105* .079Team Forming (.827) -.074 -.038 .100* .082*

Team Coordination -.006 -.026 .130** .054Communication (.807) -.127** .032 .120** .126**

Conflict Resolution (.816) -.091* -.048 .111** .067Problem Solving (.810) -.044 .011 .147** .089*

Interest1 (.850) -.014 .097* .135** .456**

Interest2 (.803) .011 .090* .037 .159*

Interest3 (.794) -.006 .118* .170** .290**

Attitude (.730) -.245** .104* .153* .076Benefit (.817) -.191** .071 .119* .055

Based on 586 samples, we observed stronger correlations between class standing and interest than between class standing and self-efficacy (significance p *: 0.05 and **:0.01).

Self-

Effica

cy

Current Work

Past

• Design & Implementation

• Testing

Present

• Beta trials• Fix the bugs• Build Additional

Rubrics

Future

• Data Mining

Conclusions

PEAR facilitates peer and self evaluations. Interest should be used as an additional

construct to measure students’ professional skill development (teamwork)

Acknowledgments

“A Modular Assessment Framework for Professional Skills Using A Model of Domain Learning Approach”-NSF DUE-1044800

“A Virtual Incubator @ Penn State Berks (VIB) to Foster Student Innovations”-VentureWell 8804-11

Questions?

konak@psu.edu, Abdullah Konak sadan@psu.edu, Sadan Kulturel-Konak Ivan Esparragoza  Gül E. Okudan Kremer Gareth Yoder Steven Magluilo