collaborative problem solving: assessment and reporting

1

Assessment of collaboration and problem

solving: a new metrics approach.

Collaborative problem solving: assessment and reporting

Patrick Griffin, Nafisa Awwal

What is it about- In the workplace - in community?ATC21S

“21st Century Skills”

• creativity and innovation

• critical thinking, problem solving, decision making

• learning to learning, metacognition

• communication

• collaboration

• information literacy

• ICT literacy

• citizenship

• life and career

• personal and social responsibility

Binkley et al. (2012) in P. Griffin, B. McGaw, & E. Care

• Literacy

• Numeracy

• ICT literacy

• Intercultural understanding

• Ethical understanding

• Critical and creative thinking

• Personal and social capability

Australian Curriculum - ACARA.

2

Why Collaborative Problem Solving?

CPS is inclusive of many other 21C skills

Collaborative Problem Solving

Collaboration

Problem Solving

Critical Thinking

Decision Making

Collaborative Problem Solving Framework

Collaborative problem solving

Social

ParticipationPerspective

takingSocial

regulation

Cognitive

Task regulation

Knowledge buiilding

3

Performance rubric CPS the curriculum KLAs and the capabilities

Nature of Collaborative Problem Solving

Collaborative Problem Solving is the act of two or more students bringing their Collaborative Problem Solving is the act of two or more students bringing their Collaborative Problem Solving is the act of two or more students bringing their Collaborative Problem Solving is the act of two or more students bringing their

own unique resources to solve a joint problemown unique resources to solve a joint problemown unique resources to solve a joint problemown unique resources to solve a joint problem

• Collaborative Process is overt – individual problem solving is covert

• Ill defined – ambiguity vs. well defined

• Shared representation

• Individual management of resources, expertise and experience

• Common goal – group planning

Assessment Style

• Human to human

• Students can collaborate irrespective of location

• Automated scoring

• Instant feedback reports for teachers

4

Skill examples to look for in this task

Part

Part ContentSocial skills

• Action• Interaction

• Task completion• Perspective taking

Cognitive skills

• Goal setting• Collecting information• Systematicity

• Hypothesis – testing a rule or general principle • Applying a rule

Balance

Children at play

The task

The data

The construct

Scenario

In the playground Online task

5

Task Analysis: Balance p.1

36

Student A Student B

Student A can see the whole balance

beam but can only interact with its left

side.

Student A begins with all the masses

and can pass them to Student B via

the transfer pad.

Student B can see the whole balance beam but

can only interact with its right side.

Student B can receive masses via the transfer

pad from Student A and subsequently transfer

weights back to Student A.

Log file

38

1669977 sngstud3968 sng4 ICP0900166 B Session start name:sngstud3968 10/04/12 17:10

1669981 sngstud3967 sng4 ICP0900166 A View page 10/04/12 17:10

1669982 sngstud3967 sng4 ICP0900166 A Session join

name:sngstud3967;

IPaddress:

202.176.202.135

10/04/12 17:10

1669984 sngstud3968 sng4 ICP0900166 B View page 10/04/12 17:10

1670127 sngstud3968 sng4 ICP0900166 B Type message how to put ? 10/04/12 17:10

1670173 sngstud3967 sng4 ICP0900166 A Place weight 500, 1, [800, 0] 10/04/12 17:10



1670224 sngstud3967 sng4 ICP0900166 A Type message drag nah ? 10/04/12 17:10

1670257 sngstud3968 sng4 ICP0900166 B Type messagegot nothing to drag o.o

10/04/12 17:11

1670258 sngstud3967 sng4 ICP0900166 A Type message4 put 50g

10/04/12 17:11

1670283 sngstud3967 sng4 ICP0900166 A Remove weight 300, 2, [1000, 0] 10/04/12 17:11


1670315 sngstud3968 sng4 ICP0900166 B Type messagesend me a mass

10/04/12 17:11

1670328 sngstud3967 sng4 ICP0900166 A Remove weight 500, 1, [1100, 0] 10/04/12 17:11

Defining the database

39

Event Data Comments

Place weight 500, 2, [1300,

0]

Indicates weight is placed, how much weight, and position it was

placed. Ex. 500g was placed in position 2. Numbers in the brackets

indicate the current total weight*position for each side. E.g.

Left=1300g, right=0g (Balance achieved is represented by same no.

on both sides)

Pass weight 500 Indicates student transferred a weight to partner. Number shows

the weight, e.g., 500g

Remove

weight

500, 2, [300, 0] Indicates student removed weight from a position. Ex. 500g

removed from position 2. Bracket numbers reflect the situation

after the weight was removed.

Move weight 500, 2->3, [300,

1500]

Indicates student moved weight from one position to another.

Arrow shows the previous position (left number) and the position it

was moved to (right number). Brackets reflect current situation

after the move.

Build rule 100g * 2 = 500g

*

Indicates student clicking on rule choices when building the

elements for the rule.

Confirm rule 100g * 2 = 300g

* 1

Indicates that student finalised the rule that was built in "Build rule”

Subskills

Strand Element Description Students Activity

Participation Action Activity within

environment

Student A passes student B a mass

Participation Task completion

Perseverance

Understands task

Follows instructions

Student understands scales must

balanceStudent follows the instructions of the

task

Perspective

Taking

Responsiveness Responds to

contributions of partner/s

Student B responds to incorrect masses

sent by partner, by sending back 50g when it doesn’t balance

Task

Regulation

Sets Goals Set goals for task Student asks where partner puts mass

in anticipation of needing that information later

Task

Regulation

Collects

information

Collects information Student asks partner when unsure or

needs more information

6

Indicative actions and observations –rubric

ParticipationBehavioural Engagement -

Committing to the task

Low: Explores the task only

Middle: Attempts to complete the task

High: Completes whole task

Perspective

Taking

Audience Awareness –

Clarifying the problem to

others

Low: Insufficient Evidence

Middle: Clearly states the problem to others when

explicitly asked

High: Clearly states the problem to others without being

explicitly asked

Social Regulation

Responsibility –

Encourages shared

responsibility for the task

Low: Insufficient evidence

Middle: Uses first person plurals in communication

High: Directs other to complete tasks

Task Regulation

Strategy Application –

Systematically approaches

the problem

Low: Random trialling

Middle: Trials sequentially

High: Trials sequentially and utilises joint resources

Knowledge

Building

Monitoring & Reflecting –

Monitors cause and effect

Low: Little or no understanding of cause and effect

Middle: Identifies cause and effect

High: Plans strategy based on monitored cause and effect

System Delivery Process

Registration System

• Administrative access control

• Multi level end-user access tier

• Authentication protocols

• User access creation

• Language settings

• Test creation

• Test controls

• Administrative access control

• Multi level end-user access tier

• Authentication protocols

• User access creation


• Test creation

• Test controls

Assessment Server

• DB structure

• Storage process

• Multiplayer communication layer

• Tracking process

• Multilingual switch

• Prototypes

• Survey

• DB structure

• Storage process

• Multiplayer communication layer

• Tracking process

• Multilingual switch

• Prototypes

• Survey

Scoring Engine

• Coding method

• Scoring algorithm

• Automatic process

• Coding method

• Scoring algorithm

• Automatic process

Reporting Tool

• Report formats & style

• Delivery mechanism

• Report formats & style

• Delivery mechanism

Intervention Module

• Guidance materials

• Resources

• Guidance materials

• Resources

© Assessment Research Centre, University of Melbourne

Schematic diagram of the platform

Game

Server

Game Lobby

Game

Session

Rooms

DBMS

Scoring

Engine

Reporting

Module

Student

Portal

Teacher

Portal

TaskTaskN

Studenti A

View

TaskN

Studenti B

View

TaskTaskN

Studentj A

View

TaskN

Studentj B

View

.........

.........


System Features

Developer

• Create new (e.g. country, projects)• Access control• Create tests• Control and allocate tests

Project/ National

• Create access (e.g. school or teacher)• Access control

• Control and allocate tests


School/ Teacher

• Create access (e.g. students) Access feedback (e.g. reports)• Assign class Monitor progress

• Access control PD modules

• Control and allocate tests Language control

Student

• Test access (according to allocation)• Self Assessment (e.g. survey)


7

• Blend subject content and 21st century skills

• Prepare selves and students for an information link community

• Encourage students to think beyond discipline content

• Assist teacher education to shift towards c21 emphasis

– Reform and exemplify

• classroom practice

• School organisation

• Assessment practices

• Reporting to different audiences

Teacher – led reform needed Teacher-led reform needed

• Resources needed

– Time

– Budget

– Professional development

• Targeted teaching

• Developmental learning

• Use of progressions

• Embedding / blending

• Explicit teaching

– Materials

– School leadership and organisation

– System leadership and commitment

Teacher-led reform needed C21 modules

1. Defining and Assessing 21st Century Skills

2. Using a Developmental Model3. ATC21S Assessments: Getting Started

4. Interpreting Reports5. Teaching and Learning 21st Century Skills

• Log on

• Identity and password

• Complete the clowns and olive oil task

• Reflect on skills used

• Teaching?

• First steps?

A practice session?

8

Assign tasks

55

Task Difficulty

level

Approximate time

(minutes)

Additional notes

Laughing Clowns Easy 5-8 CPS Task (recommended as first task)

Plant Growth Easy 10-15 CPS Task

Shared Garden Easy 5-8 CPS Task

Sunflower Easy 15-20 CPS Task

Olive Oil Medium 15-30 CPS Task

Hot Chocolate Medium 10-12 CPS Task

Balance Medium 25 CPS Task

Warehouse Medium 20-30 CPS Task

Game of 20 Hard 25-30 CPS Task

Hexagons Hard 25-30 CPS Task

Pyramids Hard 25-35 CPS Task

Lightbox Easy 5-8 Example task only, will not contribute

towards report

Arctic Trek Medium 35-40 ICT Task collab

Poetry Medium 35-40 ICT Task collab

Skill examples to look for (Clowns and Oil)

Part

Part ContentSocial skills

• Action• Interaction

• Task completion• Perspective taking

Cognitive skills

• Goal setting• Collecting information• Systematicity

• Hypothesis – testing a rule or general principle • Applying a rule

© Copyright The University of Melbourne 2015

[email protected]@unimelb.edu.au

http://education.unimelb.edu.au/arc

collaborative problem solving: assessment and reporting

Documents