Authentic Learning and Assessment in Applied Statistics

Elke Thönnes,

University of Warwick

UCL/RSS Symposium on Teaching Statistics in Higher Education. 03 April 2019

ASA Curriculum Guidelinesfor undergraduate programs in statistical science (2014)

The main goal of our recommendations is to ensure under-graduate statistics students remain useful in a world with increasingly more complex data.

Pedagogy should emphasise authentic real-world data and substantive applications related to the statistical analysis cycle.

Authentic achievementArchbald & Newmann(1988) Beyond standardized testing: Assessing authentic academic achievement in the secondary school.

Does the information collected (in assessment) present an accurate estimate of worthwhile knowledge and (meaningful) forms of mastery?

Characteristics of authentic achievement:

• disciplined enquiry

• integration of knowledge

• value beyond education

Authentic assessment

Wiggins (1990) The case for authentic assessment. Practical assessment, research & evaluation, 2(2), 1-6.

• based on tasks that mirror priorities and “ill-structured” challenges in “real-world” tests of ability;

• expects polished, thorough and justifiable answers, performances or products;

• achieves validity and reliability.

Elements of authenticity

Cumming and Maxwell (1999) Contextualising authentic assessment. Assessment in education: Principles, Policy & Practice, 6(2), 177-194.

• higher order thinking

• specific domain knowledge

• emulation of ‘real-world’ activities

• communication of understanding

• complex holistic activities

• mastery and emulation of expert performance

Framework for authenticityGulikers et al (2004): A five-dimensional framework for authentic assessment. Educational technology research and development, 52(3), 67.

authentic learning

authentic achievement

perception of authenticity

authentic assessmentauthentic instruction

ASA Curriculum Guidelinesfor undergraduate programs in statistical science (2014)

Practical skills learned should include:

• technical writing, presentation skills, and visualizations

• teamwork and collaboration

• organisation and management of projects

• ability to interact with and communicate with a variety of clients and collaborators

Applied Statistics at WarwickST404 Applied Statistical Modelling (joint with D. Rossell, R. Crossman)

ST344 Professional Practice of Data Analysis (joint with D. Firth)

Aims:

• introduce the art of data analysis and statistical model-building

• give students an appreciation that real problems may not have precise clear-cut solutions;

• provide the skills to translate technical know-how into professional practice.

“Math is Music; Statistics is Literature” De Veaux and Velleman (2008) Amstat News, 378, 54-56.

Set-up• MORSE, Maths & Stats, Data Science.• sizeable proportion of international students• ST344/ST404: 100% coursework assessed • data analysis/statistical modelling projects;• data set provided, but large and messy; students may source additional data;• projects run over 4-5 weeks; ~50% based on group work;• support:

• ST404: lectures throughout term, computer practicals timed with assignment, discussion forum;

• ST344: initial lectures and computer practicals, online material, weekly non-structured supervisions;

• workshop on teamwork + digital workbook;

Framework for authenticityGulikers et al (2004): A five-dimensional framework for authentic assessment. Educational technology research and development, 52(3), 67.

Degree of resemblance to professional practice in 5 dimensions:

• physical context

• social context

• assessment task

• assessment result or form

• criteria and standards

Social contextStudents are allocated to teams by module leader to ensure diversity.

For ST344: stratified sampling based on criteria agreed with students.

In my experience …

• Students often need support to establish a functioning team and manage diversity;

• Reflection tasks help students to turn experience into learning;

• Students need to be able to communicate their teamwork competencies to employers.

A digital resource for multinational student teams

• To promote inclusion and integration in multinational student teams• To provide an interactive tool to support their reflections on their teamwork experiences• Four core components:

• Communication patterns• Trying out ideas• Work patterns• Giving feedback

Joint with Sophie Reissner - Roubicek, Thomas Greenaway, Xiaozhe Cai

Who do you trust?

Oxfam GB (2007): Building Trust in Diverse Teams: The Toolkit for Emergency Response

• 10 trust criteria

• swift trust versus deep trust

• high context versus low contextPhoto by Kazuend on Unsplash

What do you think? Use your green/red vote

I agree more than disagree with this statement

I disagree more than agree with this statement

Green

Red

Statement A

Getting something absolutely right is more important than delivering it on time

Statement B

It’s ok to interrupt other people in your team when they are saying something you disagree with

Statement C

It’s more important to share your idea … than wait until you can express it clearly

Statement D

When you don’t understand something, it’s best to keep quiet about it

Framework for authenticityGulikers et al (2004):

Degree of resemblance to professional practice in 5 dimensions:

• physical context

• social context

• assessment task

• assessment result or form

• criteria and standards

Assessment tasksThe UK had a General Election in June 2017, to elect members of the House of Commons. The previous General Election had been in 2015; and between those two elections, in 2016, there was the Referendum on the UK’s continued membership of the European Union. A local political party and a regional newspaper group have combined to commission from your team a report on electoral change between the 2015 and 2017 elections, including analysis of how the changes in voting (between the two elections) were related to the 2016 referendum results and to constituency-level demographic data. The report’s regional sponsors are interested specifically in whether (and if so how) such relationships in their region differ from the national picture.

Happiness Economics is a strand of Economics concerned with happiness, wellbeing and life satisfaction. National and international surveys such as the Gallup World Poll are aimed at measuring well-being. Your team has been commissioned by global policy makers to write a report on the relationship between various socio-economic variables and national life satisfaction as measured by the mean value on the Cantril Life Ladder. They are also interested in whether there are regions in the world in which the relationships differ from the general global picture.

Assessment formST404:• Reports with a findings section addressed at lay audience and a methods section and

technical appendix for a peer audience• Oral presentation for a peer audience• Poster presentation for mixed audience• Anonymous feedback to team members• Reflective writing on team work experience

ST344:

• Project plan

• 1 x report addressed at a lay audience; 1 x report addressed at a peer audience

• Technical appendices in RMarkdown

• Oral presentation for peer audience

• Data storyboard for lay audience

• Reflective writing on team work experience

Authentic assessment

• Have real-world relevance, are ill-defined

• Comprise complex tasks investigated over time, using variety of resources

• Create polished products valuable in their own right

• Allow competing solutions and diversity of outcome

Herrington, Reeves, and Oliver, R. (2010) A Guide to Authentic e-Learning. Routledge, New York.

Criteria and Standards

Students may decide on differential weighting (within limits) of the group mark across team members.

• Sound and well explained statistical analysis supported by appropriate numerical and graphical evidence.

• Clear and well structured presentation appropriate to the target audience.

• Clear, well documented and functioning code in RMarkdown document.

Criteria and Standards

Adapted from James and Brookfield (2014). Engaging imagination: Helping students become creative and reflective thinkers. John Wiley & Sons, 2014.

• Checking the assumptions that inform your actions and judgements. 

• Trying to understand how another person reasons.

• Looking for blind spots and omissions in your thinking.

• Identifying what is justified and well grounded in your thinking.

• Integrating emotions and intuitions into a cognitive analysis.

Challenges

• assessing process versus assessing product

• task orientation

• time frame

• sequential nature of tasks

• team roles

• comfort zones