Problem & Need Statement

Results

Conclusions & Further Findings

Assessment of Soccer Referee Proficiency in Time-Sensitive Decision-MakingNathan Jones, Andrew Cann, Saud Almashhadi, Hina Popal

System Engineering & Operations Research, George Mason University

Method of Analysis

Junior level referees do not receive assessments for game flow understanding or fitness attributes

as predictors of call accuracy.

An assessment method is needed to evaluate junior referees based on fitness and/or game flow

understanding attributes.

# Design Alternatives Description Tests

1 Fitness TestA baseline fitness test equivalent to those administered to senior

refereesFitness

2 Game Flow Evaluation

Video performance assessments conducted by official assessors GFU

3 Combined Evaluation

Combination of first two evaluations

FitnessGFU

4 No Assessment Not conducting any referee evaluations (status quo) None

A two part analysis was conducted to determine alternative utilities:Utility is defined as the average call accuracy of the top 100 junior referees

identified using each alternative.Part I: A regression for call accuracy based on fitness and GFU developed using

a discrete event soccer game simulator.Part II: Utility of each alternative determined through a Monte Carlo analysis

using regression from part I.

Part I: Soccer Game SimulatorFitness and GFU range from 0

(worst possible) to 100 (best possible)

Simulation output regression analysis (R2 = 99.51):Call Accuracy (Fitness, GFU) = 0.713491 + 0.000923486

Fitness + 1.28791e-5 GFU - 6.4846e5 Fitness2 + 1.12504e-6 GFU2 + 1.26193e-6 Fitness3 - 6.75305e-9 Fitness4

 

Alternative Utility 95 % Utility Half-Width

Fitness Test 0.74926 0.00012Game Flow Evaluation 0.72693 0.00028

Combined Evaluation 0.74174 0.00021

No Assessment 0.72099 0.00004

Part II: Monte Carlo Analysis

“Fitness Test” dominates all other assessment based alternatives.

Fitness Test vs. No Assessment (status quo) Marginal CostFitness Test: $26,990

Marginal Utility Fitness Test:

0.028 Utility Improvement

It is not cost effective to implement fitness tests on junior referees.

Recommendation

Team combination and game flow have a significant impact

on referee call accuracy.

Further Findings I:Impact of Teams

Context

Referee Attributes Attribute Assessment Method

Fitness Fitness Test (senior referees only)

Call Decision Making (CDM) Written exam on rules (all referees)

Game Flow Understanding (GFU) Indirectly using on field assessment(senior referees only)

Further Findings II: RecommendationWhen evaluating referee quality based on

game performance, team combination must be considered as a potential confounding

variable in the analysis.

Referee Call Making Process Analysis Part I: Soccer Game Simulator

Simulation developed in Java