![Page 1: Linking Examinations to the CSE - britishcouncil.cn · Preliminary Analysis • Aptis Listening & reading ranges across approx. CSE 3 to 7 • IELTS Listening & Reading ranges across](https://reader036.vdocuments.mx/reader036/viewer/2022081723/5b89e6547f8b9a5b688e996b/html5/thumbnails/1.jpg)
Linking Examinations to
the CSE
Barry O’Sullivan British Council
Assessment Research Group
Plenary New Directions Follow-Up Event Daqing – April 25, 2008
Guangzhou – April 27, 2018
![Page 2: Linking Examinations to the CSE - britishcouncil.cn · Preliminary Analysis • Aptis Listening & reading ranges across approx. CSE 3 to 7 • IELTS Listening & Reading ranges across](https://reader036.vdocuments.mx/reader036/viewer/2022081723/5b89e6547f8b9a5b688e996b/html5/thumbnails/2.jpg)
Overview
• Introduction
• Theoretical Basis of the CSE
• Conceptualising Linking
• Applying the Approach
• Conclusions
![Page 3: Linking Examinations to the CSE - britishcouncil.cn · Preliminary Analysis • Aptis Listening & reading ranges across approx. CSE 3 to 7 • IELTS Listening & Reading ranges across](https://reader036.vdocuments.mx/reader036/viewer/2022081723/5b89e6547f8b9a5b688e996b/html5/thumbnails/3.jpg)
INTRODUCTION why i am talking to you now…
![Page 4: Linking Examinations to the CSE - britishcouncil.cn · Preliminary Analysis • Aptis Listening & reading ranges across approx. CSE 3 to 7 • IELTS Listening & Reading ranges across](https://reader036.vdocuments.mx/reader036/viewer/2022081723/5b89e6547f8b9a5b688e996b/html5/thumbnails/4.jpg)
The Linking Project
MEMORANDUM OF UNDERSTANDING
on Conducting the Collaborative Research on Linking UK English Tests to China’s Standards of English
NEEA British Council
Identify an appropriate set of procedures through which test developers should demonstrate a theoretical and
empirical link between a specific test and the CSE
![Page 5: Linking Examinations to the CSE - britishcouncil.cn · Preliminary Analysis • Aptis Listening & reading ranges across approx. CSE 3 to 7 • IELTS Listening & Reading ranges across](https://reader036.vdocuments.mx/reader036/viewer/2022081723/5b89e6547f8b9a5b688e996b/html5/thumbnails/5.jpg)
The Learning System
Curriculum
Delivery Assessment
Teacher training Text books
Classroom design Etc.
Formative [developmental]
Summative [judgemental]
STANDARDS
Linking Project
![Page 6: Linking Examinations to the CSE - britishcouncil.cn · Preliminary Analysis • Aptis Listening & reading ranges across approx. CSE 3 to 7 • IELTS Listening & Reading ranges across](https://reader036.vdocuments.mx/reader036/viewer/2022081723/5b89e6547f8b9a5b688e996b/html5/thumbnails/6.jpg)
The Learning System
Curriculum
Delivery Assessment
China Standards of
English
Across ALL education levels
Teachers, Textbooks, Classroom etc.
Zhongkao, Gaokao, CET4, CET6 etc.
![Page 7: Linking Examinations to the CSE - britishcouncil.cn · Preliminary Analysis • Aptis Listening & reading ranges across approx. CSE 3 to 7 • IELTS Listening & Reading ranges across](https://reader036.vdocuments.mx/reader036/viewer/2022081723/5b89e6547f8b9a5b688e996b/html5/thumbnails/7.jpg)
Standards
Test A Curriculum A Textbook A Common
Understanding
Test X Test Y Test Z
Curriculum X Curriculum Y Curriculum Z
Textbook X Textbook Y Textbook Z
Common Understanding
C S E
![Page 8: Linking Examinations to the CSE - britishcouncil.cn · Preliminary Analysis • Aptis Listening & reading ranges across approx. CSE 3 to 7 • IELTS Listening & Reading ranges across](https://reader036.vdocuments.mx/reader036/viewer/2022081723/5b89e6547f8b9a5b688e996b/html5/thumbnails/8.jpg)
THEORETICAL BASIS OF THE CSE underlying models
![Page 10: Linking Examinations to the CSE - britishcouncil.cn · Preliminary Analysis • Aptis Listening & reading ranges across approx. CSE 3 to 7 • IELTS Listening & Reading ranges across](https://reader036.vdocuments.mx/reader036/viewer/2022081723/5b89e6547f8b9a5b688e996b/html5/thumbnails/10.jpg)
Moving Beyond Bachman
LANGUAGE MODEL
LANGUAGE USE MODEL
LEARNER MODEL
Implied in contextualisation of CSE in China’s Education System
![Page 11: Linking Examinations to the CSE - britishcouncil.cn · Preliminary Analysis • Aptis Listening & reading ranges across approx. CSE 3 to 7 • IELTS Listening & Reading ranges across](https://reader036.vdocuments.mx/reader036/viewer/2022081723/5b89e6547f8b9a5b688e996b/html5/thumbnails/11.jpg)
A Socio-Cognitive CSE
LANGUAGE USE MODEL
LEARNER MODEL
Implied in contextualisation of CSE in China’s Education System
LANGUAGE MODEL
![Page 13: Linking Examinations to the CSE - britishcouncil.cn · Preliminary Analysis • Aptis Listening & reading ranges across approx. CSE 3 to 7 • IELTS Listening & Reading ranges across](https://reader036.vdocuments.mx/reader036/viewer/2022081723/5b89e6547f8b9a5b688e996b/html5/thumbnails/13.jpg)
CONCEPTUALISING LINKING the approach
![Page 14: Linking Examinations to the CSE - britishcouncil.cn · Preliminary Analysis • Aptis Listening & reading ranges across approx. CSE 3 to 7 • IELTS Listening & Reading ranges across](https://reader036.vdocuments.mx/reader036/viewer/2022081723/5b89e6547f8b9a5b688e996b/html5/thumbnails/14.jpg)
Sources & Types of Evidence
Construct Related The trait/ability being tested
Measurement Related Consistency & accuracy
Criterion Related Comparison with other estimates
![Page 15: Linking Examinations to the CSE - britishcouncil.cn · Preliminary Analysis • Aptis Listening & reading ranges across approx. CSE 3 to 7 • IELTS Listening & Reading ranges across](https://reader036.vdocuments.mx/reader036/viewer/2022081723/5b89e6547f8b9a5b688e996b/html5/thumbnails/15.jpg)
Overview of the Process
What is the test testing? How is it doing this?
Not OK BIN
What does a panel of experts say about the
level of the test?
OK
How do learners perform on the test (with teacher
estimates of level)?
OK
Do I have evidence of level from other research
projects?
OK
OK Accept Claim
Not OK BIN
Not OK
Not OK BIN
Weak Claim only
![Page 16: Linking Examinations to the CSE - britishcouncil.cn · Preliminary Analysis • Aptis Listening & reading ranges across approx. CSE 3 to 7 • IELTS Listening & Reading ranges across](https://reader036.vdocuments.mx/reader036/viewer/2022081723/5b89e6547f8b9a5b688e996b/html5/thumbnails/16.jpg)
APPLYNG THE APPROACH linking aptis and ielts to the cse (initial phases)
![Page 17: Linking Examinations to the CSE - britishcouncil.cn · Preliminary Analysis • Aptis Listening & reading ranges across approx. CSE 3 to 7 • IELTS Listening & Reading ranges across](https://reader036.vdocuments.mx/reader036/viewer/2022081723/5b89e6547f8b9a5b688e996b/html5/thumbnails/17.jpg)
4 Skills (+ G&V) Reports 5 papers 0-50 Reports CEFR by skill Reports overall CEFR (4 skills only) Soon to report CSE
![Page 18: Linking Examinations to the CSE - britishcouncil.cn · Preliminary Analysis • Aptis Listening & reading ranges across approx. CSE 3 to 7 • IELTS Listening & Reading ranges across](https://reader036.vdocuments.mx/reader036/viewer/2022081723/5b89e6547f8b9a5b688e996b/html5/thumbnails/18.jpg)
Project Description
Steering Group
NEEA – CSE – British Council – Cambridge
Working Group
Establish the linking process Pilot the process (using Aptis)
Apply the process to IELTS
Objectives
![Page 19: Linking Examinations to the CSE - britishcouncil.cn · Preliminary Analysis • Aptis Listening & reading ranges across approx. CSE 3 to 7 • IELTS Listening & Reading ranges across](https://reader036.vdocuments.mx/reader036/viewer/2022081723/5b89e6547f8b9a5b688e996b/html5/thumbnails/19.jpg)
Construct Phase – Document Analysis
Comparing Construct Definition
Descriptors, Specifications & Reporting Scales
IELTS
CSE
Aptis
![Page 20: Linking Examinations to the CSE - britishcouncil.cn · Preliminary Analysis • Aptis Listening & reading ranges across approx. CSE 3 to 7 • IELTS Listening & Reading ranges across](https://reader036.vdocuments.mx/reader036/viewer/2022081723/5b89e6547f8b9a5b688e996b/html5/thumbnails/20.jpg)
Construct Phase – Expert Evidence
Expert Panel
24 members
CSE
NEEA Education System
British Council Cambridge
Judgement Round 1
Analysis & Report
Judgement Round 2
Analysis & Report
Discussion
Preliminary Decision
Analysis
Final Decision
Aptis
Judgement Round 1
Analysis & Report
Judgement Round 2
Analysis & Report
Discussion
Preliminary Decision
Analysis
Final Decision
IELTS
R E V I E W
![Page 21: Linking Examinations to the CSE - britishcouncil.cn · Preliminary Analysis • Aptis Listening & reading ranges across approx. CSE 3 to 7 • IELTS Listening & Reading ranges across](https://reader036.vdocuments.mx/reader036/viewer/2022081723/5b89e6547f8b9a5b688e996b/html5/thumbnails/21.jpg)
Construct Phase – Some Findings
Expert Panel • Overwhelmingly positive about experience
• Similarly confident in outcomes
Preliminary Analysis • Aptis Listening & reading ranges across approx. CSE 3 to 7
• IELTS Listening & Reading ranges across approx. CSE 4 to 8
![Page 22: Linking Examinations to the CSE - britishcouncil.cn · Preliminary Analysis • Aptis Listening & reading ranges across approx. CSE 3 to 7 • IELTS Listening & Reading ranges across](https://reader036.vdocuments.mx/reader036/viewer/2022081723/5b89e6547f8b9a5b688e996b/html5/thumbnails/22.jpg)
CONCLUSIONS looking back and looking forward
![Page 23: Linking Examinations to the CSE - britishcouncil.cn · Preliminary Analysis • Aptis Listening & reading ranges across approx. CSE 3 to 7 • IELTS Listening & Reading ranges across](https://reader036.vdocuments.mx/reader036/viewer/2022081723/5b89e6547f8b9a5b688e996b/html5/thumbnails/23.jpg)
Local & International Reporting
Aptis or
IELTS
![Page 24: Linking Examinations to the CSE - britishcouncil.cn · Preliminary Analysis • Aptis Listening & reading ranges across approx. CSE 3 to 7 • IELTS Listening & Reading ranges across](https://reader036.vdocuments.mx/reader036/viewer/2022081723/5b89e6547f8b9a5b688e996b/html5/thumbnails/24.jpg)
Local & International Reporting
Zhongkao or
Gaokao or
CET
![Page 25: Linking Examinations to the CSE - britishcouncil.cn · Preliminary Analysis • Aptis Listening & reading ranges across approx. CSE 3 to 7 • IELTS Listening & Reading ranges across](https://reader036.vdocuments.mx/reader036/viewer/2022081723/5b89e6547f8b9a5b688e996b/html5/thumbnails/25.jpg)
Test Comparisons?
Zhongkao or
Gaokao or
CET
Aptis or
IELTS
THAT’S A WHOLE DIFFERENT PROCESS
![Page 26: Linking Examinations to the CSE - britishcouncil.cn · Preliminary Analysis • Aptis Listening & reading ranges across approx. CSE 3 to 7 • IELTS Listening & Reading ranges across](https://reader036.vdocuments.mx/reader036/viewer/2022081723/5b89e6547f8b9a5b688e996b/html5/thumbnails/26.jpg)
Summary of Linking Work
Approach to linking agreed
Practical procedures agreed
First Phases of Listening & Reading linking completed
Project completed & published by the end of 2018
![Page 27: Linking Examinations to the CSE - britishcouncil.cn · Preliminary Analysis • Aptis Listening & reading ranges across approx. CSE 3 to 7 • IELTS Listening & Reading ranges across](https://reader036.vdocuments.mx/reader036/viewer/2022081723/5b89e6547f8b9a5b688e996b/html5/thumbnails/27.jpg)
Issues
Internationalisation of China’s education system
Improved quality of education system in general
Encourage systematic progression through the system
Pressure on curricula to clarify link to the CSE
Pressure on textbooks to clarify link to CSE
Pressure on tests to clarify link to CSE
![Page 29: Linking Examinations to the CSE - britishcouncil.cn · Preliminary Analysis • Aptis Listening & reading ranges across approx. CSE 3 to 7 • IELTS Listening & Reading ranges across](https://reader036.vdocuments.mx/reader036/viewer/2022081723/5b89e6547f8b9a5b688e996b/html5/thumbnails/29.jpg)
Thank You