aligning levels to the european framework the origó experience budapest, february 2006
TRANSCRIPT
Aligning levels to the Aligning levels to the European FrameworkEuropean Framework
The Origó ExperienceThe Origó Experience
Budapest, February 2006Budapest, February 2006
The number of ORIGÓ exams in 2005The number of ORIGÓ exams in 2005(oral and written):(oral and written):
108 401108 401
ITK ORIGÓITK ORIGÓ
• The number of languagesThe number of languages:: 2929
• Arabic, Beás- Romani, Bulgarian, Chinese, Croatian, Czech, English, Danish, • Esperanto, Finnish, French, German, Hebrew, Hungarian, Italian, • Latin, Lovari-Romani, Modern-Greek, Old-Greek, Polish, Portugese, • Romanian, Russian, Serb, Spanish, Slovakian, Slovenian, Swedish, Turkish
• 2 testing systems2 testing systems:: Hungarian as a Foreign LanguageHungarian as a Foreign Language - - ALTE-membershipALTE-membership
The ratio of languages (%) in 2005The ratio of languages (%) in 2005
English 53,0German 25,5Esperanto 6,4French 3,3Lovari 2,5Italian 2,5Spanish 1,7Russian 1,1Latin 1,4Croatian 0,4Romanian 0,3Serb 0,2Slovakian 0,3Other 2,0
The number of exam centresThe number of exam centres::
70 - accredited/registered70 - accredited/registered
Oral Written
69% 49%
Pass ratePass rate
20052005
Population characteristics: age and sexPopulation characteristics: age and sex
female
male
64%
36%20052005
0
1000
2000
3000
4000
5000
6000
7000
8000
14 21 28 35 42 49 56
férfi
nő
Distribution: age and sex, 2005Distribution: age and sex, 2005
0
1000
2000
3000
4000
5000
6000
14 16 18 20 22 24 26 28 30 32 34 36 38 40 42
Candidates
• The oldest: 1935. 06. 23., female - Basic German, C, Pass
• The youngest: 1991.12. 14., female - Basic German, A, Passed
• Maximum performance (oral): 62 candidates 100%: English, Arabic, German, Esperanto, Latin, Hungarian, Russian,
Italian, Rumanian, Spanish, Serb, Slovakian, Modern-Greek, Hebrew - Basic-, Intermediate-, Advanced levels
• Maximum performance (written): 9 candidates 100%: Basic English- (5), Basic Hungarian-, Intermediate Italian-, Basic
Spanish-, Basic Modern-Greek
The Framework levels and The Framework levels and Origó levels ???Origó levels ???
EuropeanCouncil
ITK ORIGÓ Accreditedlanguage
exams
Maturaexams
Cambridge Goethe
C2Mastery CPE
ZOP,KDS
C1Operationalproficiency
Proficiency/Advanced
Proficiency/Advanced
CAE ZMP
B2Vantage Intermediate Intermediate Advanced FCE
B1Threshold Basic Basic PET ZD
A2Waystage
Junior not accredited IntermediateKET
A1Breakthrough
notaccredited
not accredited
The Hungarian context
• the law 1st September, 2006
• the Accreditation Board July, 2006
• the Matura - one-way equivalence
• international consensus?• levels?
Schedule
• familiarization - scales• specification - documentation• standardization - scales, calibrated samples • empirical validation - calibrated samples,
level setting
Origó - Previous experience
• level setting - empirical
alinging Rigó and international exams (ESU framework, Goethe)
• quality control procedures
• accreditation (1999)
Origó - work accomplished
• schedule• familiarisation (exam developers)
• specification - modifications in Origó scales
suggestions for other modifications
• familiarisation - all oral examiners
Origó - work to follow:
• familiarisation - item writers, markers (written exams)
• benchmarking - sample materials (exam developers)
• standardisation - all examiners
• empirical validation - major languages
Origó - benefits
• positive working experience (CEFR)• useful guidelines (Manual)• useful discussions (awareness raising)• useful modifications
Origó - problem areas
• terminology• translation problems • advanced (C1)/proficiency (C2) levels• availability of sample reference materials• mediation
Terminology„claims”
• linking• relating• harmonizing• aligning• level setting
Translation problems
• „claim”• „a degree of spontaneity and fluency”
Levels
Proficiency (C2)
Advanced (C1)
Difficulties with sample materials
videos in English (Eurocentre, Cambridge)
quality (copy of a copy)
accents (international)
levels
Jihun B2 / intermediate
0
5
10
15
20
25
30
35
40
near pass at level good pass
Estelle B2- / intermediate
0
5
10
15
20
25
30
35
good basic weak interm. interm.
Origó 2004 / intermediate
0
10
20
30
B1 12 B2- 13
B1+ 19 B2 28
B2- 12 B2+ 5
Gábor, 30 Lilla, 35
Renate - / B1
0
5
10
15
20
25
30
35
40
B1 B2
Annemarie - / C1
0
5
10
15
20
25
30
B2 C1
Evgeny C1 / Proficiency
0
5
10
15
20
25
good interm. weak prof. prof.
Familiarization - scales
Pragmatic competence item 4 A1:2 A2:135 B1:6
item 3 B1:53 B2:83
Strategic competence item 1 A2:46 B1:78 B2:13
item 5 B2:26 C1:106
Language competence item 1 B2:3 C1:94
item 4 B1:34 B2:59
Interpretation of data
IRT information - Eurocentre
- Dialang, Cambridge, Finnish Matriculation
not enough background info for reproduction
no measuring instruments attached
Reference measures
Eurocentre Dialangreading
Cambridgereading
Finnishreading
C2 3.90 2.01 70-90 (80) 47-50
C1 2.80 1.23 61-81 (71) 40-46
B2 0.72 0.21 57-72 (62) 25-39
B1 -1.23 -.22 50-70 (56) 8-24
A2 -3.23 -.56 30-50 (41) 4-7
A1 -4.29 -.94 3
Origó IRT measures (grammar)
Intermediate Proficiency
English 1.10 – 1.25 1.60 – 1.80
German 1.00 –1.20 1.30 – 1.70
French 1.20 – 1.40
Spanish 1.10 – 1.40
Italian 1.50 – 1.80
Russian 1.20 – 1.50
Esperanto 1.20 – 1.30
Lovari 1.20 – 1.30
Latin 1.60 – 1.90
Origó - Mediation
• major difficulty: no CEFR scales• no Council of Europe sample materials• external validation? • accreditation?