assessments of national educational progress, school improvement and students’ learning. dynamics...
Upload: istituto-nazionale-di-documentazione-innovazione-e-ricerca-educativa
Post on 09-Aug-2015
140 views
TRANSCRIPT
Kit-Tai HauThe Chinese University of Hong Kong
1
Assessments of National Educational Progress, School Improvement and
Students’ Learning – Dynamics and Practices in Asia
Outstanding Asian Performance Converging evidence:– International studies: Asian outstanding over widely
diverse representative population– e.g., 2nd IEA: 13-yr China, Taiwan among the top – TIMSS: G.4, 8 Hong Kong, Japan, South Korea,
Singapore outperformed others– PISA: Singapore, Hong Kong, S Korea top 3– Recent PISA (2012): Shanghai (China), 1st in all 3
subjects (Reading, Maths, Science), Hong Kong 2nd, 2nd, 3rd
2
– Hong Kong, Taiwan, China, Japan, South Korea (except Singapore) have Ed Monitoring systems, some (Hong Kong, S Korea, part of Japan, some parts in China) producing individual school reports to the public (league table)
– Will use Hong Kong as an example to illustrate the dynamics /issues involved
3
Presentation OutlineI. Educational Monitoring in SE Asia/PacificII. General FrameworkIII. Hong Kong Attainment Test (HKAT)IV. Problems of HKATV. Consultancy Report: Can one test serve all?VI. Divided into: Student + System AssessmentsVII. Student AssessmentVIII. System AssessmentIX. Positive Uses of Monitoring Assessment ResultsX. Current Problems/Issues
4
I. Ed Assessment in SE Asia/PacificCountry Level Subjects First YrAustralia G 3, 5, 7, 9 Literacy, Numeracy 2008
G 6, 10 ICT 2005G 6, 10 Science (Sc) 2003G 6, 10 Civil Knowledge 2004
Bangladesh G 3, 5 Language (Lang), Math 2006G 1, 5 Lang, Eng, Math, Sc, Soc Sc 2004
G 8 Lang, Eng, Math 2008Bhutan G 2, 4 Lang, Eng, Math 2007
G 4, 6, 8 Eng, Math, Sc 2008G 6 Eng, Math 2003G 10 Lang, Eng, Math, History 2006
5
Country Level Subjects First YrCambodia G 3, 6, 9 Lang, Math 2005China G 4, 8 Chinese, Eng, Math, Sc, Psycho
health, physical Ed/ Health2007
Cook Isld G 4, 6 Lang, Eng, Math 1999G 4, 5, 6 Eng, Lang 1994G 3, 7 Math 1994
Fuji G 4, 6, 8 Literacy, Numeracy 2007India G 3 Lang, Math 2004
G 5 Lang, Math, Enviro Sc 2002G 8 Lang, Math, Sc, Soc Sc 2003Age 5-16 Literacy, Numeracy 2005
Japan G 6, 9 Jap, Math, Sc 2007Kiribati G 4, 6 Lang, Eng, Numeracy 2004Lao G 1, 2, 5 Lang, Math, Sc 1996Maldives G 4, 7 Lang, Eng, Math, EnvSc, SocSc 2008
G 9 Phys, Chem, Bio, Hist, Geog 20136
Country Level Subjects First YrMyanmar G 3, 5 Lang, Math 2007Nepal G 3, 5, 6,
8, 10 Nepali, Math, Soc Studies, Eng, Sc, Health
1999
N Zealand G 4, 8 Sc, Visual Arts, ICT, Lang, Tech, Music, Math, Soc Studies, Health
1995
Pakistan G 3, 4, 8 Lang, Maths, Sc, Soc Studies 2003Philippines G 3, 6, 12 Eng, Filipino, Math, Sc, Soc Sc,
Critical thinking2004
S Korea G 6, 9, 11 Korean, Eng Math, Sc, Soc Studies 2000Samoa G 4, 6 Samoan, Eng, Numeracy 1996Singapore G 6, 10,
12Lang, Math, Sc, Soc Sc, Applied subjects
1960
Salomon I G 4, 6 Literacy, Numeracy 2005Sri Lanka G 4, 8, 10 Singalese, Tamil, Eng, Math, Sc,
Tech2003
7
Country Level Subjects First YrThailand G 2, 3, 5,
6, 9, 11, 12
Thai, Foreign Lang, Math, Sc, Soc Sc, Religion, Health, PE, Art, Career, Occupational Studies
2005
Tonga G 4, 6 Tongan, Eng, Numeracy 2011Tuvalu G 4, 6 Literacy, Numeracy 2000Vanuatu G 4, 6 Literacy, Numeracy 2007Vietnam G 5, 6, 9,
11Vietnamese, Math, Eng, Physics, Bio
2001
8
II. General Framework: Monitor + Feedback
9
Level Purpose/ Help……
Comparison targets
Student Student learn
Own past, classmate, national norms
Class Teacher teach
previous yr, other classes, national norm
School School Accountable
previous yr, other schools, national norm
State/City
Plan, accountable
previous yr, other states, national norm
Country Monitor progress
previous yr, other countries (e.g., PISA,TIMSS)
III. Hong Kong Attainment test (HKAT)• 1976-2003• One test at each grade, G.1 - 9• Subjects: Chinese, English, Maths• Operation: – test/questions left with school– changed every few years – marked by own teachers– school submit 1/30 (e.g., birth day 13th) for Ed Bureau
to mark (build HK norm, monitor HK)– later change to 1/3 to monitor school
• Main purpose: to monitor whole Hong Kong
10
• Little information provided:– for students: question paper too short, no
diagnostic value– for schools: only a relative ranking (cf. HK norm)– for whole Hong Kong: only a summary total (2015
M=87.6, no other information)• Once a year, insufficient to help daily learning• Teachers already have other means (school tests) to
understand own students’ strength/weakness• Teachers’ heavy load in marking• Paper too easy for top schools
11
IV. Problems of HKAT
V. Consultancy Report (review 2000): Can one test serve both purposes (Monitor/ Feedback)?
12
• Seek consultant’s advice
• Visit ETS, etc.• Public
consultation
Prof W. J. van der Linden
Can one test serve all purposes?Student School Country
Once a year Not enough Too much, has to rotate subjects as in PISA
paper long short
Good, but expensiveCoverage not enough
Each student answers part of the questions
Difficulty:key competencemid difficulty
Not enough discrimination, low qualityBest psychometric properties, but students, parents feel too difficult
High Stake Narrowing of curriculum, teachers cheat results not valid
13
Ideal Monitoring System• To Student: test daily, auto-marking, low stake, can retake, in
line with self-regulation• To school/city/country: should attempt to reduce the
unavoidably high stake, can change focus each year (academic subject on rotation, as in PISA)
• Schools have to supervise test administration, to reduce incentive in cheating, should not allow school to publicize results
• If cheating too serious/pressure on students too large, do not report school results, only provide higher level summaries
• But too costly if the whole scheme is just to monitor the country and does not produce school reports
14
VI. Divided into Student + System Assessments
Decision of the HK government in 2000:•Will monitor Chinese, English, Maths basic competence•To help learning + monitor school/provide support
15
(A)Secured test (G.3, G.6, G.9)System Assessment (slightly high stake)
(B)Web based adaptive test(G.1-G.9)(Student Assessment)
Hong Kong Attainment test
VII. Student Assessment• Web based• Purpose: help teachers in assessing their students• Supplement and adapt to students’ need,
difficulty appropriate to students’ ability• Can fully serve the daily need throughout the
school term
16
17
18
19
20
Problems of Student Assessment
21
• In-charge by the Examination Authorities (HK Examinations and Assessment Authorities): high quality but too few items
• Not enough promotion of usage, self-regulation, self-monitoring function
• Student mark recording system too complicated (does not facilitate widespread use)
• Does not link to learning or remedial packages, thus cannot be a fully standalone learning system
VIII. System Assessment• Mandatory tests for all students in all government
funded schools• To understand the achievement of basic competence• Only report % students (e.g., 85%) meeting basic
competence in each school, cannot publicize to avoid building league table and creating additional pressure to students/schools
• On Chinese/English/Maths, 30-90 min G.3, 65-1 – 2 hour G.9
• Only at G.3, 6, 9 (no G.1, cannot calculate value added for primary schools)
22
• One external teacher help invigilation• Centrally marked• Include even oral exam for Chinese, English (only randomly select 10%-20% of
students)
• Example report at system level (% meeting Basic Competence
23
G.3 G.6 G.9Chinese 85% 77% 76%English 79% 71% 69%Maths 87% 84% 80%
IX. Positive uses of Monitoring Assessment Results
• Analyses on item/domain performance on student + system assessment can generate useful feedback on school performance (on their strength/weakness)
• Exam pressure, over-drilling already serious in Asia Singapore no monitoring tests, other countries careful not add pressure through school reports, league tables
• In PISA, Hong Kong 1st in equity - Social Economic Status (SES) has least effect on students’ achievement
24
• Help for new migrants, low SES students through– Extra funds when schools have (i) new migrants,
(ii) low SES, (iii) SEN (special ed need) students – but must allow migrants to join System Assessment (for accountability)
– Ed Bureau (government) special team to provide school based professional support: jointly prepare teaching/ assessment materials
– University-school partner teams (supported by government funds) help needy schools
25
X. Current Problems and IssuesSystem Assessment •Only % pass/fail basic competence, no fine grade •After a few years, as these figures are quite stable, no one is interested, no one care (good, no pressure on students), but little diagnostic information as well•Items focus basic competence too easy low discrimination low quality in monitoring whole Hong Kong more difficult items added recently•To enrich the reports, now providing item performance information for schools in System Assessment but schools drill on identical item types (possible solution generate item analyses from Student Assessment)
26
• Not rotating academic subjects across years yet• questionnaire now added on students’ attitude,
family background, etc.• Low monitoring in test administration schools may
cheat, not totally cheating-proof• Item types too narrow, not creative enough, not
having a positive effect on teaching• Population declines primary schools great
pressure to drill to get good results asked government not to conduct test/or release results now stop releasing results to primary schools
• Secondary schools (G.9-12) have a high stake university entrance exam anyway results continued to be released to them
27
• Student Assessment– Not widely used by students– Insufficient coverage of fully curriculum– Item types not attractive– No linkage to remedial teaching packages (to enable self-
learning)– Insufficient convenience for students’ self-usage
• Conclusion– A good system is a compromise of educational,
psychometrical (measurement), and political considerations
28