aei summer institute - educational assessment
TRANSCRIPT
![Page 1: AEI Summer Institute - Educational Assessment](https://reader030.vdocuments.mx/reader030/viewer/2022032620/55c9a7febb61ebf51b8b45e4/html5/thumbnails/1.jpg)
Educational Assessment
Stephen Parks, Learning Specialist - Mathematics
Assessment and Evaluation Branch (Anglophone Division)
Department of Education and Early Childhood Development
Atlantic Education International
July 21, 2015
0
0 .2
0 .4
0 .6
0 .8
1 .0
-3 -2 -1 0 1 2 3
b
Ab i l i ty
Pro
bab
ilit
y
Item Characteristic Curve: ITEM 0001
a = 0 .5 3 9 b = 0 .2 0 2
![Page 2: AEI Summer Institute - Educational Assessment](https://reader030.vdocuments.mx/reader030/viewer/2022032620/55c9a7febb61ebf51b8b45e4/html5/thumbnails/2.jpg)
1. Educational Assessment
2. New Brunswick Assessment Program - NBAP (Current)
3. Realignment of the NBAP
4. Grade 9 ELPA – Grade 11/12 ELPA/R
5. Protocols for Accommodations and Exemptions
6. Assessment and Item Development Process (Grade 6)
7. Provincial Mathematics Assessment at Grade 8
TOPICS
![Page 3: AEI Summer Institute - Educational Assessment](https://reader030.vdocuments.mx/reader030/viewer/2022032620/55c9a7febb61ebf51b8b45e4/html5/thumbnails/3.jpg)
© 2005, Dynamic Measurement Group 3
GPS On Where are we?
You are Here
Port
Where do we want to be ?
How do we get there?
How are we doing?
ActualCourse
DesiredCourse
![Page 4: AEI Summer Institute - Educational Assessment](https://reader030.vdocuments.mx/reader030/viewer/2022032620/55c9a7febb61ebf51b8b45e4/html5/thumbnails/4.jpg)
Assessment is the GPS for Educators
10
20
30
40
Dec.Scores
Feb.Scores
Jan.Scores
M archScores
AprilScores
MayScores
JuneScores
60
50
Rea
ding
Aimline
![Page 5: AEI Summer Institute - Educational Assessment](https://reader030.vdocuments.mx/reader030/viewer/2022032620/55c9a7febb61ebf51b8b45e4/html5/thumbnails/5.jpg)
New Brunswick Assessment Program (NBAP)
Mission StatementThe mission for the Provincial Assessment and Evaluation Branch is to inform all stakeholders of student achievement by constructing valid, educationally beneficial, evaluations at critical points in the system. The purpose of the New Brunswick Assessment Program (NBAP) is to: determine if students demonstrate the skills and concepts outlined in New Brunswick curriculum documents; report provincially on how well students have achieved provincial standards at given points in their schooling; assist schools, districts, and the province in monitoring student learning and; coordinate the administration of national and international assessments.
![Page 6: AEI Summer Institute - Educational Assessment](https://reader030.vdocuments.mx/reader030/viewer/2022032620/55c9a7febb61ebf51b8b45e4/html5/thumbnails/6.jpg)
Current Assessment Program
•Pan-Canadian Assessment Program (PCAP) as well as the Programme for International Student Assessment (PISA).
•Scientific literacy
•Administering assessments as single units.
•Feedback for high schools.
•The current regime of provincial assessments
•French Language Programs
![Page 7: AEI Summer Institute - Educational Assessment](https://reader030.vdocuments.mx/reader030/viewer/2022032620/55c9a7febb61ebf51b8b45e4/html5/thumbnails/7.jpg)
Realignment of the Provincial Assessment Program
![Page 8: AEI Summer Institute - Educational Assessment](https://reader030.vdocuments.mx/reader030/viewer/2022032620/55c9a7febb61ebf51b8b45e4/html5/thumbnails/8.jpg)
Recommendation # 1
• To create a New Brunswick Balanced Assessment Framework document incorporating assessment as a holistic concept from formative to benchmarking to large scale and to develop a systemic implementation plan that includes accountability measures.
![Page 9: AEI Summer Institute - Educational Assessment](https://reader030.vdocuments.mx/reader030/viewer/2022032620/55c9a7febb61ebf51b8b45e4/html5/thumbnails/9.jpg)
Balanced Assessment
• No single assessment approach is 'better' than another as each serves a slightly different role.
• All types of assessment must be planned with its purpose in mind.
Standardized testing, particularly, is justified as one resource to aid learning and should not be mistaken for a comprehensive verdict on learning or school quality. ~ Peter Grimmett & Paul Shaker
![Page 10: AEI Summer Institute - Educational Assessment](https://reader030.vdocuments.mx/reader030/viewer/2022032620/55c9a7febb61ebf51b8b45e4/html5/thumbnails/10.jpg)
![Page 11: AEI Summer Institute - Educational Assessment](https://reader030.vdocuments.mx/reader030/viewer/2022032620/55c9a7febb61ebf51b8b45e4/html5/thumbnails/11.jpg)
![Page 12: AEI Summer Institute - Educational Assessment](https://reader030.vdocuments.mx/reader030/viewer/2022032620/55c9a7febb61ebf51b8b45e4/html5/thumbnails/12.jpg)
Recommendation #2To realign the Provincial Assessment Program to correspond with PCAP and PISA as well as respond to the current challenges.
![Page 13: AEI Summer Institute - Educational Assessment](https://reader030.vdocuments.mx/reader030/viewer/2022032620/55c9a7febb61ebf51b8b45e4/html5/thumbnails/13.jpg)
![Page 14: AEI Summer Institute - Educational Assessment](https://reader030.vdocuments.mx/reader030/viewer/2022032620/55c9a7febb61ebf51b8b45e4/html5/thumbnails/14.jpg)
“New” New Brunswick Assessment Program (NBAP)
Grade 6 and Grade 10 (Census) and Grade 4 (Field Test)
•Reading, Mathematics and Scientific Literacy Grade 6 Reading will be by program of instruction; PIF booklets will include a short section in French Reading; Math and Scientific Literacy will include bilingual booklets for students in the FI Program.
Grade 10 Math and Scientific Literacy will include all students and booklets will be bi-lingual where appropriate. English literacy will be assessed in the ELPA/R at Grade 9 and will not be included.
Grade 2 Reading (annual, census, administered in May)Grade 2 Reading will be in English to ensure the gains in early literacy are sustained and continued growth is monitored.
Grade 9 ELPA (Graduation Requirement) - Grade 11/12 ELPA/R
Grade 6, 10 French Oral Proficiency (10% stratified by Program)
Grade 12 French Oral Proficiency CertificationThe existing Grade 12 will become mandatory for all students in French Immersion.
![Page 15: AEI Summer Institute - Educational Assessment](https://reader030.vdocuments.mx/reader030/viewer/2022032620/55c9a7febb61ebf51b8b45e4/html5/thumbnails/15.jpg)
English Language Proficiency Assessment – ELPA/R
![Page 16: AEI Summer Institute - Educational Assessment](https://reader030.vdocuments.mx/reader030/viewer/2022032620/55c9a7febb61ebf51b8b45e4/html5/thumbnails/16.jpg)
![Page 17: AEI Summer Institute - Educational Assessment](https://reader030.vdocuments.mx/reader030/viewer/2022032620/55c9a7febb61ebf51b8b45e4/html5/thumbnails/17.jpg)
![Page 18: AEI Summer Institute - Educational Assessment](https://reader030.vdocuments.mx/reader030/viewer/2022032620/55c9a7febb61ebf51b8b45e4/html5/thumbnails/18.jpg)
![Page 19: AEI Summer Institute - Educational Assessment](https://reader030.vdocuments.mx/reader030/viewer/2022032620/55c9a7febb61ebf51b8b45e4/html5/thumbnails/19.jpg)
![Page 20: AEI Summer Institute - Educational Assessment](https://reader030.vdocuments.mx/reader030/viewer/2022032620/55c9a7febb61ebf51b8b45e4/html5/thumbnails/20.jpg)
ACCOMMODATIONS AND EXEMPTIONS
Provincial assessments are compulsory for all students enrolled in New Brunswick schools.
In certain circumstances, it is necessary and appropriate for students to be exempted from participating in provincial assessments if the testing format does not allow demonstration of their learning, even with accommodations.
Exemptions fall into three broad categories: academic, social/emotional and unforeseen. Typically, academic exemptions are sought for students following an individualized Special Education Plan (SEP-I). Students who are exempt due to an SEP-I should be assessed on those goals that are set out in their Special Education Plan(SEP).
![Page 21: AEI Summer Institute - Educational Assessment](https://reader030.vdocuments.mx/reader030/viewer/2022032620/55c9a7febb61ebf51b8b45e4/html5/thumbnails/21.jpg)
ACCOMMODATIONSAccommodations are those supports, methods and practices that are used in the classroom enabling a student to participate as independently as possible.
There are many ways in which accommodations can be used to support students when they are participating in a provincial assessment. Within the context of provincial assessments, students may require either universal or justifiable accommodation.
For the purpose of large scale assessment, accommodations are alterations that do not jeopardize the integrity or content of the test, yet give students an equal opportunity to demonstrate what they know and can do as part of the assessment process.
![Page 22: AEI Summer Institute - Educational Assessment](https://reader030.vdocuments.mx/reader030/viewer/2022032620/55c9a7febb61ebf51b8b45e4/html5/thumbnails/22.jpg)
Types of Accommodations
UNIVERSAL
Universal accommodations are those strategies, technologies or adjustments that enable a student to reach prescribed outcomes and can be used at the teacher’s or student’s discretion. Universal accommodations do not necessitate an SEP or prior approval for provincial assessments.
JUSTIFIABLE
Justifiable accommodations are documented strategies, technologies or adjustments without which the student would not be able to demonstrate knowledge. These accommodations are documented within the SEP and require prior approval to be accessed during provincial assessments.
![Page 23: AEI Summer Institute - Educational Assessment](https://reader030.vdocuments.mx/reader030/viewer/2022032620/55c9a7febb61ebf51b8b45e4/html5/thumbnails/23.jpg)
EXEMPTIONS
In certain circumstances, it is not appropriate for students to participate in provincial assessments. Any decision to exempt a student from any provincial assessment must be made in collaboration with the student’s parent(s) or guardian(s).
Within the context of provincial assessments, students may require
an exemption for academic, social/emotional or unforeseen reasons.
![Page 24: AEI Summer Institute - Educational Assessment](https://reader030.vdocuments.mx/reader030/viewer/2022032620/55c9a7febb61ebf51b8b45e4/html5/thumbnails/24.jpg)
Types of Exemptions
ACADEMICThe student has been identified with exceptionalities and has a current individualized Special Education Plan (SEP-I) that identifies the need for the requested exemption in the assessed subject area.
Typically, these students are following an SEP-I and are not expected to attain the curricular outcomes and standards. This has been communicated to parent(s) or guardian(s) and an alternate evaluation of progress has been determined within the SEP-I.
Typically, this constitutes the largest category of exemption requests.
SOCIAL/EMOTIONAL
If the student has been identified with a significant social, emotional or behavioural impairment and cannot be accommodated by such means as alternate location, extra time, etc., an exemption may be sought.
Typically, these students are following an Individual Behaviour Support Plan (IBSP) and are not expected to attain the curricular outcomes and standards at their grade level. This has been communicated to parent(s) or guardian(s) and an alternate evaluation of progress has been determined within the IBSP.
.
UNFORESEEN
In the event of serious illness, bereavement or other exceptional and unforeseen circumstance, an on-the-spot exemption can be granted.
These situations require that the director of education be contacted immediately..
![Page 25: AEI Summer Institute - Educational Assessment](https://reader030.vdocuments.mx/reader030/viewer/2022032620/55c9a7febb61ebf51b8b45e4/html5/thumbnails/25.jpg)
![Page 26: AEI Summer Institute - Educational Assessment](https://reader030.vdocuments.mx/reader030/viewer/2022032620/55c9a7febb61ebf51b8b45e4/html5/thumbnails/26.jpg)
New Brunswick Assessment Program
Operational Standards Processes How items are chosen
![Page 27: AEI Summer Institute - Educational Assessment](https://reader030.vdocuments.mx/reader030/viewer/2022032620/55c9a7febb61ebf51b8b45e4/html5/thumbnails/27.jpg)
Standards for Educational and Psychological Testing
The Standards for Educational and Psychological Testing, 2008 edition are a collaborative publication from the American Educational Research Association, the American Psychological Association and the National Council on Measurement in Education. The intent of the Standards is to promote the sound and ethical use of tests and to provide a basis for evaluating the quality of testing practices.
The New Brunswick Assessment and Evaluation Branch adheres to the Standards when developing, administering, scoring and analysing large scale assessments within the scope of our mandate.
Operational Standards
![Page 28: AEI Summer Institute - Educational Assessment](https://reader030.vdocuments.mx/reader030/viewer/2022032620/55c9a7febb61ebf51b8b45e4/html5/thumbnails/28.jpg)
Standards for Educational and Psychological Testing
![Page 29: AEI Summer Institute - Educational Assessment](https://reader030.vdocuments.mx/reader030/viewer/2022032620/55c9a7febb61ebf51b8b45e4/html5/thumbnails/29.jpg)
The Assessment Cycle
There are four (4) stages within the overall Assessment Cycle for each of the 13 assessments; the (1) Development Stage, (2) Administration Stage, (3) Data Stage and the (4) Communication Stage.
Within each of the stages, there are several phases which require the expertise from a variety of staff.
Each of these stages require strict adherence to the Standards for Educational and Psychological Testing and the Principles for Fair Student Assessment in Canada.. These standards and principles will be referenced where appropriate.
Processes
![Page 30: AEI Summer Institute - Educational Assessment](https://reader030.vdocuments.mx/reader030/viewer/2022032620/55c9a7febb61ebf51b8b45e4/html5/thumbnails/30.jpg)
![Page 31: AEI Summer Institute - Educational Assessment](https://reader030.vdocuments.mx/reader030/viewer/2022032620/55c9a7febb61ebf51b8b45e4/html5/thumbnails/31.jpg)
0
0 .2
0 .4
0 .6
0 .8
1 .0
-3 -2 -1 0 1 2 3
b
Ab i l i ty
Pro
bab
ilit
y
Item Characteristic Curve: ITEM0001
a = 0 .5 3 9 b = 0 .2 0 2
![Page 32: AEI Summer Institute - Educational Assessment](https://reader030.vdocuments.mx/reader030/viewer/2022032620/55c9a7febb61ebf51b8b45e4/html5/thumbnails/32.jpg)
1 - 7
8 - 14
15 - 21
22 - 28
29 - 35
36 - 42
43 - 48
Matrix Plot of Item Characteristic Curves for Book A
1 - 7
8 - 14
15 - 21
22 - 28
29 - 35
36 - 42
43 - 48
Matrix Plot of Item Characteristic Curves for Book A
![Page 33: AEI Summer Institute - Educational Assessment](https://reader030.vdocuments.mx/reader030/viewer/2022032620/55c9a7febb61ebf51b8b45e4/html5/thumbnails/33.jpg)
Development of the Provincial Assessment at Grade 6
![Page 34: AEI Summer Institute - Educational Assessment](https://reader030.vdocuments.mx/reader030/viewer/2022032620/55c9a7febb61ebf51b8b45e4/html5/thumbnails/34.jpg)
Blue PrintAssessment Blue Prints are developed to guide the item development process. The Blue Print may include: (Mathematics Example)
Assessment Type:
![Page 35: AEI Summer Institute - Educational Assessment](https://reader030.vdocuments.mx/reader030/viewer/2022032620/55c9a7febb61ebf51b8b45e4/html5/thumbnails/35.jpg)
Items Per Booklet: Mathematics
![Page 36: AEI Summer Institute - Educational Assessment](https://reader030.vdocuments.mx/reader030/viewer/2022032620/55c9a7febb61ebf51b8b45e4/html5/thumbnails/36.jpg)
Assessment Format – 4 Booklet Matrix SampleScience and Mathematics will be written in the same booklet by Prime and French Immersion students. Each booklet will be bilingual (English on the left-hand side and French on the right-hand side).
•Mathematics section•Scientific Literacy section•Literacy (English) section•Literacy (French) section
Booklet AScience
and Math
Booklet BScience
and Math
Booklet CScience
and Math
Booklet DScience
and Math
![Page 37: AEI Summer Institute - Educational Assessment](https://reader030.vdocuments.mx/reader030/viewer/2022032620/55c9a7febb61ebf51b8b45e4/html5/thumbnails/37.jpg)
2014-2015 Provincial Mathematics Assessment at Grade 8
Items and Rubrics