common core institute #2 assessment literacy. agenda welcome announcements and introductions...
TRANSCRIPT
Common Core Institute #2
Assessment Literacy
Agenda• Welcome• Announcements and Introductions• Introduction to Assessment• Assessment Types• Assessment System
– Break• Technical Quality• Assessment and Accountability• Building an Assessment Plan
– Lunch• Q & A/Closing Remarks/Announcements
What is ASSESSMENT?• The systematic collection, review and use of
information about educational programs to improve student learning (Palomba & Banta)
• A process embedded within larger systems in a school/district/state
• Noun: tools and strategies used to collect evidence (Heritage & Herman)
• Verb: process of using evidence to improve decision-making (Heritage & Herman)
Purpose• Assessments are always developed with
purpose in mind:– What is to be assessed?– Why is assessment taking place?– How will student learning be measured?– How will the results of the assessment be used?
Types of Assessment• Summative Assessment• Interim Assessment• Formative Assessment
Tiers of Assessment
Formative (minute-by-minute, integrated into the lesson)
Interim (instructional, evaluative, predictive)
Summative
Scop
e an
d D
urati
on
Frequency of Administration
Wide
Narrow
Less Often
MoreOften
Adapted from Perie et al.(2009) and Talbot (2011)
Summative Assessments• Given once at end of a longer period of time• Are typically large scale assessments, which are
given state wide or nationally, but may be district wide or semester exams
• Generally used to evaluate performance against a defined set of content standards
(ASSESSMENT OF LEARNING--Stiggins, Arter, Chauppuis and Chauppuis , 2006)
Summative Assessments• Can meaningfully aggregate/disaggregate and
report results• Use results for accountability and policy
information• Least flexible
Arkansas Assessment Overview 2011-2012
Grade Tested
Criterion-Referenced Tests NAEP Norm-Referenced and National Tests
KEnglish Language Development Assessment
Kindergarten Screening: Qualls Early Learning Inventory (QELI)
1 English Language Development Assessment
Norm-Referenced Test: Iowa Test of Basic Skills (ITBS)
Reading, Language and Mathematics
2 English Language Development Assessment
Norm-Referenced Test: Iowa Test of Basic Skills (ITBS) Reading, Language and Mathematics
3
Augmented Benchmark Exam: Literacy and Math
Alternate Portfolios for IEP English Language Development Assessment
Augmented Benchmark Exam: Abbreviated ITBS Reading, Language and Mathematics
4
Augmented Benchmark Exam: Literacy and Math
Alternate Portfolios for IEP English Language Development Assessment
Long-Term Trend (9 year olds)
Reading and Math
Augmented Benchmark Exam: Abbreviated ITBS Reading, Language and Mathematics
5
Augmented Benchmark Exam: Literacy, Math, and Science
Alternate Portfolios for IEP English Language Development Assessment
Augmented Benchmark Exam: Abbreviated ITBS Reading, Language, Mathematics and
Science
6
Augmented Benchmark Exam: Literacy and Math
Alternate Portfolios for IEP English Language Development Assessment
Augmented Benchmark Exam: Abbreviated ITBS Reading, Language and Mathematics
Arkansas Assessment Overview 2011-2012
Grade Tested
Criterion-Referenced Tests NAEP Norm-Referenced and National Tests
7
Augmented Benchmark Exam: Literacy, Math, and Science
Alternate Portfolios for IEP English Language Development Assessment
Augmented Benchmark Exam: Abbreviated ITBS Reading, Language, Mathematics and
Science
8
Augmented Benchmark Exam: Literacy and Math
Alternate Portfolios for IEP English Language Development Assessment
Long-Term Trend (13 year olds) Reading and
Math
Augmented Benchmark Exam: Abbreviated ITBS Reading, Language and Mathematics
ACT EXPLORE
9
End-of Course Algebra I Grade 9 Math Portfolio for IEP
English Language Development Assessment
Iowa Test of Educational Development (ITED) Complete Battery
Reading and Mathematics
10
End-of-Course Geometry End-of-Course Biology
Grade 10 Science Portfolio for IEP English Language
Development Assessment
ACT PLAN/
PSAT
11
Grade 11 Literacy Grade 11 Literacy Portfolio for IEP
English Language Development Assessment
Long-Term Trend (17 year olds)
Reading and Math
ACT** SAT** **AP (Advanced Placement)
12English Language Development Assessment
ACT** SAT** **AP (Advanced Placement)
Interim Assessments• Fall between formative and summative assessments• Given periodically throughout the year or course• Are typically medium-scale assessments, which are
given at the district or school level• Generally used to evaluate performance against a
defined set of content standards(ASSESSMENT OF LEARNING--Stiggins, Arter, Chauppuis and Chauppuis , 2006)
Interim Assessments• Can meaningfully aggregate/disaggregate and
report results• Use results for program evaluation, resource
allocation, and instructional information on past practice
• More flexible than summative
Formative Assessment• Occurs moment-to-moment as part of instruction• Is a process used by teachers and students
frequently and is embedded in the current unit of instruction
• Are small scale, short cycle assessments given in the classroom to diagnose where students are in their learning
(ASSESSMENT FOR LEARNING--Stiggins, Arter, Chauppuis and Chauppuis, 2006)
Formative Assessment• Cannot aggregate/disaggregate and report
results• Use results to provide descriptive feedback or
to modify instruction• Most flexible
Formative AssessmentFormative assessment is a process used by teachers and students during instruction that provides feedback to adjust ongoing
teaching and learning to improve students’ achievements of intended instructional outcomes (FAST SCASS, October 2006)
Five Critical Features to Guide Educators Toward Effective Use of
Formative Assessment• Learning Progressions – clearly articulates the trajectory
along which students are expected to progress• Learning Goals and Success Criteria – immediate goals to
be learned• Descriptive Feedback – ideas, strategies, and tasks students
can use to close the “gap” between his or her current learning level and the next level
Heritage, M. Formative Assessment and Next-Generation Assessment Systems: Are We Losing an Opportunity. National Center for Research on Evaluation, Standards, and Student Testing (CRESST).
Five Critical Features to Guide Educators Toward Effective Use of
Formative Assessment• Self and Peer Assessment – Students evaluate
their own learning and their peers learning based upon a success criteria
• Collaboration – All participants share responsibility for learning
Heritage, M. Formative Assessment and Next-Generation Assessment Systems: Are We Losing an Opportunity. National Center for Research on Evaluation, Standards, and Student Testing (CRESST).
Formative Assessment Cycle
Heritage, M. Formative Assessment and Next-Generation Assessment Systems: Are We Losing an Opportunity. National Center for Research on Evaluation, Standards, and Student Testing (CRESST).
What are the major differences aMONG the assessment types?
Dimensions of AssessmentType of Assessment
Assessment for Learning Assessment of Learning
Dimension Formative Interim Summative
Purpose • Instructional • Most designed for managerial uses• Some designed for instructional
uses
• Managerial
Implementation • Driven by moment-to-moment decisions; generated or selected by teacher; individualized
• Regulated by protocols developed in or out of the classroom; teacher-generated or externally generated
Timing • During instruction• High frequency
• After instruction or during a break in instruction
• Moderate frequency
• After instruction• Low frequency
Scope • Narrow; one or very few learning objectives at a time
• Moderate; a manageable number of objectives
• Broad; comprehensive set of objectives
Audience • Classroom (teachers and students) • Administration and/or classroom • Public• Administration• Classroom
Adapted from Talbot (2011)
The Assessment System
Comprehensive Assessment System
• Consists of three types of assessments– Summative– Interim– Formative
• Assessment of learning and assessment for learning
Assessment System
Assessments in the Assessment System
Learning Goals/Progressions
Herman, J. & Heritage, M. Assessment Systems That Work. Joint Presentation sponsored by the National Center for Research on Evaluation, Standards, and Student Testing (CRESST) and Assessment and Accountability Comprehensive Center (AACC).
The Three C’s• Comprehensiveness• Coherence• Continuity
Pellegrino, J., Chudowsky, N. & Glaser, R. Knowing What Students Know. National Research Council.
Building Complementary Assessment System
• Role of Classroom Assessment• Role of District Assessment• Role of State Assessment
Role of Classroom Assessment• Monitor ongoing progress• Make ongoing adjustments
– Adapting and responding to learning needs– Use different teaching methods– Use different teaching materials
Sheinker, J. & Redfield, D. Handbook for Professional Development in Assessment Literacy. Council of Chief State School Officers, Comprehensive Assessment Systems State Collaborative.
Role of District AssessmentFor Districts
• Monitor progress• Identify needed changes in:
– policies/procedures– funding– school organization
• Guide curriculum changes• Guide professional
development
For Schools• Set school improvement
goals• Determine staffing needs• Guide instructional changes• Guide professional
development
Sheinker, J. & Redfield, D. Handbook for Professional Development in Assessment Literacy. Council of Chief State School Officers, Comprehensive Assessment Systems State Collaborative.
Role of State AssessmentFor States
• Monitor standards• Monitor equity of opportunity• Guide changes in:
– laws and regulations– policies and funding– organizational structures
• Set improvement goals• Guide professional
development
For Districts• Monitor standards• Anchor district standards and
assessments• Guide changes in:
– board rules– policies and funding– organizational structures
• Monitor school improvement efforts
Sheinker, J. & Redfield, D. Handbook for Professional Development in Assessment Literacy. Council of Chief State School Officers, Comprehensive Assessment Systems State Collaborative.
Break:We will return in 15 minutes
Technical Quality
Why does technical quality matter?• Test construction• Test administration• Usefulness of results
– Credibility– Accuracy of score interpretation– Sound basis for adjusting instruction
Adapted from Sheinker, J. & Redfield, D. Handbook for Professional Development in Assessment Literacy. Council of Chief State School Officers, Comprehensive Assessment Systems State Collaborative.
Assessment Design and Format• Items, tasks, strategies, rubrics must be
purposely designed to serve purpose• Various assessment formats and contexts:
– Paper-pencil, performance– Selected, constructed, extended response– Formal “tests,” analysis of work, homework
• Format does NOT = quality• Good assessments DO well serve their purpose(s)Herman, J. & Heritage, M. Assessment Systems That Work. Joint Presentation sponsored by the National Center for Research on
Evaluation, Standards, and Student Testing (CRESST) and Assessment and Accountability Comprehensive Center (AACC).
Credibility
• Valid and reliable for purpose intended
• Ensuring fairness and eliminating bias
Validity and Reliability• Validity is measuring what is intended to be
measured and using data to make accurate inferences.
• Reliability is consistency of test results from one test to another, from one part to another, from one time to another, and from one score to another.
Sheinker, J. & Redfield, D. Handbook for Professional Development in Assessment Literacy. Council of Chief State School Officers, Comprehensive Assessment Systems State Collaborative.
Bias and Fairness• Bias is giving or not giving an advantage to
one group or another.• Fairness refers to the purposes and uses of
tests in ways that are clearly communicated, ethical, and valid for all students to demonstrate their knowledge and skills.
Sheinker, J. & Redfield, D. Handbook for Professional Development in Assessment Literacy. Council of Chief State School Officers, Comprehensive Assessment Systems State Collaborative.
Accommodations and Validity• Individuals with Disabilities Education Act (IDEA)
Section 300.160(b)--Accommodation guidelines–(1) A State (or, in the case of a district-wide assessment, an LEA) must develop guidelines for the provision of appropriate accommodations.–(2) The State's (or, in the case of a district-wide assessment, the LEA's) guidelines must:
• (i) Identify only those accommodations for each assessment that do not invalidate the score; and
• (ii) Instruct IEP Teams to select, for each assessment, only those accommodations that do not invalidate the score.
• Elementary and Secondary Education Act (ESEA), Title ISection 200.2(b)(2) requires that a State’s assessment system be designed to be valid and accessible for use with the widest possible range of students, including students with disabilities and students with limited English proficiency.
Dos and Don’ts When Selecting Accommodations
Do…make accommodation decisions based on individualized needs.
Don’t…make accommodations decisions based on whatever is easiest to do (e.g., preferential seating).
Do…select accommodations that reduce the effect of the disability to access instruction and demonstrate learning.
Don’t…select accommodations unrelated to documented student learning needs or are intended to give students an unfair advantage.
Do…be certain to document instructional and assessment accommodation(s) on the IEP or 504 plans.
Don’t…use an accommodation that has not been documented on the IEP or 504 plans.
Do…be familiar with the types of accommodations that can be used as both instructional and assessment accommodations.
Don’t…assume that all instructional accommodations are appropriate for use on assessments.
Do…be specific about the “Where, When, Who, and How” of providing accommodations.
Don’t…simply indicate an accommodation will be provided “as appropriate” or “as necessary.”
Thompson, S., Morse, A., Sharpe, M. & Hall, S. Accommodations Manual: How to Select, Administer, and Evaluate Use of Accommodations for Instruction and Assessment of Students with Disabilities. Council of Chief State School Officers, Assessing Special
Education Students State Collaborative.
Dos and Don’ts When Selecting Accommodations
Do…refer to state accommodations policies and understand implications of selections.
Don’t…check every accommodation possible on a checklist simply to be “safe.”
Do…evaluate accommodations used by the student. Don’t…assume the same accommodations remain appropriate year after year.
Do…get input about accommodations from teachers, parents, and students, and use it to make decisions at IEP team or 504 planning committee meetings.
Don’t…make decisions about instructional and assessment accommodations alone.
Do…provide accommodations for assessments routinely used for classroom instruction.
Don’t…provide an assessment accommodation for the first time on the day of a test.
Do…select accommodations based on specific individual needs in each content area.
Don’t…assume certain accommodations, such as extra time, are appropriate for every student in every content area.
Thompson, S., Morse, A., Sharpe, M. & Hall, S. Accommodations Manual: How to Select, Administer, and Evaluate Use of Accommodations for Instruction and Assessment of Students with Disabilities. Council of Chief State School Officers, Assessing Special
Education Students State Collaborative.
Accuracy of Score Interpretation• Use results based on the design and intended
use of assessment• Avoid overinterpretation/misuse of results• Factors affecting trend lines: change in test
blueprint; change in content standards; change in scale; change in cut scores; change in student population
Sheinker, J. & Redfield, D. Handbook for Professional Development in Assessment Literacy. Council of Chief State School Officers, Comprehensive Assessment Systems State Collaborative.
What is the relationship between the assessment system and
accountability?
Accountability System• Data system for reporting selected pieces of data
about student learning and factors affecting learning to determine success of the educational system
• Used as means of holding schools and their supports (district, state, legislation, policies, procedures, practices, funding) accountable for how well students are learning
• Assessment is one part of the accountability system
Sheinker, J. & Redfield, D. Handbook for Professional Development in Assessment Literacy. Council of Chief State School Officers, Comprehensive Assessment Systems State Collaborative.
Accountability System
Tiers of AccountabilityState District Schools Students
How well the laws and funding passed by legislature have
positively impacted student achievement of the standards
How well students have achieved standards
How well students have achieved the standards
How well students have achieved the standards
How well the State Department’s policies and procedures have
resulted in student achievement of the standards and high functioning
schools
How well funds are used to carry out laws, policies, and procedures in supporting student achievement
How effectively the school has maximized students’ opportunity to
learn by keeping them in school
How diligent students have been in using the opportunities to learn
provided by the school
How well qualified teachers are to teach the standards
How effectively the school has created a safe and positive learning
environment
How well schools have used given resources to maximize students’
opportunity to learn
Adapted from Sheinker, J. & Redfield, D. Handbook for Professional Development in Assessment Literacy. Council of Chief State School Officers, Comprehensive Assessment Systems State Collaborative.
Building an assessment plan
Sample Assessment PlanCentral Bucks School District, Doylestown, PA
Sample Assessment PlanCentral Bucks School District, Doylestown, PA
Sample Assessment Plan
Sample Assessment Plan
Sample Assessment Plan
Assessment Plan TemplateAssessment Cycle Assessment type(s) Assessment tools Expectations for
student use of dataPrimary Expectations for teacher use of data
Expectations for school and/or district level use of data
Emphasis area for professional development
Continuous or Daily
Weekly or Monthly
Unit
Quarterly
Annual
Ultimate goal of a coherent assessment approach: Create an aligned system of assessment so that all assessments are measuring agreed upon learning goals and discrete measurement takes place along the learning progression embedded within each goal.
Courtesy of Syracuse School District in Syracuse, NY, and the New York Comprehensive Center. http://www.nycomprehensivecenter.org/docs/form_assess/SyracuseAssessmentFramework.pdf
Ultimate goal of a coherent assessment approach:
Create an aligned system of assessment so that all assessments are measuring agreed upon learning
goals and discrete measurement takes place along the learning progression
embedded within each goal.
Assessments in the Assessment System
Learning Goals/Progressions
Herman, J. & Heritage, M. Assessment Systems That Work. Joint Presentation sponsored by the National Center for Research on Evaluation, Standards, and Student Testing (CRESST) and Assessment and Accountability Comprehensive Center (AACC).