student learning assessments - unesco · outlinei 1 de ning assessment 2 types of assessment 3...
TRANSCRIPT
Student Learning Assessments
Dr. Kelvin D. Gregory
Education ConsultantAdjunct at Flinders [email protected]
Dr. Kelvin D. Gregory Student Learning Assessments 1 41
Outline I
1 Defining Assessment
2 Types of assessment
3 Large scale assessmentsConstruct definitionScale characteristicsScale regions
4 Contextual FrameworkContextual FrameworkBroad areasQuestionnaires
Dr. Kelvin D. Gregory Student Learning Assessments 2 41
The primary purposeThe primary purpose is:
Building capacity for large scale nationalassessments
And driving that is a desire to improve Nepal’s primaryand lower and higher secondary education systems.And we will focus on:
Reasons for national assessments
Writing assessment frameworks and items
Analysis systems
Dr. Kelvin D. Gregory Student Learning Assessments 3 41
A benchmark interventionWhat if, in addition, you:
Worked with teachers before the start of theteaching block
Clarified the learning topics (and objectives),developed an assessment framework, designedoutcome measures (assessment items and tests)
The Ministry sent benchmark tests, withinstructions and interpretation guides, to schools2-3 weeks before the end of the teaching block
Teachers administered the tests, scored them,and used them as part of the reporting process
Then you repeat the exercise, building onwhat you have learned
Dr. Kelvin D. Gregory Student Learning Assessments 4 41
Defining Assessment”Assessment is the systematic collection,interpretation and use of information to givea deeper appreciation of what students knowand understand, their skills and personal capabilities, andwhat their learningexperiences enable them to do.”(Northern Ireland Curriculum Guidance for Assessment inthe Primary School)
How is assessment defined in the officialdocuments of Nepal?Do people use this definition?How would students and teachers define assessment?
Dr. Kelvin D. Gregory Student Learning Assessments 5 41
Dann (2012)”Fundamental ... is the idea that learning, ultimately, isconstructed and controlled bypupils. Without teachers willingness toengage with the curriculum and pupilsdeveloping range of cognitive competenciesand experiences, learning will not proceed.If assessment genuinely seeks to give someindication of pupils levels of learning anddevelopment, in ways which will furtheradvance learning, pupils will need tounderstand and contribute to the process.”(p. 2)How do Nepal’s students contribute tothe assessment process?
Dr. Kelvin D. Gregory Student Learning Assessments 6 41
Using Assessment Information”Many people want to use assessmentinformation, and they want to use it in manyways.” (Stiggins (2007),p.29)
Who uses assessment information in Nepal?What information do they use?When and how do they use it?Is the information used appropriately andin a timely manner?Can you offer a constructive critique of theuse of assessment information in Nepal?
Dr. Kelvin D. Gregory Student Learning Assessments 7 41
Formative and SummativeAssessment DefinitionSummative Assessments are givenperiodically to determine at a particular pointin time what students know and do not know.
Formative Assessment is part of theinstructional process. When incorporated intoclassroom practice, it provides the informationneeded to adjust teaching and learning whilethey are happening.
Dr. Kelvin D. Gregory Student Learning Assessments 8 41
Assessment for learning
enables teachers to use informationabout students’ knowledge, skills andunderstanding to inform their teaching
teachers provide feedback to studentsabout their learning and how to improve
Dr. Kelvin D. Gregory Student Learning Assessments 9 41
Assessment as learning
involves students in the learning processwhere they monitor their own progress,ask questions and practice skills
students use self-assessment and teacherfeedback to reflect on their learning,consolidate their understanding and worktowards learning goals
Dr. Kelvin D. Gregory Student Learning Assessments 10 41
Assessment of learning
assists teachers to use evidence ofstudent learning to assess studentachievement against learning goals andstandards
Dr. Kelvin D. Gregory Student Learning Assessments 11 41
AssessmentIf we want to use assessment as a tool forlearning students need to:
Know where they are going
Know where they are now
Know how to close the gap
Dr. Kelvin D. Gregory Student Learning Assessments 12 41
Formative and SummativeAssessment
Formative SummativeAssessment for learning Assessment of learningAnalyze strengths andweaknesses
Document achievement
Used to check students’understanding and to planfor subsequent teaching
Provides teachers, studentsand others withinformation about theattainment of knowledgeand skills.
Must be used in a teachingmodel
May be used in subsequentteaching of differentconcepts.
Low stakes but veryimportant
Stakes can range from lowto high.
Dr. Kelvin D. Gregory Student Learning Assessments 13 41
Defining large scale assessmentsThe British Columbia Government’s largescale assessment policy:
The term large scale assessment refersto any provincial, national orinternational assessment, examination or testthe Ministry directs boards ofeducation to administer.
Dr. Kelvin D. Gregory Student Learning Assessments 14 41
Large scale assessment purposeThe British Columbia Government’s largescale assessment policy:
A primary purpose of large-scaleprovincial, national and internationalassessments and examinations is toobtain information for the purposes ofpublic accountability, improving programsand certifying that students meetgraduation requirements.
Dr. Kelvin D. Gregory Student Learning Assessments 15 41
Large scale assessment focusThe British Columbia Government’s largescale assessment policy:
Large scale assessments andexaminations determine students’ knowledgeand skills in particular areasof learning.
Dr. Kelvin D. Gregory Student Learning Assessments 16 41
Large scale assessment useThe British Columbia Government’s largescale assessment policy:
Assessment information enables educationaldecision-makers at the classroom, district andprovincial levelsto make informed choices related to improvingstudent achievement.
Dr. Kelvin D. Gregory Student Learning Assessments 17 41
Using large scale studentassessment to improve studentachievementUngerleider (2006) makes the following points
there needs to be broad agreement aboutthe outcomes for all studentsthese must be detailed in the curriculumand supported by teaching materialthe educational community (students,teachers, parents) should be heldaccountable for the outcomesprovide teachers with adequate timeand training to interpret the outcomesDr. Kelvin D. Gregory Student Learning Assessments 18 41
”The validity of any assessment result isconditional on the fit between the purposefor which the assessment was designed andthe use of the results. At present there is noestablished procedure evaluating the validityof the alternative uses to which large scaleassessments might be put. It is not unusualthat an assessment developed for one purposeto be used for another. At present, thedevelopers of the original test leave it tosubsequent users to decide if its use isappropriate.”(Ungerleider (2006), p. 875)
Dr. Kelvin D. Gregory Student Learning Assessments 19 41
Assessment continuumAssessment involves professional judgementbased upon an image formed by the collectionof information about student performance.
Unstructuredchance meet-ings, conversa-tions
Slightlystructuredquestionnaires,observations,student self-assessment
More struc-turedchecklists, class-room tests,practical work,project work,case studies
Most struc-turedExaminations,standardizedtests, publishedaptitude tests
Dr. Kelvin D. Gregory Student Learning Assessments 20 41
The ContinuumA continuum attempts to capture in wordswhat it means to make progress or to improvein an area of learning or domain of knowledge.In noncognitive areas we think of the construct changingover time within a continuum.
Proficiency
T ime
Motivation
T ime
Dr. Kelvin D. Gregory Student Learning Assessments 21 41
Example developmental continuumfor recitation (Grades I and II)
Recitation is poor. Lacks expression.
Cannot recite an entire poem or story without prompt-ing. Pronunciation expression is not appropriate.
Can recite poem, or story with occasional prompting.Expression is not very strong and effective.
Can recite poem, or story with proper speed or expres-sion but makes occasional mistakes in pronuciation orforgets at times.
Can recite poem or a story with proper speed, diction,expression and tone.
Dr. Kelvin D. Gregory Student Learning Assessments 22 41
Example developmental continuumfor mathematics
2 + 2=
17 + 29=
76 - 29=
What is the difference between the largest and secondlargest numbers that can be made from the followingdigits: 3, 7, 4 and 6.
The larger of two integers is 5/2 the smaller.If twicethe smaller is subtracted from twice the larger, the dif-ference is 12. Find the two numbers.
Dr. Kelvin D. Gregory Student Learning Assessments 23 41
Traditional subjects
Dr. Kelvin D. Gregory Student Learning Assessments 24 41
Traditional subject withnoncognitive skills
Dr. Kelvin D. Gregory Student Learning Assessments 25 41
Introducing standards
We divide the continuum into areas tohelp make sense of it.
I This helps us communicate what we meanby the continuum.
I It simplifies what we are talking about andat the same time we do not exaggerate theaccuracy of what we are measuring.
In cognitive continua, we call these areas”performance standards” .
We describe what the person cando for each standard.
Band 1 Band 2 Band 3 Band 4
Dr. Kelvin D. Gregory Student Learning Assessments 26 41
Describing the standards
We describe what the person can dofor each performance standard.
We call the process of describing eachstandard ”standard setting”.
The easist way of describing whatstudents can do at each level is to get examples ofstudents’ work.
I Using experts, we might get a large rangeand sample of students’ work, sort it intofour piles of increasing standard.
I Then we describe what students in eachpile can do using terms to describe thecognitive skills.
Dr. Kelvin D. Gregory Student Learning Assessments 27 41
NAEP Generic Reporting scaleThe USA approach to developing theirachievement reporting scales has been todevelop a generic or general descriptioncovering all scale areas.
Level Description
BasicPartial mastery of prerequisite knowledge andskills that are fundamental for proficient workat each grade.
Proficient
Solid academic performance for each gradeassessed. Students reaching this level havedemonstrated competency over challengingsubject matter, including subject-matterknowledge, application of such knowledge toreal-world situations, and analytical skillsappropriate to the subject matter.
Advanced Superior performance.Dr. Kelvin D. Gregory Student Learning Assessments 28 41
NAEP Generic Reporting scaleAnd then they add subject area specificdescriptions.
Level Description
Basic
Fourth-grade students performing at the Basic level should show someevidence of understanding the mathematical concepts and procedures inthe five NAEP content areas. Fourth-graders performing at the Basiclevel should be able to estimate and use basic facts to perform simplecomputations with whole numbers, show some understanding of fractionsand decimals, and solve some simple real-world problems in all NAEPcontent areas. Students at this level should be able to use though notalways accurately four function calculators, rulers, and geometric shapes.Their written responses will often be minimal and presented withoutsupporting information.
Proficientrefer tohttps://nces.ed.gov/nationsreportcard/mathematics/achieveall.aspx
Advanced
Fourth-grade students performing at the Advanced level should applyintegrated procedural knowledge and conceptual understanding tocomplex and nonroutine real-world problem solving in the five NAEPcontent areas. Fourth-graders performing at the Advanced level should beable to solve complex and nonroutine real-world problems in all NAEPcontent areas. They should display mastery in the use of four-functioncalculators, rulers, and geometric shapes. The students are expected todraw logical conclusions and justify answers and solution processes byexplaining why, as well as how, they were achieved. They should gobeyond the obvious in their interpretations and be able to communicatetheir thoughts clearly and concisely.
Dr. Kelvin D. Gregory Student Learning Assessments 29 41
The frameworkThe development of cognitive indicator startswith the development of a frameworkgenerally involves the following steps:
widespread participation and reviews by educatorsand education officials;
reviews by steering committees whose members representpolicymakers, practitioners, andmembers of the general public;
involvement of subject supervisors from education agencies;
public hearings; and
reviews by scholars in the field, by the studystaff, and by a policy advisory panel.
Dr. Kelvin D. Gregory Student Learning Assessments 30 41
Construct definitionCognitive indicators need a definition thatwill define the construct throughout the study.The construct’s meaning should not changeduring this time; it must be stable.
Dr. Kelvin D. Gregory Student Learning Assessments 31 41
PISA mathematical literacy” Mathematical literacy is an individual’scapacity to formulate, employ, and interpret mathematicsin a variety of contexts. Itincludes reasoning mathematically and usingmathematical concepts, procedures, factsand tools to describe, explain and predict phenomena. Itassists individuals to recognisethe role that mathematics plays in the worldand to make the well-founded judgments anddecisions needed by constructive, engagedand reflective citizens.”
Dr. Kelvin D. Gregory Student Learning Assessments 32 41
Scale characteristicsGenerally, the numerical characteristics ofthe scale are described. The scale when IRTis used is generally a simple lineartransformation of the mean and standarddeviation obtained from the IRT analyses.
Dr. Kelvin D. Gregory Student Learning Assessments 33 41
Scale characteristics”PISA results are reported on a scaleconstructed using a generalised form of theRasch model as described by Adams, Wilsonand Wang (1997). For each domain (reading,mathematics and science), a scale isconstructed with a mean score of 500 andstandard deviation of 100 among OECDcountries; accordingly, about two-thirds ofstudents across OECD countries scorebetween 400 and 600 points.”
Dr. Kelvin D. Gregory Student Learning Assessments 34 41
Then describe regions of the scale
Level Location Items Description
2 409.5
ACIDRAINQ2(460)
At Level 2, students haveadequate scientificknowledge to providepossible explanations infamiliar contexts or drawconclusions based onsimple investigations. Theyare capable of directreasoning and makingliteral interpretations ofthe results of scientificinquiry or technologicalproblem solving.
Dr. Kelvin D. Gregory Student Learning Assessments 35 41
Contextual FrameworkInternational studies also include acontextual framework. From TIMSS 2011:”Because learning takes place within a context and not inisolation, TIMSS makes every attempt to collect informationabout the important factors that foster improved teaching andlearning in mathematics and science. The questionnairesconcentrate on procedures and practices that have been shownto be effective in increasing achievement in mathematics andscience. In this way, countries can better evaluate their TIMSSresults; in terms of the prevalence of the home or schoolsituation or instructional practice in their country and itsrelationship with student achievement.”
Dr. Kelvin D. Gregory Student Learning Assessments 36 41
TIMSS Contextual Frameworkbroad areasThe TIMSS contextual framework specifiesfive broad areas:
National and community contexts.
Home contexts.
School contexts.
Classroom contexts.
Student characteristics and attitudestowards learning.
Each of these areas is described using anumber of measurements in the 20 pagecontextual framework.
Dr. Kelvin D. Gregory Student Learning Assessments 37 41
Contextual FrameworkcomponentsThe TIMSS contextual framework specifiesfour questionnaire types:
Curriculum questionnaire.
Student questionnaire.
Teacher questionnaire.
Head teacher or principal questionnaire.
And describes what information eachquestionnaire is designed to gather and why.
Dr. Kelvin D. Gregory Student Learning Assessments 38 41
TIMSS school questionnaireThis questionnaire is designed to takeapproximately 30 minutes to complete andhas the following sections:
School enrollment and characteristics.
Instructional time.
Resources and technology.
School emphasis on academic success.
School discipline and safety.
Teachers in the school.
School readiness.
Principal experience and education.
Dr. Kelvin D. Gregory Student Learning Assessments 39 41
TIMSS teacher questionnaireThis questionnaire is designed to takeapproximately 30 minutes to complete.There are specific versions for specialist mathematics andscience teachers:
Characteristics of the teacher.School emphasis on academic success.Experiences of being a teacher.Experiences with the assessed class.And then questions for mathematics and science
I Experiences teaching the subject.I Homework practices.I Assessment practices.I Preparation to teach.I Use of computers.
Dr. Kelvin D. Gregory Student Learning Assessments 40 41
TIMSS student questionnaireThis questionnaire is designed to takeapproximately 30 minutes to complete andhas the following sections:
About the student.
About the school.
Mathematics in the school.
Science in the school.
Dr. Kelvin D. Gregory Student Learning Assessments 41 41