Why should cognitive learning models be the foundation for next generation assessments?

Karin Hess

Center for Assessment

Sue Bechard

Inclusive Educational Assessment

Understanding Learning Progressions and Learning Maps to Inform the Development of Assessment for Students in Special Populations

1

Cameto, R., Bechard, S., & Almond P. (Eds.). (2012).Third Invitational Research Symposium: Understanding

Learning Progressions and Learning Maps to Inform the Development of Assessment for Students in

Special Populations. Manuscript in preparation. Menlo Park, CA, and Lawrence, KS: SRI International and

Center for Educational Testing and Evaluation (CETE).

Topic 1 Work Group • Rosemary Abel

• Sue Bechard

• Diane Browder

• Chris Camacho

• Karin Hess

• Leah McConaughey

• Mike Russell

• Kelli Thomas

Cognitive learning models reflect HOW students learn • Learning models (learning progressions and learning maps)

describe the typical pathways that students take as they develop understanding from novice to mastery of concepts and skills, based on research and conceptual analysis (Smith et al., 2006).

• Learning models (learning progressions) visually and verbally articulate an hypothesis about how learning will typically move toward increased understanding for most students (Hess, 2009.)

• Learning models (learning progressions) differ in scope, breadth, and grain size (Heritage, 2010)

3

Understanding Learning Progressions and Learning Maps to Inform the Development of Assessment for Students in Special Populations

Cognitive learning models reflect HOW students learn (cont.)

• Learning models (learning progressions) propose the intermediate understandings that are “reasonably coherent networks of ideas and practices…that contribute to building a more mature understanding…the important precursor ideas may not look like the later ideas, yet crucially contribute to their construction.” (Taking Science to School, 2007)

• Learning models reflect systematic consideration of interactions among the learner, the content, and the context for learning, as well as the dynamic, cumulative outcomes of these interactions (Cameto, Bechard, & Almond, eds., 2012).

Characteristics of Learning Progressions and Learning Maps

Are organized around

“big ideas” or “unifying

ideas”

Consist of pathways

and key learning targets

Multiple pathways are

implied

Provide potential

interim learning targets

built upon earlier

learning along multiple

(and interrelated)

pathways (or strands)

Reflect the “universe” of

cognition (abilities),

concepts (knowledge), and

skills related to a domain

Consist of networks of

sequenced learning targets

Multiple pathways are

embedded

Are sometimes more

detailed than progressions

Based on cognition and

developing concepts/skills in a

domain

Multi-dimensional

Can be sequential and/or

cyclical

Can have a range of granularity

Have components that are:

Static=based on normative data

and/or Dynamic=based on

individual learning data

Do not address rate of learning

Reflect consideration of

interactions among learner

characteristics, content, and

context

Validation is data-based and

iterative

Learning Progressions Learning Maps

Cognitive Learning Models

Section of DLM Mathematics Map

DLM Mathematics Map on Transformations

The Learning Progressions Frameworks (LPFs) Designed for Use with the Common Core State Standards

Developed by researchers and content experts synthesizing available research

• Began by framing Big Ideas of each content discipline, then described grade span learning targets, and indicators of progress found in the research

• Organized by major strands & grade span, not grade level

• Each strand includes a short research summary with key ideas

• K-12 Progress Indicators (PIs) in the LPF

– Generally sequenced using empirical research

– Larger grain size than what would be used for developing a single test item – best for unit design, curriculum mapping, instructional planning

– Linked to related CC standards or parts of CC standards

– Some PIs are not linked to any standard, because they do not represent a learning endpoint, but do describe critical stages of learning

ELA Excerpt from the Learning Progressions Frameworks (LPFs) – grades 3-4 [with some grade 3 & 4 CCSS]

Write Informational Texts, By gr 4… Read Informational Texts, By gr 4…

Apply organizational strategies (e.g., sequence, description, definition, compare-contrast, cause-effect) to develop, summarize, and communicate factual information about topics and events for authentic audiences.

Recognize and use knowledge of expository text structures and genre-specific features to read and comprehend informational texts: Identify, compare, and draw inferences about concepts, central ideas, point of view, and supporting details.

WI.j generate own ideas for writing; using

strategies to clarify writing [3.W-5; 3.SL-1d, 3; 3.L-3, 6; 4.W-5; 4.SL-1d, 3; 4.L-3, 6]

RI.h locate relevant key ideas using text features to answer questions and expand

understanding [3.RI-1, 5, 7; 4.RI-1, 7]

WI.k locate information from at least two reference sources (print/ non-print) to obtain information on a topic; list sources

RI.i identify, paraphrase, summarize central ideas and supporting details; determine

importance of information [3.RI-2; 4.RI-2]

WI.l use note-taking and organizational strategies to record and meaningfully organize information (e.g., relate subtopics to facts)

RI.j attend to signal words, text structure, and semantic cues to interpret and organize

information [3.RI-3, 7, 8 ; 4.RI-5, 7]

WI.m write introduction of several sentences that sets the context and states a focus/ controlling idea about a topic/ subtopics (e.g., “Many sports can be played outside in winter.”)

RI.k use supporting evidence to analyze or compare texts or parts of texts: author’s purpose, points of view, key ideas/details,

different accounts [3.RI-6, 9; 4.RI-3, 6, 8]

Excerpt from the Grade 7 ELA Learning Progressions Framework: Progress Indicators (PIs) for Reading Informational Texts, with related grade 7 Common Core standards, and possible instructional Building Blocks for each step (PI) in the progression

PI: M.RI.j use supporting evidence to

summarize central ideas, draw inferences, or analyze connections within or across texts (e.g., events, people, ideas) 7.RI-1, 2, 3, 9

PI: M.RI.i utilize knowledge of text Instructional Building Blocks for this PI might include:

Locate text evidence /key details to

support inferences about central ideas in a text 7.RI-1

Summarize central ideas 7.RI-2 Find similar OR different

information about a topic in two or more texts 7.RI-9

Draw inferences from or explain connections among ideas, events, or individuals d i s c u s s e d i n a t e x t 7.RI-3

structures and genre features to locate, organize, or analyze important information 7.RI-5

PI: M.RI.h flexibly use strategies to Instructional Building Blocks for this PI might include:

Locate & use informational text features to answer questions (captions, titles, headings, etc.)

Identify the kind of information found in different informational texts (e.g., newspaper versus magazine)

Recognize structures that help to organize information (e.g., introduction, body, conclusion; signal words for compare-contrast, proposition-support, etc.) 7.RI-5

derive meaning from a variety of print/non-print texts 7.RI-4; 7.L-4, 5a; 7.SL-2

Instructional Building Blocks for this PI might begin with

Listen for key ideas and details in news/media stories 7.SL-2

Interpret visuals in informational texts (e.g., arrows in a graphic organizer showing how a plant grows or how water evaporates) 7.SL-2

Answer the question: does this word make sense? 7.RI-4; 7.L-4

CCSS describe WHAT students should know and be able to do

• The Common Core identifies endpoints – grade level targets for learning

• The Common Core does not suggest an instructional sequencing plan:

“…just because topic A appears before topic B in the standards … does not necessarily mean that topic A must be taught before topic B. A teacher might prefer to teach topic B before A, or might choose to highlight connections… of her own choosing that leads to A or B” (CCSSM p. 5)

A standard…grade 3

• 3.OA-1: Interpret products of whole numbers,

e.g., interpret 5 × 7 as the total number of

objects in 5 groups of 7 objects each. For

example, describe a context in which a total

number of objects can be expressed as 5 ×

7.

• 3.OA-5: Apply properties of operations as

strategies to multiply and divide

A learning progression…

Connecting Assessment Design to Instruction and Cognitive Models of Learning

“Ideally, an assessment should measure what students are actually being taught, and what is actually being taught should parallel the curriculum one wants students to master. If any of the functions are not well synchronized, it will disrupt the balance and skew the educational process. Assessment (inferences) will be misleading, or instruction will be ineffective.

Often what is lacking is a central theory about the nature of learning and knowing around which these three functions can be coordinated” (Pellegrino, 2010).

Assessments for accountability based on CCSS

• A key objective of CCSS assessment is to examine whether or not students are proficient relative to agreed-upon standards at particular grade levels.

• Because of this design, the achievement of students at the highest and lowest levels can not be accurately measured.

Assessments based on cognitive learning models • Learning model assessment seeks to identify where a student

falls along a learning continuum while also maintaining the ability to determine whether the student is proficient on a given grade level standard.

• Learning models (progressions) “represent a promising framework for developing meaningful assessments, allowing both large-scale and classroom assessments to be grounded in models of how understanding develops in a given domain” (Alonzo & Steedle, 2009).

• Learning models provide a cohesive cognitive framework that is crucial for correspondence between the components of a comprehensive assessment system (Cameto et al., 2012).

Cognitive learning models have traditionally been used for formative and instructionally embedded assessment

• Formative assessment means assessment questions, tools, and processes that are embedded in instruction and are used by teachers and students to provide timely feedback for purposes of adjusting curriculum to improve learning (RTTT, 2009).

• Formative assessments are based on learning progressions that clearly articulate the sub-goals of the ultimate learning goal (Heritage, 2008).

• Formative assessments are given during the learning process – “minute-by-minute”.

• Use of learning progressions can increase use and frequency of formative strategies in making instructional decisions; and shift perceptions away from a deficit model of education (Hess, 2012a).

Suggested areas of new LP research (Hess, 2012b)

1. Essential content: Is there content that is most essential to learn and build upon over time? Can some “chunks of content” be skipped and not become insurmountable learning gaps later on?

2. Learning time & conditions: How much content is reasonable to learn in a school year? Is there optimal sequencing of content or instructional approaches to support that learning?

3. Assessment design and use: How can knowing more about what to teach and how best to teach it change how we design and interpret assessment evidence, monitor progress, and describe learning growth?

4. Affect on educator knowledge, practice, or perceptions: To what degree can use of learning progressions in the classroom change educator practice?

In summary: There will be new opportunities to learn more about how ALL students learn

• For the first time, the numbers of students in participating in all of the assessments will allow a closer investigation of how students in special populations are learning. Specifically, we will have large numbers of SWCD taking the same alternate assessments. Let’s take advantage of this “perfect storm” in assessment history!

• Data from interim/benchmark assessments have traditionally been aggregateable to the district level, and will allow closer investigation of special populations subgroups.

• Adaptive assessments proposed in several consortia provide expanded opportunities to examine the pathways students take in order to validate various learning models.

In summary… (cont.)

• Learning progressions and learning maps have the potential to transform how we construe learning, instruction, and assessment in all content areas and for all students.

• It takes a substantial amount of time and resources for researches to develop empirically-based learning progressions and aligned formative and summative assessment tools. There will never be enough time…unless we work together.

• A collaborative approach to development—educators and researchers working as partners—is recommended.

References Alonzo, A. C. & Steedle, J. T. (2009). Developing and assessing a force and motion

learning progression. Science Education, 93(3), 389-421.

Cameto, R., Bechard, S., & Almond P. (Eds.). (2012). Third Invitational Research Symposium: Understanding Learning Progressions and Learning Maps to Inform the Development of Assessment for Students in Special Populations. Manuscript in preparation. Menlo Park, CA, and Lawrence, KS: SRI International and Center for Educational Testing and Evaluation (CETE).

Duschl, R., Schweingruber, H., and Shouse, A. (Eds.). (2007). Taking science to school: Learning and teaching science in grades K-8. Washington, DC: National Academies Press.

Heritage, M. (2008). Learning progressions: Supporting instruction and formative assessment. Council of Chief State School Officers. Washington D.C.

Heritage, M. (2010). Formative assessment and next-generation assessment systems: Are we losing an opportunity? Council of Chief State School Officers. Washington D.C.

References (cont.)

Hess, K. K. (Ed. & Principle Author), (2011). Learning Progressions Frameworks (LPFs) for ELA. Retrieved 2/1/12, from http://www.nciea.org/publications/ELA_LPF_12%202011_final.pdf

Hess, K. K. (2012a). Learning progressions in K-8 classrooms: How progress maps can influence classroom practice and perceptions and help teachers make more informed instructional decisions in support of struggling learners. Retrieved 2/1/12 from http://www.cehd.umn.edu/NCEO/OnlinePubs/Synthesis87/SynthesisReport87.pdf

Hess, K. K. (2012b). Equitable assessment of special populations: The importance of learning progressions research for students with cognitive disabilities. Invited Diversity Panel Presentation at the 2012 Annual Meeting of the National Council of Measurement in Education, Vancouver, B.C.

References (cont.)

Hess, K., Kurizaki, V., & Holt, L. (2009). Reflections on tools and strategies used in the Hawai’i progress maps project: Lessons learned from learning progressions. Retrieved 2/1/12, from http://www.nciea.org/publications/Hawaii%20Lessons%20Learned_KH09.pdf

Pellegrino, J. W. (2010). The challenges of conceptualizing what low achievers know and assessing that knowledge. In M. Perie (Ed.), Teaching and assessing low-achieving students with disabilities: A guide to alternate assessments based on modified achievement standards (pp. 67-109). Baltimore, MD: Paul Brookes Publishing Co.

Smith, C., Wiser, M., Anderson, C., & Krajcik, J. (2006). Implications of research on children’s learning for standards and assessment: A proposed learning progression for matter and atomic-molecular theory. Theory Measurement, 14 (1&2), 1-98.