ASSESSING TRANSFER: USING REFLECTION TO EVALUATE TRANSFER OF KNOWLEDGE AT CRITICAL 

TRANSITIONS IN WRITING PROGRAMS  B O B B R O A D , I L L I N O I S S T A T E U N I V E R S I T Y H E I D I K E N A G A , W A Y N E S T A T E U N I V E R S I T Y D A N A D R I S C O L L , O A K L A N D U N I V E R S I T Y J O S E P H P A S Z E K , W A Y N E S T A T E U N I V E R S I T YW E N D Y D U P R E Y , W A Y N E S T A T E U N I V E R S I T Y L I A N E R O B E R T S O N , W I L L I A M P A T E R S O N U N I V E R S I T YG W E N G O R Z E L S K Y , W A Y N E S T A T E U N I V E R S I T Y D A V I D S L O M P , U N I V E R S I T Y O F L E T H B R I D G EJ A R E D G R O G A N , W A Y N E S T A T E U N I V E R S I T Y K A R A T A C Z A K , U N I V E R S I T Y O F D E N V E RA D R I E N N E J A N K E N S , W A Y N E S T A T E U N I V E R S I T Y T H O M A S T R I M B L E , W A Y N E S T A T E U N I V E R S I T Y

WORKSHOP OVERVIEW

• Defining & assessing transfer• Key principles for measuring transfer• Case studies on assessing transfer• Working Groups• Critical transitions• Developing goals• Developing assessment plans

To access all workshop materials online:http://assessingtransfer.pbworks.com

BASIC DEFINITIONS

• Smit (2007) argues that the ability to transfer knowledge is what the term “learning” actually means (p. 130).

• The National Research Council (1999) argues that “The ultimate goal of learning is to have access to information for a wide set of purposes—that the learning will in some way transfer to other circumstances” (p. 61).

• Other transfer terms: boundary crossing,

knowledge building

TRADITIONAL COGNITIVE AND TASK-BASED TRANSFER

• Definition: “The application of knowledge learned in one situation to a new situation.”

• Research method: “improved performance on tasks” (primarily through experimental design)

• Research questions: “Was transfer obtained? What conditions facilitate transfer?”

(Loboto 2003)

Briel

Jimmy

Rhetorical and

Genre Analysis in FYC

Analysis of PresidentialCandidates

Product Analysis

Whitepaper in Business

Class

Jimmy: Each situation is viewed as unique and nothing is carried to the next situation.

Briel: Transfers knowledge between tasks and builds her knowledge of analysis.

CONTEXT-BASED TRANSFER (ACTIVITY THEORY)

• Actor-Oriented Transfer focuses more on the context of learning and works within the realm of activity theory

Definition: “The personal construction of relations of similarity across activities (i.e., seeing situations as the same).”

Research method: “Researchers look for the influence of prior activity on current activity and how actors construe situations as similar.”

Research questions: “What relations of similarity are created? How are they supported by the environment?” (Loboto, 2003)

Briel

Jimmy

University Activity System

American Civic Engagement Activity

System

BUS300 Activity System

FYCActivity System

PoliticalVoting

Activity System

CONTEXTUAL-DISPOSITIONAL (HYBRID) APPROACHES

• Wells and Driscoll (under review) argue that both task-based and contextual approaches are useful, but provide an incomplete picture of transfer.

• Rather, they argue it is the intersection of the task, context, and the individual learner’s dispositions. These include:• motivation, self efficacy, help-seeking, willingness to engage

in mindful abstraction, developing a metacognitive mindset, beliefs, attitudes etc.

• This approach examines the relationship between learner, the task, and the context and is particularly useful to assessment.

• Bio-ecological assessment can also fit within a hybrid approach (as described next)

BIO-ECOLOGICAL THEORY OF TRANSFER

Dispositions

Demand Characteristics

Resources

Individual

Proximal Processes• Processes

through which learning occurs (within local environment)

Transaction

Bronfenbrenner and Morris (2006); Slomp (2012)

C H A L L E N G E S A N D C O N S I D E RAT I O N S

ASSESSING TRANSFER

3 CHALLENGES

1. Choosing a theory of transfer that captures a full picture of the factors that support or inhibit transfer.

2. Defining/adopting a socially-situated construct model of writing ability.

3. Overcoming technocentric limitations in current writing assessment practices

CHALLENGE 1:DEFINING A ROBUST THEORY OF TRANSFER

THEORIES OF TRANSFER

• Clearly define how you are conceptualizing transfer:• The theory of transfer you adopt will determine your

research/assessment focus.

• Carefully consider which theory of transfer you base your assessment on.• Traditional conceptions of transfer have

underrepresented what people know and are able to do (Schwartz, Bransford & Sears, 2005).

TASK-BASED CONCEPTION OF TRANSFER

Task 1:

Complete worksheet on rules for using commas.

Task 2:

Complete worksheet requiring student to properly insert commas on a page of unpunctuated sentences.

Are students able to draw on knowledge of comma rules developed in task 1 to complete task 2?

Do students perceive that what they learned in FYC has helped them with later writing assignments across the university? How does the curriculum structure influence generalization?

CONTEXTUAL THEORIES OF TRANSFER

Context 1: FYC

Organization skills, process knowledge, critical reading, subject knowledge

Context 2: Writing Across the University

Students did not generalize from FYC because the activity system did not encourage them to do so.

Activity Systems: (Wardle, 2007)

CONTEXTUAL THEORIES OF TRANSFER

• Transformation and repurposing (Roozen, 2010)

Context 1: Religious engagements

Prayer journaling, verse-copying, sermon outlining

Context 2: Academic engagements

Note taking, outlining, and organizing academic arguments.

How does prior knowledge inform current practices? How is prior knowledge repurposed when applied in new contexts?

BIO-ECOLOGICAL THEORY OF TRANSFER

In what ways do characters of the developing individual, proximal processes, and the ecological systems in which students learn shape their capacity to generalize, repurpose and/or transfer knowledge about writing?

Context 1: FYC

Writing process knowledge, metacognitive knowledge, discourse community knowledge,

Context 2: Writing Across the University

Barriers to transfer included: curriculum structures, prior experience, personality issues, challenging home environments.

Slomp and Sargent (2009, forthcoming)

CHALLENGE 2:DEFINING A CONSTRUCT MODEL

CONSTRUCT VALIDITY

• The construct writing ability is defined through the lenses of developmental theories (Camp 2012):Syntactic Maturity = Complexity of syntactic

constructions

Stage models of development

= Cognitive maturation

Socially situated theories of writing

= Discourse community knowledge &

metacognitive knowledge

Metacognitive Knowledge

Discourse Community Knowledge

Rhetorical Knowledge

Writing Process

Knowledge

BEAUFORT’S (2007) TRANSFER-ORIENTED CONSTRUCT MODEL

Genre Knowledge

Subject Matter

Knowledge

CHALLENGE 3:OVERCOMING TECHNOCENTRIC L IMITATIONS

METHODOLOGICAL CONSIDERATIONS

• Writing assessment has traditionally been limited by its technocentric orientation (Huot 2002):• Emphasis on achieving high degrees of reliability• Constrained by reliance on current technologies of

assessment

METHODOLOGICAL CONSIDERATIONS

• Assessment-as-research (Huot, 2002):

• Focus on defining information needs• Choose assessment methodologies that help you achieve

those information needs.

• Shift away from technocentric views of reliability toward a rhetorical orientation (Parkes, 2007).• Emphasis on validity

METHODOLOGICAL CONSIDERATIONS

ECOLOGICAL ASSESSMENT DESIGNWardle and Roozen (2012)

BIO-ECOLOGICAL ASSESSMENT DESIGN

METACOGNITION, TRANSFER, AND A NEW RESEARCH PARADIGM

TWO TYPES OF TRANSFER

High Road

• Results from mindful abstraction• Can occur quickly, without long-

term practices• Example: applying the “count to

ten” rule learned in childhood to inhibit tantrums during adulthood to prevent impulse buying

• Promotes greater understanding, reflective evaluation, and conscious adaptation of previously learned concepts and skills

- Salomon and Perkins (1989)

Low Road

• Results from extended practice• Spontaneous, automatic, with

little need for reflective thinking

• Example: driving a car to driving a truck

• Increased speed and efficiency• Decreased long-term memory

and analytic reflection• Potential for negative transfer

MINDFUL ABSTRACTION

Mindful Abstraction• Using metacognitive

thinking to decontextualize information to construct principles, patterns, strategies, or procedures

-- Salomon and Perkins (1989)

Abstraction• Identifying key qualities,

attributes, or patterns• Decontextualizing

information and re-representing it as a set of principles or schemas

Mindfulness• Thinking guided by

metacognition and conscious reflection on target task, context, known strategies, and potential adaptations

PRINCIPLES FOR PROMOTING TRANSFER THROUGH METACOGNITION

• Requiring students to actively monitor their learning • Providing feedback on students’ use of new knowledge• Showing contrasting cases to highlight key features• Foregrounding the transfer potential of new knowledge• Teaching new knowledge in multiple contexts • Moving from specific to general levels• Helping students abstract principles• Balancing specific examples with general principles - --National Research Council (1999)

PROMOTING TRANSFER THROUGH REFLECTION

Reflection – both a theory and a practice:

• From the work of Schon – on reflective practitioners

• From the work of Yancey – students as agents in their own learning process

PROMOTING TRANSFER THROUGH REFLECTION

Reflection’s Connection to Transfer: Our Starting Point• Significant research on each separately• Absence of research that explicitly explores

reflection’s connection to transfer • Beaufort’s knowledge domains - reflection

discussed as important for metacognition but not explicitly pursued• Schon and Yancey - reflection both

theory/practice

PROMOTING TRANSFER THROUGH REFLECTION

Reflection’s Relationship to Transfer:

• Kara’s Research: questions whether or not reflection is one of the vehicles by which students transfer knowledge and practices of writing to other academic writing situations.

• Liane’s Research: questions which content transfers effectively and how reflection as a reiterative practice fosters the transfer of that content.

PROMOTING TRANSFER THROUGH REFLECTION

Definition of Reflection:• We define as: systematic, explicit, intentional • Both an intellectual act and a physical act• aligns with Perkins and Salomon’s claim that “conditions

of a classroom affect transfer”• aligns with the notion of mindful abstraction – active

self-monitoring arouses mindfulness

• Students’ ability to monitor their own thinking processes is what leads to mindful abstraction – alertness to the activity in which they are engaged

PROMOTING TRANSFER THROUGH REFLECTION

Practice of Reflection:• We use reflection as: reiterative practice• by which students learn to define and apply their own

theory of writing as a way to foster transfer of knowledge and practices from one academic writing situation to another.

• Theory of Writing – main reflective practice for our FYC course• Students create a framework of writing knowledge• Students begin to develop metacognitive ability

PROMOTING TRANSFER THROUGH REFLECTION

Our Theory of Writing Component:• Systematic, explicit, intentional• Explicitly encourages transfer• Asks students to be mindful about what they are

learning• Reiterative assignment feature - ten writing assignments

related to theory of writing throughout the semester

• Combines learning about writing theory, through a set of key terms and through reading reflective theory, with the practice of systematic, explicit, intentional reflection

PROMOTING TRANSFER THROUGH REFLECTION

3. Ends As• Metacognitive thinking;

Abstraction• Development of a theory

of writing knowledge and practice

4. Continues As• Application of knowledge

and practice in new writing contexts

1. Begins As• Practice more than

theory • Not mindful• No abstraction, direct

application

2. Progresses Toward

• Increased active self-monitoring

• Key terms understood as writing concepts

• Mindfulness develops

PROMOTING TRANSFER THROUGH REFLECTION

Sample Reflective Activities:

• Early Guided Reflection

• Reiterative Reflection

MOTIVATION AND DISPOSITIONS

Factors

• Task difficulty level• Perceived relevance• Performance vs.

learning orientation -- National Research Council (1999)

Teaching Approaches

• Devising challenging but do-able assignments• Designing tasks that

demonstrate relevance• Providing social

support for risk-taking

DEVELOPING THE NEW RESEARCH PARADIGM: INTEGRATING INDIVIDUAL, SOCIAL, AND

DEVELOPMENTAL ASPECTS

PFL Enables Writing Studies Researchers to:

• Evaluate students’ metacognition in relation to writing studies conceptual and procedural knowledge

• Evaluate how students use this knowledge to learn about unfamiliar genres and rhetorical situations

• Triangulate data from assessments of individual factors like dispositions and motivations and of social factors like curriculum

Preparing for Future Learning (PFL)

• Focusing on Interpretive Knowledge

• Evaluating Interpretive Knowledge

• Incorporating opportunities for learning into assessment

• Focusing on both “transfer in” and “transfer out”

-- Schwartz, Bransford, and Sears (2005)

DEVELOPING THE NEW RESEARCH PARADIGM: INTEGRATING INDIVIDUAL, SOCIAL, AND DEVELOPMENTAL

ASPECTS

• Pre- and post-semester surveys evaluating students’ motivation and dispositions related to writing instruction

• Text-based interviews that ask students to describe their drafting choices, particularly the conceptual and procedural knowledge they used

• Textual analyses comparing discourse features of students’ reflective writing with those of their academic writing

• Textual analyses of students’ reflective writing on their uses of conceptual and procedural knowledge about writing.

• Tasks used and knowledge required• Learners’ prior experiences and dispositions• Classroom and curricular contexts

Develop a hybrid, locally focused model that takes into account all of the factors affecting transfer.

ASSESSING TRANSFER

REFERENCESDriscoll, D. and Wells, J. (under review). Toward a Dispositional

Model of Writing Transfer: The Impact of the Individual Learner. Loboto, J. (2003). How Design Experiments Can Inform a

Rethinking of Transfer and Vice Versa. Educational Researcher, 32(1), 17-20.

National, R. C. (1999). How people learn: brain, mind, experience, and school. . Washington D.C.: National Academy Press.

Royer, J. M., Mestre, J. P., & Dufresne, R. J. (2005). Introduction: Framing the transfer problem. . Greenwich, CT: Information Age Publishing.

Salomon. D. N., and Perkins, G. S. a (1989). Rocky Roads to Transfer: Rethinking the Mechanisms of a Neglected Phenomenon. Educational Psychologist, 24(2), 113-142.

Smit, D. (2007). The End of Composition studies. Carbondale, IL: Southern Illinois University Press.

Tuomi-Grohn, T., & Engestrom, Yrjo (2003). Between school and work : new perspectives on transfer and boundary-crossing (1st ed.). . Boston, MA: Pergamon.