Reflection, Integration, Identity, and Institutional Change: Lessons for Seven Years of the Inter/National

Coalition for Electronic Portfolio Research

Barbara Cambridge, NCTEKathleen Blake Yancey, Florida StateDarren Cambridge, George Mason

AAEEBL, Boston, July 21, 2010

Context, Results, and Future Directions

• Barbara – Contextualizing the Inter/National Coalition for Electronic Portfolio Research

• Kathi – Selected Findings • Darren – Future directions

Contextualizing the Inter/National Coalition for Electronic Portfolio Research

Three premises of the I/NCEPR

• Building social capital is essential for change.• Intermediaries are essential to explain the

local.• Defining the “non-negotiable core” is essential

for the future of eportfolios.

Building Social Capital is Essential for Change

• Bonding social capital– Institutional research teams

• Bridging social capital– Cohort of multiple teams– Network for cohorts– International members of a cohort– Potential expanded membership

Intermediaries are Essential to Explain the Local

“The strongest theme that emerged form each group was that research—whether scientifically based or best practices—must be viewed in relation to the local context.” W. T. Gates Foundation Report, 2010

Designed inquiry emerging from a local question and leading to findings shared with a wider audience exemplifies the scholarship of teaching and learning

Educational capital is “the progressive accumulation, in forms usable by educators, to validate experience and knowledge about successful educational ideas and strategies”

Intermediaries, a category of knowledge builders, “can offer the stability, expert depth, and field-wide research to make assembling and circulating elements of educational capital a signal contribution to their constituents.”

Ray Bachetti and Tom Ehrlich, Reconnecting Education and Foundations, 2007, p. 23, 43

“Both the research literature and the participants identified the importance of intermediaries and trusted individuals to increase communication between policy makers and practitioners.”

“All participants in the study seldom, if ever, go to reports of research findings. They almost always seek research indirectly through intermediaries and translators.”

W. T. Grant Foundation report, 2010

I/NCEPR as Intermediary

• Internally– Calls– Critical friend exercises– Synthesizing and recirculating

• Externally– Ten-page reports– Need for interpretation for policy makers

Defining the “Non-Negotiable Core” is Essential for the Future of Eportfolios

• Are there certain features of eportfolios, eportfolio practice, and/or assessment of eportfolios that must be present to justify the name eportfolio?

• Cohort VI will take up a common research question about the special capacities of eportfolios for assessment

(Selected) Findings from the Inter/NationalCoaltion for Electronic PortfolioResearch

•Common question played across different sites•The rhythm of the questioning•The importance of documenting practice•The role of inquiry and the willingness

to engage in it•The importance of connecting the question and its results with larger sets of data

The Power of Collective Expertise

Beginning/Continuing Questions

What difference, if any, do eportfolios make in student engagement?

What difference, if any, do eportfolios make in student retention?

How can/do we define reflection?

How can/should we “assign” reflection?

What is the relationship between eportfolios and institutional cultures?

On virtually every question, students in these ePortfolio-intensive courses tended to score higher in engagement than the collegewide reported by the LaGuardia Office of Institutional Research. For example, on one critical thinking-related question--Question 5a from the 2005-6 survey: “How much has your coursework emphasized synthesizing and organizing ideas, information, and or experiences in new ways?”--the collegewide mean is 2.85 (a substantial .18 points above the national mean of 2.67). The mean for students in ePortfolio courses was 3.12, an additional .27 points higher than the already positive college mean. The pattern was similar for questions about writing, effort, technology and classroom collaboration.

The six questions that were significantly more positive than national and local benchmarks addressed the use of values, critical thinking, writing, teamwork, and a level of engagement. [In addition] Students’ spontaneous, open-ended comments in the survey support our hypothesis that working on the ePortfolio with the values approach is leading students to engage more deeply in their learning, to mention the values in their reflections, and to relate the values to their understanding of their learning.

Retention data is similarly positive. For example, analysis of the transcripts of a sample of nearly 2,000 students in ePortfolio-intensive courses in 2005-6 showed an average one-semester return rate that is 5.6 percentage points higher than the college average.

“Not only did I gain technical skills, but I learned how to express myself as a student. The different sections of my ePortfolio made me realize important things about how I see myself starting at LaGuardia, how I see myself now and in my future. My experience with ePortfolio at LaGuardia has made me

see more of who I want to be.”

Seton Hall’s contribution to this line of research is to focus on the use of the eportfolio as a site for students’ recording and reflecting upon non-cognitive traits, specifically five such traits, including familial support for success in college and social integration during students’ first year. The intent in this eportfolio project, then, is to foster the development of these non-cognitive factors so that students stay in school. Initial data from this project show two important outcomes: (1) that scoring guides keyed to these traits can be developed and applied to eportfolios and (2) that students who score well on such traits are in fact more likely to stay in school.

Learning to reflect is a key skill that needs to be scaffolded at key points in the course in a way in which

students are clear about the purpose of reflection, and through which the linguistic mean, and technical knowhow are achieved.

• Sharing private thoughts in public spheres

• PD in ways to promote reflection and assess reflective practice

• Reflection and reflective practice should be built in at the course design stage. Where this is not possible courses should be redesigned.

Implications for Practice: Waterloo and Sheffield Hallam

Reflection and GenreUnder what conditions do low-stakesportfolios generated for programmaticassessment support student learning?

What is the relationship between reflectionand argument in portfolio cover essays?

“In many ways, our research method highlights the concepts of process and reflection that we now seek to investigate in

student portfolios.”

ePortfolios and Threshold Concepts

1. The Role of Purpose2. The Role of Learning Activity Design3. The Role of Processes4. The Role of Ownership5. The Disruptive Nature of ePortfolios

(Selected) Findings from the Inter/NationalCoaltion for Electronic PortfolioResearch

Cambridge, Cambridge, and Yancey, eds. Electronic Portfolios 2.0.Washington, DC: Stylus. 2009.

www.ncepr.org

Joyes, Gray, and Hartnell-Young, “Effective Practice with E-Portfolios.”Australasian Journal of Educational Technology 26.1 (2010): 15-27.

Future Directions for the Inter/National Coalition for Electronic Portfolio

Research

Possible Future Research Focuses

• Assessment that capitalizes on the distinctive features of eportfolios

• From individual to collective self-representation

• Eportfolios in medical education • Longitudinal impact of eportfolio practice • Methodology for reading across local research

University of Cincinnati

• Comparison of rubric-based assessments of work from eportfolios and CLAP scores

• Generative patterns of similarity and difference

• Through sharing work in progress, realized that their results were about authentic assessment of samples of work, not assessment of eportfolios as such

Beyond VALUE

• VALUE provides a powerful set of metarubrics for conversations about liberal learning grounded in authentic evidence

• Metarubrics designed for authentic assessment of individual samples of work rather than portfolios

• Little attention to reflection, diversity of evidence, and the digital and networked medium

VALUE Intercultural Rubric

From individual to collective

• The eportfolio is strongly associated with an individualist perspective

• Eportfolios can also have value as collective represtations– Demonstrated by the Urban Universities Portfolio

Project • Social media enable new collectivist designs

http://augustaportfolio.com/

Eportfolios in Medical Education

• Fast, international uptake of eportfolios• Existing tradition of reflective practice• Experiential learning with both cognitive and

affective dimensions• Could provide a model for other professions and

for developing practical reasoning more generally• Challenges of access and cost shared with higher

education and illuminated by international dialogue

Longitudinal Impact of Eportfolios

• Lifelong learning a ubiquitous goal of eportfolio projects, but little existing research beyond the original context of the project

• Do graduates continue to use eportfolios after leaving the institution and, if so, how?

• Are there habits of mind instilled through eportfolio practice that persist and continue to develop?

• How does eportfolios practices transfer between educational levels and settings?

A meta-research topic

• Can we systematize the way we read across multiple local research projects to draw more general conclusions?

• Can we find more effective ways to translate those results-in-coalition for broader audiences, such as policy makers?

Electronic Portfolios 2.0: Emergent Findings and Shared Questions

• Collection of 24 chapters detailing research from cohorts I, II, and III of the Coalition

• Published by Stylus in 2009

Coalition website: ncepr.org

• Member campuses with project descriptions

• Final reports from Cohorts I-IV

• Slides at ncepr.org/darren