An Introduction to the ePET e-Portfolio

and lessons learned along the way…

Simon Cotterill Senior Research Associate School of Medical Sciences Education DevelopmentNewcastle University

http://www.eportfolios.ac.uk

Overview

Background and Overview of ePET

Case Studies:

Medicine at Newcastle

EPICS-2: blog linked to skills

Dental ePortfolio

Dynamic Learning Maps

Lessons Learned

Interoperability

A long history….

ePET project

FDTL-4 Project

EPICS regionalePortfolio projectEPICS-2 regionalePortfolio project

Dental ePortfolios

Mobileportfolioprojects

Hostedservices

FDTLTransferability

Project

PIOP projects x 3DynamicLearning Maps

NARNnetwork

Newcastle-Nottingham InternetPersonal & Academic Records x 21998 - present

JISC

T&L Fund Projects x 2

Postgrad.ResearchTraining ePortfolio

Teaching FellowshipePortfolio for appraisal

Postgraduate eProgression

Institutional roll-out project

Mini project(Bioscience)

Mini Projects(Medicine)

eDoctoringportfolio(California)

Embedding in the curriculum = better engagement and effective use of ePortfolios

Driessen E et al. Portfolios in medical education: why do they meet with mixed success? A systematic review. Med Educ. 2007;41:1224-33.

Atlay, M. Embedding PDP practice in the curriculum, in Personal Development Planning and Employability, Higher Education Academy, York. 2006

Cotterill SJ, Horner P, Teasdale D et al. Effective embedding and integration of ePortfolios in medical and dental curricula. IJCOS in press

Cotterill SJ, Lowing K, Cain K, Lofthouse R, et al. Blogs and e-Portfolios: can they support reflection, evidencing and dialogue in teacher training. Journal of Learning Development in Higher Education, 2, 2010.

… but, what do we mean by embedding?

Discrete ‘bolt on’ – optional and additional to the curriculum

+ Low cost- Risk low engagement

Linked Run in parallel to the curriculum but linked to it

+ No disruption to curric.- Risks low engagement

Embedded Included in specific module(s) + Better consistency of experience- Risks being fragmented

Integrated Underpins and used throughout the curriculum

+ Becomes integrated part of staff/student thinking- Difficult to achieve!

Extended Integrates activities in the curriculum along with wider life experiences

+ Draws together study, work & life experiences- Unpredictable outcomes

Atlay (2009) classified ways in which PDP can be associated with curricula:

Opportunity to customise portfolio for specific programmes:

Select componentsfor your programme(eg. CV, blog,SWOT, meetings etc)

Design new componentsfor your course-create proformas via simple Web forms.

Customise look-and feel, terminology and text

Specify Skill-sets/ Learning Outcomes

Build on core features, including:

• Content sharing – add formative comments• Integrated action planning• Uploading files

ePET: A flexible component-based ePortfolio

Cotterill SJ, Horner P, Hammond GR, et al. Implementing ePortfolios: adapting technology to suit pedagogy and not vice versa ! Proc. ePortfolio 2005

Case Studies

Cotterill SJ, Bradley PM, Stacy R. Using ePortfolios to support annual appraisal in undergraduate medicine. Proc. ‘ePortfolios, identity and personalised learning in healthcare education’, HE Academy MEDEV Subject Centre, Newcastle, 2008. pp 57-63. ISBN 978-1-905788-66-2

Medicine: Portfolios to support F2F Annual Appraisals

70% of students (Yr 1) found it a positive experience

useful for reflection, interaction/discussion & planning. instructive on how to evidence their development for

future appraisals/assessment Students really appreciated personal interaction in the

appraisal 58% felt it would influence their learning in Yr 2

Some unclear on purpose/rationale Discussion too focussed on portfolio process – for some Timing – near exams – was an issue (changed

subsequently)

Summaries of evidence used as the basis of a 20 minute face-to-face annual appraisal.

Questionnaire Study (Medicine): 213 Respondents (90% response rate)

“The fact that you actually spoke about your progress made you feel that it wasn't just about exams”

Personal / Interaction

“It was really good to talk about the course and how I was doing and what I could do better. I really found it beneficial to have someone to bounce things off.”

What did you most helpful about the appraisal session ?

Reflection

“It made me think about what I have achieved this year.”

“I was able to clarify to myself via the facilitator both the things that have gone well regarding my learning this year, and the things that I needed to improve.”

Holistic / Overview

“Brought things together. Made me sum up my overall progress of the year, which was a really nice thing to do. Positive reflection.”

“The appraiser helped me realise that there were other activities that I did which would also constitute evidence…”

“It linked in the extra curricular activities to the medical course.”

“It encouraged me to really give thought to what I wanted to achieve during the [SSC] ....As a result of the portfolio I think I got much more out of the [SSC] than I would have otherwise.”

“It made me concentrate on creating aims at the start of the [SSC] and allowed me to plan the [SSC] with my supervisor in a defined way. Overall it made my learning for the [SSC] more organised and focused.” “Most of the things I learned couldn't

be 'measured/quantified' so I felt it was of little value to try and invent a way in which they could.”

Printed output from the ePortfolio (partial)

Summative Assessment: ePortfolio for SSC Placements

Cotterill SJ, Bradley PM, Hammond GR ePortfolios: Supporting assessment in complex educational environments. In: Bryan C and Clegg K, ed. Innovative Assessment in Higher Education. London: Taylor and Francis Group Ltd, 2006.

North East regional collaboration for personalised, work-based, and life-long learning

October 2007 to February 2009

Partners include: 5 HEIs FE colleges (Comport Project) CETL4HealthNE

Project Director: Prof. Geoff HammondProject Manager: Simon CotterillProject Officers: Paul Horner

Martin Edney

http://www.epics.ac.uk

EPICS-2

ePET: Unstructured Blogbut with explicit links to skills/outcomes

Cotterill SJ, Horner P, Gill S, et al. Beyond the Blog: getting the right level of structure in an ePortfolio to support learning. Proc. ePortfolios 2007, Maastricht

Evaluating use of ePortfolio to support personalised learning pathways

• Cohort: students commencing October 2007 (n=119)

• Modest engagement: av. 28 logins per student in 18 months (range 1 -168)

• Minimal engagement prior to assessed assignment in Semester 2, Year 1

• Mostly used CV/skills (assessed),limited use of cross-module learning log

Case Study: Combined Studies

“It helped me see the skills I was using that my modules shared and sort of helped me develop those further”.

“I had to write a CV and it helped me think about what modules and what aspects of each module I enjoy, while writing about them”.

“didn't really understand what I needed to put down as evidence”

“too time consuming”

Evaluating use of ePortfolio/blog to support placement learning

• Cohort: BSc & MSc students (all years) from October 2007

• Medium engagement: av. 30 logins per student (range 2 -142)

• Steady use over time; recording clinical goals, placements and clinical skills

• Virtually no sharing of blog entries in the community areas

Case Study: Speech & Language Sciences

“being able to put in my placement goals, and have a format to review my progress in these at each stage”.

“Ability to put in different placements into the portfolio alongside your goals and whether you achieved them. Ability to make blogs private to evaluate yourself without others seeing.”.

“I find I do not have enough time to use the ePortfolio regularly.”

“You had to be careful to tick a load of boxes saying that your blog was private because it could get posted in the community blog otherwise”

Evaluating use of ePortfolio/blog to support placement learning

• Cohort: PGCE students (all subjects) from Sept 2008 (n=156)

• High engagement: av. 41 logins per student (range 4 -178) in Semester 1, 1096 files uploaded

• Good use of blog: av. 16 entries. Many linked to Skills av. 76 links to QTS standards

• Many blogs published to community areas: 825 entries, 262 comments

Case Study: PGCE (Secondary)

“the ePortfolio is well set-out with regards to being able to link blogs to the skills pages. The ePortfolio in a sense guides you through the necessaries”.

“What I do like about the ePortfolio is that it is designed for the purpose of building a skills repertoire, and allows you to connect thoughts and experiences to the Key Skills.”

“It's not very clear exactly what parts of it are mandatory and what parts are optional”

“It is tedious having to fill in a weekly blog especially when I have many other things to do.”

“good for staying in touch whilst on teaching practice”.

91% of respondents use social networking sites (58% on a daily basis). Use of these sites was predominantly for social reasons, though:

62% sometimes used them to communicate with classmates about course-related topics 9% had used them to communicate with teaching staff about course-related topics

Use of Social Networking and perceptions of its use in learning

“…most people see Facebook etc. as an escape from work and it really should stay that way.”

BSc Speech & Language Sciences student

“It must never be enforced. Emphasis on ‘social’ networking. Informality is key.”

Combined Studies student

“I prefer to keep social networking sites for personal use and for engaging in general conversation about essays etc in a non-official/non-university domain where it's friends discussing a course.

Professional dialogue, opinions on educational matters, lesson plans, theory discussions I prefer to engage in face to face or via the official, nominated online spaces.”

PGCE studentn=163 (30% response rate)

Factors related to engagement with ePortfolio / blog

Group 1

High Engagement

Group 2

Medium Engagement

Group 3

Modest Engagement

I have a clear understanding of the purpose of the ePortfolio 87% 74% 58%

I have a clear understanding of how the ePortfolio is used in my programme 91% 76% 42%

I received adequate information on how to use the ePortfolio 58% 50% 58%

I had a clear understanding of the skills being evidenced in the ePortfolio 66% 65% 17%

The skills included in the ePortfolio are important in studying for my degree 69% 94% 25%

The skills included in the ePortfolio are important for my longer-term career 59% 94% 33%

The ePortfolio is important for my programme 75% 82% 33%

Course handbooks and study guides refer to the ePortfolio 58% 65% 25%

Teaching staff regularly refer to the ePortfolio 84% 41% 33%

Questionnaire results n=163 (30% response rate, 3 subjects – non-medical)

EPICS-2 Projecthttp://www.epics.ac.uk

• Porfofolio accessed using wireless laptops/ Mobile devices in clinics

• Students record procedures & grade their own performance on clinic

• Staff grade students’ performance & feedback

• Additional reflections can be added & grade summaries accessed later via Web

Assessment at the point of learning:Dental ePortfolios & mobile technologies

• Facilitated roll-out across specialty clinics at Newcastle University

• Supports both self assessment & evaluation/feedback from staff • Now used by 430 dental undergraduates & their supervisors/tutors

• Integration

• Used on a day-to-day basis at the point of learning• Supports assessment across much of the curriculum

• High levels of engagement & highly valued by students and staff

Impact ♯1

Ellis J, Teasdale DW, Vernazza C, Cotterill SJ, Drummond P, Moss JD, Scott L, Thomason M. Effective use of ePortfolios and mobile technologies to support learning and assessment in Dentistry. Proc.'ePortfolios, identity and personalised learning in healthcare education', HE Academy MEDEV Sub 2008

Vernazza C, Ellis, J, Teasdale D, Cotterill S, Scott L, Thomason M, Drummond P, Moss J. Introduction of an e-portfolio in clinical dentistry: Staff and student views. Eur J Dent Education 2011;15:36-41

Synchronous & asynchronousaccess

Dynamic Learning Mapshttp://learning-maps.ncl.ac.uk

Add notes and reflections in any topic in a curriculum map

-Linked with ePortfolio using latest standards

Lessons Learned

Lessons Learned: Engagement and Effective Use of ePortfolios

• Clarity of purpose- Students- Staff

• Perceived value – what’s in it for me?- Assessment? Careers / employability?

• Embed & relate to the curriculum - not a ‘bolt on’

• Training- A brief mention at induction is unlikely to be enough- Set a specific task early on

• Plan in a check-point - don’t leave it all to the end of the year!

• Implementation – take the longer view– ‘a good fit takes lots of time and lots of practice’– Change management – its about learning cultures not just

technology (e.g. independent, life-long learning)

• Getting the right level of structure– Some structure may provide useful ‘scaffolding’– Too much structure often perceived as ‘form filling’ and may

reduce engagement

• Changing expectations and technical literacies– Blogs – dialogues within ePortfolios– ‘Digital natives’ > Facebook generation & beyond– Personalised learning

Lessons Learned: Engagement and Effective Use of ePortfolios #2

Life Long Learning(interoperability)

Interoperability• Standards and specifications for transferring life-

long learning records between systems: • EUROPASS• IMS ePortfolio• LEAP2a http://www.leapspecs.net

• Extended the work in EPICS-2 into 5 JISC-CETIS collaborative projects:• PIOP1..3 (completed)• XCRI mini project (completed)• Competencies (ongoing)

Horner P, Cotterill SJ. Meeting the ePortfolio Interoperability Challenge. Proc. ePortfolios, identity and personalised learning in healthcare education. 2008 Newcastle

Horner P, Cotterill SJ, Peterson J, Skelly G. Interoperability in Action – ePortfolios, Leap2a and Dynamic Learning Maps Proc. London Learning Forum; ePortfolio – Key Competencies – Identity, 2010, London.

Continuity in life-long learning (LLL)

School HE Continuing Development

• Explicit recognition of prior learning• Focus on continuous development, not episodic learning

Integration with other systems

Educational Benefits of Interoperability

Life-long ‘personal learning space’Serial transfer of data Aggregators of multiple portfolios / blogs

Alternativemodels

• Linking into the wider MLE• Integration with other systems

e.g. Workshop admin portfolio/CVe.g. CV recruitment services

Summary• Case studies add to our knowledge of the use of

ePortfolios/blogs to support learning

• Novel approach to link blogs to structured outcomes in ePET

• Work on interoperability standards for lifelong learning helped shape national standards.

• Dynamic Learning Maps – curriculum maps linked to ePortfolio

Further information

Full list of publications and resources:http://www.eportfolios.ac.uk

Dynamic Learning Mapshttp://learning-maps.ncl.ac.uk/

Simon.Cotterill@ncl.ac.uk