(with apologies to masterchef), a feast of formative video

(With Apologies to Masterchef), A Feast of Formative Video Feedback

Served Five Ways,on a Bed of Enterprise Education

with a Side of Researchand Drizzled with Impact on

Student Learning

Vic Curtis & Rob MoonDerby Business School

www.derby.ac.uk/business

1. Our previous attempts at video feedback

2. Some research around formative feedback

3. Formative video feedback, served five ways

4. Feedback on the feedback

5. How can you implement video feedback?


What We Propose To Do Today…

• 78 summative feedback videos for two modules – 32 hours over 11 days – Average time 6½ - 7 mins per video (+ 50 - 75%) – Took too much time and was too repetitive!

• But, feedback from students – For 50% who accessed, watched multiple times – Video feedback was of greater length/quantity,

and had better depth/richness/quality – Overall, video feedback was preferred

What We Have Done Before


• Almost all positive • Anecdotal evidence reinforced this view

Student View on Video Feedback


Pros • Extends student

multimedia experience

• More senses used so more comprehension

• Richer feedback as more detailed and longer

• Viewable at time / place of student choosing

Cons

• Being self conscious and self critical

• Many feedback points repeated as similar issues identified

• Takes a long time! • Summative feedback,

so little to feed-forward

Our View on Video Feedback


“When the cook tastes the soup, it is formative; when the guests taste the soup, it is summative”

(Bob Stake)

How can formative feedback inform a new vision for Enterprise Education by using multimedia technology to help students quickly and easily

“taste the soup” and, importantly, to help them feed-forward and see if they need more salt?

Let’s look at some of the research on feedback…


Tasting the Soup

• Is there evidence that improving formative assessment raises standards? – Yes, as evidenced by effect sizes of typically 0.4-0.7

• Is there evidence that there is room for improvement? – Yes, by avoiding superficial learning, not focusing on

marks and not competitively comparing students

• Is there evidence about how to improve formative assessment? – Yes, by focusing on quality of work, how to improve,

student self-assessment and allowing time for reflective thinking

Black and Wiliam (1998)


• Effect Size, d: d = (Meanb – Meana) Pooled Std Dev

wh a = some initial time / control group b = some later time / other group

Hattie (2009)• Synthesizes over 800 meta-analyses encompassing

52,637 studies about influences on achievement Hinge point, d = 0.40


Hattie (2012)InfluenceConcentration/persistence/engagementFeedbackProviding formative evaluationSelf-reported gradesStudent control over learningStudy skillsTeaching test-taking and coachingTime on taskWorked examples

Activity: Can you identify the top three influences on student achievement here and put them in order of size of effect?


Hattie (2012)Influence Effec

tRank

Self-reported grades 1.44 1Providing formative evaluation 0.90 4Feedback 0.75 10Study skills 0.63 22Worked examples 0.57 32Concentration/persistence/engagement

0.48 54Time on task 0.38 75Teaching test-taking and coaching 0.27 98Student control over learning 0.04 144


• Results suggest FI has overall moderate positive effect on performance

• But over 38% of FI effects were negative (ie reduced performance)

• Wide variability in effects

Kluger and DeNisi (1996)• Feedback intervention (FI) effects on performance

from 131 papers involving 12,652 participants Average effect, d = 0.41


Hattie & Timperley (2007)• Effect sizes relating to feedback


“…the most effective forms of feedback provide cues or reinforcement for learners; are in the form of video-, audio- or computer-assisted

instructional feedback; and/or relate to goals” (Hattie and Timperley, 2007 p.84)


Which Means That...

Advantages of Video Feedback• Better depth and quality of feedback • More extensive, informative and nuanced • Key points better emphasised • Easier / clearer to understand • Better level of student engagement • Reviewable multiple times • More accessible • More personal • Video feedback preferred


Various sources inc: Thompson & Lee (2013); Turner & West (2013); Crook et al, (2012); Mathisen (2012)

“Teachers will not take up attractive sounding ideas, albeit based on extensive research, if these

are presented as general principles which leave entirely to them the task of translating them into everyday practice… What they need is a variety of

living examples of implementation, by teachers with whom they can identify”

(Black and Wiliam, 1998 p. 15-6)


Why Show You Examples?

Formative Video Feedback, Five Ways• Examples 1 & 2 – “The Open University”

o Set up a static video camera with microphone on a tripod in the corner of the room, and/or

o Turn on lecture capture in the background o Do your normal feedback o Upload to University intranet o Students can review at their leisure



Open University - 1

http://revision.derby.ac.uk/Panopto/Pages/Viewer.aspx?id=47ec19d2-50cd-4b17-b34f-a6cb7914b3b6

http://revision.derby.ac.uk/Panopto/Pages/Viewer.aspx?id=47ec19d2-50cd-4b17-b34f-a6cb7914b3b6


Open University - 2

http://revision.derby.ac.uk/Panopto/Pages/Viewer.aspx?id=438227fb-352b-4660-ac49-502e94c6cbff

http://revision.derby.ac.uk/Panopto/Pages/Viewer.aspx?id=438227fb-352b-4660-ac49-502e94c6cbff

Formative Video Feedback, Five Ways• Example 3 – “The Good, The Bad and The

Ugly” o Web cam with microphone at your desk o Turn on lecture capture in the background o Turn the video camera on or off, your

choice o Give feedback about what’s on the screen

(eg slides, report/essay, spreadsheet, video etc)

o Upload to University intranet o Students can review at their leisure



The Good, The Bad and The Ugly

http://revision.derby.ac.uk/Panopto/Pages/Viewer.aspx?id=f0c5c94c-e070-43a1-9d67-f50c7bb219d5

http://revision.derby.ac.uk/Panopto/Pages/Viewer.aspx?id=f0c5c94c-e070-43a1-9d67-f50c7bb219d5

Formative Video Feedback, Five Ways• Example 4 – “Smith and Jones”

o Find an empty classroom and connect web cam with microphone to PC

o Turn on lecture capture in the background o Give feedback about what’s on the screen o Two points of view works well to bounce

ideas o Upload to University intranet o Students can review at their leisure



Smith and Jones

http://revision.derby.ac.uk/Panopto/Pages/Viewer.aspx?id=61f4b389-8fdb-40ae-aab2-a16b28112fa7

http://revision.derby.ac.uk/Panopto/Pages/Viewer.aspx?id=61f4b389-8fdb-40ae-aab2-a16b28112fa7

Formative Video Feedback, Five Ways• Example 5 – “Parkinson”

o Contact media productions dept and arrange to use their studio for more formal discussion

o They will video what you want o Give feedback to an “interviewer”, who

has questions on the ground! o Upload to University intranet or YouTube o Students can review at their leisure



Parkinson

https://www.youtube.com/watch?v=Awep_2D0f1s


Viewing Impact


Issues in Global Business

International Entrepreneurship Hand in dates

Grades Impact


Note: there will be other things affecting grades!

Student Views on Video Feedback

• Anecdotally, virtually all positive


Feedback on the Feedback


• We conducted some interviews with students into the use of feedback via video, asking… o How did lecturers use video for feedback? o What did the students think about using

video for feedback? o What feed-forward effects did it have?

• Naturally, we used video to get feedback on the video feedback


www.derby.ac.uk/businesshttp://revision.derby.ac.uk/Panopto/Pages/Viewer.aspx?id=d4eac215-142c-40bb-a1be-31065c09b21b&tid=82e35719-6783-484e-acf2-4d39af4cdab7

http://revision.derby.ac.uk/Panopto/Pages/Viewer.aspx?id=d4eac215-142c-40bb-a1be-31065c09b21b&tid=82e35719-6783-484e-acf2-4d39af4cdab7



• Feedback from students captured using lecture capture system on an iPad

• Could also use video on iPad, but be careful of file sizes with Apple devices

• Very easily used – point and press start

• So, to prove how easy, we’d like to capture some feedback… o Following today, what are your

thoughts on using video for feedback?

Guo et al (2014)


• Recommendations for using video eg for feedback

So How Do You Get Started?


• The two fears you first need to conquer: o The technology

- Practice using the technology - Talk to your IT dept for 1-2-1 support

o Being self conscious - Students will hear your normal voice - Practice speaking to camera - Personality and enthusiasm count

• Then go for it…

Finally…


• Formative video feedback is typically longer, deeper and more nuanced than written feedback, improves student engagement, can be viewed multiple times and, importantly, is preferred.

• This can contribute to student learning by helping them “taste the soup” before the guests arrive!

• Enhanced formative feedback should inform a new vision for Enterprise Education as it helps students apply their learning, leading to increased self-efficacy and confidence to ‘cook’ in the future

Appendix – Full Videos• Open University

• The Good, The Bad and The Ugly

• Smith and Jones

• Parkinson


http://revision.derby.ac.uk/Panopto/Pages/Viewer.aspx?id=4fd598ae-d14b-469f-8acc-1cc93472f966

http://revision.derby.ac.uk/Panopto/Pages/Viewer.aspx?id=6511e522-697d-413c-a9c7-8e6f1dedb0a1

http://revision.derby.ac.uk/Panopto/Pages/Viewer.aspx?id=a17720df-a839-4a70-9b21-502d722fda69


http://revision.derby.ac.uk/Panopto/Pages/Viewer.aspx?id=4fd598ae-d14b-469f-8acc-1cc93472f966

http://revision.derby.ac.uk/Panopto/Pages/Viewer.aspx?id=6511e522-697d-413c-a9c7-8e6f1dedb0a1

http://revision.derby.ac.uk/Panopto/Pages/Viewer.aspx?id=a17720df-a839-4a70-9b21-502d722fda69


Hattie & Timperley (2007)


(with apologies to masterchef), a feast of formative video

Documents