visusalisation of video, by ben kraal
Post on 19-Oct-2014
1.840 views
DESCRIPTION
We (Design Researchers at the People and Systems Lab, QUT) do a lot of work with video, collected in our usability lab and in the field, but we rarely show that video to other people. We’ve found that by turning video into visualisations it’s easier to compare different videos. When we have visualistions that “sit still” the patterns of use that people engage in become more apparent. In order to make the video “sit still” we use the common technique of coding behaviours and activities that we are interested in observing.In this presentation Dr Ben Kraal will show some of these visualisations and talk about how they work and how we’ve used them to develop insights into the activities we observed.In particular this presentation will show how we used these visualisations to understand the different levels of expertise shown by nurses doing compression bandaging to treat venous leg ulcers.TRANSCRIPT
Visualisations of Video.
Ben KraalPeople + Systems LabQUT
What I’m going to talk about today is how to turn video into data, make that data useful and get some insights from it.
[fade in] In other words, how to create visualisations of video.
Hi, I’m Ben.
@bjkraal
I work at thePeople + Systems Lab, QUT.
We do Design Research.
These are some of the projects that have been associated with the P+S Lab in the past. We’re focussed on learning about context, activity, culture and people.
context activity
culture people
And we use those as our basis to move our focus from objects to doing design research about systems and experiences.
early design stage
before design started finished artifact
design and development
This is an (overly simple) explanation of how design research fits into practice.
On the left, you can do design research before you’ve started design. On the right, you can do DR with a finished artefact. In the middle is practice. Practice is where you’ll find techniques like user research in the early stage of design or usability testing in the development phases.
Visualisations of Video.
What I’m going to talk about today is how to turn video into data, make that data useful and get some insights from it.
[fade in] In other words, how to create visualisations of video.
UX is off the desktopand in the world.
Designers will domore research.
Use Video tocollect data.
Video makes data. Lots of data. This is the tapes from two small studies. At P+S we will often use two cameras concurrently to capture two different views simultaneously. Depending on how we record the footage we either mix the two cameras picture-in-picture or we use the power of our analysis software to play the footage simultaneously.
Purely from a logistical point of view, we generate hundreds of gigabytes of video each year, pushing into terabytes, and that’s before we do anything with it.
Visualise your video.
The best way to deal with all this video is to to turn it into data and then create visualisations to describe that data.
More than you everwanted to know about
I’m going to use a case study to explain these ideas. The case study is about a project we started a few years ago. It’s about Leg Ulcers.
All about Leg Ulcers:* sores* venous insufficiency (varicose veins etc)* compression therapy* nurses* assessed by instrumenting healthy legs and testing whether achieves ideal pressure
Here’s a still from some of the footage I took.
Nurse on the floor. Patient on the chair. Patient has her left leg completely bandaged and the right is partially done.
Visualise your video.
Back to visualising your video.
Codeyourvideo.
Before you can visaulise your video, you need to code it.
Coding data is a practice that comes from the social sciences. A lot of the time they can code transcripts of interviews. Coding video is quite similar.
Tagyourvideo.
Another way to consider coding data is to think about tagging it.
Tag parts ofyourvideo.
Or, more fully, to think about tagging parts of it.
Turn video into data.
And this tagging, or coding, is what turns video into data.
We useNoldus Observer.
Noldus Observer looks like this.
Coding scheme in the centre.
The way it works is to allow you to establish a coding scheme and then as the parts of the video that are described in the coding scheme occur, the software makes it very easy to track.
But you don’t have to.
Now, we use Observer, but you don’t have to.
Visualisations make video “sit still”.
Represent the data in different ways.
And let you represent data in different ways.
00:00
01:00
02:00
03:00
04:00
05:00
06:00
07:00
08:00
09:00
10:00
11:00
12:00
13:00
14:00
15:00
16:00
17:00
18:00
19:00
20:00
21:00
22:00
23:00
24:00
25:00
materials
reacting
doing
planning
Eg, like time-series charts. Like this one that describes some bandaging activity.
Aggregate your data.
And then you can graph it, or slice and dice it in new and interesting ways.
00:00
01:00
02:00
03:00
04:00
05:00
06:00
07:00
08:00
09:00
10:00
11:00
12:00
13:00
14:00
15:00
16:00
17:00
18:00
19:00
20:00
21:00
22:00
23:00
24:00
25:00
materials
reacting
doing
planning
This chart describes 25 minutes of activity. So you can see the power already. 25 minutes of video on one slide that you can take in and see the shape of in 30 seconds.
Now, to explain this a little better, I’m going to make up a fictional example.
00:00:00 00:05:00
This shows 10 minutes of (fictional!) bandaging activity.
But that’s not really very exciting.
stocking
00:00:00 00:05:00
stockingtype 2 bandageundercast
It’s much more interesting if we can describe the activity in more detail.
Eg: which bandaging materials are being used and for how long.
We’ll assign colours to different bandage types. Yellow can be a thin cotton stocking. Orange can be undercast. Purple can be a type 2 bandage. And then another thin cotton stocking to hold it all together.
[these should fade in.]
00:00:00 00:05:00
stocking stockingtype 2 bandageundercast
If you wanted, you could code for other behaviours, or activities, too. You might code for taping bandages down. [fade in]
Or for using scissors to cut bandage to length. (stocking comes in a roll, so you cut it before you use it.)
Or tearing it to length, as you can do with undercast.
Visualisations let you make comparisons.
Compare tasks across observations.
Orwithin observations.
Combinelevels of detail.
Ask questions about activity and task relationships.
Visualisations add dimensionto analysis.
Visualisations add dimension to analysis.
And let you add dimensions to analysis.
00:00:00 00:05:00
stocking stockingtype 2 bandageundercast
Which brings us back to here.
We didn’t find that knowing where bandages were fixed, taped or torn was useful, so we stopped looking for it. [fade out]
And we even found that looking only at the bandaging activity wasn’t that useful (because the way that bandages are used is proscribed, so we weren’t going to learn about non-standard practices, nor did we want to), so we took a longer view.
00:00:00 00:10:00
s st2uc
doing
preparing
[This should start with just the vertical bars and the 00.00.00. Zoom the 10.00 in.
Taking this back to a wider view, of ten minutes, and bringing the same bandaging activity [click] in I can introduce some other codes, for activity at a higher level than bandages. First I’m introduce a code that encompasses all the bandaging activity: doing [fade in].
Which leaves all this time before the bandaging took place. Now, in the video of the nurse she spends quite a lot of time before doing the bandaging preparing the work space. So we can call that preparing.
Everyone with me?
00:00
01:00
02:00
03:00
04:00
05:00
06:00
07:00
08:00
09:00
10:00
11:00
12:00
13:00
14:00
15:00
16:00
17:00
18:00
19:00
20:00
21:00
22:00
23:00
24:00
25:00
materials
reacting
doing
planning
Now, you’ve seen this before, but now I trust you can follow it.
[talk through chart]
Now, there’s something on here that I haven’t explained yet, and that’s the code that’s shown in pink about 3/4 the way through. And right now it doesn’t matter, and I’ll get to it later.
Expertise.
The chart I’ve just shown you was of a nurse who demonstrated high expertise in executing the application of the bandages.
Less Expertise.
Now let me show you one with slightly less expertise.
00:00:00 00:10:00
st2uc
doing
p
s
d d
preparingp
[fade in from l-r and t-b] Preparing, doing bandaging, preparing next two, then doing. Then preparing last one, then doing.
We say this demonstrates less expertise because pattern of preparing and doing is broken up. There is not a demonstration of the nurse being able to fully imagine the end-to-end bandaging process and so prepare everything before executing it. Instead, we see a partial sort of preparation and doing.
00:00
01:00
02:00
03:00
04:00
05:00
06:00
07:00
08:00
09:00
10:00
11:00
12:00
13:00
14:00
15:00
16:00
17:00
18:00
19:00
20:00
21:00
22:00
23:00
24:00
25:00
materials
reacting
doing
planning
Now, what I showed hypothetically before is seen in this actual chart. Let me zoom it in.
00:00
01:00
02:00
03:00
04:00
05:00
06:00
07:00
08:00
09:00
10:00
11:00
12:00
13:00
14:00
15:00
16:00
17:00
18:00
19:00
20:00
21:00
22:00
23:00
24:00
25:00
materials
reacting
doing
planning
Now, this is a different nurse and a different set of bandaging materials. As you can see, it’s basically the same pattern with four different types of compression therapy working together. And, during the bandaging you can see that the nurse takes some brief pauses in the middle to gather the next materials she needs.
Now, because she only takes breaks after completing each bandage, we say that she demonstrates experience because she knows the individual component tasks, but she doesn’t pull them together as fluidly as the first nurse.
Less Expertise.
So that was a demonstration of lower expertise.
Inexperience.
Now, what inexperience looks like. I’ll show you a real chart and not to a hypothetical.
00:00
01:00
02:00
03:00
04:00
05:00
06:00
07:00
08:00
09:00
10:00
11:00
12:00
13:00
14:00
15:00
16:00
17:00
18:00
19:00
20:00
21:00
22:00
23:00
24:00
25:00
materials
reacting
doing
planning
First, in this chart, there’s no initial preparation because the activity started before I turned on the camera. ;) But in the video it’s possible to see that the nurse has prepared most of what she needs on a small tray.
She begins and then has a break in the “doing” with no corresponding turn to “preparing” -- she’s asking another nurse for help. And that’s what the pink is, not “asking for help” but “breaking away” or shifting focus from the completion of the bandaging activity to some other activity. Then she resumes the activity and there is a large break at about the 5 minute mark. She’s demonstrating the bandage shown in green to the patient. Then she begins but soon after has a large focus shift *while doing* the bandaging. She’s winding it up the patient’s leg, deciding that it’s not right and winding it back down again.
Then at the end she has to stop and gather the final material before applying it.
00:00
01:00
02:00
03:00
04:00
05:00
06:00
07:00
08:00
09:00
10:00
11:00
12:00
13:00
14:00
15:00
16:00
17:00
18:00
19:00
20:00
21:00
22:00
23:00
24:00
25:00
materials
reacting
doing
planning
00:00
01:00
02:00
03:00
04:00
05:00
06:00
07:00
08:00
09:00
10:00
11:00
12:00
13:00
14:00
15:00
16:00
17:00
18:00
19:00
20:00
21:00
22:00
23:00
24:00
25:00
materials
reacting
doing
planning
00:00
01:00
02:00
03:00
04:00
05:00
06:00
07:00
08:00
09:00
10:00
11:00
12:00
13:00
14:00
15:00
16:00
17:00
18:00
19:00
20:00
21:00
22:00
23:00
24:00
25:00
materials
reacting
doing
planning
And to go back, here’s all three charts at once. The nice regular chart of the nurse who demonstrated high expertise, the slightly less regular and more “busy” chart of the demonstration of slightly less expertise and the chart of the inexperienced nurse.
I should say that we did do more analysis than just these three observations. I’ve shown these three because they illustrate the story.
Also, the signifigance and innovation of this approach needs to be underscored. Creating these visualisations is not especially hard, but using them in this way for analysis is new and adding the extra dimensions to describe activities at a higher level while maintaining the lower-level descriptions is where the power lies.
Sowe did this research.Now what?
From doing the research we learned three things.
Expertise is visiblein the organisationof activity.
First, we learned that expertise is visible in the organisation of activity.
This was something that the expert nurses could see as well, but they had no way to describe it. They just “knew” if someone was good. Maybe they saw something else, too, but our new heuristic is very useful, not least because it means that there is a shortcut to seeing bandaging expertise in the field, rather than in a lab.
Changethe training?the assessment?
The second thing is we were able to ask whether the nurses should change their training or assessment for compression therapy.
Develop a new treatment?
And finally, we were able to start asking questions about how it would be possible to develop new treatments that were, for example, less labour-intensive, while still taking into account all the really valuable knowledge that the nurses have about what they do.
Investigating these existing practices in order to develop something new isn’t something we’ve completely pursued, so I can’t describe what the new product or system would look like.
Design research with video is hard.
To bring this to a close,
Design research with video is hard. It takes a long time and you end up with a lot of unwieldy data.
Create visualisations to work with video.
One way to work with that data is to create visualisations of the video in order to turn it into something that’s easier to work with.
Creating the visualistions takes time, but if you’re going to really work the data and look for something new, taking that time is worth it.
When doing design research, particularly of this type which is of an existing product or situation but with a view to develop something new, this kind of in-depth analysis is valuable and can lead to significant insights. But you can’t have the insights without the research and analysis.
00:00
01:00
02:00
03:00
04:00
05:00
06:00
07:00
08:00
09:00
10:00
11:00
12:00
13:00
14:00
15:00
16:00
17:00
18:00
19:00
20:00
21:00
22:00
23:00
24:00
25:00
materials
reacting
doing
planning
Ben KraalPeople + Systems Lab, QUT@bjkraal
Understand and analyse interaction through visualisation.
This is the one thing I want you to take away.
You can understand and analyse interaction through visualisation of activities.
Further ReadingPopovic, Vesna and Kraal, Ben J. (2008) Focus-Shift: Interaction and Expertise Level.In: Design Research Society 2008, 16-19 July 2008, Sheffield, UK.http://eprints.qut.edu.au/14673/
Kraal, Ben J. and Popovic, Vesna (2007) Looking for Expertise in Physical Interactions.In: OZCHI 2007 - Australasian Computer-Human Interaction Conference, November 28-30, Adelaide, Australia.http://eprints.qut.edu.au/11083/