1 annotating digital documents l3. 2 3 ink annotations are different to text annotations ink stands...
TRANSCRIPT
![Page 1: 1 Annotating Digital Documents L3. 2 3 Ink Annotations are different to text annotations Ink stands out from the original It is free form – the annotator](https://reader031.vdocuments.mx/reader031/viewer/2022032803/56649e3b5503460f94b2d546/html5/thumbnails/1.jpg)
1
Annotating Digital Documents
L3
![Page 2: 1 Annotating Digital Documents L3. 2 3 Ink Annotations are different to text annotations Ink stands out from the original It is free form – the annotator](https://reader031.vdocuments.mx/reader031/viewer/2022032803/56649e3b5503460f94b2d546/html5/thumbnails/2.jpg)
2
![Page 3: 1 Annotating Digital Documents L3. 2 3 Ink Annotations are different to text annotations Ink stands out from the original It is free form – the annotator](https://reader031.vdocuments.mx/reader031/viewer/2022032803/56649e3b5503460f94b2d546/html5/thumbnails/3.jpg)
3
Ink Annotations are different to text annotations
• Ink stands out from the original• It is free form – the annotator can
– Emphasize – underline, highlight, asterisk– Question– Agree– Add side notes to explain, clarify
• The authorship remains with the original author
![Page 4: 1 Annotating Digital Documents L3. 2 3 Ink Annotations are different to text annotations Ink stands out from the original It is free form – the annotator](https://reader031.vdocuments.mx/reader031/viewer/2022032803/56649e3b5503460f94b2d546/html5/thumbnails/4.jpg)
4
Why do we annotate?
• Active reading– Adding annotations to a document helps the
reader to process the information• Communicate with others/self
– When developing the document collaboratively
– When evaluating the work (eg marking assignments)
![Page 5: 1 Annotating Digital Documents L3. 2 3 Ink Annotations are different to text annotations Ink stands out from the original It is free form – the annotator](https://reader031.vdocuments.mx/reader031/viewer/2022032803/56649e3b5503460f94b2d546/html5/thumbnails/5.jpg)
5
Expressiveness and usefulness
• Annotations are very individualistic• However they can generally be understood by
others.
• Interesting studies of students choosing old – already annotated books – and looking carefully at the style of annotations
• Also students understanding pre-annotated material more quickly than the original document
![Page 6: 1 Annotating Digital Documents L3. 2 3 Ink Annotations are different to text annotations Ink stands out from the original It is free form – the annotator](https://reader031.vdocuments.mx/reader031/viewer/2022032803/56649e3b5503460f94b2d546/html5/thumbnails/6.jpg)
6
Visual message
• Looking at an annotated document in an instant you understand the position of the annotator. Are they:
• Being critical?• Correcting the spelling and grammar?• Adding their own explanatory notes
![Page 7: 1 Annotating Digital Documents L3. 2 3 Ink Annotations are different to text annotations Ink stands out from the original It is free form – the annotator](https://reader031.vdocuments.mx/reader031/viewer/2022032803/56649e3b5503460f94b2d546/html5/thumbnails/7.jpg)
7
Example Software
• Penmarked– Assignment marking
• RCA– Code review
![Page 8: 1 Annotating Digital Documents L3. 2 3 Ink Annotations are different to text annotations Ink stands out from the original It is free form – the annotator](https://reader031.vdocuments.mx/reader031/viewer/2022032803/56649e3b5503460f94b2d546/html5/thumbnails/8.jpg)
8
Penmarked Motivation
• Electronic submission and marking of assignments is attractive to faculty and students alike
• Particularly programs assignments when a digital copy of the assignment is required for marking
• However providing meaningful feedback to students is more difficult
• And individualised feedback is important
![Page 9: 1 Annotating Digital Documents L3. 2 3 Ink Annotations are different to text annotations Ink stands out from the original It is free form – the annotator](https://reader031.vdocuments.mx/reader031/viewer/2022032803/56649e3b5503460f94b2d546/html5/thumbnails/9.jpg)
9
Design Brief
• Annotation of the script• Score recording• Work practice support
– Who wants to ‘waste time’ marking?
![Page 10: 1 Annotating Digital Documents L3. 2 3 Ink Annotations are different to text annotations Ink stands out from the original It is free form – the annotator](https://reader031.vdocuments.mx/reader031/viewer/2022032803/56649e3b5503460f94b2d546/html5/thumbnails/10.jpg)
10
Implementation
![Page 11: 1 Annotating Digital Documents L3. 2 3 Ink Annotations are different to text annotations Ink stands out from the original It is free form – the annotator](https://reader031.vdocuments.mx/reader031/viewer/2022032803/56649e3b5503460f94b2d546/html5/thumbnails/11.jpg)
11
Annotation Pane
• Ink anywhere• Erase• Multiple files
• A technical nightmare!– Two layers– Lots of Windows API calls
• We have more-or-less got pdf going
![Page 12: 1 Annotating Digital Documents L3. 2 3 Ink Annotations are different to text annotations Ink stands out from the original It is free form – the annotator](https://reader031.vdocuments.mx/reader031/viewer/2022032803/56649e3b5503460f94b2d546/html5/thumbnails/12.jpg)
12
Scoring Pane
• Marking rubric• Numeric scores• Entered
– in score box with pen and recognized using Microsoft recognition engine with numeric factoid feature also range checked
– Or through the keyboard
![Page 13: 1 Annotating Digital Documents L3. 2 3 Ink Annotations are different to text annotations Ink stands out from the original It is free form – the annotator](https://reader031.vdocuments.mx/reader031/viewer/2022032803/56649e3b5503460f94b2d546/html5/thumbnails/13.jpg)
13
Work Practices SupportPre-marking• Set up of marking rubric, directories, file type/name
filters• Collection of assignments from a dropbox or directory
structureDuring• All assignments listed in student list pane
– Can be marked as opened and complete• Automatic unpackaging of zip files• Direct link to student’s folder for executing programOn completion • Exporting of marks as xml file• Conversion of assignments to pdf files (includes mark
rubric and annotated files)• Email return of assignment to student
![Page 14: 1 Annotating Digital Documents L3. 2 3 Ink Annotations are different to text annotations Ink stands out from the original It is free form – the annotator](https://reader031.vdocuments.mx/reader031/viewer/2022032803/56649e3b5503460f94b2d546/html5/thumbnails/14.jpg)
14
Evaluation 1
• Iterative informal during development• Usability testing with think aloud
– Talking while marking a program is challenging– A few interaction problems
• pen/keyboard input• ‘do you want to save’ dialogue
– Recognition required correct formation of digits– Using the pen to interact with non-tablet programs
(the students assignments) is slow!• Focus group
– Student list and workflow support invaluable– Accurate writing required for score recognition
![Page 15: 1 Annotating Digital Documents L3. 2 3 Ink Annotations are different to text annotations Ink stands out from the original It is free form – the annotator](https://reader031.vdocuments.mx/reader031/viewer/2022032803/56649e3b5503460f94b2d546/html5/thumbnails/15.jpg)
Evaluation 2• One large programming class ~ 180 students• Six markers• Three assignments
– increasing and size and complexity• Three treatments
– Paper– ‘Database’– Penmarked
• Balanced treatment– Each student had one assignment marked by each process– Each marker used each process – Rotated so student – marker different for each assignment
![Page 16: 1 Annotating Digital Documents L3. 2 3 Ink Annotations are different to text annotations Ink stands out from the original It is free form – the annotator](https://reader031.vdocuments.mx/reader031/viewer/2022032803/56649e3b5503460f94b2d546/html5/thumbnails/16.jpg)
Marking Time
Assignment 1 Assignment 2 Assignment 30
5
10
15
20
25
30
PaperDatabasePenmarked
min
utes
![Page 17: 1 Annotating Digital Documents L3. 2 3 Ink Annotations are different to text annotations Ink stands out from the original It is free form – the annotator](https://reader031.vdocuments.mx/reader031/viewer/2022032803/56649e3b5503460f94b2d546/html5/thumbnails/17.jpg)
Number of Annotations
Assignment 1 Assignment 2 Assignment 30
100
200
300
400
500
600
PaperDatabasePenmarked
Tota
l num
ber
of a
nnot
atio
ns
![Page 18: 1 Annotating Digital Documents L3. 2 3 Ink Annotations are different to text annotations Ink stands out from the original It is free form – the annotator](https://reader031.vdocuments.mx/reader031/viewer/2022032803/56649e3b5503460f94b2d546/html5/thumbnails/18.jpg)
Student Satisfaction – process
Pa
pe
r
Da
tab
ase
Pe
nm
ark
ed
Pa
pe
r
Da
tab
ase
Pe
nm
ark
ed
I found the submission easy I found the return of work easy
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
DisagreeNeutralAgree
They did not like paper!
Voluntary survey 42 respondents
![Page 19: 1 Annotating Digital Documents L3. 2 3 Ink Annotations are different to text annotations Ink stands out from the original It is free form – the annotator](https://reader031.vdocuments.mx/reader031/viewer/2022032803/56649e3b5503460f94b2d546/html5/thumbnails/19.jpg)
Student opinion aid to learning
Paper Database Penmarked Paper Database PenmarkedI understood where I had lost or gained marks The feedback helped my learning
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Rarely
Some-times
Mostly
Correlation between number of comments and students’ opinion of aid to
learning.
(statistically significant)
![Page 20: 1 Annotating Digital Documents L3. 2 3 Ink Annotations are different to text annotations Ink stands out from the original It is free form – the annotator](https://reader031.vdocuments.mx/reader031/viewer/2022032803/56649e3b5503460f94b2d546/html5/thumbnails/20.jpg)
Overall Preference
Overall preference for Penmarked
Paper Database PenmarkedI prefer to get my assignments marked this way
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Not at allA bitCompletely
![Page 21: 1 Annotating Digital Documents L3. 2 3 Ink Annotations are different to text annotations Ink stands out from the original It is free form – the annotator](https://reader031.vdocuments.mx/reader031/viewer/2022032803/56649e3b5503460f94b2d546/html5/thumbnails/21.jpg)
What else we could do with digital ink
• IDEs
• Web
![Page 22: 1 Annotating Digital Documents L3. 2 3 Ink Annotations are different to text annotations Ink stands out from the original It is free form – the annotator](https://reader031.vdocuments.mx/reader031/viewer/2022032803/56649e3b5503460f94b2d546/html5/thumbnails/22.jpg)
22
RCA
• One of the difficulties with Penmarked is that the markers have to go to another application to compile and execute the program
• RCA is an annotation addin for Visual Studio
![Page 23: 1 Annotating Digital Documents L3. 2 3 Ink Annotations are different to text annotations Ink stands out from the original It is free form – the annotator](https://reader031.vdocuments.mx/reader031/viewer/2022032803/56649e3b5503460f94b2d546/html5/thumbnails/23.jpg)
23
RCA in Visual Studio
![Page 24: 1 Annotating Digital Documents L3. 2 3 Ink Annotations are different to text annotations Ink stands out from the original It is free form – the annotator](https://reader031.vdocuments.mx/reader031/viewer/2022032803/56649e3b5503460f94b2d546/html5/thumbnails/24.jpg)
24
Technically
• Transparent overlay– Another technical nightmare!
• Each annotation consists of:– A linker – line or circle– Annotation – group text or diagram– A severity indicator
![Page 25: 1 Annotating Digital Documents L3. 2 3 Ink Annotations are different to text annotations Ink stands out from the original It is free form – the annotator](https://reader031.vdocuments.mx/reader031/viewer/2022032803/56649e3b5503460f94b2d546/html5/thumbnails/25.jpg)
CodeAnnotator
• Same idea but trying to use Eclipse.• Added some navigation and character
recognition• Pretty much the same problem!
• No extensibility points from the code windows!
25
![Page 26: 1 Annotating Digital Documents L3. 2 3 Ink Annotations are different to text annotations Ink stands out from the original It is free form – the annotator](https://reader031.vdocuments.mx/reader031/viewer/2022032803/56649e3b5503460f94b2d546/html5/thumbnails/26.jpg)
CodeAnnotator UI
26
![Page 27: 1 Annotating Digital Documents L3. 2 3 Ink Annotations are different to text annotations Ink stands out from the original It is free form – the annotator](https://reader031.vdocuments.mx/reader031/viewer/2022032803/56649e3b5503460f94b2d546/html5/thumbnails/27.jpg)
27
Related Work• Paperless environments
– Sellen and Harper (2002)• Ink interaction
– Plimmer and Apperley (2003) • Freeform - UI sketching environment
– Jarrett and Su (2003) • Annotation
– Marshall (1997)– Shipman, Price et al (2003)– Wolfe (2000)
• Recognition– Tablet OS
• Marking software– Heinrich and Lawn (2004)
![Page 28: 1 Annotating Digital Documents L3. 2 3 Ink Annotations are different to text annotations Ink stands out from the original It is free form – the annotator](https://reader031.vdocuments.mx/reader031/viewer/2022032803/56649e3b5503460f94b2d546/html5/thumbnails/28.jpg)
Browser Annotation
• http://prj-hcha155.cs.auckland.ac.nz/DemoWebsite/annotation.aspx • Login and password ‘new3’ send bug reports
28
![Page 29: 1 Annotating Digital Documents L3. 2 3 Ink Annotations are different to text annotations Ink stands out from the original It is free form – the annotator](https://reader031.vdocuments.mx/reader031/viewer/2022032803/56649e3b5503460f94b2d546/html5/thumbnails/29.jpg)
Browser Annotation
• Goal – an annotation toolbar– Failed – can’t fire silverlight from the toolbar
• Revised goal – copy web page and then annotate – iFrame– Java script– Silverlight– ASP .Net – MySQL
29
![Page 30: 1 Annotating Digital Documents L3. 2 3 Ink Annotations are different to text annotations Ink stands out from the original It is free form – the annotator](https://reader031.vdocuments.mx/reader031/viewer/2022032803/56649e3b5503460f94b2d546/html5/thumbnails/30.jpg)
30
Discussion
• Why is it so difficult?– There is a fundamental digital divide between
text and images– Trying to combine the two and keep them
consistent is really difficult– Knowing what to do with ink when the
underlying document changes its problematic– Many standard windows do not easily support
non-text
![Page 31: 1 Annotating Digital Documents L3. 2 3 Ink Annotations are different to text annotations Ink stands out from the original It is free form – the annotator](https://reader031.vdocuments.mx/reader031/viewer/2022032803/56649e3b5503460f94b2d546/html5/thumbnails/31.jpg)
31
Future Work
• Visual studio 2010– Yeah it works!
• Annotation in web browsers …….
![Page 32: 1 Annotating Digital Documents L3. 2 3 Ink Annotations are different to text annotations Ink stands out from the original It is free form – the annotator](https://reader031.vdocuments.mx/reader031/viewer/2022032803/56649e3b5503460f94b2d546/html5/thumbnails/32.jpg)
32
Finally
• Going paperless requires– Providing informal inking – Reliable recognition (sometimes)
• and – Attending to work practices
which are more varied than we first imagine!