User Evaluation of SADIe

User Study of SADIe Transcoding EngineDarren Lunn, Sean Bechhofer and Simon Harper

1

SummarySADIe transcodes content based upon meaning of the visual rendering.User evaluation supports SADIe transcoding as a way to aid visually impaired users.Observations from the study also provide insights into further transformations.2

The WebFocus on presenting information visually.Images.Columns.Chunks.Some knowledge is only available implicitly from the page rendering.3

Implicit Knowledge

Advertisement

Banner

Menu

Main Content4

Assistive TechnologiesVisually impaired users can make use of assistive technologies e.g. screen readers.Render pages sequentially in audio.Achieved by accessing the underlying HTML.Focus on visual presentation rather than content hampers this.Particularly if attention is not paid to coherent design.Subtleties of visual presentation can be lost.5

Assistive TechnologiesTraversal of content is serial.Top-to-bottom.Left-to-right.Important information may not be encountered until later on.Some information such as menus can be repeated for every page.6

TranscodingRules / Patterns.Can adapt a large number of pages.Can suffer from reduced accuracy.Annotation.Accurate.Time consuming as every page is annotated.7

SADIe ApproachThe visual rendering of a Web element informs the user of its purpose.The CSS defines the visual rendering.Identifying the purpose of the CSS definition implicitly defines the purpose of the Web element.A single CSS definition is applied to every page within the Website.8

Mention what transformations we can do: Defluff, reorder, menu8

SADIe

BannerBannerTabsAdvertisement Main StoryImageHeadingMenuOverviewHeadingOverviewMain Story9

9

Is This Useful?Previously technically feasible of SADIe.Conducted user studies to support transcoding from user perspective.Identify future transcoding areas to investigate.10

Remote Or Lab-Based Study?Remote Studies.Long term.Potentially more users.Can see what users did.Lab-based study.More controlled.Glean information from users.Can understand why users performed actions.11

StimuliEight text-based Websites.News; Blogs; Search results.Taken from Alexa top 100.All pages transcoded in the same way.User presented with original or modified version.Tasks involved finding information on the page.12

UsersFour visually impaired usersFrequent computer users.Used the Web daily.All used JAWS screen reader.13

TasksUsers were presented with a page.They did not know which page.They did not know which version.Users were asked to complete a task.E.g. What is the first search result.Task completion time was calculated.14

User InteractionTask index page

Page 3

15

Time to Task Completion16

1) About; 2) BBC News; 3) Yahoo; 4) Google; 5) CNN; 6) Lady Xanga Blog; 7) New York Times; 8) Cultured Left Foot blog16

T-Test AnalysisMean TimesOriginal: 163.25SADIe: 54.19t(7.6) = -2.14p = 0.033As p < 0.05 statistically significantSupports the use of our transcoding17

Difference between the two means 109.0617

A Good Start PointEvaluation supports the use of SADIe transcoding.Defluff; Reorder; Menu.User discussions and observations highlight future areas of investigation.18

Importance of StructureParticipants frequently discussed the importance of page structure.If you told me these were the results with a heading that would help me.- P1 When you say heading level 4 or whatever, you need to describe what is under it. - P319

P3 used headings a lot and was frustrated when the heading didnt match the content underneath.19

Importance of StructureP3 found list structures much easier to navigate than paragraphs of text.Scope for not only reorganising page but restructuring elements.20

Importance of CuesBBC News page required users to find story headline.SADIe version took longer.P2 reached headline within 3 seconds.Did not know it was the headline as page had changed.Required cue of Africa before realising they had gone too far.21

On the age found a heading with Africa and knew they had gone too far.21

Importance of Cues

22

Importance of CuesP1 identified search results from the word cached.P3 knew they had gone too far from the word feedback.Can we make use of these cues to guide people around the page?23

User StrategiesBasic strategy is to use find but this only really works when you know the page. P1P2 quickly tabbed through the page to get an overview of the content.Can these strategies be utilised to help users?24

24

SummarySADIe transcodes content based upon meaning of the visual rendering.User evaluation supports SADIe transcoding as a way to aid visually impaired users.Observations from the study provide insights into further transformations.25

Questions?http://hcw.cs.manchester.ac.uk/darren.lunn@cs.manchester.ac.uk