GenURC: Generation Platform for Personal and Context-Driven User Interfaces
15
Cloud platforms Lead to Open and Universal access for people with Disabilities and for All www.cloud4all.info GenURC Generation Platform for Personal and Context‐Driven User Interfaces Presentation at the W4A conference, 2013‐05‐13, Rio de Janeiro Gottfried Zimmermann, Stuttgart Media University Co‐authors: Bern Jordan, Parikshit Thakur & Yuvarajsinh Gohil Note: Textual descriptions for the figures in these slides are available as PDF comments at the beginning (top left corner) of the slide.
Presentation at the W4A conference, 2013-05-13, Rio de Janeiro. Gottfried Zimmermann, Stuttgart Media University Co-authors: Bern Jordan, Parikshit Thakur & Yuvarajsinh Gohil
TRANSCRIPT
Cloud platforms Lead to Open and Universal access for people with Disabilities and for All
www.cloud4all.info
GenURCGeneration Platform for Personal and
Context‐Driven User Interfaces
Presentation at the W4A conference, 2013‐05‐13, Rio de JaneiroGottfried Zimmermann, Stuttgart Media University
Note: Textual descriptions for the figures in these slides are available as PDF comments at the beginning (top left corner) of the slide.
SLIDE 2 ZIMMERMANN – GENURC PLATFORM
What is URC?
What are GPII and Cloud4all?
GenURC
GenURC sample user interfaces
Conclusion & Outlook
Overview
Universal Remote Console (URC)
A device or software application, such as– a tablet– a smart phone– TV + IR remote– car head unit– smart clothes– etc.
Vorführender
Präsentationsnotizen
Pictures shown: A tv with a complex and a simple remote control A windows smartphone An iPhone An Android tablet A signing avatar A microphone of the size of a pin head A car head unit A PDA with a tactile overlay A Braille notetaker
Universal Remote Console (URC)
… through which a user accesses and controls
… any electronic & IT product or service, such as– a TV, a DVR– a washing machine, a thermostat, an alarm clock– a telephone, a cell phone – a calendar or alarm management system– a copy machine, an elevator, a ticket vending machine– etc.
Vorführender
Präsentationsnotizen
Pictures shown: A TV A refridgerator A washing machine A microwave oven A calendar (symbolizing an electronic calendar service) An eletronic thermostat A light An oven An elevator
Resources(Labels,
help, etc.)
Target Application
Target
User Interface Implementation
Description
Networking Platform
Networking Platform
Client (URC)
Resource ServerCompany A
Internet
Resource ServerCompany B
User Interface Socket Mirror
User Interface Socket
Socket Elements
Vorführender
Präsentationsnotizen
Illustration of the core URC principles: A device or service to be remotely controlled is called "Target". The Target has a Target Application which exposes its user interface as a "user interface socket" with socket elements (variables, commands, notifications). These socket elements are not specific to any user (no language dependence, no specific modality, etc.) A URC client is a unit with which a user can remotely control a target. The Client has a "user interface socket mirror" which is connected to the Target's user interface socket through a networking platform (URC can use any suitable networking platform). In the user interface socket mirror, the values of the socket elements are synchronized with the corresponding values on the target's user interface socket. On the client, the user interface socket mirror, the socket elements have atomic resources (e.g. labels, help texts, icons). In the client, a concrete user interface is rendered as "user interface implementation description" on top of the user interface socket mirror. The atomic resources and the user interface implementation description are retrieved from resource servers on the Internet. These can be hosted by companies and other organizations.
SLIDE 7 ZIMMERMANN – GENURC PLATFORM
Global▫ Worldwide initiative
Inclusive▫ For everyone, no barriers
Public▫ Results will be available publicly, no‐royalty
Infrastructure▫ Assistive technologies for all, open‐source implementations as basis
▫ Support for developers and service providers
▫ Store user settings (preferences) locally or in the cloud
"We build streets, not cars"
What is GPII?
Raising the Floor International
GPII & Cloud4all
GPII
Cloud4all
Trace Center IT RERC
IDRC Fluid project
IDRC FLOE project
Vorführender
Präsentationsnotizen
GPII: Initiative Raising the Floor: Organization in Geneva, Switzerland GPII is part of Raising the Floor International. Projects contributing to GPII: Cloud4All, European research project Trace Center IT RERC IDRC Fluid project IDRC FLOE project
Cloud4All Partners
Vorführender
Präsentationsnotizen
„Cloud Platforms Lead to Open and Universal Access for People with Disabilities and for All“ Funding by European Commission (FP7) 24+3 project partners from Europe & North America Project lifetime: Nov. 2011 – Oct. 2015 Project Partners: Technosite Raising the Floow CERTH Philips Hochschule der Medien Fundación Vodafone Espana inclusive design Technische Universität Dresden FULL measure Fondazione Don Carlo Gnocchi Onlus bdigital SingularLogic IESE Business School Fraunhofer IAO LifeSTech astea SOLUTIONS enLogic textHELP Omnitor codefactory EMERGYA HÖFT & WESSEL Fundación ONCE stiftung digitale-chancen Microsoft Accessibility serotek mozilla
SLIDE 11 ZIMMERMANN – GENURC PLATFORM
Extend the Universal Remote Console (URC) platform* by
automatic user interface generation
* The URC platform supports pluggable user interfaces for the control of appliances and services (targets)
Pragmatic approach for personal user interfaces
Personal user interfaces
▫ Tailored to a person's preferences and needs
Context‐driven user interfaces
▫ Adapted to a specific device (controller)
▫ Adapted to a specific situation of use
GenURC Goals
Vorführender
Präsentationsnotizen
Personal device: Could change at runtime, e.g. screen orientation Use situation: Changes dynamically Brightness, noise, location, etc.
www.cloud4all.info
Natural language(s)
Output modalities
Input modalities
Audio description
Text size, zoom factor
Keyboard settings
Line spacing
Speech volume
Contrast settings
Groupingstructure
Captions
Widget set
Use of color& color coding
Level ofassistance
Level ofsimplification
Key shortcuts
TTSparameters
Brailleparameters
CONTENT
& SEM
ANTICS
STRU
CTURE
& GRA
MMAR
PRESEN
TATION
& IN
PUT EV
ENTS Gesture alphabet
Button size& distance
URC
Cloud4all
Vorführender
Präsentationsnotizen
This animated slide walks through the following stages: An arbitrary set of user interface adaptation aspects (e.g. text size, etc.). The background consists of three colors that blur into each other (with gradients). From top to bottom: yellow, orange, blue. On the left, the names of user interface layers are shown, corresponding to the three background colors. From top to bottom: Presentation & input events, structure & grammar, content & semantics. Now, the UI adaptation aspects are re-ordered to map onto the underlying UI layers. Two technologies are superimposed over the UI adaptation aspects and the layers: "URC" on the lower part (mapping to the layers "structure & grammar" and "content & semantics", and "Cloud4all" on the higher part (mapping to the layer "presentation & input events"). Other notes: Output modalities: visual text, visual symbols, speech, braille, sign language, +captions, +audio description (often requires authoring) Input modalities: typing, pointing, speech, gesture, location sensing Speech volume: dependent on ambient noise level Widget set: slider vs. input field, radio buttons vs. drop-down menu Level of assistance: Tips, help texts TTS parameters: Character vs. word echo, speech rate, speech character Braille parameters: 6-dot vs. 8-dot, contraction level Grouping: Patterns for navigation (one window drill down vs. all inline, etc.) URC: substituation & augmentation Cloud4all: Configuration & tweaking
SLIDE 14 ZIMMERMANN – GENURC PLATFORM
GenURC Adaptation Process
ResourceServer
UCHWeb
Browser
GenURC Server
Rich Grouping
Target Description,Socket Description,
Grouping Sheet,Resource Sheet
ProprietaryProtocol
Target
URC‐HTTP
HTML Container forRich Grouping
HTML Container forRich Grouping
Correctionsat Runtime
Static userpreferences
Dynamic userpreferences,
device & situationalparameters
Vorführender
Präsentationsnotizen
This diagram shows the user interface adaptation process as distributed chart: The GenURC server retrieves the following from the resource server: Target Description, Socket Description, Grouping Sheet, Resource Sheet. The GenURC server builds the "Rich Grouping", and stores it on the "Resource Server". The UCH retrieves an "HTML Container for Rich Grouping" from the "Resource Server". The UCH is connected to a Target (with communication in both directions) via a "Proprietary Protocol". A "Web Browser" retrieves the "HTML Container for Rich Grouping" from the UCH. It then connects to the UCH (with communication in both directions) via the "URC-HTTP" protocol. "Static user preferences" affect the information in the "Rich Grouping" coming from the "GenURC Server". "Dynamic user preferences, device & situational parameters" affect the user interface as rendered in the "Web Browser". A user may make "corrections at runtime", thus affecting the user interface as rendered in the "Web Browser".
SLIDE 17 ZIMMERMANN – GENURC PLATFORM
GenURC Sample User Interface (1)
Vorführender
Präsentationsnotizen
Three screenshots are shown, illustrating three different renditions of a thermostat control user interface on a Safari browser on an iPhone. 1. The screenshot on the left shows the following content: - Window title: "iPhone Grouping Client" - The address bar with "192.168.2.50:8088/UCH..." - The title "Main Group" and an "Edit" button - "Operating Mode" and a drop-down menu showing the selection "COOL" - "Fahrenheit/Celsius" and a toggle control with selection "F" - "Current Temperature" and value "86" - Desired Temperature and an input field with value "89" "Temperature Settings" with a right arrow to click for navigating to another screen. 2. The screenshot in the middle shows the following content: - Window title: "iPhone Grouping Client" - The address bar with "192.168.2.50:8088/UCH..." - The title "Home Page" and an "Edit" button - "Operating Mode" and a drop-down menu showing the selection "COOL" - "Fahrenheit/Celsius" and a toggle control with selection "F" - "Current Temperature" and value "86" - Desired Temperature and an input field with value "89" - "Temperature Settings" (indented) with an arrowing pointing down (indicating that its content has been unfolded beneath) - "Day: Monday" (indented) with an arrow pointing right (indicating that its content can be unfolded by clicking the arrow) - "Day: Tuesday" (indented) with an arrow pointing right (indicating that its content can be unfolded) - "Day: Wednesday" (indented) with an arrow pointing right (indicating that its content can be unfolded) - "Day: Thursday" (indented) with an arrow pointing right (indicating that its content can be unfolded) 3. The screenshot on the right shows the same content as the screenshot in the middle, but now the days are further expanded. For "Day: Monday", the following unfolded information is shown as part of a framed group: A clock icon, "Time: 08:00 AM", and a sun icon "Temperature Program", and an input field with value "85" A clock icon, "Time: 04:00 PM", and a sun icon "Temperature Program", and an input field with value "79"
SLIDE 19 ZIMMERMANN – GENURC PLATFORM
GenURC Sample User Interface (3)
Vorführender
Präsentationsnotizen
The three screenshots on this slide show the home screen of the thermostat user interface, each with a different adaptation regarding presentation or structure. On the left, the same screen is shown as on the previous slide. Text is shown in black on white. In the middle, the font size and contrast has changed – now bigger font and white on black. On the right, some widgets are swapped. Instead of a drop-down menu, a group of vertically arranged radio buttons is shown, with the following items: "HEAT", "COOL", "AUTO", "OFF". Instead of an iPhone switcher (toggle), a group of vertically arranged radio buttons is shown, with the following items: "Fahrenheit", "Celsius".
SLIDE 20 ZIMMERMANN – GENURC PLATFORM
URC + Automatic user interface generation = GenURC
GenURC + Cloud4all = User interface adaptations on all layers
