ui toolkits - github pagesuwdata.github.io/hcid520/17wi/lectures/01-uitoolkits.pdf · for...

UI Toolkits

HCID 520 User Interface Software & Technology

http://www.cryptonomicon.com/beginning.html

Xerox Alto 1973

Command Line (UNIX shell, DOS prompt) Interaction driven by system User prompted for input when needed Text-based input and output

Event-Driven Interfaces (GUIs) Interaction driven by user System waits for user action and responds Pointing & text input, graphical output More complex architecture & development

Evolution of User Interfaces

Component Model

Encapsulate interactive components (widgets) Component library (button, slider, container) Interface built as a hierarchy of components

Widgets drawn by underlying graphics library

Input event generation and dispatch Historically mouse & keyboard, now touch, …

Bounds management & damage/redraw Model geometry, redraw updated regions only

Component (Widget) Model

User Interface Components

Each component (widget) is an object with: A bounding box A method for drawing itself Drawn in the component’s coordinate system

Methods (callbacks) to process input events

public void paint(Graphics g) { g.fillRect(…); // interior g.drawString(…); // label g.drawRect(…); // outline }

User Interface ComponentsLabel TextArea

ButtonsEach component has its own 2D coordinate system.

Containment HierarchyWindow

Panel

TextArea Panel

Button Button

Label

Component LayoutWindow

Panel

TextArea Panel

Button Button

Label

Each container is responsible for positioning its contents.

Event-Driven Architecture

User input is modeled as events. Events are dispatched to components for processing.

Events


Hardware Events Mouse: move, button-down, button-up Keyboard: key-down, key-up

Events


Hardware Events Mouse: move, button-down, button-up Keyboard: key-down, key-up

Inferred Events Mouse: click, double-click, enter, exit, drag Keyboard: key-press Window: move, resize

Events

Events encapsulate the information needed for components to react to input or changes.

Event Type (mouse moved, key down, etc) Event Target (the input component) Timestamp (when did event occur) Modifiers (Ctrl, Shift, Alt, etc) Event Content: Mouse: x,y coordinates, button pressed, # clicks Keyboard: which key was pressed

Anatomy of an Event

Go to http://mhcid.washington.edu Open the JavaScript console, enter: var h = document.querySelector("header"); h.addEventListener("click", function(event) { console.log(event); });

Click the header region at the top of the page. Inspect the events in the JavaScript console.

Demo: Browser DOM Events

http://mhcid.washington.edu

Event Dispatch Loop

Event Queue List of input events

Component Invoked callback method Update application state Request repaint, if needed

Mouse moved (t0,x,y)

Event Loop (runs in dedicated thread)

Remove next event from queue Determine event type Dispatch to proper component(s) Invoke callbacks on components Continue, or wait until event arrives

Event Dispatch Loop

Event Dispatch Loop




Component Invoked callback method Update application state Request repaint, if needed


Window

Panel

TextArea Panel

Button Button

Label

Event Queue Mouse moved (t0,x,y) Mouse pressed (t1,x,y,1) Mouse dragged (t2,x,y,1) Key typed (t3, ‘F1’) …

Event Dispatch

Window

Panel

TextArea Panel

Button Button

Label

Event Queue Mouse moved (t0,x,y) Mouse pressed (t1,x,y,1) Mouse dragged (t2,x,y,1) Key typed (t3, ‘F1’) …

/* callback for TextArea */ function mouseMoved(evt) { // process mouse moved event }

Event Dispatch

UI Tools Discussion

Brief UI tool history, establish vocabulary in wide use today, and chart future trajectories.

At any point did they “miss the mark”?

Myers, Hudson & Pausch

Brief UI tool history, establish vocabulary in wide use today, and chart future trajectories.

At any point did they “miss the mark”? Web: subsuming many desktop applications Design vs. Dev: In practice often done by teams with varied skills (interaction design, graphic design, implementation) 3D: Mostly limited to games? BUT fabrication now likely to drive new 3D design tools. VR?

Myers, Hudson & Pausch

Not just for desktop GUI widgets & events Mobile applications (iOS, Android) Information visualization (d3.js, Protovis, Prefuse) Tangible user interfaces (Papier-Maché) Gestural user interfaces (Kinect API) Speech user interfaces (MS Speech API) Proxemic user interfaces (Proximity Toolkit)

Many exhibit similar tensions around expressiveness, efficiency and ease-of-use

Toolkits Abound…

Simplify & accelerate UI development Lower the threshold of specification difficulty Raise the ceiling of what can be expressed But hard to do both! Establish path of least resistance Makes common / successful designs easy Makes unusual / ineffective designs harder Can limit & shape what we conceive / design Enforce consistency of design Maintain familiar widgets and interactions

Toolkit Benefits & Shortcomings

E X E R C I S E 1 Widget Design

Hello

Default State

Hello

Default State

Data: styling, content, etc.

Hello

Default State

Data: styling, content, etc.

https://dribbble.com/shots/361683-Color-checkboxes

http://pixelmonarchy.com/resources/form-checkboxes-and-radio-buttons-sky-ui-kit-psd/ https://www.filterforge.com/forum/read.php?

FID=9&TID=10550 Origin Unknown.

Hello

Checked State

https://dribbble.com/shots/361683-Color-checkboxes

http://pixelmonarchy.com/resources/form-checkboxes-and-radio-buttons-sky-ui-kit-psd/

https://www.filterforge.com/forum/read.php?FID=9&TID=10550 Origin Unknown.

Disabled-UncheckedDisabled-CheckedFocusMaybe-Checked

https://www.google.com/design/spec/components/selection-controls.html

https://developer.mozilla.org/en-US/docs/Web/Guide/HTML/Forms/The_native_form_widgets

https://www.filterforge.com/forum/read.php?FID=9&TID=10550 Origin Unknown.

http://developer.att.com/technical-library/ui-elements/html5-checkboxes

http://www.telerik.com/forums/how-to-select-checkbox-as-indeterminate-father

State Diagram to the Rescue

Default

CheckedClick

ClickDisabled


Default

CheckedClick

Disabled

Maybe

Click

Click


Default

CheckedClick

Disabled

Maybe

Click

Click

Shift- Click

Your Turn! Submit

! In pairs, brainstorm the states + transitions for a submit button.

! What does the button look like in each state? ! What are the transitions between states? ! What actions trigger the transitions? ! How does a user know what to do?

! Regroup in ~10 mins to share ideas.

Your Turn!

! In pairs, brainstorm the states + transitions for a textbox.

! Hint: States+transitions for the textbox but also its contents (e.g., text selection).

! Regroup in ~10 mins to share ideas.

I am a textbox

Your Turn: Interactive Prototypes

http://uwdata.github.io/hcid520/17wi/ex1/

http://uwdata.github.io/hcid520/17wi/ex1/

ui toolkits - github pagesuwdata.github.io/hcid520/17wi/lectures/01-uitoolkits.pdf · for...

Documents