very quick introduction to organization of ui software

Very Quick Introduction to Organization of UI Software

The basics of what goes into a user interface (mostly GUIs)

Tasks and components to implement them

2

The User Interface Typically want to think of “UI” as only

one component of an overall system The part that “deals with the user” Distinct from the “functional core”

(AKA the “application”)

3

Separation of UI from “Appl” Really good reasons to want separation

of UI from “application”

In general want “separation of concerns”

Modularity (good software design) Different expertise needed Don’t want to iterate the whole thing

4

Unfortunately this is typically very hard to do in practice More and more of interactive programs

are tightly coupled to UI In some cases everything is “in” the UI

Why? Need to structure around user concepts UI structure “sneaks into” application Tight coupling can offer benefits to user

(better feedback)

5

Separation of concerns is a central theme of UI org A continual challenge A continual tension and tradeoff

Real separation of UI from application is almost a lost cause

6

UI tasks So far have:

Clearly there is more structure

UI Appl

7

UI tasks Basic parts of UI

ApplInput

Output

Appl

Inter

UICore

8

UI tasks Basic flow

ApplInput

Output

Appl

Inter

UI

Core

9

Canonical code structure for an interactive program

Initialize();Repeat

Evt := Wait_For_Input();Dispatch_Input(Evt);If something_has_changed Then

Redraw_All();Until time_to_exit;

Find appropriate object to deliver the input to

Object decides what to do with it based on:

•What it is•What state it is in

Ask each object whose appareance might have changed to redraw itself (based on what it is and

what state it is in)

10

This is the basic “Event/Redraw Loop”

Initialize();Repeat

Evt := Wait_For_Input();Dispatch_Input(Evt);If something_has_changed Then

Redraw_All();Until time_to_exit;

Used in some form by almost all interactive systems

11

Need to use system infrastructure to implement

Layered system components (for GUI)

I/O Devices

Layered Drawing/Windows

ApplInput

Output

Appl

Inter

UI

Core

Win

dow

Sys

OS

Hard

ware

Toolkit

OSetc.

Input Abstraction

“Most of the Work”

12

Need to use system infrastructure to implement

Unfortunately market forces give us:

ApplInput

Output

Appl

Inter

UI

Core

Hard

ware

Win

dow

Sys

OS

Toolkit

OSetc.OS

13

Need to use system infrastructure to implement

But conceptually these are the right layers

ApplInput

Output

Appl

Inter

UI

Core

Win

dow

Sys

OS

Hard

ware

Toolkit

OSetc.

14

Look mostly at toolkit level Tasks remain:

ApplInput

Output

Appl

Inter

UICore

15

How do we connect these disparate parts into a working whole

Tempting to architect systems around these boxes A module for input, one for output,

one for application interface

Things like this have been tried • “Seeheim model”

Didn’t work real well

16

Architectures with “3 big boxes” don’t work well because...

Modern (“direct manipulation”) interfaces tend to be collections of quasi-independent agents Each “object of interest” is separate e.g. a button

• produces “button-like” output• acts on input in a “button-like” way• etc.

Each object does its tasks based on• What it is• What its current “state” is

• Context from prior interaction or application

17

Leads to object-based architecture Interactor objects

AKA components, controls, widgets Each object implements each aspect

In a way that reflects what it is Objects organized hierarchically

Normally reflecting spatial containment relationships

“Interactor trees”

18

Interactor Tree Operation

Interface represented by tree

Toolkit

Infra

structu

re

Ap

plica

tion

19

Interactor Tree Operation

Input

Toolkit

Infra

structu

re

Ap

plica

tion

20

Interactor Tree Operation

Application response (manipulate tree)

Toolkit

Infra

structu

re

Ap

plica

tion

21

Interactor Tree Operation

Objects update selves & declare damage

Toolkit

Infra

structu

re

Ap

plica

tion

22

Interactor Tree Operation

Redraw (top-down traversal)

Toolkit

Infra

structu

re

Ap

plica

tion

23

Interactor Tree Operation

Complete “input-redraw-cycle” again

Toolkit

Infra

structu

re

Ap

plica

tion

24

Challenge: separation of concerns Challenge is doing all this different stuff in a

single object without creating a hopelessly large and complicated beast

Three general approaches several models in each not mutually exclusive (can / should use multiple)

Composition Inheritance Aggregation

25

Composition Put together interactive objects at

larger scale than interactors

Container objects e.g., row and column layout objects

Containers can also add input & output behavior to things they contain

26

Composition (containers) Can also have more sophisticated

containers that change input/output

Composition approach separates concerns into different interactors

27

Approaches at the interactor level Inheritance

all concerns in one object inherit / override them separately works best with multiple inheritance example: draggable_icon

• inherit appearance from “icon”• output aspects only

• inherit behavior from “draggable”• input aspects only

28

Inheritance Don’t have multiple inheritance in most

languages (i.e. java) but can still (partially) take this approach

Inheritance tends to give tighter coupling between aspects together in the code, no boundaries

Inheritance is the most common approach

29

Another object level approach:Aggregation

Actually separate out different concerns into separate objects Treat collection as “the interactor”

Classic architecture: “model-view-controller” (MVC) from Smalltalk 80

Cntr

View

Model

30

Model-View-Controller Each is separate object

Controller

View

Model

Output

Input

31

Model-View-Controller Model

the “underlying” or “application” information we interact with

MVC takes approach of “editing” this information

Fits with direct manipulation interface paradigm (model is object user manipulates, but not representation)

32

Model-View-Controller Model

Simple examples• text editor: model is text string• slider: model is an integer

Model is “data only” no input or output aspects but may include behavior (semantics)

33

Model-View-Controller View

mechanism needed to map model data to rendition (view / display)

all output aspects here when model changes, view object is

informed view arranges to update screen

• Declare damage• Redraw when requested

34

Model-View-Controller Controller

listens to user input translates into changes to model all input aspects here

Controller almost always has to “talk to” view Why?

35

Model-View-Controller Controller almost always has to “talk

to” view need geometry of output to interpret input

(e.g., picking) need to do feedback!

As a result, VC tend to be very tightly coupled

36

In theory should be able to plug different views and controllers good property in practice VC tend to be written together

and be too tightly coupled

Typical modern view of MVC combine VC into one object M(VC)

Model-View-Controller

37

Tasks again in more detail Core functions (cross cutting)

Hierarchy management• Again: will be trees of objects

Geometry management• coordinate systems• bounds

Object status / information management• Visible, enabled, selected, …

38

Tasks again in more detail Output

Damage management• knowing what needs to be redrawn

Layout• establishing size and position of each object

(Re)drawing

39

Tasks again in more detail Input

Picking• Figuring out what objects are “under” a screen

point

Event dispatch, translation, handling• A lot of the work is in here• In a typical toolkit “handling” is often majority of

the code you write

40

Tasks again in more detail Application interface

(Not very well developed) Callback model Command objects

41