Model Driven Techniques for User Interface Generation

Model-Driven Techniques for User Interface GenerationJacob AdamsTopic PaperDepartment of Computer ScienceSouthern Illinois University EdwardsvilleModel-Driven DevelopmentWikipedia- software development methodology which focuses on creating models, or abstractions, more close to some particular domain concepts rather than computing (or algorithmic) conceptsModel-Driven DevelopmentBenefits

Model is typically more declarative

Model is typically described in terms related to the problem domain

Required less technical knowledge to createReasons for Model-Driven User InterfacesSame benefits as regular model-driven development

Creating user interfaces (UIs), is still a largely manual process

Creating UIs for several different platforms can lead to redundant work

My Interest in Model-Driven UI GenerationBasler Electric 2005-2008

Many products had hundreds of screens

Needed similar user interfaces for both desktop and embedded applications

Developed multiplatform, database-driven screen generatorResearchEvaluate techniques for model-driven UI generation from a software engineering standpoint.

Important CriteriaExtensibilityMaintainabilityEfficiencySimplicityPrevious WorkThe User Interface GeneratorAlan Heirich -1987Use declarative commands to describe system:Token command describe type of data to be enteredDescriptive phrasesActions that can be executedAcceptance tests-performs validation logicThe User Interface GeneratorCan also define dependencies between fieldsUI elements are limited to a small set of widgets (textbox, selection, button, menu, dialog box)Can also create command line version of the applicationWolff, Forbig, Dittmar, ReichartAllows an application to be developed in an evolutionary fashion

Provides quick prototypes

10Wolff, Forbig, Dittmar, ReichartOriginally requires a task model

Wolff, Forbig, Dittmar, ReichartTask model is turned into a dialog graph

Wolff, Forbig, Dittmar, ReichartDialog graph is used to create abstract UI (AUI)AUI is defined in XULAUI is originally just placeholder widgetsPlaceholders are replaced by real widgets

Wolff, Forbig, Dittmar, ReichartMaking updates

Changes must be propagated back through task model, dialog graph and AUI

SUPPLEGajos and Weld

Requires 3 ModelsInterface specification constraints (e.g. type, widget used) places on user interface elementsDevice model information and constraints for the particular device that the UI will be generated forUser model information, such as usage patterns about the intended user of the applicationSUPPLEAfter the data is collected, a pruning algorithm is performed to find the rendering of the UI the reduces the expected effort required to use the user interface

UI generation and rendering is performed at runtime and is performed dynamically.SUPPLE

User Interface Generated for Mouse/Pointer-Based InputSUPPLE

User Interface Generated for Touch-Based InputSUPPLE

Different User Interfaces Generated Based on Different Usage PatternsARNAULDGajos, Weld, and WobbrockUsed to generate effort estimations used by SUPPLEAsks user to choose between renderings

SUPPLE++Gajos, Weld, and Wobbrock

Another method of determining effort estimate for SUPPLE

Measures users motor abilities

In experiment with 11 participant with motor impairmentsUsers were 26% fasterMade 73% fewer errorsSUPPLE++http://www.youtube.com/watch?v=B63whNtp4qc&feature=player_embeddedFeuerstack, Blumendorf, Schwartze, AlbayrakRequires context model, task tree, domain model, abstract user interface, and dialog models

Provides a tool to generate layout statement from these models.

Layout statements can define containment, order, orientation and sizeFeuerstack, Blumendorf, Schwartze, Albayrak

Feuerstack, Blumendorf, Schwartze, AlbayrakContainmentOrderOrientationSize

Statement are given a scope (screen, set of elements, entire application)Models and statements are evaluated at runtime

KalvaldjianFully automated process of generating UI from model

Requires discourse model

Screens and state machine are created from discourse model

Discourse model contains set of actionsCommon types request, informing, question, answer, etc.

KalvaldjianATLAS transformation language converts actions and other information into an AUI

Model2Code transformation language converts AUI into concrete UI (CUI)

Widgets are selected based on type of information to be input or displayedStocq and VanderdoncktRequires domain model (relational database)Wizard style toolSelect list or datasheet style UISelect data sourceSelect data source for lookup widgetsChoose which widgets update dataChoose relative position of widgetsMake optional modifications to sizes

Stocq and VanderdoncktWidgets are generated based on their type

Widgets are sized based on their longest possible value

Tool can create both desktop and mobile UIs

MastermindStirewalt and RugaberRequires Presentation model information presented to userApplication model information and functions available to UIDialog model interactions between user and application, relationships to other models

Automatically creates UI from these modelsMastermindAn agent is created to handle each model

Creates event to act as connection pointsContain information about actions and callbacksSpecified in generic, model independent way

UI is generated if models and connections are valid.

Created two proof of concept applicationsSimple print and save widgetAir traffic controller applicationMahfoudhi, Abed, and AbidAlso generated UI completely from the models

RequiresStatic structural task model (SSTM) hierarchy of tasks and what they are supposed to do.Dynamic structural task model (DSTM) sequencing and synchronization of tasks in SSTMMahfoudhi, Abed, and AbidOperational model converts task models into component objectsContain state machine built from actions and hierarchy defined in SSTM and transitions defined in DSTM

Component objects are aggregated into larger objects.

Translated into information need for concrete UI: user models, local interface models, abstract interface models, and interface implementation models

Analysis of Previous WorkModels Required to Generate UISome techniques required only one model, others required several

The detail involved in the models also varied significantly

There are tradeoffs between having a simpler or more complex model

Model Detail TradeoffsSimple modelsEasier to develop models

Detailed ModelsEasier to develop the rest of the applicationInformation is more declarativePossible to require more work that traditional development techniques

Automatic Generation of UINo outside steps requiredRequires detailed model or application may become rigid and/or have poor qualityMay be difficult or impossible to model UI entirely

Manual changesAllows more flexibility and customizationCan make updates difficult

Making Updates and ChangesMaintainability is a primary aspect of software engineering

System needs to be flexible to change over time

Automatic UI generation allows for quicker changes

Making updates in multiple places defeats purpose of the using a modelDependency on ModelApplications with automatic generation are often tightly coupled to their modelsCan cause problems if model cannot handle future changedCan also make getting rid of model difficult

Tradeoff: tightly coupled vs. harder to update

Use of Abstract User InterfaceCan help reduce dependencies on model

Helps decouple generation process from concrete UI specifics

Allows easier changes of UI widgets

Provides easier creation of UI for different environments, including different form factors

40Support for Different Form FactorsSome approaches (Stocq and Vanderdonckt, User Interface Generator) produce UIs for a few platforms

Others, such as SUPPLE, support creation on an unlimited number of platforms

Several even allow different input mechanisms (keyboard, mouse, touch)

Increasingly important with rise of mobile computingSupport for Different Form FactorsUser may want different UIs for different platforms

Allowing customizations to UI can help solve this problem

Using sample usage patterns can also address this problemUsage PatternsContains information such as frequency and order of actions performed by a user.

Can make using application much more efficient.

Requires more work.

Can be generated dynamically and at runtime.

Runtime UI CreationAllows UI to change over time, which recompilation

Can change with user as usage requirements change

Can not easily be performed if manual steps are required

Can be confusing to have dramatic changes to the UI

Gajos, Weld, and Wobbrock showed that using a hybrid approach can be a good comprimisePossible Future EnhancementsLess manual involvement of the developer

Provide richer interfaces

Make more improvements from software engineering perspectiveConclusionConsiderable research has already been done.

However, UI generation is still a relatively new and expanding field

There is still significant improvements that can be made.