The Design of a GUI Paradigm based onTablets, Two-hands, and Transparency

Gordon Kurtenbach, George Fitzmaurice, Thomas Baudel, and Bill Buxton

Alias | Wavefront110 Richmond Street EastToronto, Canada, M5C 1P1

<gordo, gf, tbaudel, buxton>@aw.sgi.com1+ 416 362-9181

ABSTRACT

An experimentalGUI paradigmis presentedwhich is basedon the design goals of maximizing the amount of screenused for application data, reducing the amount that the UIdiverts visual attentions from the application data, andincreasingthequality of input. In pursuitof thesegoals,weintegratedthenon-standardUI technologiesof multi-sensortablets, toolglass, transparent UI components, and markingmenus. We describe a working prototype of our new para-digm, the rationale behind it and our experiences introduc-ing it into anexistingapplication.Finally, wepresentssomeof the lessons learned: prototypes are useful to break thebarriers imposed by conventional GUI design and some oftheir ideas can still be retrofitted seamlessly into products.Furthermore, the added functionality is not measured onlyin terms of user performance, but also by the quality ofinteraction, which al lows artists to create new graphicvocabularies and graphic styles.

KEYWORDS: two-handed input, toolglass, tablets, trans-parency, marking menus, task integration, divided attention

INTRODUCTION

Thebasiccomponentsof aGUI reflectthecharacteristicsorsubtasks of a user’s workflow. For example, in the drawingdomain, original interfaces like MacPaint have UI compo-nentslikeatool paletteandscrollabledrawing surface.Thisroughly reflects the way an artist would work with pencilsandpaper. Theusermovesbetweenthepaletteanddrawingsurface,drawing andchangingtheir focusof attention(nav-igating)to differentportionsof thedrawing. Sinceselectionfrom the tool palette, drawing and navigation are frequenttasks, GUI designers make these functions readily accessi-ble generally by constantly displaying the UI widgets forthese functions.

While thisdesignapproachhasbeenverysuccessfulit doescreate some design tensions. First, it introduces a competi-tion for screenspacebetweentheUI widgetsandtheuser’sart work (Figure 1). Second, i t produces a dichotomy

betweenUI widgetsandtheartwork wherea largemajorityof theUI widgetsexist aroundtheedgeof theartwork. Thefirst design tension could be addressed by a larger screenwith the cost being the expense of a larger display. How-ever, as screen size increases the second design tensionbecomes a problem. As the screen and artwork becomelarger the distance a user must travel to/from a tool paletteor menu increases. This results in longer task times. Fur-thermore,auser'sfocusof attentionmustconstantlychangefrom some point on the artwork to a UI widget at the edgeof the screen and then refocus on the artwork again. Divid-ing attention in this manner requires additional time toreacquire the context and can also result in users missingsome message from the system or some change in the art-work performedby thesystem.Webelieve thatthis dividedattentionproblemsignificantlyaffectsthequalityof auser'sinteraction1.

In additionto thesedesigngoalswe alsowantedto addressthe issue of the quality of input in a traditional GUI. Origi-nal GUIs such as the Xerox Star and the Macintoshassumed the mouse and keyboard to be the basic inputdevices.A hugeamountof thepowerof thetraditionalGUIcomes from the fact that the mouse allows continuous 2dimensional input from one of the user’s hands. We wereinterested in how providing continuous input for the otherhand would improve or affect the design of a GUI.

In this paper we describe an experimental GUI whichattempts to address these issues. We designed a GUI para-digm (model of interaction) with the fol lowing designgoals:

Artwork: Maximize the amount of screen used for artwork

Focus: Avoid forcingtheuserto divert theirvisualattentionfrom the artwork

Input: Increasethedegreesof manipulationandcomfortofinput.

These goals have driven much of the recent research in theareas of two-handed input [3, 8, 9, 11], toolglass[2], trans-

1. Thisproblemis notspecificto artwork andin generalappliestoany sortof applicationdata.Weusethetermartwork in thispapersince our application domain is artists and artwork.

parency[7] , marking menus[10], graspable UI [5] andmulti-sensor tablets. The work described in this paper is afirst attemptto integratethis researchinto single(albeitpro-totype) application.

For theremainderof thispaperwedescribeandanalyzeourprototype GUI called T3. This name is derived from thefact that the three major technologies used in our systemstart with the letter "T" (tablets, two-hands, and transpar-ency) andthuswe referto our systemasT3. We provide anoverview of the prototype’s application functionality anddiscusshow thecombinationof thetechnologiesanddesignconcepts contribute to our three design goals. We concludethe paper by describing our experiences introducing theparadigminto a full-featuredprofessionalgraphicsapplica-tion.

THE APPLICATION

T3 allowssimple2D graphicsto becreatedandeditedsuchas circles, rectangles, triangles and polyl ines. Figure 2shows thescreenof T3 andsomesimplegraphics.Ourgoalwas not to produce a full featured drawing program butrather to provide enough functions to “outline” the experi-mental paradigm. Many standard GUI features such asobject handles and a multiple selections mechanism wereleft out. There were two rationales for leaving out someobvious features. For some features, like selection handles,wesaw noproblemusingcurrentGUI paradigms.For otherfeatures, however, it wasn’ t clear how to fit them into theparadigm.Thesewereleft for futureresearch.Theseissueswi l l be further discussed in the fol lowing sections.

INPUT DEVICES

Our prototype uses Wacom digitizing tablets and custom-ized input devices to satisfy our design goals while at thesame time respecting practical i ties for the end user (interms of cost, accuracy, and avoiding intrusive/immersivesolutions). The use of tablets has many design benefi tsincluding: minimizes the onset of fatigue by allowing handand arms to operate and rest on the desk surface, minimaldevice interference from working environment, familiarand robust technology which is capable of small, wireless,batteryless devices, and finally the abil ity to sense morethan one sensor on a tablet. These properties will be dis-cussed in the following sections.

One Button Rotation-Sensi tive Pucks and Tablets. Wechoseto usetwo customizedWacompuckdevices(oneperhand) for our input devices which contributes to our Inputdesign goal. The input devices used are shown in Figure 3.The pucks used on both tablets sense both x and y position

and rotation. While Figure 3 shows two tablets, ideally, wewouldhave likedto useasingletabletbut thiswasnot tech-nically possible without sacrificing sensing rotation.

Although the pucks we used for T3 have four buttons oneach we decided our paradigm would be based on a singlebuttonon eachpuck.We choosethis simplificationbecauseit makes explaining and learning the fundamental mappingof input device statesto functionsimple.Usingtwo buttonsresults in only four different possible combinations of but-ton state.Theintentionwasthata limited numberof buttonstates would allow a user to quickly try all combinationswhenexploring theinterface.If wehadusedtwo buttonsoneachpuckthis would have resultedin sixteenstates.In thiscase,learningandtrying all combinationsof buttonswouldtake a long time.

As mentioned previously, T3's pucks also sense rotation.We chose to sense this degree of freedom for several rea-sons.First, two dimensionalrotationof artwork andobjectsin the artwork was a very desirable function. Second, thehand grasp required to hold a puck affords rotation.

Another motivation for choosing one button per hand wasthatwewantedit to bepossibleto substituteastylusfor the

Figure 1: A popular word processor with most toolbars turned on(this is the default configuration). Note that only nine lines of textcan be displayed on a 800x600 screen.

Figure 2: A screen shot of the prototype of the T3 paradigm. All com-mands are located in the toolglass which is show overlapping somesimple artwork. The cursor to the right of the toolglass. The screenresolution is 1280x1040.

Figure 3: The input devices in T3. Two tablets with two rotationsensing pucks. A pen shown on the right can also be usedinstead of a puck. Just above the pen is a device called a “flip-brick.”

dominant hand puck. In this situation, pressure on the sty-lus tip triggers the button press function. A stylus is espe-cial ly effective i f the artist is performing a freehanddrawing. Thus in T3, if the artist desires they can removethe dominant hand puck from the tablet and replace it withan stylus. Note that technical ly, a stylus can be built tosense rotation, but doing this was beyond the resources ofour project.

Fundamental Input Device Mappings For every interactionin the application we try to utilize two-handed techniqueswhich reflecthow analogoustasksareperformin theevery-dayworld. Therolesof thetwo handsreflecttheasymmet-ric dominant/non-dominant (D/ND) roles of our handscharacterizedby Guiard[6]. This will bediscussedin moredetail later. First we describe how the input devices controlthe interface.

Each puck has one button and this results in four binarystates which map to four general behaviors:

00: No buttons pressed: D puck moves cursor, ND puckmoves a toolglass palette.

01: D button pressed only: as in current GUIs the cursordrags objects or carries out the function of the current tool.

10: ND button pressed only: the artwork pans according tothe movement of the non-dominate puck.

11: Both buttonspressed:theartwork zoomsin/out accord-ing to the movement of the pucks relative to one another.The metaphor is stretching the artwork by pulling it apartby grabbingtwo locations.(zoomingout is compressingtheartwork by pushing the two locations together).

Sensing rotation allows addition functions simultaneouslyin each of these states:

00: No buttons pressed: The cursor and toolglass rotateaccording to the rotation of the pucks.

01: D button pressed only: object being dragged can besimultaneouslyrotateaboutthedragpoint.Thecurrenttoolcan utilize the rotation as a tool parameter.

10: ND button pressed only: rotating the ND puck rotatesthe artwork about the center of the drag point.

11: Both buttons pressed: puck rotation not used.

UI WIDGETS

In contrast to most traditional GUI designs, we have nostatically displayed user interface control widgets. Thismeans we do not have scrollbars or a menu bar. The netresult is we are able to maximize the display space for art-work instead of using the space for control widgets. Inreplacement,we have a single,mobiletool palettebasedonthe toolglass design [2].

ToolGlassAs mentioned previously, when the ND button is notpressedaToolGlasstrackstheND puck.In effectauserhasa toolglass "in their hand" when they are not orienting theartwork.

MovingtheToolGlass.Having theToolGlassfollow theNDpuckby in thismannercontributesto our Inputdesigngoal.

Because humans are very skil led at having the one handfollow or stay close by the other hand, this skill transfersvery effectively in T3. It is very easyto keeptheToolGlassalwayscloseto thecursor(which is beingcontrolledby theD puck). An artist does not have to constantly "pick-up,move, and put-down" the tool palette as required by tradi-tional floating tool palettes.

Having the ToolGlass constantly track the ND puck alsocontributes to our Artwork and Focus design goals. TheToolGlasscaneasilybemovedawaysoit doesnot interferewith the artist viewing the artwork, thus in effect maximiz-ing the artwork. Since the user can move the ToolGlasswithout having to look at it, focus on the artwork can alsobe maintained.

TransparencyOne of the key features of the toolglass paradigm is theabil i ty to "cl ick thru" toolglass button (for example, anobject’s color can be changed by clicking the cursor overtheredcolorbuttonwhenthecursoris alsoover theobject)."Clicking thru" requiresthe"click thru" typesof buttonsina ToolGlass to be transparent (since it is important to seewhat the click thru will be applied to). T3's ToolGlass istransparentfor this reason.However, transparency alsocon-tributes to the Artwork and Focus goals. Since the Tool-Glassis transparent,evenwhenit is over theartwork, someof the artwork under the toolglass is stil l visible. This inturn contributes to the Focus design goal since even if thetoolglass is covering the desired area of focus in the art-work theusercanstill maintaintheir focusbecausesomeofthe artwork is still visible beneath the ToolGlass.

Transparency is not only used in the Toolglass, but also forprototype shapes, and the click-hold cursor, described inthe next sections.

CREATING AND EDITING OBJECTS

Objectsarecreatedin T3 usinga two-handedtechniquewecall “ two handed stretchies” which works as follows. Theartist moves the cursor over the ToolGlass button for thedesiredobjecttype(for example,therectangleobjectin theToolGlass in Figure 4). The D button is then pressed. Thesystem immediately hides the ToolGlass. When the artistdrags the D puck a rectangle is swept out in the conven-tional way, from the corner. Since the ND hand is free (thetoolglass being hidden), we can use it in the transaction, aswell. Thus, moving the ND puck stretches the rectanglefrom the corner diagonally opposite to the corner which isattached to the D puck. In effect, the artist has a hold ofboth corners of the rectangle and this allows them to trans-late, rotate and scale all at the same time. When the D but-ton is released, the object is created and “dropped” on topof the artwork. Lines, circles and rectangles can be createdwith this technique.

Figure 4: Close up a toolglass palette and cursor in T3.

Two-handed Stretchies contribute to the Input design goalby providing simultaneous control of translation, scale androtation of an object. Tasks like positioning and scaling acircle to fit inside a box are substantially easier when con-trolling bothpropertiessimultaneously. Wehavealsofoundtheit allowsartistto exploredifferentplacements,sizesandrotations of objects more easily.

The two handed stretch interaction also supports our Art-work goal. First, hiding the ToolGlass while the artist isstretchingtheobjecthelpsdisplaymoreartwork andallowsthe artist to position an object without the ToolGlass inter-fering with theoverall appearanceof theart.Second,whenan object is being created (before the artist releases the Dbutton) the object is transparent (when the button isreleased the object is created in the current color). Like theuseof transparency in theToolGlass,transparentprototypescontribute to our Artwork goal.

MovingArtworkObjects. In T3 theD puckalsosensesrota-tion. This allows usto extendtheGUI conceptof dragging.Not only can objects be translated in two dimensions butthey can simultaneously be rotated. Furthermore, the pivotpoint of the rotation is defined by the point at which thedrag started. Because this mapping corresponds so closelyto everyday manipulation of objects, adding three additionparameters to dragging (rotation angle and x, y rotationpoint) is almostinstantlylearned.Webelieve thatno longerhaving to break these operations into discrete steps (move,specific pivot point, and rotate) contributes to our Inputgoalandreflectsthenotionof phrasingandchunkinginter-actions [4].

OurArtwork andFocusgoalsarealsocontributedto by thisdesign. In particular, rotation and setting the pivot pointrequire no graphical objects, so screen space for artwork isconserved and in turn focus on the artwork is maintainedsince the user does not have to go to a graphical widget ormenu item to invoked these functions.

COMMAND EXECUTION

In T3 we were concerned about supporting applicationswith many functions. For example, PowerAnimator byAlias | Wavefront,which is aprofessional3D computerani-mation package has approximately 400 commands.Roughlyspeaking,thismeansthatwewouldhaveto fill ourToolGlasswith 400elements.Clearlythis is notpossibleordesirable.Whatis desirableis someway of displayingonlyasmallsetof commandsbut allowing theuserquickaccessto the undisplayed commands.

We accomplish this by embedding a hierarchic markingmenu [10] in the top of the ToolGlass palette which allowstheuserto selectamonga setof possibletoolglass“sheets”(see Figure 5). Assuming our tool palette could comfort-ably contain 10 commands, a two level menu hierarchywith 8 items at each level (a total of 64 items in the menu)wouldallow accessto 64differentToolGlassesor 640com-mands.Clearly, this is in thecommandcountrangeof largeapplications like PowerAnimator. Finally, changing sheetscan be done quickly by using marking menus’ abil ity toselect using quick marks.

Figure 5 shows the marking menu used in T3. Moving thecursor over the “marking menu hotspot” in the ToolGlassand pressing the D button, causes the menu to pop-up. Themenu contains the other ToolGlass palettes available in T3(a total of 6). Changing ToolGlass palettes only requires a

quick fl ip in the direction of the desired toolglass. Cur-rently, the menu is not hierarchic, so only straight l inestrokes are needed to select different ToolGlass palettes.

This designcontributesto all threeof ourdesigngoals.TheArtwork goal is contributed to in several ways. First, usinga pop-up menu only temporarily consumes screen space.Also, there is even less impact on artwork if the user per-formsa selectionwith a markratherthanby displayingthemenu.The menu items are also transparent so the artworkcan be seen beneath them. Since the user does not have togo to the edge of the screen to change palettes Focus ismaintained.Finally, if theuseris familiarwith thelayoutofthe menu, they can quickly switch palettes by inputting amark. This contributes to our Input goal.

NAVIGATION: PANNING, ROTATING, AND ZOOMING

The ability to pan and rotate the artwork by pressing anddragging the ND puck is based on our two-handed inputdesign concept described in the introduction. Specifically,the ND positions and orients the artwork while the D handdraws.

This designcontributesto our Input goalandtherearefourissues driving the design. The first issue concerns quicktask performance. First, using conventional scrollbars andscrol l arrows can be extremely inefficient in that theyrequired the user to move back and forth between thescrollbar and the artwork. Second, orientation of the art-work affects the efficiency of movement. For example,Guiard reports that handwriting is 20% faster if the papercan be manipulated by the ND hand [6].

The second issue concerns comfortable movements. Whilere-orienting the artwork may sometimes have to do withmoving the working area to different (hidden) parts of theartwork, i t is also done for comfort reasons. We haveobservedusersmoving artwork closerto whatthey deemisa comfortable work area (e.g., towards the middle of thetablet as opposed to drawing in the upper corner of a tab-let).

The third issue concerns quality of movements. We haveobserved artists rotating the artwork so lines or curves canbe drawn with a movement that is easier to perform withthearm.For example,rotationfrom theelbow affordslargesmoothcurvesto bedrawn but theresultingcurvesarehor-izontal.To use the same technique to create vertical curvesrelative to the artwork, the artwork is rotated.

The final issue is that the ability to orient the artwork mustalways be immediately accessible. If the cost of re-orient-ing the artwork is greater than the cost of working in anuncomfortable position, artists will temporarily work in an

Figure 5: A marking menu to access different toolglass palettes canbe popped up from a hotspot at the top of every toolglass.

uncomfortable position. This is why we dedicated a buttonto orienting the artwork.

By providing a physicaldevice to controlpanningwe elim-inate the need for graphical scrollbars. This contributes tothe Artwork goal since standard scroll bars along the sideand bottom of the application window consume about 6%of thewindow'sspace.Furtherscreenrealestateis savedbynot requiring graphical gadgets for rotating and zoomingthe artwork.

The disadvantage is that these features are not self-reveal-ing. That is, there are no graphical elements that suggestand remind the user how to accomplishing scrolling, rotat-ing and zooming. In general, using graphical elements torevealfunctionalityto theuserhasbeenthebackboneof thesuccess of GUIs. However, our approach has been, ratherthan making T3 "walk-up-and-use" we assume that a newuser must be given a small amount of instruction to definethe "fundamentals" before beginning to operate the inter-face. The key observation here is that the "fundamentals"then do not have to be self-revealing and hence we candesign these interactions to contribute to our three designgoals.

Finally, having a physical device to control panning, rotat-ing andzoomingtheartwork contributesto our Focusgoal.The user does not have to divert attention from their art-work to a scroll baror othergraphicalwidgetto pan,rotateor zoom. Visual focus can be (and must be) maintained onthe artwork to control the operation.

CURVE GUIDES

The T3 prototype supports the notion of curve guides. Acurve guide is a tool that emulates the way a ruler, frenchcurve or frisket is used in traditional paper-based illustra-tions. That is, the curve guide is a “controlling element” or“dynamic constraint” that is mostly managed by the NDdevice and is used in conjunction with ink generation toolsbeing controlled by the D hand. This two-handed interac-tion technique facilitates the production of curves.

In T3 we have defined a set of french curves and custom-ized curves that can be used as a curve guide. Each curveresides on a toolglass sheet (see Figure 6) which can bepositioned and rotated with the ND device. A scale widgeton thetoolglasssheetallows theentiresheet(i.e., curve) tobe scaled. Note that all three affine transformations (posi-tion, rotationandscale)canbeperformedat thesametime.After the toolglass sheet has been positioned, the D deviceis usedto lay down ink by runningtheink cursoralongthe

contour of the curve. The inking cursor is automaticallysnapped to the contour of the curve.

This two-handed interaction technique touches all threedesign goals. First, the artwork is always visible since thecurve guide toolglass sheet is transparent. Secondly, theuser’s focuscanbemaintainedon theartwork sincethetooland artwork can be superimposed. The only diversionoccurs when the user must acquire the scale widget on thetoolglass sheet. Finally, Translate and rotate operations forthe sheet are always available through manipulation of theinput device.

MOVING TO THE REAL WORLD: STUDIOPAINT

EvaluatingT3 presentsa challenge.SinceT3 is a toy appli-cation, user testing under more realistic “ real work” condi-tionsis notmeaningful.However, it is importantto notethetypes of evaluation besides user testing that have alreadyoccurred and their value. First, prior to the construction ofT3 many of theindividual input techniquesusedin T3 havebeen empirically evaluated [3, 4, 5, 7, 8, 10, 11] and haveshown advantage. Second, artists participated in the designof T3, so user evaluation has been intrinsic in our designprocess(for example,theability to directly andfluidly pan/zoom/rotate the artwork is derived directly from artistrequests). Third, as UI designers we evaluated the para-digm. For example, can it handle a large number of func-tions? How much of the interface can be learned bydiscovery? Are the mapping of input devices to functionsconsistent, etc.?

User-testing under “ real work” conditions would be a criti-cal evaluation of a real application based on T3. However,building a real application from scratch with new technol-ogy is a huge task involving significant risk. We have cho-sen to minimize our risk by incremental ly adding T3features into an existing application and evaluating.

Hence our approach is incremental and iterative (i.e., T3prototype,portionsof T3 into anappropriateexistingappli-cation and eventually a whole application based on T3).What we describe in the remainder of this paper is theincrement from prototype to portions of T3 in a real prod-uct. It is also critical to note that the process of trying tointegrate portions of T3 into an existing application is initself a realistic and valuable design evaluation. While thisis not a replacement for user testing it is a valuable metricon theroadto usertesting,andis thesubjectof theremain-der of this paper.

The application chosen was StudioPaint, a high end paintsystemaimedat replacingpaperbasedillustrationin designstudios. StudioPaint suited our needs because the focus onquali ty of interaction is particularly important. Typicalusers have little or no training with conventional GUIs andare ready to switch back to paper if they feel the productdoes not suit their needs. StudioPaint also had some inter-esting features from an experimental point of view. Forinstance, it had been designed not to use any modal dia-logues.

However, integratingT3 functionalitiesinto anexistingandwidely used program involved compromises. From thetechnological point of view, some features could not beimplemented. For example, it was impossible StudioPaintto rotate the artwork in real time, so we had to drop thatfunctionality rather than providing lower quality interac-tion. We alsocouldn’t useour customtwo tabletconfigura-Figure 6: Curve guide on a toolglass sheet.

t i on. For tunatel y, most Studi oPai nt users havecommercial ly avai lable Wacom tablets. With these wecouldsensebotha puckandpenbut rotationwasnot avail-able.

Finally, thebiggestchallengeis to provide a smoothtransi-tion betweentheconventionalGUI thatusersalreadyknewand the new T3 paradigm. We had to preserve all the tradi-tionalwidgets,while allowing theuserto evolve towardtheT3 paradigmat theirown pace.To allow auserto maximizetheir artwork, UI widgets such as scribblers, menus barsand tool palettes could be removed from the display with asingle command selection, and the setup is saved acrosssessions.Figure7presentsthetypical setupmostartistsusewhen drawing with StudioPaint.

In hindsight, these limitations justified the need for our T3prototype. If we had tried to implement the paradigm inStudioPaintdirectly, wewouldhavemissedexploringsomeof the paradigm’s most powerful and interesting features.

StudioPaint interface controlsThe control portion of the interface (menus, palettes andscrollbars)hadto bereplacedby theirT3 equivalents.How-ever, we had to make sure we would provide enough func-tionality right from the beginning for users to accept andevolve towards the proposed workflow. For instance, theND hand was usually placed on the keyboard to issue hot-key commands. Requiring the same hand to control thepuck introduces a competition between the puck and key-boardfor theND hand.Thepuckwill win this competitiononly if the frequently used commands available from thekeyboard are also available from the puck. To accomplishthis, we had to make our first compromise: the ND handdevicewouldmakeuseof threebuttonsinsteadof one.Thisworked as follows:

• The left button is usedto invoke global commands:itacts as a modifier allowing the D hand to access a mark-ing menu that contains most of the functionality of themain menu bar. As described earlier, marking menus canbe accessed as rapidly as keyboard hotkeys.

• The right button functionslike theoriginal T3 ND but-ton: it allows navigation by panning the artwork with theND hand.This removedtheneedfor scrollbarsin Studio-Paint.

• Themiddlebuttonprovidesaccessto thetool palette.Acustomizable palette called “ the shelf” can hold various

objects, like brushes, colors, curve templates and clip-boards. This palette can be made semi transparent toreduce obtrusiveness. Pressing the third button makes itappearnearbythecurrentpositionof thepointer, sotheDhandcanrapidly “dip” in thepaletteto chooseitems(seeFigure 12).

An obviousalternative to additionalpuckbuttonswould bea toolglass sheet to replace keyboard commands. However,a combination of problems discouraged us from imple-menting toolglass sheets:

• It requireda complex rework of the userinterfacesoft-ware architecture.

• In a paint program,click-throughtools, the main fea-tures of the toolglasses are of l i ttle use: i t is ratherunlikely that an artist will use a “click-through brush” ora color by selecting it and then directly start drawing. Ingeneral, the artists need to rehearse their gestures beforeactually inking the drawing.

• Artists foundCurve Guidesmuchmoreinterestingthangeneric toolglasses.

StudioPaint curve guidesOur implementationof curveguidesin StudioPaint is muchmoresophisticatedthanin T3. StudioPaint curveguidesarecalled“sweeps”which is a termusedin designstudios.Thesweeps in StudioPaint can be created with the set of stan-dard drawing tools, similar to a MacDraw editor. The usercan create and editing shapes like rectangles, splines andellipses and then transform these shapes into sweeps. Likethe T3’s curves guides, a sweep becomes attached to theND hand and can be moved around on the artwork. How-ever, it cannot be rotated. To compensate for this sweepshave manipulationhandles(seefigure8). WhentheD handgrabs a handle of a “sweep” , two opposite corners of thebounding box for the sweep become attached to eitherhand, and the user can move, scale and rotate these shapeswith a “two-handed stretchies” style of interaction.

Snapping Sweeps. Like in T3, sweeps can be used to con-trol precisely the path of the ink while brushing freelyalongacurve.Whenusedwith varyingthicknessor opacitybrushes,thisallows theartistto giveamorelively characterto their drawing, while “snapping” to very precise outlines(see Figure 9). Finally, Sweeps can be stored on Studio-Paint’s shelf, allowing the user to build their own sets ofreference curves.

Masking Sweeps. Sweeps can also be used as a movingmask which artists commonly refer to as a “ frisket” . Artistreport seldom using a real airbrush without some sort ofmaskthatallows themto produce“sharpedge”effects.The

Puck Stylus

Keyboard

Figure 7: Typical setup for StudioPaint: the usual interactors:menus and palettes are present, but mostly to convey status infor-mation. Most of the workspace is used for the artwork.

Figure 8: A curve guide in StudioPaint. The four handles ineach corner allow the guide to be scaled and rotated. LikeT3, there is a hotspot for a marking menu (shown popped-up).

“ frisket” is usually held in the ND hand, and moved freelyto block thespraypaint from thepaper. This is usedto cre-atevariousgraphiceffects(seeFigure10).Otherpaintpro-grams, l ike Photoshop, usual ly provide these maskingfeatures,but becausethey make useof only onecontinuousinput device, they can’ t provide the seamless interactionavailable with paper-based tools. StudioPaint’s sweeps,however, begin to emulate the fluidity and spontaneity ofreal airbrushes.

CombinationSweepsFinally, thecombinationof bothsnap-ping and masking introduces novel graphical effects thatcannot be produced with paper based i l lustration, andwould have beenpreviously tediousto achieve with a paintprogram.As shown in Figure11,a hardedgecanbedrawneasily along a smooth predefined path, to produce a glow-ing effect. These would have required multiple maskingeffects, and cautious stroking if sweeps were not available.However, one simple stroke is required when using asweep.

Marking Menus. Like T3, sweeps have a marking menuembedded in a hot spot at the top center of the sweep’sbounding box (see Figure 8). While the marking menu canbe used to change to another shape of sweep (like T3), inaddition to this it is used for commands that apply to thecurrent sweep. For example, there are menu items to turnmaskingandsnappingon andoff, andto copy thecurrentlyselectedgeometryinto thesweep.In all, thereare12 menuitems that affect the current sweep.

An obvious design alternative would be to have the 12 but-tonsdisplayedonthesweepitself insteadof 12menuitems.However, therewerethreemajorreasonsfor not doingthis.First, addingbuttonscreatesscreenclutter. Second,theart-ist would have to be careful while inking along the sweepnot to accidentallyclick on a button.Finally, sincea sweepcanbeanarbitraryshape,it wascomplicatedto alwaysfinda good place to put the buttons.

Pragmatics of two-handed input in StudioPaintIn T3 we supported both left handed and right handed art-ists by simply having the user explicitly specify a prefer-ence. In StudioPaint we discovered that many times artistswork togetherat thesameworkstationtakingturnsworkingon the artwork. In this case, having to explicitly set thehandednesswasirritating andquickly fell into not utilizingthe ND puck.

To overcome this problem, we developed a method forautomatically detecting the handedness of a user and toinstantly reconfigure StudioPaint. Because we use a puckand a stylus, relative device positions can be detected andareassignedrespectively to thenon-dominantandthedom-inant hand. Then, we can infer the handedness of the user.We usethis informationto choosewhereto pop-uppalettesor which anchor points to use when doing “ two-handedstretchies” editing. Figure 12 shows an example.

In T3, we used a separate tablet for each hand. In Studio-Paint we used a single tablet and we encountered the prob-lem of the two hands (or the pen and puck) occasionallycolliding with oneanotherwhendrawing alongasweep.Tocure this problem, we offset the attachment of the puck tosweepsuchthattheperimeterof thesweepdoesnotoverlapthe footprint of the puck. Because we can automaticallydetecthandedness,we canintelligently offsetthepuck.Forexample,for aright handedpersonthesweepis offsetto theright and above the puck. For a left handed person thesweep is offset above and to the left.

Figure 9: Brushing along a curve guide. The spline above the car wasplaced along the upper edge of the car and used to trace along theguide, with repeated, varying width brush strokes.

Figure 10: A masking sweep. An airbrush was applied with thefrench curve masking the bumpers of the car. Note the “hard edge”effect produced by the mask.

Figure 11: Brushing along a curve guide with a mask set up:the resulting effect, that of a hard edge which follows exactly agiven contour, is very difficult to produce with a real airbrush.

StylusPuck

PointerPalette

Puck

Pointer Palette

Stylus

Right handed Left handed

Figure 12: The ”shelf” tool palette is popped nearby the current cursor location when the user depresses the middlepuck button. The palette disappears when the button isreleased. Note the implicit detection of the user’s preferredhand.

SUMMARY

Table 1 shows a summary of how the major features of theT3 paradigmcontributedto our threedesigngoalsof maxi-mizing the artwork, minimizing diversion of visual focuson the artwork and enhancing the quality of input. In addi-tion it shows how the features were realized in StudioPaint.

CONCLUSIONS

In generalmostfeaturesof T3 contributeto all threedesigngoals. We feel this is a result of the general approach ofreplacing graphical widgets wi th physi cal widgets(devices). This, in turn, provides more room for the art-work. Furthermore,if thechoiceof physicaldevicesis donecarefully, theusercanoperatethesedeviceswithout havingto look at them, thus allowing them to stay focused on theartwork. Finally, if thedevicessensemanipulationsthatwearevery skilled with, complex manipulations(like simulta-neous scaling, translation and rotation) can be performedthus contributing to the quality of input.

T3 is an interesting paradigm not because it provides newfunctionsto users(for example,theability to scroll andpanartwork is not a new function) but because it provides ahigher quality way of performing the functions. This isanalogous to the desktop paradigm which didn’ t providenew functions (for example, the abil i ty to organize fi leswasn’t anew function)but providedahigherqualitywayofperforming those functions. In this paper we have tried todescribe what we believe are the design principles contrib-ute to this notion of quality.

Our implementation of T3 into StudioPaint has shown usthat providing artists with new ways of interacting withapplication data (i.e., the sweeps) encourages them to cre-ate new graphic vocabularies and styles of illustration. In asense,by enhancingtheUI thefunctionalityof theapplica-tion becomes enriched.We have already observed this phe-nomenon in the “Ligne Claire” mark-based spline editingtechnique [1].

FUTURE RESEARCH

Work continues on the concepts surrounding T3. We aremainly focusing on applying these concepts to other appli-cationdomainslike3D modellingandcomputeranimation.

We are also beginning to gather usage experiences fromautomotive graphics artists using StudioPaint and its T3features.Currently, StudioPaintwith T3 featureshasnotyetbeenreleased.However, wearealreadycollectingreactionsfrom our beta users. Initial results are encouraging.

ACKNOWLEDGEMENTS

We grateful ly thank Azuma Murakami and Yasuhi roFukasaki at Wacom technology for supplying inspirationalideas, tablets, and the custom pucks.

REFERENCES

1. Baudel, T. A Mark-basedInteractionTechniqueforFree-Hand Drawi ng. Proceedi ngs of the ACMUIST’94 Conference on User Interface Software andTechnology, 185-192.

2. Bier, E. A., Stone,M. C., Fishkin, K., Buxton, W.,Baudel, T., A Taxonomy of See-Through Tools. Pro-ceedings of the ACM CHI’94 Conference on HumanFactors in Computing Systems,358-364.

3. Buxton, W., & Myers, B. A., (1986)A study in two-handed input. Proceedings of the ACM CHI’86 Con-ference on Human Factors in Computing Systems,321-326.

4. Buxton, W. (1986) Chunking and phrasingand thedesign of human-computer dialogues. InformationProcessing ‘86 Proceedings of the IFIP 10th WorldComputer Congress, 475-480.

5. Fitzmaurice,G. W., Ishii, H., Buxton,W., Bricks:Lay-ing Foundations for Graspable User Interfaces. Pro-ceedings of CHI’95, (Denver, CO), ACM Press, 442-449.

6. Guiard,Y. (1987).AsymmetricDivision of Labor inHuman Ski l led Bimanual Action: The KinematicChain as a Model. Journal of Motor Behavior, 19(4),486-517.

7. Harrison,B., Kurtenbach,G., Vicente,K. (1995) AnExperiment Evaluation of Transparent User InterfaceTools and Information Content. Proceedings of theACM UIST’95Conferenceon UserInterfaceSoftwareand Technology, 81-90, ACM.

8. Kabbash,P., Buxton,W., andSellen,A., (1994)Two-HandedInput in aCompoundTask.Proceedingsof theCHI’94 Conference on Human Factors in ComputingSystems, 417-423.

9. Kabbash,P., MacKenzie, I.S. & Buxton, W. (1993).Human performance using computer input devices inthe preferred and non-preferred hands. Proceedings ofACM InterCHI ‘93. Amsterdam, 474-481.

10. Kurtenbach,G., Buxton, W. (1993) The limits ofexpertperformanceusinghierarchicalmarkingmenus.Proceedingsof CHI ‘93 Conferenceon HumanFactorin Computing, 482-487.

11. Leganchuk,A., Zhai,S.,Buxton,W., BimanualDirectManipulation in Area Sweeping Tasks. Submitted forpublication.

TABLE 1.

T3 feature A F I StudioPaint

no peripheral UIwidgets

yes yes ability to hideshelf, tool ar andscrollbars

drag/rotate objects yes yes yes drag onlyND hand pans/rotates Art

yes yes yes panning only

marking menu tochange tool palette

yes yes yes on sweeps

D/ND handszooming artwork

yes yes yes no, performancelimitation

toolglass palettes yes yes sweepsresizable toolglass resizable sweeps1 button per hand yes 3 buttons on ND

handtwo handedstretches

yes yes yes not used

curve guides yes yes sweepsTools lock on cur-sor

yes already had toolmodes

Transparency yes yes used in sweeps