one-dimensional handwriting: inputting letters and words...

12
One-Dimensional Handwriting: Inputting Letters and Words on Smart Glasses Chun Yu Ke Sun Mingyuan Zhong Xincheng Li Peijun Zhao Yuanchun Shi Key Laboratory of Pervasive Computing, Ministry of Education Tsinghua National Laboratory for Information Science and Technology Department of Computer Science and Technology, Tsinghua University, Beijing, 100084, China {chunyu, shiyc}@tsinghua.edu.cn, {k-sun14, zhongmy14, lixc12, zhaoym13}@mails.tsinghua.edu.cn ABSTRACT We present 1D Handwriting, a unistroke gesture technique enabling text entry on a one-dimensional interface. The challenge is to map two-dimensional handwriting to a reduced one-dimensional space, while achieving a balance between memorability and performance efficiency. After an iterative design, we finally derive a set of ambiguous two- length unistroke gestures, each mapping to 1-4 letters. To input words, we design a Bayesian algorithm that takes into account the probability of gestures and the language model. To input letters, we design a pause gesture allowing users to switch into letter selection mode seamlessly. Users studies show that 1D Handwriting significantly outperforms a selection-based technique (a variation of 1Line Keyboard) for both letter input (4.67 WPM vs. 4.20 WPM) and word input (9.72 WPM vs. 8.10 WPM). With extensive training, text entry rate can reach 19.6 WPM. Users’ subjective feedback indicates 1D Handwriting is easy to learn and efficient to use. Moreover, it has several potential applications for other one-dimensional constrained interfaces. Author Keywords Text entry; one-dimensional input; unistroke gestures. ACM Classification Keywords H5.2. Information interfaces and presentation: User inter- faces — Input devices and strategies. INTRODUCTION Nowadays, many new smart personal devices are emerging. Such devices typically have a constrained input interface due to the limited size of the form factor. Consequently, text entry is difficult on these devices, which prohibits their broader use. In this paper, we focus on one-dimensional text entry for devices that have only one-dimensional input signals, or the input capacity on one dimension is much greater than that on the other. Examples include the touchable spectacle frame of a smart glass (e.g. Google Glass), the edge of a side screen of a smart phone (e.g. GALAXY Note Edge), and a smart wristband. Researchers have proposed various techniques to enable text entry on constrained interfaces (e.g. a keypad, a joystick, a tracking ball, and air interaction etc.). These techniques include coding letters [16], unistroke gestures [2], ambiguous keyboards [18], and gesture keyboards [12, 31]. However, most of these techniques are for two- dimensional constrained interfaces or interfaces with a limited number of buttons or gestures. In this paper, we present 1D Handwriting, which enables users to perform unistroke gestures on a one-dimensional interface to input text. We research, implement and evaluate 1D Handwriting with Google Glass, a representative one- dimensional input device. The biggest challenge encountered is that many letters look similar, if not identical, after projected into a reduced one-dimensional space. Therefore, our goal is to derive a set of one- dimensional gestures that strikes a balance between memorability, input accuracy and input speed. After a careful and iterative design, our result is a set of ambiguous unistroke handwriting gestures based on sub- stroke of two-level lengths. To complete the input, we design a one-dimensional gesture recognition algorithm that classifies input strokes in a probabilistic way. For letter input, we design a smooth transition gesture allowing the user to first gesture a stroke (representing a group of letters) and then select the target letter (within the group) without lifting his/her finger. For word input, we use a Bayesian approach that takes both the probability of gesture and the language model into account to interpret users’ input. To evaluate the performance of 1D Handwriting, we conduct three lab experiments. Results show that 1D Handwriting provides immediate usability: With limited training, users can achieve 4.67 WPM for letter level input (error rate = 0.78%) and 9.72 WPM for word level input (error rate = 0.25%). These speeds are comparable to those proposed for two-dimensional constrained interfaces, and significantly outperforms the one-dimensional selection- based technique (the Baseline in our experiments) by 11.4% Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. CHI'16, May 07-12, 2016, San Jose, CA, USA © 2016 ACM. ISBN 978-1-4503-3362-7/16/05…$15.00 DOI: http://dx.doi.org/10.1145/2858036.2858542 How Fast Can You Type on Your Phone? #chi4good, CHI 2016, San Jose, CA, USA 71

Upload: others

Post on 24-Jun-2020

15 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: One-Dimensional Handwriting: Inputting Letters and Words ...pi.cs.tsinghua.edu.cn/lab/papers/p71-yu.pdf · One-Dimensional Handwriting: Inputting Letters and Words on Smart Glasses

One-Dimensional Handwriting: Inputting Letters and Words on Smart Glasses

Chun Yu Ke Sun Mingyuan Zhong Xincheng Li Peijun Zhao Yuanchun Shi Key Laboratory of Pervasive Computing, Ministry of Education

Tsinghua National Laboratory for Information Science and Technology Department of Computer Science and Technology, Tsinghua University, Beijing, 100084, China

{chunyu, shiyc}@tsinghua.edu.cn, {k-sun14, zhongmy14, lixc12, zhaoym13}@mails.tsinghua.edu.cn

ABSTRACT We present 1D Handwriting, a unistroke gesture technique enabling text entry on a one-dimensional interface. The challenge is to map two-dimensional handwriting to a reduced one-dimensional space, while achieving a balance between memorability and performance efficiency. After an iterative design, we finally derive a set of ambiguous two-length unistroke gestures, each mapping to 1-4 letters. To input words, we design a Bayesian algorithm that takes into account the probability of gestures and the language model. To input letters, we design a pause gesture allowing users to switch into letter selection mode seamlessly. Users studies show that 1D Handwriting significantly outperforms a selection-based technique (a variation of 1Line Keyboard) for both letter input (4.67 WPM vs. 4.20 WPM) and word input (9.72 WPM vs. 8.10 WPM). With extensive training, text entry rate can reach 19.6 WPM. Users’ subjective feedback indicates 1D Handwriting is easy to learn and efficient to use. Moreover, it has several potential applications for other one-dimensional constrained interfaces.

Author Keywords Text entry; one-dimensional input; unistroke gestures.

ACM Classification Keywords H5.2. Information interfaces and presentation: User inter-faces — Input devices and strategies.

INTRODUCTION Nowadays, many new smart personal devices are emerging. Such devices typically have a constrained input interface due to the limited size of the form factor. Consequently, text entry is difficult on these devices, which prohibits their broader use. In this paper, we focus on one-dimensional text entry for devices that have only one-dimensional input

signals, or the input capacity on one dimension is much greater than that on the other. Examples include the touchable spectacle frame of a smart glass (e.g. Google Glass), the edge of a side screen of a smart phone (e.g. GALAXY Note Edge), and a smart wristband.

Researchers have proposed various techniques to enable text entry on constrained interfaces (e.g. a keypad, a joystick, a tracking ball, and air interaction etc.). These techniques include coding letters [16], unistroke gestures [2], ambiguous keyboards [18], and gesture keyboards [12, 31]. However, most of these techniques are for two-dimensional constrained interfaces or interfaces with a limited number of buttons or gestures.

In this paper, we present 1D Handwriting, which enables users to perform unistroke gestures on a one-dimensional interface to input text. We research, implement and evaluate 1D Handwriting with Google Glass, a representative one-dimensional input device. The biggest challenge encountered is that many letters look similar, if not identical, after projected into a reduced one-dimensional space. Therefore, our goal is to derive a set of one-dimensional gestures that strikes a balance between memorability, input accuracy and input speed.

After a careful and iterative design, our result is a set of ambiguous unistroke handwriting gestures based on sub-stroke of two-level lengths. To complete the input, we design a one-dimensional gesture recognition algorithm that classifies input strokes in a probabilistic way. For letter input, we design a smooth transition gesture allowing the user to first gesture a stroke (representing a group of letters) and then select the target letter (within the group) without lifting his/her finger. For word input, we use a Bayesian approach that takes both the probability of gesture and the language model into account to interpret users’ input.

To evaluate the performance of 1D Handwriting, we conduct three lab experiments. Results show that 1D Handwriting provides immediate usability: With limited training, users can achieve 4.67 WPM for letter level input (error rate = 0.78%) and 9.72 WPM for word level input (error rate = 0.25%). These speeds are comparable to those proposed for two-dimensional constrained interfaces, and significantly outperforms the one-dimensional selection-based technique (the Baseline in our experiments) by 11.4%

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. Copyrights forcomponents of this work owned by others than ACM must be honored.Abstracting with credit is permitted. To copy otherwise, or republish, topost on servers or to redistribute to lists, requires prior specific permissionand/or a fee. Request permissions from [email protected]. CHI'16, May 07-12, 2016, San Jose, CA, USA © 2016 ACM. ISBN 978-1-4503-3362-7/16/05…$15.00 DOI: http://dx.doi.org/10.1145/2858036.2858542

How Fast Can You Type on Your Phone? #chi4good, CHI 2016, San Jose, CA, USA

71

Page 2: One-Dimensional Handwriting: Inputting Letters and Words ...pi.cs.tsinghua.edu.cn/lab/papers/p71-yu.pdf · One-Dimensional Handwriting: Inputting Letters and Words on Smart Glasses

ftrlWingain

Iwpr1cf

R

LHabcMa

MoingWtoWQrs

F

UFuf[R[sWug

for letter inputraining focuseetters, 7 distin

WPM after thndicate that 1

gestures are eaapply 1D Handnterfaces.

In the remaindwork relevant tprocess for derreport three us1D Handwritinconclude our rfuture search di

RELATED WO

Letter input onH4-Writer [16]and leverages Hby considering could be achievMeanwhile, a gachieve 5.3-6.6

Multi-step interon constrained nput letters (e.

gestures (5.4 WWriter [7] uses o recursively

WPM) (FigureQWERTY keybregions. A firsspecifies the ch

Figure 1. Matrix(left and

Unistroke letteFigure 2 showunistroke vocabfast to write, a2] was desi

Researchers sh15]; but after

sessions), UnisWPM vs. 11.4used for eyes-gesturing in the

t and 20.0% foed on a singlenct gestures), the tenth input.D Handwritingasy to remembdwriting to othe

der of this papto our researchriving the 1D Hser studies thatng for inputtingresearch with irections.

RK

n constrained ] uses four buHuffman codinletter frequenc

ved after ten 35gesture-based v6 WPM using a

raction is anotuser interface.

.g. with a WiimWPM) (Figure

horizontal andreduce the lete 1, right). board on a verst swipe specifharacter within

x-based layout d middle) [11]; L

ers were first pws some samp

bularies. Unisand less prone igned to res

howed users cor extensive trstrokes could WPM) [5]. Ufree input on

e air [17].

or word input.e phrase (6 wotext entry rate. Users’ subjeg is fun to usber. In the futer one-dimensi

per, we first rh. We then introHandwriting get examine the g letters and wdiscussion of

interfaces ttons to input ng to minimizcy. A 20.4 WPM5-50 minutes pversion of H4-a touchpad and

ther means for . GesText [11]

mote) with twoe 1, left and md vertical movetter range to thSwipeboard [ry small touchfies the regionthat region.

and tri-center lLURD-Writer (

proposed for pple letters of ttrokes [8] wasto recognition

semble handwould learn Graraining (twenty

be significanUnistroke letter

mobile devic

. With extensivords, 14 distine can reach 19ective feedbacse and its strokture, we plan ional constrain

review previooduce the desigestures. We thperformance

words. We finalf limitations an

letters (base 4e key sequencM text entry ra

practice session-Writer [3] coud a Wiimote.

inputting letteallows users

o-step directionmiddle). LURDements of curshe target one [6] divides th

hscreen into ninn, and a secon

layout of GesTe(right) [7]

pen-based inputwo well-knows designed to bn errors. Graffwritten Romaaffiti in minuty fifteen-phra

ntly faster (15rs could also b

ces [21] and f

ve nct 9.6 ks ke to ed

us gn en of lly nd

4), ces ate ns. uld

ers to

nal D-sor (8 he ne nd

ext

ut. wn be fiti an. tes ase 5.8 be for

Figu

EdgeWto enabmotionThe firwhere (Figuresequendesignlearnindevice[24], an

UniGe[23], wprimitigestureEvaluaWPMvisual designi

Word iAmbigis too numbeto consto overT9) anto possof lette

Gesturusing ainputspecialhave toword. interacdifficukeyboaUsers tcontrol

One-diselectioreducereduce

ure 2. Sample g

Write [28] was ble unistroke inn stability for rst version fixusers traversed

e 3). Gesture rnce of corners ted to mimic i

ng. Edgewrite s such as joystnd spatial keys

Figure 3. Sa

est [4] definedwhere each gesive motions. es based on oation results safter a brief lsimilarity to hing the gesture

input on consguous keyboard

small to accoer of signals (estraints from brload individu

nd resolve wordsible words in ers to keys can

re keyboards [a single unistrusing both lal advantage ofo lift up the peTherefore, a g

ction (e.g. Vultult. MotionInpuard that arrangtraverse the keller.

imensional texon-based. The

e the size of thes the QWERT

gesture of Unistr(right) [5]

originally propnput while prov

individuals wxed a square hd the edges anecognition wasthat were hit. Eits handwrittencould also be

ticks and touchs [29].

mple gestures o

a gesture alpsture was desig

Writing-withon-boundary anshowed text elearning sessio

handwritten lettes of individual

strained interfads are widely uommodate all e.g. keys or geboth the deviceal keys with md ambiguity bya predefined vbe done in dif

[12, 31] enablroke gesture. anguage and f gesture keyboen or finger be

gesture keyboature [17]) wherut [19] realizeges eight ambieys to input wo

xt entry techniq1Line Keyboa

he screen occuTY keyboard i

rokes (left) and

posed by Wobbviding high ac

with motor imhole over a to

nd diagonals tos performed ba

EdgeWrite’s alpn counterpartse adapted to ohpads [26, 30]

of Edgewrite [28

habet based ogned to have ah-joystick [10nd off-boundaentry rate achon. For both tters was the gul letters.

aces used when the letters or only

estures) are avae and/or user. Tmore than one y matching keyvocabulary. Thfferent ways [1

le users to inpThe algorithmgesture infor

oard is that usetween lettersard is suitable fre inputting a des an ambiguoguous keys intords with a Le

ques already exard [13] was p

upied by the keinto one row

d Graffiti

brock et al. curacy and

mpairments. ouchscreen o input text ased on the phabet was s for quick other input , Trackball

8]

n Wiimote at most two 0] defined ary strokes. hieved 4.5 techniques, uideline for

screen size y a limited ailable due The idea is letter (e.g.

y sequence he mapping 8].

put a word m interprets rmation. A sers do not within one for mid-air delimiter is ous gesture to a circle.

eap Motion

xist; all are proposed to eyboard. It containing

How Fast Can You Type on Your Phone? #chi4good, CHI 2016, San Jose, CA, USA

72

Page 3: One-Dimensional Handwriting: Inputting Letters and Words ...pi.cs.tsinghua.edu.cn/lab/papers/p71-yu.pdf · One-Dimensional Handwriting: Inputting Letters and Words on Smart Glasses

eight keys, with each key overloaded with multiple letters. The Minuum keyboard [21], based on 1Line Keyboard, enables text entry on Google Glass. However, there is only a demo video describing it; no implementation details nor performance data are available. Circle keyboard [20] organizes letters into a circle in alphabetical order. Users rotate a Wiimote to select target letters. The text entry rate was 10.2 WPM. Walmsley et al. [22] applied the source-channel approach to Circle keyboard to allow fast and imprecise input. Experiment results showed expert users could reach 21WPM after 6.7 hours practicing.

In sum, an input interface can be constraint in terms of the number of input signals (keys and usable gestures) or the number of dimensions of the input space. In this work, we target the constraint interface where touch can be performed only in one-dimensional space. Besides, we focus on handwriting approaches rather than selection-based ones for text entry.

STUDY 1: DESIGN OF 1D HANDWRITING GESTURES The goal was to gain an understanding about user acceptance of 1D Handwriting gestures as well as the input capacity of Google Glass, the device we used to experiment our idea in this research. To achieve this, we first derived a set of 1D Handwriting stroke gestures, and then asked individual participant to assess their intuitiveness by performing them on Google Glass. Based on the subjective and objective results, we agreed on the design guidelines for the final handwriting gestures, and revised our design.

Phase 1 —Initial design of handwriting gestures In our first trial, we adopted a user-participatory approach [25]. We recruited eight participants and asked them to design a set of one-dimensional gestures for Google Glass. The process lasted for one hour for each participant to design gestures for all twenty-six letters of the alphabet. In the end, four participants proposed a code-based design, while the other four proposed a handwriting design. Unfortunately, none of the obtained gesture sets was satisfactory, because they were not memorable. This result revealed the difficulty of designing appropriate 1D Handwriting stroke gestures, especially within a very limited time. However, according to the post-experiment interviews, participants (including those who proposed the coding-based design) consistently agreed that the handwriting design was much more recognizable and memorable than the code-based design.

We then decided to design a set of handwriting stroke gestures by ourselves. The guideline was to design a 1D Handwriting stroke gesture for each letter to best mimic their two-dimensional counterparts. Figure 4 summarizes the handwriting gestures. For some letters, the design was easy and intuitive. For example, the handwriting stroke gesture of “w” should be a four-stroke gesture “back—forward—back—forward”. However, for others it was not an easy task since several letters were similar, if not identical, when projected into a reduced one-dimensional

space. In such cases, we designed strokes with more details (e.g. adding a short stroke at the end to distinguish “q” from “y”), or use a dot to distinguish them, such as “o” and “v”, “b” and “k”, “i” and “s”, and “j” and “l”. The design of handwriting gestures employed strokes of three levels of lengths (i.e. short, medium and long) and a dot.

Figure 4. Handwriting gestures used in Study 1. Dots (“·”) were included in this design. However, these were

consequently dropped in favor of faster input speed. Arrows represent the directions for each sub-stroke in a gesture. They

are drawn continuously in a gesture.

Phase 2 — User assessment and data collection

Apparatus We used Google Glass for this experiment. As shown in Figure 5, there is a touchpad mounted on the right spectacle frame. The touchpad (7.0 centimeters long and 0.8 centimeters wide) supports up to three touch points with an input resolution of 1366×160. Although Google Glass is not a strict 1D input device, the vertical space is very limited for input. In addition, there is a virtual screen (resolution: 640×360) and a built-in speaker on the device. The Google Glass system was XE 22 running on Android 4.4.4. We developed a software that recorded all touch events, including time and coordinate information.

Figure 5. The touchpad configuration of Google Glass

Design and procedure Twelve participants (six males and six females; Age: 18-25) were recruited from Tsinghua University campus. None had previously used a Google Glass. The experiment took about one hour to complete. At first, each participant was asked to memorize the handwriting gesture set (Figure 4). Then, they were instructed to perform these gestures (five times per letter) on Google Glass at their comfortable speed. For reference purposes, a sheet containing every letter-gesture pair was placed beside participants in case they forgot a gesture during the experiment. At the end of the

How Fast Can You Type on Your Phone? #chi4good, CHI 2016, San Jose, CA, USA

73

Page 4: One-Dimensional Handwriting: Inputting Letters and Words ...pi.cs.tsinghua.edu.cn/lab/papers/p71-yu.pdf · One-Dimensional Handwriting: Inputting Letters and Words on Smart Glasses

eow

RMraucaoUlcdmsinin

Idsssbpbmino

Fm

PBrwsetowrwggd

experiment, paof each individwe collected 15

Results Memorability. resembled twadjustments (i.eusers to rememcomments, the an arbitrary waout. ParticipanUnfortunately, earning the s

constantly referdifficulty. Mormemorability wstroke gesturesndividual partnterpreting the

Input accuracdistinguish betwshows the diststrokes after besub-stroke witbetween the thparticipants occbeginning or emore difficult tntend to drop s

of a two-level d

Figure 6. Beta medium and sho

Phase 3 — FinBased on the requirement ofwhich posed astroke gestureefficiency. Insto represent mo

was inspired bresolving the lwith a small nugathered duringgestures, we fodesign.

articipants rateual gesture on

560 handwritin

Although thewo-dimensionae. short strokes

mber actual strosmall adjustm

ay in that no cnts had to rem

this was not estroke gesturerred to the geseover, participwas diverse, es, (e.g. “q”, “ticipants shoule handwriting s

cy. It was dween three levtribution of seing normalizethin a gestureree-level lengtcasionally perfend of a strokto recognize byshort strokes frdesign of strok

distribution foort strokes.

nal design of tabove results

f one-to-one ma major challenes with gooead, we alloweore than one ley an ambiguolimitation of inumber of buttog the initial dermed the follow

ed the perceivea 5-point Like

ng gestures for

e overall handl handwritins and dots) mad

okes. Accordingments seemed tconsistent rule member the steasy given the s. The fact tsture sheet alsoants’ subjectivven for the sa

“h”, “g” etc.). d have discrep

stroke gestures

difficult for vels of stroke short, mediumed to the lengte. There wasths of sub-stroformed unintenke, making shoy the algorithmrom our designe length (medi

or the relative

he handwritins, we decided mapping of genge for designd memorabiled a handwritinetter if necessarus keyboard thnput resourcesons). Based onesign of the hanwing guideline

ed memorabiliert scale. In tota26 letters.

dwriting strokng, the smade it difficult fg to participanto be definedcould be draw

trokes by healimited time f

that participano confirmed th

ve assessment oame handwritin

This suggestpant models f.

participantslength. Figure

m and long suth of the longes much overlkes. In additio

nded hooks at thort strokes evm. Therefore, wn and be in favium and long).

lengths of lon

ng gestures to remove th

stures to letterning handwritinity and usagng stroke gestury. This decisiohat was also fs (e.g. a keyp

n the informationdwriting strokes for the gestu

ity al,

kes all for ts’ in

wn art. for nts his on ng ed for

to e 6 ub-est ap

on, he en we vor

ng,

he rs, ng ge

ure on for ad on ke

ure

(1) Mgetraea

(2) Mstr

(3) Mpe

(4) Sineff

The debetweeprovidpromotwe firsletter performin eacselectiostroke dimens

Figureof Goo

stahan

counte

Figure The demost owe des26 lettlettersdealt din a olettersdegreedimensthus as“n” to

Mimic traditionaestures shouldaditional handwasy to be remem

Minimize levelsroke gestures a

Minimize numberform stroke gngle stroke inp

fficiently.

esign goal waen speed, accue immediate te simplicity rst designed a “b

without concmed a few adjch group to ron within a ggestures to m

sional handwri

e 7. Gesture desogle Glass is plaarting with a cirndwriting. Letteerclockwise. Bla

for on

7 illustrates tesign was baseof these are pesigned 13 strokters of the alpassociated wit

differently withone-dimensiona“e”, “z”, and

es counterclocsional space. Lssigned the ge it. We also i

al handwritingd be easy towriting, the obmbered. s of stroke lengaccurately and er of sub-strok

gestures efficieput: In order to

as to obtain a uracy, learnabil

usage. One rather than distbest” stroke gecerning abouustments to lim

reduce ambigugroup. We de

mimic the sub-iting and to be

sign for individuaced to the left frcle are the firsters “e”, “s”, “z”ack arrows reprne-dimensional

the final stroked on glyphs oerformed with ke gestures, wh

phabet. There th each handwh letters that lal space if m“s”, their glyp

ckwise and thLetter “x” has esture of anothintroduce shor

g: 1D Handwrio learn. By btained strokes

gths: In order efficiently.

kes per letter: ntly. o perform strok

comprehensivlity and memobasic principtinction. To acesture for each

ut ambiguity.mit the numbeuity and facilesigned 1D H-stroke sequenas succinct as

ual characters. for reference. Rt strokes in trad” are rotated 90resent the propol gestures.

ke design for ef lower-case lea single strokhich were mapwere no more

writing stroke glacked optimal

mapped directlyphs were first hen mapped a two-stroke her two sub-strthand in the

iting stroke mimicking

s should be

to perform

In order to

ke gestures

ve trade-off orability to le was to

chieve this, h individual

We then er of letters litate letter

Handwriting nce in two-

possible.

The image Red arrows ditional degrees osed design

each letter. etters since

ke. In total, pped to the e than four gesture. We l mappings y. For the rotated 90 to a one-glyph. We

troke letter design. In

How Fast Can You Type on Your Phone? #chi4good, CHI 2016, San Jose, CA, USA

74

Page 5: One-Dimensional Handwriting: Inputting Letters and Words ...pi.cs.tsinghua.edu.cn/lab/papers/p71-yu.pdf · One-Dimensional Handwriting: Inputting Letters and Words on Smart Glasses

trawo

TIthapltaWin

GThrNreb

T

SWoppdspa

SIarp

SWdln

SWspws

wg

raditional handa backward (towith a backwaour design for s

THE TECHNIQIn this section,he technique u

algorithm to recprobability of etters. We thenarget words b

We also descndividual char

Gesture recogThe purpose ihandwriting strrather than to dNote that accorepresent moreestimated resulbased word-lev

The recognition

Step #1: FilterinWe first identioccur when strpoints of a stroproceed to elidistance is lessegmentation ispoints. The deras … .

Step #2. RemoIn order to deaat the beginniremove the firspixels (7.5 mm

Step #3. NormaWe normalize tdividing the lenength of sub

normalized in a

Step #4. CompWe only compsub-strokes is eprobability of we assume thestrokes to be in

where the probgiven according

dwriting, letteroward the bodyard stroke. Thesimplicity and

QUE , we describe

used in 1D Hancognize gesturan observed

n present our y taking advaribe the interacters and wor

gnition s to estimate roke input ( )discretely classording to our de than one lt to produce le

vel input.

n algorithm has

ng and segmenify the “turninroke direction oke are two deiminate consess than 20 pixs performed brived sequence

val of unintentil with unintentng and end ost and last sub) in length.

alization the length of angth of the lonb-strokes in advance.

utation of a pospute the templaequal to the deother templatee estimation o

ndependent from|bability densityg to the result o

rs “n”, “m”, any) stroke, whileese strokes weusability.

the implemenndwriting. We fres, which estim

gesture givenBayesian algo

antage of the lraction designrds respectively

a posteriori p) given a stroksify it into a spdesign, a strokletter. We canetter-level inpu

s four steps:

ntation. ng points” in tchanges. The efault turning pecutive turningxels (1mm) asased on the ree of sub-stroke

ional strokes tional hooks thof a handwriti-strokes that a

an individual sungest sub-strok

the gesture

steriori probabate gestures werived gesture. es is set to zerof individual nm each other. W|

y function (PDof Study 1.

nd “p” start wie letter “d” en

ere eliminated

ntation details first describe thmates a posterin by individuorithm to predilanguage moden, for inputtiny.

probability of ke template (pecific templatke template man then use tht and dictionar

the stroke, thestarting and enpoints. Then wg points whos noise. Finallemaining turnines is represent

hat usually occing gesture, w

are less than 15

ub-stroke ( ) bke. Note that th

templates a

ility whose number

The a posterioro. Furthermornormalized suWe have

DF) of |

ith nds

in

of he ior ual ict el. ng

f a ),

te. ay he ry-

ese nd we ose ly, ng ed

cur we 50

by he

are

of ori re,

ub-

(1)

is

PredicA userone wletters,numbeinput aapproacan be

Becaus

We theother. W

That is

where

If no mauto-cothat hawordspresent

InteracFigure on Goois dividrow) ddisplaybottom

The hatranspaText reof the time ldimensdimensthe scr

cting target wor inputs a sequ

word ( ) from, that is, er of likely woand language

ach [9], the poscomputed as f|

se we have no ,en assume eachWe have

,s,

, is the stro

matches are foompletions byave more letters

are sorted,ted to the user

ction design 8 shows the in

ogle Glass thaded into three

displays the inpys the recogn

m row) displays

Figure 8. Inte

andwriting stroarent overlay oegion is directl

touchpad. Thline. Therefosional stroke gsional visual freen, providing

ords uence of strok

m a vocabulary… . Thords based on t

model. Similsterior probabilfollows: , ∗knowledge of ∗h letter input i

∗∗

oke template o

found, the algoy computing ths than the inpuaccording to in descending

nterface of theat users can seee regions. The put text. The Lnized letters. s the five most

erface design of

oke is renderedon top of Text ly mapped to thhe horizontal

ore, although gesture, users feedback of thg guidance duri

kes ( …y ( ). The w

he algorithmthe probabilitylar to Goodmlity of I given

, we can com/

is independent

|

|

of letter . orithm attempthe probability

ut sequence. Altheir probab

order [27].

e 1D Handwrite on the virtuaText region (

etter region (thThe Word relikely words.

f 1D Handwriti

d in real-time region. The hehe entire touchdirection reprthe input i

can still obsehe handwritinging input.

) to input word has n

predicts a y of gesture

man et al.’s a word (w)

(2)

mpute

(3)

from each

(4)

(5)

ts to make y of words ll predicted bility, and

ting system al screen. It the middle

he top row) egion (the

ing

on a semi-eight of the hable range resents the is a one-rve a two-

g stroke on

How Fast Can You Type on Your Phone? #chi4good, CHI 2016, San Jose, CA, USA

75

Page 6: One-Dimensional Handwriting: Inputting Letters and Words ...pi.cs.tsinghua.edu.cn/lab/papers/p71-yu.pdf · One-Dimensional Handwriting: Inputting Letters and Words on Smart Glasses

Irthfstopwwto

Imfasrththg

Femsof

DboMpto

AFbasstosdsthawtass

TGcrthfs

Input a wordrespective strokhat most likel

five) are showselect the mosouchpad with

performing a tword, followedword is selecteo the end.

Input a letter. mode. In orderfinger on the toafter performinswitch to letterregion highlighhe touchpad tohe selection.

gesture for inpu

For some letterend near one omovement in thsystem arrangopposite direcfrequently used

Deletion of a leby clicking observation, mMost likely duperpendicular to be performed

A VARIATION For the sake obased keyboardan ambiguous shown in Figusimilar to 1D ouch-selection

switch betweendown gesture switched). To ihe ambiguous

and then tap twword, users swap one finger t

select a word,selection.

To improve tarGlass, we impcontrol/display respectively. What users coul

fast stroke or strokes.

d. Users inpuke gestures of ly correspond

wn in the Worst likely word

two fingers, two-finger mod by a two-fined, the system

The system isr to input a leouchpad for 30ng a stroke. Ar input mode hted) allowing o select letters.

This design utting letters.

rs, the handwriof the edges ofhat direction des letters in

ction of the d letter selected

etter is performthe camera

most users prefue to the facto the main inpd on the touchp

OF THE 1LINEof comparison,d, referring to

keyboard basure 9, The inte

Handwriting n, rather than n letter mode a

(in our expnput a letter, ukey, swipe tw

wo fingers to cowipe one fingeto confirm the and then tap

rget selection lemented cursratios for slo

We empirically d traverse the

perform acc

ut words by each letter in sto the current

rd region. Usd out of them

or select the ovement to senger tap to coautomatically

s set by defauetter, users hav00ms (empiricAt that point, (with the framusers to moveLifting the finoffers a seam

iting stroke gef the touchpaddifficult. To de

frequency orlast stroke,

d at the endpoin

med with a swibutton. Acc

fer to use the ct that a swipput direction, thpad.

E KEYBOARD we implemen1Line Keybo

sed on QWERerface and theexcept that a 1D gesturing: and word moderiment, the

users swipe onewo fingers to onfirm the seler to select an selection, swip

p two fingers

with the touchsor accelerationow and fast fin

determined thentire keyboa

curate selectio

performing thsequence. Wort input (at moers can directby tapping thother words b

elect the desironfirm. When appends a spa

lt to word inpve to place theally determinethe system w

me of the Lette their finger o

nger will confirmless one-fing

esture is likely , making furth

eal with this, thrder toward thwith the mo

nt of the stroke

ipe-up gesture ording to ocamera butto

pe-up gestureherefore not ea

D nted a selectioard design [13RTY layout. Ae interaction a

user inputs bUsers manual

de with a swipmode was n

e finger to seleselect the lette

ection. To inputambiguous ke

pe two fingers to confirm th

hpad on Googn. We used twnger movemenhe parameters ard with a singns with slow

he rds ost tly he by ed

a ace

put eir ed)

will ter on rm ger

to her he he ost e.

or our on.

is asy

on-3], As are by lly pe-not ect er, t a ey, to he

gle wo nts so

gle wer

Figselecte

t

Becaus“ambigsimulaWe usthe soillustraKeybovocabuexperimdiction

Fi

USER In ordeinputtinof 1Lin

ApparaSixteenfrom campubefore.

F

gure 9. A variatied key (“edc”) ithe interface de

se both 1Lineguous” for ation to quantifed a dictionar

ource code ofates the resultoard exhibited ularies containments, we unary.

igure 10. Accur

STUDY 2: INPer to evaluate tng letters, we ne Keyboard.

atus and partin participants 18-30) were

us. None of the.

Figure 11. The

ion of 1Line Keis highlighted. Fesign is similar

Keyboard aninputting wofy their abilityry (with frequf the Androidts. Both 1D high Top-1 a

ning as many aused the top

racy for vocabul

PUT LETTERSthe performanccompared its

icipants (eight males recruited fro

em had previou

experiment sett

eyboard. The cuFor consistencyto 1D Handwri

nd 1D Handwrords, we pery to specify tarency data) derd Keyboard. Handwriting accuracy (ove

as 178,000 wor10,000 word

laries of varyin

S ce of 1D Handperformance a

and eight femom Tsinghua usly used a Go

ting of User Stu

urrently y purposes, iting.

riting were rformed a rget words. rived from Figure 10 and 1Line r 95% for rds). In the ds as the

ng sizes

dwriting for against that

males; aged University

oogle Glass

udy 2

How Fast Can You Type on Your Phone? #chi4good, CHI 2016, San Jose, CA, USA

76

Page 7: One-Dimensional Handwriting: Inputting Letters and Words ...pi.cs.tsinghua.edu.cn/lab/papers/p71-yu.pdf · One-Dimensional Handwriting: Inputting Letters and Words on Smart Glasses

Fsearrsth

DWo(fMsthinbrigt

BptfGBwDpoetm

TthcpppwlKGmst

RTapwbop

Figure 11 illusetting. The syenter on a laptaway from therested their elrequired inputssynchronized ahe laptop PC th

Design and PrWe used a witonly one indep(1D Handwritinfor each parMacKenzie ansame phrases fhe order of phrn one session.

block containirequired to breagnored for pechnique was c

Before the expparticipant. Weechniques wa

familiarize theGlass, which Before each tewarm-up sessiDuring the exphrases as fast of the experievaluation forechniques. Th

minutes.

The warm-up shat of 1Line

containing eveparticipants. Npractice the performing eacwere asked to etters appeared

Keyboard, partGlass until thmovement coselecting/deletiechniques laste

Results The main depeand uncorrecteprovide more dwe further divbetween the coof the next performing the

ustrates the eystem displayetop PC. Partici

laptop PC. Thlbow onto thes. The softwareall touch eventhrough Wi-Fi.

rocedure thin-subject dependent factor ng and 1Line Krticipant was d Soukoreff's for the two terases randomiz Sessions cons

ing two phrasak after finishinpractice. The counterbalance

periment, we e did not informs ours. Moreomselves with took approxi

echnique was ion with the xperiment, sub

and accurate aiment, particirm and provhe total experim

session of 1D Keyboard. F

ry letter-gestuext, we asked strokes on

ch gesture fivcomplete a shd twice in a raticipants practhey became ontrol and ng a letter). ed for about th

endent measureed errors), inpdetailed inform

vided its input ompletion of th

stroke), gestue handwriting

experiment ened phrases foripants sat apprhey wore a Ge desk while e on Google Gts during the e

esign for this ewas the Tech

Keyboard). Thrandomly g

phrase set [14chniques per pzed. Each technsisted of six bses for input.ng a block. Thpresentation

ed.

introduced itsm participants wover, we allowthe user intermately five tpresented, patested text en

bjects were reas possible. Fiipants filled

vided feedbackment lasted fo

Handwriting For 1D Handwure pair was fi

participants toGoogle Glasse times. Fina

hort practice taandom order. Aiced input phrfamiliarized wother input

Warm-up sehirty minutes.

es were input eput speed and mation about 1

time into elahe previous strouring time (stroke gesture

nvironment anr participants roximately 0.5oogle Glass anperforming th

Glass logged anexperiment wi

experiment. Thhnique employhe tested phrasgenerated fro4]. We used thparticipant, winique was testlocks, with ea Subjects we

he first block worder of ea

s goals to eacwhich of the twwed subjects rface of Googto ten minute

articipants had ntry algorithmequired to inpinally, at the en

a NASA-TLk on the twr roughly nine

was longer thwriting, a sheirst presented o memorize ans by correctally, participanask where all 2As for the 1Linrases on Googwith the tou

gestures (e.essions of bo

errors (correctinput time. T

1D Handwritinapsed time (timoke and the sta(time spent oe) and selectio

nd to

5m nd he nd ith

he ed

ses om he ith ed ch

ere was

ch

ch wo to

gle es.

a ms. put nd

LX wo ety

an eet to nd tly nts 26 ne

gle ch .g.

oth

ed To ng, me art on on

time (importnecessagesturelocatiohandw

CorrecFigure error ra

There werror rlow unHandwThe reif they

ANOV(F1,15=6correctTechniParticipthan w(SD=3performp<.005effect p=.29)

Figur

Input SFigure over thincludeaverag1D HaKeyboinput a4.20 WHandwaveragblock significtechniqKeyboeffect.(F4,60=

(time spent otant to note ary pause bees (to completon) and the t

writing stroke ge

cted and uncorr12 illustrates

ates migrated w

was no significratio (F1,15=1.7ncorrected erro

writing and 0.4eason was that

noticed it on t

VA showed 68.92, p<.000ted error rique Block wapants had more

with 1Line Key.69%). Howevm better (i.e. h5) with 1D Hafor 1Line Key.

re 12. The error

Speed 13 shows the

he five blocked the time u

ge, participantsandwriting an

oard. If we conat 4.67 WPM

WPM (SD=0.6writing was 8ge (F1,15=5.31,

(F1,15=8.70, pcant main effeques (1D Han

oard: F4,60=15.3There was no 1.37, p=.26) on

on performingthat the elap

etween succeste a stroke gestime participaestures.

rected errors s how the cowith blocks for

cant effect of T76, p=.20). Boor ratios: 0.78

48% (SD=0.65participants te

the virtual scre

significant 01) and Blockratio. The as also significe corrected err

yboard, 17.67%ver, through prhaving less corrandwriting. In yboard was no

rs for both techntesting result

e text entry raks. The calculusers spent ons input at 4.04

nd 3.71 WPMsidered only th(SD=0.61) w

61) with 1Lin8.9% faster th

p<.05), and p<.01). Mean

ect of block onndwriting: F4,60

39, p<.0001), interaction eff

n text entry rat

g the selectipsed time incsive handwritsture and resetants spent to

orrected and ur letter-level in

Technique on uoth techniques8% (SD=1.12%5%) for 1Line ended to correcen.

effects of k (F4,60=3.80,

interaction cant (F4,60=4.22rors with 1D H% (SD=6.10%)ractice, participrected errors) (contrast, such

ot significant (

niques with thets.

ates for both lation of text n correcting e4 WPM (SD=

M (SD=0.46) whe last block, p

with 1D Handwe Keyboard. H

han 1Line Ke11.2% faster

nwhile, we on text entry rat0=14.50, p<.00suggesting th

ffect of Techniqtes.

on). It is cludes the ting stroke t the finger

recall the

uncorrected nput.

uncorrected s exhibited %) for 1D Keyboard.

ct mistakes

Technique p<.01) on effect of 2, p<.005).

Handwriting vs. 5.51%

pants could (F4,60=4.68,

a learning (F4,60=1.39,

e statistical

techniques entry rate

errors. On =0.58) with with 1Line participants writing and Hence, 1D eyboard in in the last

observed a es for both 001; 1Line he learning que Block

How Fast Can You Type on Your Phone? #chi4good, CHI 2016, San Jose, CA, USA

77

Page 8: One-Dimensional Handwriting: Inputting Letters and Words ...pi.cs.tsinghua.edu.cn/lab/papers/p71-yu.pdf · One-Dimensional Handwriting: Inputting Letters and Words on Smart Glasses

i

InIti(Efinscsptile

Fslinthmdgwd“pgtir

Figure 13. Textand 1Line Keyin the 4th block

resulted fro

nput times of 1In average, elaime was 504m

(SD=151) inclElapsed time, gfor 45%, 16% as, the majority

next letter, recselecting lettercould be impsignificant effep<.0001), gestuime (F4,60= 11.earning curve

elapsed time an

Figure 14. Elaacross five bloc

Furthermore, aselection time fetter. From Fncreased as thehe structure o

medium lengthdependent on gestures as welwas no obvioudetected that fo“z”) elapsed participants spegesture. Interesime was short,

rather large.

t entry rates acryboard. There w

for 1D Handwrom uncontrolled

1D Handwritingapsed time wasms (SD=96), anluding 300ms gesturing timeand 39% of th

y of input timecalling the hans. Fortunately,

proved (Figureects of Block uring time (F4,6

04, p<.0001). Aof gesturing ti

nd selection tim

apsed time, gestcks for letter-lev

analysis of elapfor individual igure 15, we e number of suf sub-strokes i

h sub-strokes). the number oll as the order ous pattern foror some letterstime was relent more time rstingly, for the , both gesturing

ross blocks for was a sudden periting. We interd factors in the

g s 1397ms (SD=nd selection tim

spent on swe and selectionhe input time ree was spent onndwriting stro with practicee 14): RM-A

on elapsed t60= 5.41, p<.00A noteworthy ime was not asme.

turing time andvel 1D Handwr

psed time, gesletters was concan see that,

ub-strokes or thincreased (withSelection tim

f letters repreof letters insidr elapsed times that were selatively high, recalling its haletter “j”, althog time and sele

1D Handwritinerformance droprpret it as a nois

experiment.

=354), gesturinme was 1206m

witching moden time accountespectively. Thn considering thoke gesture, an, all input tim

ANOVA showtime (F4,60=8.701) and selectioaspect is that ths steep as that

d selection time riting in Study 2

sturing time annducted for ea, gesturing timhe complexity h both long an

me was primariesented by inpe a group. Thee. However, wldom input (e.indicating th

andwriting strokough the elapsection time we

ng p se

ng ms es. ed

hat he nd

mes ed

73, on he of

2.

nd ch

me of nd ily put ere we .g. hat ke ed

ere

Figurediffere

SubjecNonpacompaHandwp<.000much other m

Althouto permanagpracticintervigestureIn addifatigueon the 1D Hascreen participThese1D Han

USER The goHandwsixteenrecruiteseven appara

The dethat paletter-lrequire

Prior tohandwand to that pa

e 15. Elapsed timent letters. The c

same handwr

ctive feedbacksarametric Wilcared to 1Linwriting to be 01). On the othbetter evaluat

measures were

ugh at first it srform the hanged to quickly ce rounds for ew, every parties to be intuiti

dition, most pare with 1Line Ke keyboard to andwriting req

while performpants were mfindings sugge

andwriting.

STUDY 3: INPoal of this stu

writing and 1Lin participants ed again for tto fourteen da

atus as in Study

esign and procarticipants perflevel input. Ined to enter 18 p

o the experimewriting stroke ge

write them dowarticipants cou

me, selection timcolors separate

riting stroke for

s coxon signed-rne Keyboard,

more mentalher hand, theytion (z=-42.0, not statisticall

seemed difficulndwriting strok

master the ter each letter. icipant reporteve to recognizrticipants repo

Keyboard since control cursor

quired a lesserming stroke g

more familiar wested eyes-free

PUT WORDS udy was to evine Keyboard

who participthis study. Thays after Study 2.

cedure were siformed word-ln this experimphrases (divide

ent, participantestures for all 2wn on a piece

uld correctly r

me and gesturine letters with resr easy comparis

rank tests sugg, participantsly demanding

y gave 1D Hanp<.01). Diffe

y significant.

lt to rememberke gestures, pechnique in les

In the post-eed 1D Handwrize and easy to orted higher lev

more focus war movement. Ir degree of focestures, espec

with the strokee input was po

valuate and cofor inputting w

pated in Stude experiment

dy 2. We used

imilar to Studylevel input as oment, participed into nine blo

ts were asked to26 letters of thof paper. Resu

recall 89% of

ng time for spect to the on.

gested that felt 1D

g (z=-52.5, ndwriting a erences on

r and learn participants ss than ten experiment iting stroke remember. vels of eye as required In contrast, cus on the ially when e gestures. ssible with

ompare 1D words. The dy 2 were took place

d the same

y 2, except opposed to pants were ocks).

o recall the he alphabet, ults showed

the stroke

How Fast Can You Type on Your Phone? #chi4good, CHI 2016, San Jose, CA, USA

78

Page 9: One-Dimensional Handwriting: Inputting Letters and Words ...pi.cs.tsinghua.edu.cn/lab/papers/p71-yu.pdf · One-Dimensional Handwriting: Inputting Letters and Words on Smart Glasses

gwtredo

PatorT

RTas

CFe

TeloH

Wceas(s(

F

InFo

I1K1la9

gestures. Four well were “x”raining each p

each letter in adays, this resulof 1D Handwri

Participants fama warm-up sesso fifteen minu

required to inpThe total exper

Results The main depeand uncorrectedstandard deviat

Corrected and uFigure 16 illuerror rates migr

There was no serror ratio (F1,

ow uncorrecteHandwriting an

We observed acorrected errorerror ratio wasand 11.87% (SDsignificant eff(F7,105=1.15, p=significant int(F7,105=0.46, p=

Figure 16. The

nput Speed Figure 17 showover the eight b

In average, par1D HandwritinKeyboard. Tha1Line Keyboarast two blocks

9.72 WPM (SD

letters that pa, “d”, “j” andparticipant hadverage) and thlt positively inditing stroke ges

miliarized themsion before theutes for each

put phrases as riment lasted fo

endent measured errors) and intions. Therefor

uncorrected erstrates how thrated with bloc

ignificant effec,15=1.21, p=.29ed error ratiosnd 0.07% (SD=

a marginal signr ratio (F1,15=s 17.38% (SDD=5.81%) for fect of Block=.034), suggeteraction effe

=.86) was obser

corrected and utechniques a

ws the text enblocks.

rticipants inputng and 7.14 Wat is, 1D Handwrd in average (Fs were consideD=2.67) for 1D

rticipants could “t”. Consided (no more thhe interval of sdicated the goostures.

mselves with the task, which l

technique. Pfast and accur

or about eighty

es were input enput speed. Ale, no outliers w

rrors he corrected acks for word-le

ct of Techniqu9). Both techns: 0.25% (SD==1.42%) for 1-L

nificant effect o4.51, p=.051)

D=8.67%) for 1Line Keyboa

k on correctsting the learect of Techrved.

uncorrected erracross blocks

ntry rates for

t at 8.84 WPMWPM (SD=1.writing was 23F1,15= 6.92, p<ered, text entryD Handwriting

ld not remembering the limitehan 20 times f

even to fourteod memorabili

he techniques asted around tarticipants werate as possibl

y minutes.

errors (correctll data are withwere removed.

and uncorrectevel input.

e on uncorrectniques exhibit=0.07%) for 1Line Keyboard

of Technique o. The correct1D Handwritinard. There wasted error rat

rning effect. Nhnique Blo

ror rates for bot

both techniqu

M (SD=2.79) wi.21) with 1Lin3.8% faster tha

<.05). If only thy rates raised

g and 8.10 WP

ber ed for en ity

in en

ere le.

ed h 2

ed

ed ed

1D d.

on ed ng s a tio No ck

th

ues

ith ne an he to M

(SD=1was 20blocks

ANOVtext enlearninof Tech

F

Input tiOn ave(SD=6be intewhen c1Line 1D HaStudy techniq

Figua

Auto-coAuto-c1D Hadifferenp<.000input vphrase5.17 WThe recompleHandwstroke the lap

.34) for 1Line0.0% faster th (F1,15=5.86, p<

VA also showedntry rates (F7

ng effect. Therhnique Block

Figure 17. The

imes erage, the elaps

66) longer thanerpreted as thecompared to tKeyboard, the

andwriting was2, we observe

ques (p<.05), a

ure 18. Slide timacross eight blo

ompletion completion accandwriting an

ence was st01). In other wvia auto-comps at 8.30 WPM

WPM (SD= 0.8eason why partetion for 1Lin

writing, as pargestures, they

ptop PC screen

e Keyboard. Than 1Line Ke<.05).

d a significant7,105=11.43, p<re was no sign

k on text entry r

learning curve

se time of 1D n that of 1Linee recall time fthe slide time e gesturing tims much smallered significant as shown in Fig

me, gesturing timocks for both tec

counted for 6.nd 38.10% fortatistically sigwords, if we

pletion, particiM (SD=2.70) f86) for 1-Line kticipants took

ne Keyboard warticipants perfy tended to loon instead of th

That is, 1D Heyboard in th

t main effect o<.0001), suggnificant interacrates (F7,105=1.2

of both techniq

Handwriting we Keyboard, wfor gestures. In

(1,593 ms, Sme (427 ms, Sr. Similar to thelearning effectgure 18.

me and elapsed chniques in Stu

51% of inputr 1Line Keybgnificant (F1

did not consiipants would hfor 1D Handwkeyboard over more advantag

was as followsformed the hok at the test phe Google Gla

Handwriting e last two

of Block on gesting the ction effect 25, p=.28).

ques.

was 289 ms which could

n addition, D=132) of

SD=38) for e results of ts for both

time and udy 3.

letters for board. The ,15=191.49, ider letters have input

writing, and all blocks.

ge of auto-s: With 1D handwriting phrases on

ass display.

How Fast Can You Type on Your Phone? #chi4good, CHI 2016, San Jose, CA, USA

79

Page 10: One-Dimensional Handwriting: Inputting Letters and Words ...pi.cs.tsinghua.edu.cn/lab/papers/p71-yu.pdf · One-Dimensional Handwriting: Inputting Letters and Words on Smart Glasses

Tnpcpthwpath

UTbfinatwn[wsao

RWa1lrla

AorposMthsru

LTp

They only turneeded to selecparticipants cocontrol cursor mparticipants to he post-experim

with participanparticipants maauto-completioheir eyes reste

USER STUDY The goal of thibe following exfive participantnput a single

and Soukoreff’wo trials were

number of word6]. However, i

was based on aserved as a rouauto-completioof a word with

Figure 19. The

Results showedWPM (SD=3.5achieved 24 W16 WPM. As earning effect

rate. In averagast block than

According to pobservations, arates was the liperforming fasovershot the toslow down finMoreover, we he design of

speed and expreasonably reflused 1D Handw

LIMITATIONS AThis research point to the dire

rned to the vct words. In constantly lookmovement. Thnotice the au

ment interviewnts. We think ay be more liken function witd on the virtua

4: EXTENSIVEs study was to xtensive trainits who participphrase (random’s phrase set [1e ignored as prds, the trainingit is important

a single phrase,ugh estimation.n so that partichandwriting st

learning curve

d the mean tex50) for the last

WPM, while theFigure 19 shof Block (F9,6

ge, participantsin the first one

participants’ suba major factor imited input cast gestures, thouchpad. In sunger movemenconfirmed wiexperiment: P

perience obtainlect their profiwriting for a lon

AND FUTUREhas the follo

ection of future

virtual screen ontrast, with 1

ked at the viherefore, it wasuto-completion w, we confirme

in actual reaely to take full th 1D Handwr

al screen.

E TRAINING estimate how

ing. To achievpated the Studymly selected f14]) for twelveactice. By focu

g time could beto note that gi

, the obtained s In this study, cipants had to troke gestures.

e of 1D Handwr

xt entry rate cophrase. The fae slowest part

hows, there w3=6.16, p<.000

s entered 33.95e.

bjective feedbafor prohibiting

apacity of Goohe finger ofteuch cases, parnts to produceth participants

Participants thned in the ex

ficient input spnger period.

E WORK owing limitatioe works.

whenever th1Line Keyboarirtual screens more likely f

suggestions. ed this differenal-life scenarioadvantage of thriting by havin

fast a user coue this, we asky 3 to repeatedfrom MacKenze times. The firusing on a smae greatly reduciven the practispeed could onwe switched oinput each lett

riting in Study 4

ould reach 19.6astest participaicipant achiev

was a significa01) on text ent5% faster in th

ack and our owg high text entgle Glass. When went off rticipants had e correct inpus with regardhought the inpxperiment coupeed if they h

ons, which al

ey rd, to

for In

nce os, he ng

uld ed

dly zie rst all ed ice nly off ter

4.

61 ant ed

ant try he

wn try en or to ts. to

put uld ad

so

First, ofor lowmore cnon-Labetter band 1Dto remeoverloaaddrescharactthe “fllower c

Secondparticipand rethere iemploy[1]) to to testtraining1D Hlearnin

Third,handwbe notadvantas the sensorshandwsmartpedge),

Finallyinputtin1D hakeyboa

CONCWe exform interfachandwanalyzeHandwimplemGlass. suggesrememfeasibi

ACKNO

This wof ChiTsinghand No

our current 1Dwer-case Englcharacter setsatin alphabets.be an easy-to-r

D strokes; otheember. In addiading a single s this with ter sets. For elip up” gesturcase, or even b

d, regarding lpants could lea

ecall 89% of tis still space foy a well-desigfacilitate gestu

t the performg. Now, in allandwriting di

ng time.

in this researcwriting. Howev

t optimal for tage of the vert

three-point tos) for input.

writing techniqphones with a

smart wristban

y, in current resng letters. It isandwriting forard. This might

LUSION xplored the po

of constraince. The basic

writing strokesed the usabili

writing gestumented and ev

In user studiested 1D Hand

mber and efficiility of 1D Han

OWLEDGMEN

work is supportina under Grahua Universityo. 2015130077

D handwritinglish letters. It , such as num One requiremrecognize maperwise, the 1Dition, supportin stroke with mmode selectio

example, on Gre to switch bbetween numbe

learnability, ouarn the 1D strothe strokes aft

for improvemengned learning iure learning. M

mance after lol three experimid not conver

ch, we confirmver, the current

Google Glasstical input and ouch and even

It is also que on other

curved-edgednds, or even 2D

search, we focus interesting to r word level t further impro

ssibility of texned interface:

idea was to into a one-ity issues of

ures. Subsequvaluated 1D Hs, both objecti

dwriting strokeient to use. O

ndwriting.

NTS

ted by the Natant No. 61303y Research Fun71.

g gestures wereis important

mbers, punctument is that thpping between D strokes will bng more characmore characteron to specify

Google Glass, wbetween upperers and punctua

ur research shokes with a fewfter one week.nt. For exampinterface (e.g. Moreover, it is ong-term and

ments, the inpurge due to th

med the feasibt interaction ds: We did notother input senn the embeddinteresting to1D interface

d screen (e.g. D touchscreens

us on 1D handexplore the poinput, i.e. 1

ove the text ent

xt entry on an: the one-dproject two-d

-dimensional sdesigning uniuently, we

Handwriting wiive and subjecte gestures we

Our research v

tural Science F076 and No. nding No. 20

e designed to support

uations and here should 2D strokes be difficult cters means s. One can y different we can use r case and ations

howed that w practices, However,

ple, we can OctoPocus interesting extensive

ut speed of he limited

ility of 1D design may t take full nsors (such

ded motion o test 1D es such as Galaxy S6 s.

dwriting for ossibility of D gesture

try speed.

n emerging imensional imensional space. We istroke 1D

designed, ith Google tive results re easy to

verified the

Foundation 61272230, 151080408

How Fast Can You Type on Your Phone? #chi4good, CHI 2016, San Jose, CA, USA

80

Page 11: One-Dimensional Handwriting: Inputting Letters and Words ...pi.cs.tsinghua.edu.cn/lab/papers/p71-yu.pdf · One-Dimensional Handwriting: Inputting Letters and Words on Smart Glasses

REFERENCES 1. Olivier Bau and Wendy E. Mackay. 2008. OctoPocus: a

dynamic guide for learning gesture-based command sets. In Proceedings of the 21st annual ACM symposium on User interface software and technology (UIST '08), 37-46. http://dx.doi.org/10.1145/1449715.1449724

2. Conrad H. Blickenstorfer. 1995. Graffiti: Wow! Pen Computing Magazine, pp. 30-31.

3. Steven J. Castellucci and I. Scott MacKenzie. 2013. Gestural text entry using Huffman codes. In Proceedings of the International Conference on Multimedia and Human-Computer Interaction (MHCI '13), 119.1-119.8. http://www.yorku.ca/mack/mhci2013e.html

4. Steven J. Castellucci and I. Scott MacKenzie. 2008. Unigest: text entry using three degrees of motion. In CHI '08 Extended Abstracts on Human Factors in Computing Systems (CHI EA '08), 3549-3554. http://dx.doi.org/10.1145/1358628.1358889

5. Steven J. Castellucci and I. Scott MacKenzie. 2008. Graffiti vs. unistrokes: an empirical comparison. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '08), 305-308. http://dx.doi.org/10.1145/1357054.1357106

6. Xiang 'Anthony' Chen, Tovi Grossman, and George Fitzmaurice. 2014. Swipeboard: a text entry technique for ultra-small interfaces that supports novice to expert transitions. In Proceedings of the 27th annual ACM symposium on User interface software and technology (UIST '14), 615-620. http://dx.doi.org/10.1145/2642918.2647354

7. Torsten Felzer and Rainer Nordmann. 2006. Alternative text entry using different input methods. In Proceedings of the 8th international ACM SIGACCESS conference on Computers and accessibility (Assets '06), 10-17. http://dx.doi.org/10.1145/1168987.1168991

8. David Goldberg and Cate Richardson. 1993. Touch-typing with a stylus. In Proceedings of the INTERACT '93 and CHI '93 Conference on Human Factors in Computing Systems (CHI '93), 80-87. http://dx.doi.org/10.1145/169059.169093

9. Joshua Goodman, Gina Venolia, Keith Steury, and Chauncey Parker. 2002. Language modeling for soft keyboards. In Proceedings of the 7th international conference on Intelligent user interfaces (IUI '02), 194-195. http://dx.doi.org/10.1145/502716.502753

10. Zhenyu Gu , Xinya Xu, Chen Chu, Yuchen Zhang. 2015. To Write not Select, a New Text Entry Method Using Joystick. Human-Computer Interaction: Interaction Technologies, 35-43.

11. Eleanor Jones, Jason Alexander, Andreas Andreou, Pourang Irani, and Sriram Subramanian. 2010. GesText: accelerometer-based gestural text-entry systems. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '10), 2173-2182. http://dx.doi.org/10.1145/1753326.1753655

12. Per-Ola Kristensson and Shumin Zhai. 2004. SHARK2: a large vocabulary shorthand writing system for pen-based computers. In Proceedings of the 17th annual ACM symposium on User interface software and technology (UIST '04), 43-52. http://dx.doi.org/10.1145/1029632.1029640

13. Frank Chun Yat Li, Richard T. Guy, Koji Yatani, and Khai N. Truong. 2011. The 1line keyboard: a QWERTY layout in a single line. In Proceedings of the 24th annual ACM symposium on User interface software and technology (UIST '11), 461-470. http://dx.doi.org/10.1145/2047196.2047257

14. I. Scott MacKenzie and R. William Soukoreff. 2003. Phrase sets for evaluating text entry techniques. In CHI '03 Extended Abstracts on Human Factors in Computing Systems (CHI EA '03), 754-755. http://dx.doi.org/10.1145/765891.765971

15. I. Scott MacKenzie and Shawn X. Zhang. 1997. The immediate usability of Graffiti. Graphics Interface 97: 129-137.

16. I. Scott MacKenzie, R. William Soukoreff, and Joanna Helga. 2011. 1 thumb, 4 buttons, 20 words per minute: design and evaluation of H4-writer. In Proceedings of the 24th annual ACM symposium on User interface software and technology (UIST '11), 471-480. http://dx.doi.org/10.1145/2047196.2047258

17. Anders Markussen, Mikkel Rønne Jakobsen, and Kasper Hornbæk. 2014. Vulture: a mid-air word-gesture keyboard. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14), 1073-1082. http://dx.doi.org/10.1145/2556288.2556964

18. Franck Poirier and Mohammed Belatar. 2007. UniGlyph: only one keystroke per character on a 4-button minimal keypad for key-based text entry. HCI International 2007, 479-483.

19. Shuo Qiu, Kyle Rego, Lei Zhang, Feifei Zhong, and Michael Zhong. MotionInput Gestural Text Entry in the Air. http://www.columbia.edu/~sq2144/MotionInput.pdf

How Fast Can You Type on Your Phone? #chi4good, CHI 2016, San Jose, CA, USA

81

Page 12: One-Dimensional Handwriting: Inputting Letters and Words ...pi.cs.tsinghua.edu.cn/lab/papers/p71-yu.pdf · One-Dimensional Handwriting: Inputting Letters and Words on Smart Glasses

20. Garth Shoemaker, Leah Findlater, Jessica Q. Dawson, and Kellogg S. Booth. 2009. Mid-air text input techniques for very large wall displays. In Proceedings of Graphics Interface 2009 (GI '09), 231-238.

21. The Minuum Keyboard. http://minuum.com/

22. William S. Walmsley, W. Xavier Snelgrove, and Khai N. Truong. 2014. Disambiguation of imprecise input with one-dimensional rotational text entry. ACM Trans. Comput.-Hum. Interact. 21, 1:4 http://dx.doi.org/10.1145/2542544

23. Wiimote, https://en.wikipedia.org/wiki/Wii_Remote

24. Jacob Wobbrock and Brad Myers. 2006. Trackball text entry for people with motor impairments. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '06), 479-488. http://dx.doi.org/10.1145/1124772.1124845

25. Jacob O. Wobbrock, Htet Htet Aung, Brandon Rothrock, and Brad A. Myers. 2005. Maximizing the guessability of symbolic input. In CHI '05 Extended Abstracts on Human Factors in Computing Systems (CHI EA '05), 1869-1872. http://dx.doi.org/10.1145/1056808.1057043

26. Jacob O. Wobbrock, Duen Horng Chau, and Brad A. Myers. 2007. An alternative to push, press, and tap-tap-tap: gesturing on an isometric joystick for mobile phone text entry. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '07), 667-676. http://dx.doi.org/10.1145/1240624.1240728

27. Jacob O. Wobbrock, Brad A. Myers, and Duen Horng Chau. 2006. In-stroke word completion. In Proceedings of the 19th annual ACM symposium on User interface software and technology (UIST '06), 333-336. http://dx.doi.org/10.1145/1166253.1166305

28. Jacob O. Wobbrock, Brad A. Myers, and John A. Kembel. 2003. EdgeWrite: a stylus-based text entry method designed for high accuracy and stability of motion. In Proceedings of the 16th annual ACM symposium on User interface software and technology (UIST '03), 61-70. http://dx.doi.org/10.1145/964696.964703

29. Jacob Wobbrock, Brad Myers, and Brandon Rothrock. 2006. Few-key text entry revisited: mnemonic gestures on four keys. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '06), 489-492. http://dx.doi.org/10.1145/1124772.1124846

30. Jacob O. Wobbrock, Brad A. Myers, Htet Htet Aung, and Edmund F. LoPresti. 2003. Text entry from power wheelchairs: edgewrite for joysticks and touchpads. In Proceedings of the 6th international ACM SIGACCESS conference on Computers and accessibility (Assets '04), 110-117. http://dx.doi.org/10.1145/1028630.1028650

31. Shumin Zhai and Per-Ola Kristensson. 2003. Shorthand writing on stylus keyboard. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '03), 97-104. http://dx.doi.org/10.1145/642611.642630

How Fast Can You Type on Your Phone? #chi4good, CHI 2016, San Jose, CA, USA

82