RENOTE ACCESS TO VOICE AND TEXT MESSAGES

Chris Schmandt Research Associate

Media Laboratory Massachusetts Institute of Technology

3 August 1984

The "Voiced Mail" system of M.I.T.'s Architecture Machine Group uses synthesis-by-rule to allow remote access to an electronic message system. A voice storage audio message system is integrated with the text message database, to allow users to send and receive voice replies to text messages and vice versa. Message access is interactive, controlled by Touch-Tones sent by the user, and incorporates many features of conventional on-line mail systems.

ffect of the use of compute s for storage and nt of a wide of textual dat

ls to access th informatnon. ce in office ere. This limits red on-line.

The ability o text-to-speech nthesizers to verbalize ASCII text streams and the pervasive nature o one network suggests au access to suc lines. Thi aper describes access to an on-line electron

ility sf the dateba output and human factors concerns with the utility of such output.

T r s Architecture achine Group computer syste access their

electronic wai xt-to-speech synthesizer (Speech PlusDs Prose 2 ch-Tone telephone. Text messages

es gathered by a voice mail sub- system in a manner which is transparent to the caller. In an environment of heavy on-line mail usage, this system has gained acceptance by a community of about 30 subscribers needing to both read and transmit messages.

Although this system was developed as a component of a wider research activity in telephone access to multi-media message systems, it was found useful enough as an internal utility to be implemented as a stand-alone system and left running 24 hours a day on one of the laboratory" minicomputers. Major portions sf Voiced Mail have been incorporated as is into a personalize text and voice messa e storage and answering machine ( 1 , 2 ) .

To use Voiced Mail a user calls in and gives a unique identi- fier (home phone number) and a password by Touch-Tones, much bike using an automatic bank teller. Messages, both text and voice recordings, are merged, sorted by source, and played sequentially within each group. The caller may interact with the system by Touch-Tones, to jump to the next message or next sender, obtain more information about the sender, repeat a sentence, or a make a reply (figure 1).

u r e 4 . T e l e g R command l a y o u t .

e n e r a t e d . T h e c a l l e r may o f t h e f o r m "I r e a d y o u r

l i n e > a n d t h e e s w , o r no, o r e l e p h o n e n u m b s t h e n k e y e d

c o r d a v o i c e r e p l y f o r a n y s e n d e r , $ e x $ m e s s a g e i n f o m i n g t h e m o f t h e

s t r u c t i o n s f o r t e b p h o n e a c c e s s .

h s p e e c h r e e s i t i s n w a s i n c l u d e d i n t h e h a r d w a r e it was n o t u s e d e x c e p t f o r d e m o n s t r a -

s e o f p o o r p e r o r m a n c e . T h i s c a n b e a t t r i - b u t e d t o t h e f a c t t h a t m c s t r e a c c e s s i n t h e e a r l y s t a g e s o f d e v e l o p m e n t w e r e o v e r l o n g a n c e p h o n e l i n e s , w h i c h t e n d

f a c t t h a t t h e r e c o g n i z e r t e l e p h o n e u s e .

i b i s a n o v e r l a p $ 0 a n e x i s t i n t e x t m e s s a g e s y s t e m , P e a w i t h i t s o p e r a i o n . I n h e r e n t i n t h e % h e b e l i e f t h a t i n eractios w i t h a n d

e d i f f e r e n t f o r s c r e e n a n d e n t i f i e d , t h e i r m a i l f i l e i s i l V s own i n t e r n a l d e s c r i p -

h e m a i l f i l e i n i c a t e t h e b o s a r e n o t p r s e n t e d i n t h

n o r i s a l l t h e t e x t o f t h e f i l e

Figure 2. Voiced Mail hardware configuration.

One of the main motivations for electronic mail systems is convenience of access; on-line message systems enable quick and reliable communication between parties or groups in dis- parate locations and perhaps on differin schedules. Rapid message delivery and 24 hour availability make it ideal for communication or decision making on issues requiring timely response; to use conventional electronic mail systems, how- ever, one must carry a terminal and modem to read and send messages. A telephonic text-to-speech interface removes this restraint.

Speech synthesis can gain acceptance as a viable new means of access to existing text databases much faster than it can in a totally new environment or application. Users already accus- tomed to some service seem much more tolerant of the limita- tions of synthesis when it clearly makes that service more available, while experiments using synthesized speech in completely new situations (e.g., talking supermarket cash registers) have been much less successful. It is important to note that all the users of Voiced Mail were already experi- enced and regular users of one or more electronic mail sys- tems.

Many of the limitations in synthetic speech can be overcome, or at least compensated for, by the interface which allows a user to control information access. Only if these limitations are acknowledged can a system robust enough to satiasfy users other than dedicated speech researchers be implemented.

F o u r m a i n p r o d w i t h s y n t h e t i c s p e e c h i a l o g u e : i t s o v e r a l l q u a l i t y , f s p e e c h , a n d t h e f a c t t h a t a 1 i n n a t u r e .

a r y c o n c e r n w i t h v o i c e o u t p u t m u s t b e i n t e l l i f f i c u l t t o q u a n t i f y ( 1 . M o d i f i e d Rhyme

s i n g l e w o r d , a n d a l i s t o f s i m i l a r e f u l f o r e v a l u a t i n g t h e t s c o r e s o n s u c h a t e s t l l i g i b i l i t y , a s o n e may ill u n d e r s t a n d t h e e d i f f i c u l t y u n d e r -

s t a n d i n g u n f a m i l i a r w o r d s , n a m e s , o r a c r o n y m s .

A s a l i s L e n e r i s e x t o a p a r t i c u l a r s y n t h e t i c s p e e c h h e r a l , a n d b e c e d t o i t , m i s u n d e r s t a n d i n g

u c h a s o n e i m p r o v e s i n a b i l i t y i o n a P o r f o r e i g n a c c e n t (5). S t u d i e s i n d i -

e v e n t h o u g h w o r e r s t a n d i n g may t h i s t a k e s s o m e t h a t a l i s -

f t h e s e n t e n c e s p o k e n (6).

Of p a r t i c u l a r l i c a t i o n s u s i n g s y n t h e s i s i s t h e s e n t e n c e l e v e l p r o s o d y g e n e r a t i o n i n v o i c e m e s s a g e s e x c e e d o n e o r t w o

s e n t e n c e s , t h e f l e s s s o p h i s t i c a t e d s y n - p t i o n , w h i c h i n t u r n l i m i t s

t a c t i c c o n t e x t t o i m p r o v e u n d e r s t a n d - e r s a r e b a r e l y i n t e l l i g i b l e

An i m p o r t a n t c o n s i d e r a t i o n f o r t h e u s e o f s p e e c h o u t p u t i s i t s s l o w a s c o m p a r e d t o c o m p u t e r t e r m i n a l

s p e e d s . I n s i t u a t i o n s r e q u i r i n g b e k e p t t e r s e . I n a p p l i c a - e a m o u n t o f t e x t ( e . g .

a s a n y e x t r a n e o u s i n f o r m a - e f s s e i t i s s p o k e n t o t h e u s e r .

F i n a l l y , i n a n t h e s i s % a a p p b i c a t i o n s w h i c h f o r m e r l y u s e d ---- o r a l ------ n a t u r e o f s p e e c h m u s t b e k e p t . i n m i d i s p l a y a n u m b e r o f menu

u l t a n e a u a l y o n a s c r e e n , t h e y m u s t b e s p o k e n s e q u e n - c c e s s . I f t h e r e a r e a n u m b e r o f c h o i c e s ,

o t t e n b e f o r e t h e l a s t i s s p o k e n , o f t h e c o n c e n t r a t i o n r e q u i r e d f o r u n d e r -

a l i s t sf r e c i t e d o p t i o n s

serial, whereas th eye can (and does) skip lace on the gage or screen scannin

required information.

I H P E E

To minimize short-term memory loads and simplify the concep- tual framework of the control func ions, a modeless, single- keystroke telephone keypad menu wa devised (see figure 1 ) . The slow, serial nature o speech output mitigates against complicated b-menus. T e menu chosen provides csmprehen-

y functiona ity at the rice of a few extra features whi could not be implemented. Slightly expanded functionality was provided when speech recognition was used for input.

Grouping messa e recital by the sender, rather t normal time se uential order, is also done to simplify organization. t is often the case that a single sender will mail a series of related messa es about a common issue, and it makes sense to hear those messages together. Commands, such as next or revious, can operate on a single message or a group.

It is important to minimize the amount of information trans- mitted with each message to speed up the interaction, as users consistently found the system slow. The headers associated with each message were not transmitted (they are often longer than the messa e body itself), although a more info key recites the sender's full name, mailing address, and time of message.

The synthesizer warns "This is a very long messagen if the body exceeds 200 characters; such messages are often ignored until the recipient is at a terminal. Likewise, the subject of a long message will be spoken, and the caller will be asked whether to continue, e.g. "This is a very long mes- sage. It's about the price of widgets in China. Do you wish to hear it?" A short message will simply be spoken, as it takes les time to hear the message than it would to answer questions about it.

Another aid to responsiveness is that the system is ---- alwalg interru~tible an aspect which is vital to its success. A ,,----- -,,--9

ieue command allows speech to be stopped at any is resumed from the beginning of the current

sentence. In fact, any command will be responded to immedi- ately at any time. It is important to be able to abandon playing a Ion message which is not urgent and hence may be

ristely read at a terminal.

l l o w s a c o n a p t a b i l i t y b e t w e e n e d u s e r s . n s t r u c t i o n s ,

a s s w o r d o r a % e n u o p t i o n s , i m m e d i a t e l y c u t s h e e x p l a n a -

t i o n . The n a i v e u s e r c a n r e c e i v e a l e n o r i a l , w h i l e j u s t k e y s i n a s e commands a n d

A 1 1 u s e r s f o u n d t h c o u l d u n d e r s t a n d m o s t o f t h e i r m e s s a m o s t o f % h e t i m e , i n f o r m a l l y , t h e o b s e r v a t i o n t h a t s p e e c h u n d e ~ s t a c r e a s e s r a p i d l y a f t e r e v e n a s h o r t e x p o s u r e . mmand r e p l a y s t h e l a s t s e n t e n c e more s l o w l y , w h i e more i n t e l l i g i b l e , a n d a l l o w s t h e r e s t o f t h e d i a The s e c ~ n d i n v o e a t

T h i s s y s t e m s u p p o r t s a u s e r p o p u l a t i o n o f a b o u t 30, s f w h i c h h a l f a r e o c c a s i o n a l a n e r c a l l i n t h r e e t o t e n o r more t i m e s p e r w e e k , all r e s e a r c h g r o u p , many u s e r comments w e r e o f f e r s u g g e s t i o n s i n c o r p o r a t e d i n t o s u c c e s s i v e v e r s i o n s o f t h e m a i l f a c i l i t y .

e s s g e n e r a l l y fsun o k e menu s y s t e m e q u a t e a n d e a s y t o e u s e r s s i m p l y l i s t e n

t o t h e i r m e s s a g e s i n w i l l d o t h i s e v e n i f no k e y s a r e n u s e o f t h e r andom a c c e s s and

In a n e a r l y v e a i l , i n w h i c h o n l y t e x t y l i t t l e u s e was made o f t h i s

n e r a l i n t e r e s t i n n i f i c a n t p o r t i o n

o f t h e c e l l s i n a r e p r i m o r t h e p u r p o s e o f l e a v i n g a s a g e f o r a n o t h ; t h e s e m e s s a g e s a r e g e n e r -

r i c h e r t h a n t t i f a r e c e p t i o n i s t t r a n - s c r i b e s a m e s s a

f o r u s e r s o f a n e x i s t i n g d e m o n s t r a t e s t h a t a p p r o p r i a t e

c h n l g u e s a s w e l l a s i n c o r p o r a t i o n o f u s e r o f command s t r u c t u r e c a n r e n d e r $Be means o f r e m o t e a c c e s s i n a

on t e l e c s m m u n i c a -

Barry Arons an Caren Baker, as graduate and undergraduate students respe contributed th software and design input to the Vo ail system. bers of the Architec- ture Machine uinea pigs and in turn offered additional in lected in the system's final design.

Portions of this work were funde TT, the Nippsn Tele- graph and Telephone Public Cor oration. Speech Plus assisted this project with hardware support.

1 . Schmandt, C. and Arons, B A Conversational Telephone essaging System. In Digest of Technical Papers. IEEE International Conference on Consumer Electronics, 1984.

2. Schmandt, C. and Arons, B. Phone Slave: A Graphical Telecommunication Interface. In Digest of Technical Papers. SIR International Symposium, 1984.

3. Miller, G.A., Heise, G.A. and Licten, W. The intelligibility of speech as a function of the context of the test materials. Journal of Experimental Psychology 4 1 :329-3359 1951.

4. House, W.S., Williams, C.E., Hecker, M.H.L., and Lryter, K.D Articulation testing methods: consonantal differentiation with a closed response set. Journal of the Acoustical Society of America 37~158-166, 1965

5. Slowiaczek, L.M. and Pissni, D.B. Effects of practice on speech classification o natural and symthetic speech. Journal sf the Acoustical Society of America 7 1 , 1982.

6. Luce, P.A., Feustel, T.C., and Pisoni, D.B. Capacity demands short-term memory for symthetic and natural word lists.

a n F a c t o r s 2 ( 1 ) : 1 7 - 3 2 , 1983

cPeters, D . L . and T h a r p , A.L. sf rule-generated s t r e s s o n computer

achine S t u d i e s

Chris Schmandt Research Associate

Nedia Laboratory Massachusetts Institute of Technology Room 9-516 77 Massachusetts Avenue Cambridge, MA 02139

Mr. Schmandt received his B.S. in Computer Science and his Y.S . in Computer Graphics from M.I,T. He has continued there-as a Research Associate at the Architecture Yachine Grou~, a component of the new Media Laboratory. His research interests there are focused on interactive systems and human-interface issues, with emphasis on voice interaction and telecomunieations. Some of the concepts presented in this paper are being developed by the author in conjunction with Active Voice of Seattle, WA.