Instant Messaging and Haptic Presence

Riki Kawakami

University of Tampere Department of Computer Sciences Computer Science / Int. Technology Seminar "Haptic Communication and Interaction in Mobile Contexts" December 2008

i

University of Tampere Department of Computer Sciences Computer Science / Interactive Technology Riki Kawakami: Instant messaging and haptic presence Seminar paper, 14 pages December 2008

Key words and terms: haptics, instant messaging (or IM), presence awareness, computer-mediated communication.

ii

Contents

1. INTRODUCTION .......................................................................................................1

2. INSTANT MESSAGING............................................................................................1 2.1. INSTANT MESSAGING BASICS...............................................................................1 2.2. MOBILE INSTANT MESSAGING .............................................................................2 2.3. POPULARITY OF INSTANT MESSENGERS...............................................................3

3. PRESENCE AWARENESS & INTERPERSONAL COMMUNICATION ..............3 3.1. PRESENCE STATUS................................................................................................3 3.2. AFFECT OF PRESENCE AWARENESS......................................................................4

4. HAPTICS AND INSTANT MESSAGING.................................................................5 4.1. CURRENT SITUATION............................................................................................5 4.2. HAPTICONS ...........................................................................................................5 4.3. HAPTICS IN INSTANT MESSAGE CONVERSATION .................................................6 4.4. HAPTICS AND PRESENCE.......................................................................................7

5. CONCLUSIONS AND SUGGESTIONS ...................................................................9

References......................................................................................................10

1

1. Introduction Instant messaging (or IM) is widely spread form of communication over a network, such as the Internet. An important factor in IM is presence awareness enabling the users to see the status of their contacts. The users find value in knowing who else is online even without further contact with them. Haptics in IM can augment the non-verbal cues, such as smileys, in textual communication. Applications incorporating haptics and instant messaging require additional hardware and frameworks, such as the HIM [11]. With the help of examples we review how haptics can enhance computer-mediated textual communication, solve some ambiguities and drawing attention when multitasking.

2. Instant Messaging

2.1. Instant Messaging Basics Instant messaging (also known as IM) allows users to communicate in real-time over the Internet or Intranet. Instant messaging allows text-based communication between two or more users through an instant messenger system. Instant messengers usually consist of the main window displaying the user’s contacts and windows where the real-time communications – or chats – take place between the user and the contacts (see Figure 1).

Modern instant messengers have support for voice calls, as well as for video conferencing. Some instant messengers also allow VoIP (Voice over IP) calls where the user is able to make phone calls over the Internet, and avoid (long-distance) call charges. A microphone and a web camera are required to use these services.

Individual accounts with the instant messenger service provider are required in order to use the service.

2

Figure 1. Chat window of an instant messenger

2.2. Mobile Instant Messaging Instant messengers are also available for mobile devices with Internet connection. The connection can be, for example, over GPRS or Wi-Fi connection. Mobile instant messengers also allow VoIP calls (see Figure 2.). Proprietary or third party software is used to enable instant messaging. Fring, one of the third-party instant messenger providers reports to have over 200 000 new users each month [4], which reflects the popularity of IM usage.

Figure 2. Mobile instant messaging by Fring

3

2.3. Popularity of Instant Messengers There are more than 385 million instant messenger users worldwide, according to ComScore Inc. [1]. The most popular instant messenger applications include Windows Live Messenger, Yahoo! Messenger, Skype and QQ [1,3,11]. Unfortunately, the total amount of users varies depending on the source. Also, the number of active users is not always distinguished from the total number of registered accounts depending on the source. Tencent Inc., a Chinese Internet service portal, have up to 822,2 million instant messaging accounts for their QQ instant messenger service but only 341,9 million of them are active [13]. It is worthwhile noticing that this figure would leave only 43 million users for rest of the world if subtracted from the 385 million users as reported by ComScore Inc. Despite this, it is obvious that instant messaging has a vast and ever-growing user base.

3. Presence Awareness & Interpersonal communication

3.1. Presence Status Instant messenger users are provided by awareness information of their contacts’ presence. This information is displayed in the main window (or contacts list) of the instant messenger system, under each listed contact (see Figure 3.). By monitoring the presence status, we can see whether a contact is online or offline. It is also possible to set a visual notification being displayed on the screen and an audio alarm to signal a contact coming online.

When users are logged into their instant messaging account, they are able to change their presence status displayed for other users. This is useful, for example, in business environment where setting the status from “Available” (usually the default status upon log in) to “Busy” blocks notifications – audio and visual – of an incoming messages from other users that might be disruptive. The messages will stay on in the background enabling them to be read later. By setting the status to “Busy”, not only does it block incoming messages but it also shows to the user’s contacts that this might be engaged.

Other presence status options include the following (variances between different instant messenger services can be found): “Do not Disturb”, “Away”, “In a Call” and “Invisible”.

Some instant messengers automatically switch users’ status to “Away” when they have been inactive for a predetermined time. This indicates that the user is possibly away from the computer.

“Invisible” status option allows the user to be online without any contacts being aware of this. For them the user with status set as “Invisible” is displayed as “Offline”. This is useful option in a situation where users are possibly busy and do not want

4

anyone to contact them, while they themselves might be expecting a certain contact to come online.

Figure 3. Presence status under each contact as indicated by the arrows.

3.2. Affect of Presence Awareness Dey and De Guzman discovered that instant messaging applications are used as a way to maintaining an awareness of the contacts, and not just to communicate with them [2]. This was backed up by a research from Nardi et al. who found that instant messenger users wanted to maintain their sense of connection with others, and that they found value in knowing who else was around without wanting to interact [8]. This shows that instant messaging is more than just a text-exchanging medium such as e-mail.

Paulos’ observations [9] support the findings [2,8], where presence awareness gives sufficient satisfaction and no further direct communication is needed.

In the context of work environments, about 60% of workplace phone calls fail to reach intended recipients because they were not at their place or they were already speaking to someone else [8]. Asymmetry in conversation arises when the recipient is engaged in another task or conversation when it would be convenient for the initiator to converse.

Instant messenger presence information allows the initiators to judge if the recipients are available for conversation. Instant messaging communication also allowed the recipients to decide when it was the most suitable for them to respond to the queries, or to ignore them.

Instant messaging was also found to be useful in organising a face-to-face meeting or a phone conversation with content found to be too in-depth for instant messaging. With the help of short instant message exchange, it is easy to organise the meeting time for a suitable time for both parties. In these kinds of cases instant messaging increases work efficiency because the initiators are able to view if the other party is available based on their presence status.

5

4. Haptics and Instant Messaging

4.1. Current Situation You are most likely to come across with haptics and instant messaging if you have IM client – such as earlier mentioned Fring – installed on your mobile phone. Fring enables the integrated vibrator alarm of the phone to be activated when an instant message is received (see Figure 4.).

Logitech’s iFeel vibrotactile mouse and force feedback joysticks could suit for haptic instant messaging use. However, it is not very common to have home computers equipped with haptic peripheral devices yet. Hence most of the products presented in the following sections are only prototypes.

Figure 4. Vibrating alert menu option of Fring mobile IM application

4.2. Hapticons

Hapticons can be described to be “small programmed force patterns that can be used to communicate a basic notion in a similar manner as ordinary icons are used in graphical user interfaces” [11]. It is not yet know to what extent hapticons can be used as an alternative to emoticons in instant messaging.

As emoticons, hapticons are to extend even further the loss of non-verbal cues in textual communication, and they can resolve some ambiguity arising during chatting.

A special output device needs to be connected to the computer to allow the vibration pattern output. The vibration pattern is consisted of frequency, amplitude and duration (see Figure 4.). Salminen et al. [12] claimed that the hapticons lack the ecological validity of the emotional qualities because they were based on the intuition of the authors themselves. However, Rovers and van Essen emphasise that a pattern becomes a hapticon only after it is recognised and associated with a special meaning [11]. They also allow users to create their own hapticons from scratch, which allows the users to have individual or private hapticons, unfamiliar to external users. (More information of the HIM framework in section 4.3.)

6

Figure 4. Some emoticons and proposed hapticons [11]

4.3. Haptics in Instant Message Conversation

4.3.1. Custom-made IO Finger-controlled Device Haptic Instant Messaging (HIM) is a framework, which uses haptic effects and hapticons. In a scenario where two users are engaging a chat, instead of sending a message to initiate the chat, the initiator tickles a touchpad input device, which activates the recipient’s output vibration device worn in the pocket. In this scenario haptics are being used to draw attention [11].

The chat proceeds and at some point the other user types in the big smile emoticon “:D” to express enthusiasm. This hapticon generates a haptic effect at the recipient’s vibration device. In this case, a fast vibration with increasing amplitude burst that ends abruptly (see Figure 4. for illustration of the big smile haptic effect).

The devices used in the experiment [11] were custom-made IO devices. The vibration device, which could be worn in the pocket, has a simple DC motor controlled by a microcontroller. The touchpad is a simple circuit board with pattern of lines etched onto it. When the board is touched, a signal is sent to the vibration device via a computer (see Figure 5.).

Figure 5. Custom-made devices. Vibration device (a) and touch pad (b)

7

4.3.2. FootIO – Foot Interaction Device FootIO is an output device that can be connected to the HIM framework [10]. FootIO enables hapticons to be delivered to a foot by using vibrotactile actuators.

The foot is suitable for haptic or tactile communication, although hands are mainly used for that. Feet are well suited to perceive tactile stimuli, although initial tests showed that accurately identifying the location and pattern of stimuli was difficult with the used prototype [10]. However, the use of FootIO was generally found to be enjoyable. From the mental point of view the foot is suitable for perceiving tactile stimuli because touching another person’s feet is regarded as intimate behaviour. Hence they are well suited for intimate communication. It is also suitable for concealed and hands-free communication because the device can be set under a computer desk where it is hidden from other people.

4.4. Haptics and Presence

As many reports say [8,2,7] presence awareness is found to be important for people and not only because it supplies status information of users’ IM contacts.

When you also consider that Rovers and van Essen. [11] found haptic information to be more direct and intimate manner of interaction than vision and hearing, it seems there is demand for haptic presence applications.

Salminen et al. [12] identify means for haptic stimulation, which could be used for haptic presence awareness. Prototypes have been developed for haptic emotional interaction. The working principle of these devices is to ease the feeling of social isolation, especially among elderly and lovers who are apart from their loved ones. The test participants have found them useful and pleasant.

They also investigated what emotional experiences and behavioural responses haptic stimulation evoked. For this they used their friction-based horizontally rotating fingertip stimulator prototype (see Figure 6.).

Figure 6. Prototype of fingertip stimulator

8

The test included different stimuli of burst length, continuity and direction (see Figure 7.). The results showed that continuous stimulation was less pleasant but more dominant and drew attention. These qualities were recommended to be utilised for high priority events when working in visually overloaded environments [12].

Discontinous stimuli were rated as more pleasant and approachable. It seems that these kinds of stimuli would suit better for presence awareness in instant messaging, while continuous stimuli could be used for drawing attention.

Figure 7. Fingertip stimulator and burst direction

The type of tactility is also important. Haans et al. [6] ran tests to measure can computer-mediated touch be perceived as social touch. The tactile stimulation was provided through a neoprene vest and two arm straps equipped with vibrotactile actuators (see Figure 8.). The actuators in the vest and in the arm straps were able to simulate poke and stroke-like touch. The results showed that the test participant experienced poke-like touches less pleasant than stroke-like touches. Touches on the stomach, arm and wrist were significantly less pleasant than touches on the upper and lower back regions. The results indicate that social touch can be perceived through mediated touch. However, the researchers emphasise that verbal and non-verbal cues might have considerable effect on physical touch. An aspect that was specifically isolated from the experiment.

9

Figure 8. Tactile vest (a-c) and arm straps (d, e)

The above-mentioned studies show that there are means to incorporate haptics to instant messaging and enrich the presence awareness, which has been found to be significant factor for IM users.

5. Conclusions and Suggestions We have seen examples of haptics being incorporated with instant messaging and tactile stimuli being produced in order to find out if machine-generated touch can be perceived as social touch. This is a significant factor because presence awareness has been found to be important for the IM users, and a correct type utilisation of tactile stimuli could enhance this.

Typical user setting for instant messaging is consisted of screen, keyboard and mouse. For mobile instant messaging the user setting would be a mobile phone with traditional keyboard or touch screen for text input. The example devices are not compatible by default because the HIM framework [11] and the fingertip stimulator [12] require additional hardware. The tactile vest [6] could be used in everyday situation but wearing the vest and the arm straps is probably not comfortable for an extended time. Hence further development is needed. The HIM framework and fingertip stimulator could incorporate similar technology to enable easy tactile stimulation for IM use.

If the tactile stimulation is meant to draw attention, for example in noisy working environment, the haptic device needs to be attached to the user to enable the stimulus to be noticed. This is a vital requirement. In this case the arm straps introduced by Haans et al. might be suitable if they are lightweight enough and comfortable to wear.

If the tactile stimulation is meant to convey social touch for enhanced presence awareness, for example, from a husband or a wife, it may not be vital to have the haptic device attached to the user. There is no vital importance to receive a social touch. The

10

fingertip stimulator could work in this scenario because its design is similar to that of a mouse. In fact, similar designs have been available previously in the form of Logitech’s iFeel vibrotactile mouse.

References 1. ComScore, Inc, 2008. Instant Messengers December 2007, 2008. Reston, VA:

ComScore, Inc. Available from: http://www.comscore.com/press/release.asp?press=2196 [Accessed 9 October 2008]

2. Dey, A. K. and De Guzman, E. S., 2006. From Awareness to Connectedness:

The Design and Deployment of Presence Displays. In: Proceedings of the SIGCHI conference on Human Factors in computing systems, 2006 Montréal. New York: ACM

3. Ebay Inc, 2008. Ebay inc. Reports First Quarter 2008 Results. San Jose, CA:

EBay Inc. Available from: http://news.ebay.com/releases.cfm?ReleasesType=Earnings&Year=2008 [Accessed 9 October 2008]

4. Fringland Ltd, 2008. Choose incoming call and message tone. Tel Aviv:

Fringland Ltd. Available from: http://www.fring.com/fring_is/what_is_fring/fring_features/#downSub7 [Accessed 9 October 2008]

5. Fringland Ltd, 2008. Fring® on iPhone™ App. Tel Aviv: Fringland Ltd.

Available from: http://www.fring.com/newsroom/fringAppStore.asp [Accessed 9 October 2008]

6. Haans, A., de Nood, C., and IJsselsteijn, W. A., 2007. Investigating response

similarities between real and mediated social touch: a first test. In: CHI '07 Extended Abstracts on Human Factors in Computing Systems, 2007 San Jose, CA. New York: ACM

7. Haans, A. and IJsselsteijn, W. A., 2005. Mediated social touch: a review of

current research and future directions. In: Virtual Reality 9, 2 (Jan. 2006), 149-159.

11

8. Nardi, B. and Whittaker, S., Bradner, E., 2000. Interaction and Outeraction: Instant Messaging in Action. In: Proceedings of the 2000 ACM conference on Computer supported cooperative work, 2000 Philadelphia. New York: ACM

9. Paulos, E., 2003. Connexus: a communal interface. In: Proceedings of the 2003

Conference on Designing For User Experiences, 2003 San Francisco. New York: ACM

10. Rovers, A. and van Essen, H, 2005. Guidelines for haptic interpersonal

communication applications: an exploration of foot interaction styles. In: Virtual Reality 9, 2 (Jan. 2006), 177-191.

11. Rovers, A. and van Essen, H, 2004. HIM: a framework for haptic instant

messaging. In: CHI '04 Extended Abstracts on Human Factors in Computing Systems, 2004 Vienna. New York: ACM

12. Salminen, K., Surakka, V., Lylykangas, J., Raisamo, J., Saarinen, R., Raisamo,

R., Rantala, J., and Evreinov, G., 2008. Emotional and behavioral responses to haptic stimulation. In: Proceeding of the Twenty-Sixth Annual SIGCHI Conference on Human Factors in Computing Systems, 2008 Florence. New York: ACM

13. Tencent, Inc, 2008. Tencent Announces 2008 Interim and Second Quarter

Results. Shenzhen: Tencent, Inc. Available from: http://www.tencent.com/en-us/at/pr/2008.shtml [Accessed 9 October 2008]