Social Aspects of Joint Actions:from analysis to design of social actions

Virginia Dignum1, Frank Dignum2, and Catholijn M. Jonker3

1 Delft University of Technology, The Netherlands, M.V.Dignum@tudelft.nl2 Utrecht University

3 Delft University of Technology

Abstract. Social interaction has essential effects on social reality, whichcan even outweigh the importance of the physical effects, but which arenot directly nor objectively observable. Understanding the social contextsin which actions and interactions take place is thus of utmost importancefor planning one’s goals and activities. We claim that joint action shouldbe considered from both the social and the physical perspective jointly.These aspects are interdependent and influence each other continuously.In order to support the inclusion of social aspects into state descrip-tions for joint actions, we propose a methodological approach for socialanalysis of joint action, that enables to identify and represent the socialcharacteristics of a joint action setting. Through the use of social prac-tices it is possible to combine both physical and social aspects of jointactions. We show how these social practices can be used to design agentsand robots that take into account both the social and physical contextand goals.

1 Introduction

Coordination is essential for the success of joint action. This success is not just aconsequence of the interleaving of physical actions by the participants, but is fora large part dependent on common knowledge amongst the actors to anticipateeach other’s behaviour, thus increasing the effectiveness of the joint activity.Moreover, social interaction has essential effects on social reality, which can evenoutweigh the importance of the physical effects, but which are not directly norobjectively observable. This is an essential aspect of joint action when designinghuman-robot joint action.

Understanding the social contexts in which actions and interactions takeplace is of utmost importance for planning one’s goals and activities. Whereaspeople are pre-eminently able to understand context, robots and other computersystems are notorious for their inability to do so in general. Joint action in humansettings has been a topic of intense research in the social sciences, cognitivepsychology and philosophy.(see e.g. [6, 14, 19, 15]). In these areas the emphasisis often on the exact nature of what constitutes a joint action. E.g. are thepassengers of a bus participating in a joint action? Are the drivers of the carsin a traffic jam performing a joint action? All of them have at least a partly

shared goal related to the activity they all perform at the same time. Althoughthese are important and interesting questions, in this paper we will avoid thisdiscussion and assume it is clear that the robot and human are performing a jointaction. There is also a substantial amount of work on the mental conditions thatshould hold for performing a joint action. This relates to having joint intentions,joint goals, common beliefs, etc.(see e.g. [1, 7]). The main issue here is that iftoo many joint mental attitudes are required there will never be joint actions,while if the requirements are slackened behavior might result that would neverbe expected in joint actions. Again, in this paper we will not seek to answer thesequestions, but rather assume some shared beliefs and attitudes when necessaryand thus create a very pragmatic framework. In the area of Multi-Agent Systems(MAS) the formal aspects of joint actions, coordination, cooperation and taskdependencies have been extensively studied (e.g. [18, 10]), but mainly from thepoint of view of objectively observable effects. In this paper we will complementthis work by also taking into consideration the social aspects that are not directlyobservable.

We claim that joint action should be considered from both the social andthe physical perspective jointly. These aspects are interdependent and influenceeach other continuously. In order to support the inclusion of social aspects intostate descriptions for joint actions, we propose a methodological approach forsocial analysis of joint action, that enables to identify and represent the socialcharacteristics of a joint action setting. The aim of this social analysis is to catchthe essential elements from the social context that are needed to model the jointaction. For the case of joint action between people, this has been extensivelyresearched in Sociology and Psychology [11]: “Just as the picking up of infor-mation about our environment is fundamental to the performance of action [...],so too, we argue, is an individual in a social context fundamentally constrainedby the picking up of information about others’ incidental movements and inten-tional actions. Recognizing that our movements and actions in the world are asconstrained by others as, within our body, implies (a) a universality of dynamicalprinciples that unify components in a system - they are true for social as wellas for non-social action, (b) that linkages are not simply mechanical, but can beinformational - that what we take in with a look (or other modality) can affectour behaviour as strongly as a mechanical force, and (c) how others ‘moor’ us inspace and time defines the frames of reference for our past, present, and futurebehaviour.”.

In order to illustrate our proposal, we take as working example a blockworldscenario where a person and a robot have to jointly build a tower of four cubeswith a triangle on top. Both have two cubes and a triangle to contribute. Eventhough, this description is given purely in physical terms, in the real world, jointaction is not just about the functional end result (in this case, the completedtower), but also about what that result means to the participants, i.e. the socialand emotional results of the action. That is, in order to make joint action possi-ble, one needs not only to describe the physical context as in the example, butthe social, mental and behavioural conditions under which joint action emerges

for socially embedded individuals, and what kinds of patterns of behaviour arepossible [11].

In the case of the example, consider that the description would contain some-thing such as: both the person and the robot should feel satisfied (or happy) afterthe tower has been built. Or: the robot and person have never met before andthus do not know what the other is capable of or wants to contribute to thejoint action. These additions to the state descriptions will probably change whatwe see as acceptable ways to perform the joint action. These aspects becomeeven more prominent when both actors each have four cubes and could thus, inprinciple, build the whole tower alone. Or if they have only three cubes togetherand will never manage to build the tower according to the specification. Whatwould we expect the actors to do in these cases? The answer will depend on thesocial context and assumptions we make about the actors and their relation.

The paper is organised as follows. In the next section, we will look at thesocial analysis of joint actions and present a preliminary check-list of essentialelements that should be taken into account. In Section 3, we present preliminarymethod for social analysis of a situation, by providing a check-list of issues toconsider. In section 4, we show how the concept of social practices can be usedto combine the social and physical parts of a situation for concrete plans ofactions and mutual expectations about the joint action. In section 5 we discusshow this can be used for the design and deliberation of robots that interact withhumans. We finish the paper with some preliminary conclusions and future workto explore this area.

2 Social Analysis of Joint Actions

In sociology, social analysis is the systematic exploration of social issues, relatedto the general quality of life, social services and social justice of a society. Wedefine social analysis of joint action as the evaluation of issues related to thequality and results of joint action. It includes the identification of the socialcharacteristics, requirements and expected results of an interaction, as well asthe social meaning of the physical elements of that interaction. In this section,we will look at aspects of social analysis as they apply to joint action. I.e. wegive arguments why some aspects are relevant for joint action between humansand thus also might influence the joint actions between humans and robots.

In order to focus the discussion and give some concrete example of a joint ac-tion we use the description of a joint action in the blockworld scenario describedin the Introduction. It is clear that this description is not sufficient to guaranteethat the joint interaction between human and robot will succeed. For instance,there is no mention of social relations and status between the two actors, past ex-periences, assumed and actual capabilities, mental images each may have of thesituation, etc. The example also does not consider the consequences of successor failure. What is at stake? How can failure be countervailed if needed?

A social analysis of the situation looks at three different aspects of the socialsituation: the (general) social context, the personal social relations between theactors and the social interpretation of the physical situation.

Firstly, we explore the social context in which the scenario takes place, iden-tifying the relationships between the human and the robot and also the widersocial setting in which the joint action takes place. E.g. does it take place atschool, at home, in a research lab, at a demonstration of robots, etc.? This widersetting gives a common background and expectation about the joint action. Isit something they rarely perform together and thus the actors have little ex-perience and knowledge of what to expect from each other. Or is it an actionperformed as a demonstration and well rehearsed?

Within each setting, actors take up social roles. E.g. in the setting of a re-search lab it may be natural for the human to tell the robot what to do ateach step; whereas in a school, we might expect the robot to take the initiativeteaching a child to build a tower. As can be seen from the example the mostimportant aspect of the social context are the social roles and their (contextual)definition. Social roles describe aspects of a setting that lead to standard expec-tations for most people involved. A teacher has as objective to teach studentssome skills or knowledge. The teacher also follows norms such as being patientwhen mistakes are made and be supportive in the learning (answering questions,examplify, etc.).

Next, the personal social relations should be analysed. These include theexpectations of each actor concerning the result fo the joint action. E.g. it maybe important that all actors are happy at the end of the interaction. This isparticularly important in case the joint action is part of a longer interactionor relation. So, we need to know about the nature and expectations about thepersonal relation between the actors. Do they know each other? What is theirjoint experience? Did they perform this type of action together already? Dothey expect to build up (or continue) a social relation? The latter leads to awillingness to ”invest” in a joint action with the expectation it might lead tobenefits later. If there already is a common history the relative social statusis taken into account to determine how much to ”invest” in the current jointaction.

Finally, we need to analyse the (possible) social interpretation of the physicalsituation, actions, objects and actors. In the example scenario there might notbe many, because it is relatively simple, but we will give a few examples of eachaspect.Physical situation. The initial position of the blocks on the table suggests thatboth the human and the robot “own” two cubes and a triangle and thus candecide what to do with them. Another example is the time given for the taskor the time of the day at which it takes place which can strongly influence theinteraction. E.g. if little time is available in general the more experienced actorwill take more initiative in order to be finish in time. Also, the relative positionand measure of the actors and objects is relevant for the social situation. E.g.

imagine a huge robot and a small table with tiny blocks, or a situation in whichthe person is too far from the table to be able to handle the blocks.

Actions. The type and order of actions also have a social connotation. E.g.”taking turns” can be seen as cooperative behaviour. Or, being the one puttingthe last block in place can be interpreted as asserting the social status. I.e. theone finishing the job is the most important. Attaching social interpretations tothe actions will allow to reason about the social effects of the actions that areperformed in the joint action.

Objects. The physical attributes of the objects and the (expected) capabil-ities of the actors can lead to specific expectations about the joint action. E.g. ifthe robot is small and can hardly reach the top of the tower it seems logical toexpect the human to finish it (if that one is taller). Similar things can be statedabout the size, weight, colour, etc. of the cubes. If the robot cannot distinguishthe colours while the cubes should be stacked based on colour it is expectedthat the human should assist. Note that the actors are not always aware of thecapabilities (or limitations) of the other actors and thus the joint action shouldinclude the possibility to exchange information about these aspects.

In the next section we will describe a first version of a check list of socialaspects that should always be considered when performing a social analysis andwhich can be used to design the robot for particular types of joint actions.

3 Check-list for social analysis

Based on the discussion in the previous section we now try to give a more con-crete check list for the social analysis. The use of such a check list is twofold. Itshould be considered at the start of the joint action and also during the jointaction.At the start of the joint action it should give insight into aspects that should beconsidered when a joint action is performed. Not in every joint action all aspectswill be relevant. The more limited the interaction possibilities, the less relevantare the social aspects. E.g. in an English auction most aspects are pre-determinedand can be taken for granted by the actors. Thus they hardly influence the waythe interaction is performed or the outcome of it. When the relevant aspects aredetermined it should be established whether they are known to the participants.If aspects are unknown, the actor can assume default values for them or make ita point to acquire the information before or during the joint action. E.g. if thehuman does not know whether the robot is capable to reach the top of the towerit can assume it can (because it would otherwise not start the joint action) orthe human can ask the robot whether it thinks it can reach that high.During the joint action the check list is used again whenever unexpected eventsoccur that necessitate adjustment of the joint action (plan). In this case it func-tions as background knowledge that leads repair actions (often involving infor-mation exchange about expectations).

In order to give some structure we divide the social characteristics in threeparts: the situation, the actors and the expected/desired results.

1. Situation

(a) Physical characteristics that impact the social situation

(b) History (past encounters): have the robot and the person interactedbefore? How was that experience? What are the safety measures in theenvironment?

(c) Resources: do all actors have access to all resources? How is ownership?

(d) Understanding social situation in which the action is to take place

i. Individual goals & motives of the actors: do the actors have their owngoals for the joint action? What are their motives for those goals?

ii. Joint goals & motivesiii. Shared awareness: are the actors aware of each other’s goals and

motives? Can they reason about each other?

2. Actors

(a) Capabilities: can each of the actors take all the necessary actions or arethey dependent on each other?

(b) Physical appearance (size, shape,...): Relative size of robot, blocks andthe human (huge robot, small human, small blocks, vs. tiny robot andblocks)

(c) Emotional state: does the human feel intimidated, dominating, superior,etc. to the robot?

(d) Trust: have the actors bases to trust or mistrust each other?

(e) Social network, relationships: are there relationships between the actors?Is there a team of several actors or just a pair? How is the balance human-robot in the team?

(f) Social status (power, authority...)

3. Results

(a) Consequences of success (for each individual and for the group)

(b) Consequences of failure (for each individual and for the group)

(c) Risk assessment measures

(d) Risk mitigation (alternative plans)

This check list is based on the well-accepted idea that the success of a jointaction depends on the actors performing it, the physical possibilities to performthe joint action and whether the goal of the joint is achieved for all actors.However, the check list now includes the social aspects that are involved ineach of these elements of the joint action. It is clear that including the socialaspects considerably grows the check list, which shows the importance of thesocial aspects for the success of joint actions. We do not claim that the abovecheck list is complete, but in our experience the listed aspects always play a rolein joint actions and thus should be part of any minimal check list. In the nextsection, we provide a framework to represent joint actions in a way that combinesphysical, personal and social aspects, using the theory of social practices. Theresults of this check-list are used as the input to the instantiation of joint actionas social practices.

4 Social Practices

Social practice theory seeks to determine the link between practice and contextwithin social situations [17]. Social practices refer to everyday practices and theway these are typically and habitually performed in (much of) a society. Suchpractices as “going to work”, “meeting”, or “greeting” are routinely performedand integrate different types of elements, such as bodily and mental activities,material artefacts, knowledge, emotions, skills, and so on [13]. In fact, socialpractices can be seen as recurrent joint actions performed for shared social rea-sons [19]. Social practices are similar for groups of individuals at different pointsof time and location. As such, they can be seen as ways to act in context, i.e. oncea suitable practice is identified, people will use it as a ‘short cut’ to determinean action which does not require elaborate reasoning about the plan to follow.However, social practices are not just mere scripts in the sense of [12]. They sup-port, rather than restrict deliberation about behaviour. E.g. the social practiceof “going to work” incorporates means of transport that can be used, timingconstraints, weather and traffic conditions, etc. Maybe, normally you take a carto work, but if the weather is exceptionally bad you can deliberate a new planin this situation and take a bus or train (or even stay home). That is, differentsituations give rise to other ways of enacting a social practice.

Although we will not go into the theory of how social practices arise and aremaintained (see [16] for more on this aspect) a very important aspect of themis that they are a kind of shared way of acting in context. Thus people use asocial practice also to form expectations about the roles and actions of the otherparticipants in the social practice. E.g. when a person extends his hand to greetanother person, she will expect that the other person will grab the hand andthey start shaking hands. Thus, by their very nature, social practices are usedto standardize interactions and provide a context in which expectations are setand social interpretations are given for the actions taking place.

Based on these ideas, we developed a model to represent social practices thatcan be used in social deliberation by intelligent systems [3]. The components ofthis representation model are as follows:

– Purpose refers to the expected physical and social results of the practice– Physical Context describes elements from the physical environment that can

be sensed:• Resources are objects that play a role in the practice such as cubes and

triangles in the scenario.• Places indicates where all objects and actors are located relatively to

each other, in space or time.• Actors are all people and autonomous systems involved, that have capa-

bility to reason and (inter)act.– Activities indicate the normal activities that are expected within the prac-

tice. Not all activities need to be performed! They are meant as potentialcourses of action.

– Social Context contains:

• Social Interpretation determines the social context in which the practiceis used.

• Roles describe the competencies and expectations about a certain typeof actors.

• Norms describe the rules of (expected) behaviour within the practice.

– Plan Patterns describe usual patterns of actions defined by the landmarksthat are expected to occur.

– Meaning refers to the social meaning of the activities that are (or can be)performed in the practice. Thus they indicate social effects of actions

– Competences indicate the type of capabilities the agent should have to per-form the activities within this practice. They include both the physical aswell as the mental capabilities.

Although we included the purpose component in the social practice this doesnot mean that the reason for performing the social practice is that purpose. Itcould be that the reason for the joint action of stacking blocks together is thatthe robot and human have a good time or explore each others capabilities. Thephysical result might thus be accidental to the activity. The reverse might alsobe true and the physical result might be the only important goal that happensto be satisfied (easy) through this social practice. Thus the purpose of a socialpractice should be seen more as a potential result that could match a goal ofthe participants of the social practice. However, in the more extreme case theparticipants perform the social practice as a goal in itself. As we can see infigure 1 the meaning of the social practice of stacking blocks by instruction canbe the fulfillment of the power motive of the person. Thus the social practicecan serve as an activity performed for a motive while the actual results of thesocial practice do not play a role in initiating it. The latter is closer to thetheory of social practices from social science where often it is assumed that thereason to perform a social practice is exactly the performance of it. However, thiswould assume that actors are mindlessly performing social practices and haveno goals of themselves, which seems contradictory to our experience. Thus wetake a less extreme stance in which we assume that social practices can be usedto (partially) fulfill goals of the actors, but also can be initiated based on theirmeaning and thus be performed directly for the satisfaction of their performance(”without thinking about them”).

Figure 1 shows how the above framework can be used to describe the socialand physical aspects of a joint action to stack blocks together. The two columnsshow different choices of social practices that could be employed to stack blockstogether. The components in the table are filled in based on the results of thesocial analysis described in Section 3.

As said before, in Sociology, social practice theory is mostly used as inquiryon how social activity occurs and identifies its main causes and outcomes. Ourrecent work [3] proposes to use social practices also as a model for action in(social) context. Thus we try to combine a goal directed deliberation of agentswith a standard/normative performance of social practices. In order to makethis combination we have to define the connection and use of social practices

Social practice Block stacking by instruction Block stacking by teamwork

Purpose Physical

Social

Stack of 4 cubes with pyramid in top, placed on top of the table Robot and person satisfied

Physical Context Resources

Places Actors

Blocks: cubes and pyramids; table Blocks are initially on the table; person and robot at opposite ends of the table Person, robot

Activities Pick-up blocks, stack blocks, communicate

Social Context Social

interpretation

Roles Norms

person feels superior to robot; follow-orders as cooperative behaviour Person is the leader, robot the subordinate Person decides order of blocks Robot follows person orders In case of problems, person is responsible for alternative plan

person is not intimidated by robot; turn-taking as cooperative behaviour Teammates Communication expected to determine who starts In case of problems, each can suggest action or alternative

Plan patterns Person decides stacking plan Person requests robot to place block If robot unable, then person will stack blocks If block falls, person decides on alternative

Person and robot agree on who starts Taking turns, each stacks a block If block falls, they will agree on alternative The one appointed takes the alternative action

Meaning fulfilment of power motive for person Cooperative behaviour

Competences Physical

Choice

Robot has limited range cannot reach tower top Person can reach top of tower Person can pick blocks from the ground Robot follows all orders from person Person knows how to make building plan

Robot has limited range cannot reach tower top Person can reach top of tower Person can pick blocks from the ground Person prefers to use robot blocks first Person and robot know building plan

Fig. 1. Joint action as Social Practice: different interaction styles

with the deliberation of agents. The general process for joint action using socialpractices would include the following steps:

– Perform a social analysis of the context– Perform a physical analysis of the context– Match a social practice that fits both the social and physical parameters– Within this social practice determine a course of action– Monitor the environment for the expected interactions during the joint action– Adjust to any deviations based on the information of the social practice

For the purpose of human-robot interaction, the above can be used in twoways. In the ideal case, robots would be equipped to perform all the above stepsautonomously. In order to get to that point they would have to use a cognitivearchitecture we have coined social agent in [5]. Although we cannot assume cur-rent robots to perform according to this vision yet in the next section we discusssome first steps towards designing a robot that uses social practices to performjoint actions. However, the above process can also be used by the designer of

the robot to guide building the capabilities of the robot. Due to its high levelof abstraction describing activities, social practices can facilitate the specifica-tion of flexible behaviours, e.g. describing ‘counts-as’ relations between low-levelrobot functions and high-level social behaviours. For example, programming arobot to wait for instructions before picking-up a block, is a way to representthe social behaviour of ‘subordinate’ as described in the column ‘Block Stackingby Instruction’ in Figure 1. Thus the social practice description leads to thespecification of specific capabilities for the robot that can fulfil its role in thesocial practice.

Another important aspect that is directly supported by the social practicedescription is the analysis of three important factors in human-robot interaction:Observability, Predictability and Directability (ODP) [8]. I.e. we need to knowwhat is observable by the participants, which actions of the participants arepredictable, and which actions can be delegated and how this can be done toeach participant.Observability analysis is supported by the description of the physical contextin the social practice. From this description one might deduce which objectsand actions are/will be visible by the participants and possible remedies to keepcrucial actions observable for the relevant actors.Predictability analysis is supported by several elements of the social practice.First of all it indicates the activities and plan patterns that determine the socialpractice. These, of course, directly lead to expectations of behaviour. Besidesthese elements, the roles also lead to expectations of responsibilities, rights andnorms for the actors that fulfil those roles.Directability analysis is supported by the roles which indicate which actor istaking the lead at each point and the plan patterns indicate which dependenciesexist between the actors in the social practice and thus at which points one actorshould be able to direct the other. One can also check whether capabilities existwith the actors to realize this directability. E.g. if a command should be giventhere should be a common language in which the command can be given andactuators and sensors to send the command and receive it (in time).

Finally, the description of social practices for the joint action supports theinterdependence analysis of the joint action. Awareness of the interdependencerelationships between the actors is important for designing robots because theactivities performed by one actor may influence the activities of the others.To analyse the interdependences in a joint activity, [8] presents a tool calledthe Interdependence Analysis (IA) table, which uses a colour scheme to helpidentify the interdependence relationships and describe how one agent supportsthe other in teamwork. We suggest to use a tool such as the IA table to refine theactivities and identify the interdependency relations. The designer of the jointactivity and/or robot can take into account known limitations and capabilitiesof the robot, and design communication and interaction protocols to supportteamwork.

Depending on circumstances, the designer can also equip the robot withhardware (sensor, communication devices, grippers, etc.) to support the joint

Context observation

Identity

beliefs

Reasoning processes

goals

Concrete social practice

Social practices

Plan generation

Evaluation

Plan execution

selection

revision

adapt update

trigger

salience

Fig. 2. Social reasoning process

activity and the teamwork to support that joint activity. For example, the turn-taking capability asks for some way in which the robot can know when it has toput a block on the stack. Such capability can be autonomous, e.g., by increasingthe sensory capability, or be dependent on the other actor. In the last case, therobot and the other actor need communication capabilities and the language toexchange the necessary information for turn taking.

5 Social Practice Directed Behaviour

In order for the robot to be able to use the social practices in its deliberation forjoint actions, it needs an architecture that allows for both internal motivations(or goals) to drive behaviour as well as external situations and practices tosuggest behaviour. In [3], we proposed a deliberation architecture for agentsbased on social practices, depicted in Figure 2.

In this architecture plan generation is driven by both the beliefs and goalsas well as by the applicable social practices. When the interpretation of thesensed environment in terms of the existing social practices, results in only onepossible action, the agent will perform that action directly. However, in casesthat there are several possible courses of action, the agent will take into accountits motives in order to determine possible goals that are applicable to the sensed

environment and generate a plan accordingly. This distinction resembles the fastand slow reasoning tracks as described by Kahnemann [9]. However, due to thecontext of the social practice the robot can limit the portion of the applicablerules and beliefs and also can use specialized rules for planning (patterns) thatare (only) applicable within the social practice and the social identity it has whenplaying a certain role within that social practice. The deliberation componentitself can be based on BDI or extensions thereof, such as the FAtiMA [2] orBRIDGE [4] deliberations containing emotions, goals, intentions, beliefs, roles,identity, etc.

This architecture assumes that sensing is not a passive but also an activeprocess. Active social practices direct the agent’s sensing to find objects, actorsand events that are expected within that social practice. This leads to reactivebut focused sensing based on the current situation. The drive to search for pat-terns (and thus the sensing process) can be steered from the agent’s identities.E.g., a robot with Follower identity will wait patiently for requests by the per-son, while a robot with TeamPlayer identity will actively check for changes ofthe stack of blocks in order to know whether it already can put another block.I.e. the parallel tracks of pro-active and reactive behaviour already start withthe sensing behaviour.

The architecture also show that there are feedback loops between the eval-uation of actions and both the social practices as well as the robot’s mentalattitudes. When a social practice was very successful it’s use is reinforced andit’s salience in similar circumstances will be higher. If the social practice did notwork very well it can lead to an adjustment of the social practice or giving it alower priority. In a similar way the identity of the robot and some of it’s mentalattitudes will be reinforced.

For the use of social practices it is important for the robot to determine it’srole in the social practice. This should be matched with the social identity of therobot and with possible roles in the practice. In the block stacking example therobot might assume any role, but if the robot has a servant identity with respectto the human it might not be able to match with the leader role in the blockstacking. In general the robot can follow the following (simplified) algorithm todetermine which role it should play in this particular social practice and whichrole the human will play:

In the algorithm we use expected(P, Leader) to denote a function that de-termines whether the human can be expected to take the leader role. This canbe based on past experiences, the current social identity of the human, but ulti-mately also on the human giving a first order right away. A second point to noteis that the robot has a compliant nature in that it will only take the leader roleif the human does not want it. Of course, more subtle versions could be madeof the algorithm that make this dependent on the robot’s previous experiences.

Once the robot has established it’s role in the social practice and therewithalso the role of the human it can start planning it’s actions according to the planpatterns that are assumed in the social practice. Important in this respect is thatthe social practice already indicates what can be expected of the other role(s)

Algorithm: social-identity(P)

if known(P) thenif expected(P, Leader) then

enact(Follower)else

enact(Teamplayer)end

elseif askPreference(P, Leader) then

enact(Follower)else

if askPreference(P, Team) thenenact(TeamPlayer)

elseenact(Leader)

end

end

endAlgorithm 1: Determination of Social Identity

and the plan can use the expected interaction points for efficient planning. Italso indicates exactly what aspects of the environment should be monitored forexpected actions and communication and thus when failures can be detected.In the following two figures we show the combined plans of the robot and thehuman for each of the two block stacking social practices that the robot couldchoose from.

Notice that the activities in these diagrams can still be very abstract and de-tailed plans made in several ways. Given the social practice of teamwork stackingthe pattern that is given for the robot where it will stack a next block after thestack has changed (due to the human stacking a block) has an inherent socialeffect of being cooperative. Thus the robot does not have to reason separatelyabout this social effect of the plan, it is packaged with the social practice. Simi-larly the passive waiting in the other social practice is not interpreted as beingdisinterested but in this context shows cooperative behaviour.

6 Conclusions

In this paper, we discuss the effect of social issues on joint action. We arguethat for successful joint action, not only is it necessary to describe the physicalaspects of the domain, but also, and primarily, its social characteristics. Weintroduce an initial check list for social analysis that can guide the designers ofjoint actions through the identification of the social characteristics of the domain.The methodological check-list proposed in this paper supports the identificationof what are the aims and steps of the joint action, but not how to achieve them.

Social practices, as discussed in section 3, describe the social meaning of phys-ical action and can therefore be used to describe how to act physically in ways

Rob

otP

erso

n

monitor tower

place top

stack block

monitor tower

request help

changed?

pick up blocks

can stack

crashed?

request toptop?

stack block

changed?

can stack

crashed?

top?

Fig. 3. Stacking blocks as teamplayer

that are socially acceptable, and enhance the social relationships. We propose touse the social practice architecture proposed in [3] to specify human-agent-robotinteraction in ways that are computationally interpretable and therefore be usedto design artificial agents and robots that interact with people.

We concluded the paper by providing an initial description of how socialanalysis and social practices can support the deliberation design of the robot inorder to participate in human-robot interactions.

This paper presents initial work on this topic. Immediate essential steps willbe on the validation of the social analysis check-list, extending and modifyingit by application to several case studies. Future work will focus on the furtheroperationalization of social practices for the design of robots capable of socialjoint action.

References

1. P R Cohen and H J Levesque. Teamwork. Technical report, 1991.

2. J. Dias, S. Mascarenhas, and A. Paiva. Fatima modular: Towards an agent ar-chitecture with a generic appraisal framework. In Proceedings of the InternationalWorkshop on Standards for Emotion Modeling, 2011.

3. F. Dignum and V. Dignum. Contextualized planning using social practices. InProceedings of COIN 2014, 2014.

4. F. Dignum, V. Dignum, and C.M. Jonker. Towards agents for policy making. InMABS IX, pages 141–153. Springer, 2009.

Rob

otP

erso

n

Stack request

not last

place toplast block

stack block

observe tower

check placement

request help

stack possible

crashed

wait

pick up blocks

Fig. 4. Stacking blocks as follower

5. F. Dignum, R. Prada, and G.J. Hofstede. From autistic to social agents. In AAMAS2014, May 2014.

6. Margaret Gilbert. On Social facts. Routledge, 1989.

7. B Grosz and S Kraus. Collaborative plans for complex group actions. ArtificialIntelligence, (86):269–358, 1996.

8. M. Johnson, J.M. Bradshaw, P.J. Feltovich, C.M. Jonker, B.M. van Riemsdijk,and M. Sier-huis. Coactive design: Designing support for interdependence in jointactivity. Journal of Human-Robot Interaction, 3(1):43–69, 2014.

9. D. Kahneman. Thinking, fast and slow. Farrar, Straus & Giroux, 2011.

10. V. Lesser, K. Decker, T. Wagner, N. Carver, A. Garvey, B. Horling, D. Neiman,R. Podorozhny, M.Nagendra Prasad, A. Raja, R. Vincent, P. Xuan, and X.Q.Zhang. Evolution of the gpgp/tms domain-independent coordination framework.Autonomous Agents and Multi-Agent Systems, 9(1-2):87–143, 2004.

11. Kerry L. Marsh, Michael J. Richardson, and R. C. Schmidt. Social connectionthrough joint action and interpersonal coordination. Topics in Cognitive Science,1(2):320–339, 2009.

12. Marvin Minsky. A framework for representing knowledge. In Allan Collins EdwardSmith, editor, Readings in Cognitive Science, pages 156 – 189. Morgan Kaufmann,1988.

13. A. Reckwitz. Toward a theory of social practices. European Journal of SocialTheory, 5(2):243–263, 2002.

14. Michael Schmitz, Beatrice Kobow, and Hans Bernhard Schmid. The Backgroundof Social Reality. Springer, 2013.

15. Natalie Sebanz, Harold Bekkering, and Gnther Knoblich. Joint action: bodies andminds moving together. Trends in Cognitive Science, pages 70–76, 2006.

16. E. Shove, M. Pantzar, and M. Watson. The Dynamics of Social Practice. Sage,2012.

17. A. L. B. Smolka. Social practice and social change: activity theory in perspective.Human Development, 44(6):362–367, 2001.

18. Milind Tambe. Towards flexible teamwork. Journal of artificial intelligence re-search, pages 83–124, 1997.

19. Raimo Tuomela. Philosophy of Social Practices: A Collective Acceptance View.Cambridge University Press, 2002.