Investigating the Technical and Social Challenges Faced by Large Collaborating

Groups

Pedro Antunes, Gustavo Zurita, Nelson Baloian

Abstract. Our main research goal is qualitatively investigating the influence of large

groups on decision-making. We set up an interpretative case study of three large groups,

with 42, 44 and 48 participants, accomplishing a “choose” task with no right answer. The

investigated factors considered information overload, awareness, task involvement, and

collaboration support. Furthermore, we considered these impacts on two main components

of the group task: discussion and consensus. The results indicate that the participants

understand and deal with several trade-offs, notably between stress and immediacy, and

between feedback and noise. The participants avoid information overload using multiple

data reduction and attitudinal strategies. This study contributes to understand the various

trade-offs faced by large collaborating groups.

Keywords: Large Group Collaboration, Group Decision-Making.

1. Introduction

Research on group collaboration has traditionally favoured small groups. A basic search in

SCOPUS for articles from 2007 to 2016 with the keywords “collaboration” and “small

group”, “collaboration” and “large group” and “collaboration” and “crowd”; returned about

374227, 4381 and 64168 papers, respectively, for the field of “Ccomputer science” and

“Ssocial sciences”. Similar search in WoS for articles from same period or years, with also

same topics “collaboration” and “small group”, “collaboration” and “large group” and

“collaboration” and “crowd”; for the research areas “Computer sciences” and “Social

sciences and other topics”; returned 18, 4 and 22 papers; and for research area “Education

and educational research”, returned 94, 7 and 26 papers respectively. Alike results were

obtained with Google Scholar, considering the same keywords specified above, period of

years, and for all subject areas; returned 135, 51 and 134 papers. Although this type of

search is merely circumstantial, with many false positives and negatives, it gives a

reasonable indication of the breadth of research associated with each topic. Interestingly,

the set of papers returned by searching for crowd collaboration has been mainly published

between 2011 and 2015, which also indicates that increasing the group size is a recent

endeavour.

One reason explaining the lack of studies with large groups is that traditional Group

Support Systems (GSS) pose many technical and logistical problems to evaluation [1].

However, the emergence of a new generation of GSS, where we may include Wikipedia,

Facebook, LinkedIn, Doodle, Dropbox, Twitter, Zoho, Google Docs, and Google Maps,

among others, brought new opportunities to research large group collaborations. All these

systems rely on the Web infrastructure to support collaboration and are popular, widely

accessible, and always available [2]. Thus, many of the traditional logistical problems have

been suppressed.

Another reason is that the Web infrastructure is fostering new types of organizations, work

arrangements and collaborations. Projects like Amazon Mechanical Turk challenge

traditional conceptions about organizational culture, sense of belonging, roles and

attributions, responsiveness and remuneration, just to mention a few [3].

In this paper we report on a case study with three groups of 42, 44 and 48 participants. The

main purpose of this study was obtaining insights about the technical and social challenges

faced by large groups performing a collaborative task.

The data collection was obtained from three samples. In each sample a group was requested

to fulfil an urban design assignment. To successfully complete the assignment, the group

members had to walk around a city area, finding design ideas to improve the city living,

sharing their ideas with the others, and then converging into a list of the ten best ideas.

The samples used different collaboration tools, which allowed reducing some biases caused

by analysing collaboration supported on one particular tool.

Considering the research goals, the adopted data analysis methodology was founded on the

principles of qualitative data analysis, where qualitative inquiry and serendipitous findings

are privileged over quantitative assessments [4]. The qualitative data was gathered through

a survey.

The paper is organized as follows. In the next section we provide a brief overview of the

related work. Section 3 describes in more detail the research methodology. Section 4 details

the case study. Section 5 presents the obtained results. Finally, Sections 6 and 7 discuss the

obtained results and present some conclusions from this research.

2. Related Work

The literature related with large group collaboration is dispersed through several research

fields including group decision-making, computer supported collaborative work, social

media, and also the combinations of these fields, which is the specific case of group support

systems. Even though the number of research studies specifically focussed on large groups

seems to be scarce, we have been able to identify several factors that seem to be influenced

by group size. Table 1 provides a summary of the literature review, noting the different

factors that were identified, the observed effects of group size found when studying those

factors, and the restricted context where the effects were analysed.

Table 1. Summary of factors affecting large group collaboration reported in the related

literature.

References

Factors Effects Context

[5] Behavioural cascading

Technology mMitigates coordination costs

Ordinary public goods game, and in a public goods game with punishment

[6] Participation Increases with more participants

Only when participants know each other

[7] Communication Decreases Learning[8] Critical mass Task fails with insufficient

number of contributorsCrowdsourcing

[9][10]

Quality of outcomes Decreases because of coordination costs

Crowdsourcing

[11] Production blocking Decreased Brainstorming[12] Synergy Increased Brainstorming[13] Motivation Decreases because of perceived

marginal individual and anonymous contributions

Group decision-making

[14] Awareness Decreased Manufacturing organization[15] Social loafing Increases Brainstorming[14, 16] Coordination costs Increase Manufacturing organization /

Visual constructs on larger displays[14] Coordination

mechanismsPreferred over collaboration mechanisms

Manufacturing organization

[16] Coordination strategy Moderation preferred over distributed work

Visual constructs on larger displays

[17] Usability Decreases because of problems related with resource utilization, group dynamics, social interactions, display resolution, and visualization

Interactions with Tabletop shared-display Groupware

[18] Perceived utility Increases Access to experts[19] Production blocking Decreased in both, face to face

and computer based support meetings

Face to face large oral, and computer based meetings

[19] Evaluation apprehension

Decreased in both, face to face and computer based support meetings

Face to face large oral, and computer based meetings

[19] Participation Decreased more in face to face meetings, compared with computer based meetings

Face to face large oral,and computer based meetings

[20] Generation of ideas Enhance individual novel ideas Individual brainstorming followed of access to the ideas of others

Fowler and Christakis [5] hypothesized that online technology helps mitigating the negative

effect of group size on collaboration because of increased behavioural cascading, which

magnifies particular attitudes, e.g. being helpful and punishing wrongful interactions. Based

on simulation experiments, Siegel [6] found that participation rates tend to increase with

size for a particular group structure: the village (or clique), in which every participant

knows each other. However, Shaw [7] studied a highly related factor, communication, and

found out that communication was higher for small groups than larger groups. This study

was conducted in a specific context: Computer-Supported Collaborative Learning (CSCL).

Schenk [8] reports that group size is important to achieve critical mass: large collaborative

tasks may fail if an insufficient number of individuals cannot be guaranteed to contribute to

the task. This effect was found in the context of crowdsourcing, where the participants

contribute to a common goal, often as volunteers, but working in silos. In some cases the

participants are not even aware of each other because some central entity coordinates the

production of results [21].

Goodchild [9] notes that, as the group size increases, the quality of outcomes may decrease.

It has been suggested that such loss in quality may be attributed to coordination problems

[10].

Early studies on electronic brainstorming found out that large groups would get better

results than smaller groups because they would be less affected by production blocking

[11]. Subsequent research found out that electronic brainstorming is actually affected by a

combination of positive and negative factors, but as group size increases the positive factors

become more significant [12].

Still in the context of electronic brainstorming, synergy was also identified as a factor

positively influenced by large decision-making groups [12].

Beyond the heavily studied electronic brainstorming niche, it has been reported that large

electronic groups offer lower motivation to complete a task, because technology factors like

anonymity tend to exacerbate the perception of marginal individual contributions [13].

Team size is considered the main explanation for the social loafing phenomena consistently

observed in electronic groups [15]. Furthermore, it has been observed that smaller groups

have more awareness about the team members [14].

It has been suggested that Steiner’s productivity model [22], in which productivity

decreases with group size because of coordination costs, also applies to computer supported

collaborative work [14, 16]. Sando et al. [16] report some experiments where large groups

tended to rely on a central person to perform the task with others just providing advice, as a

strategy to reduce coordination costs. Bradner [14] observed that large teams prefer

coordination to collaboration technology, while smaller teams show a preference for the

opposite.

Ryall [17] suggested that large group interactions may also be affected by usability

problems. These problems may be related with resource utilization, group dynamics, social

interactions, display resolution, and visualization.

White et al. [18] made an experiment with a synchronous Q&A system where the users’

perception of group size was investigated. They explicitly asked about the perceived system

utility. The obtained results indicate that utility increases with group size. The authors

suggest this increase may be related with having more experts to answer questions.

According to [19]{Lindblom, 2014 #29} , many factors influence the outcome of meetings,

as por example, technology, task type, individual characteristics of the participants, and size

of groups. Specifically, they found that production blocking and evaluation apprehension

are lower with “very” large groups exchanging oral comments both in a face to face (37

and 38 participants), and a computer based meetings (33 and 35 participants). Although,

participation with comments were lower in face to face meetings. Participation was

measured by the total number of comments, the number of relevant comments, and the

number of words per comment.

In [20], it is explained that a number of studies on electronic brainstorming have found that

large electronic groups can facilitate the number of ideas generated relative to control

groups of similar numbers of solitary performers. Thus far there is no clear evidence for the

basis of this facilitative effect. The most likely explanation is that group members benefit

from exposure to the wide range of ideas in large groups. Therefore the authors, designed to

assess the role of number of ideas and number of folders on individual idea generation and

to eliminate some alternative interpretations for the group size effect. The results indicated

that only the number of ideas factor was important for facilitating idea generation.

The table above highlights two gaps in the existing body of knowledge. In particular, we

note that information overload has not been studied. Information overload, i.e. the

perceived inability to process information in an efficient and timely manner, can be

overemphasized by digital media [23] and more so in a collaborative context [24]. Team

awareness in large groups has only been alluded to in the extant literature [14], and it has

generally been restricted to group awareness, i.e. the perception of who belongs to the team

members. However, the phenomena of awareness can addresses richer information such as

context awareness and activity awareness [25-27], for which we do not know much about

in a large group context.

3. Research Approach

This research adopts an interpretive case study approach. According to Klein and Myers

[28], the interpretive approach assumes that knowledge of reality is gained through the

analysis of meanings that people assign to a phenomena, through social constructions and

interaction with people, tools and other artefacts. We find this research lens particularly

adequate to an exploratory study of large collaborating groups: large groups may have

different views of what is at stake, may engage with the group in multiple ways and

different contexts, and may use time, space and tools in very different ways. Of course

these arguments do not deny the value brought by quantitative research, but since

knowledge about large group collaborations is relatively scarce and the difficulties

associated with experimental research with very large groups and time and space

dispersion, gathering qualitative data can be a good strategy to explore the area and develop

posterior quantitative studies. Next we provide additional details about the adopted

approach.

3.1. Conceptual Framework and Research Questions

Commensurate with the adopted qualitative approach, we define “large group” as a concept

referring to tenths or maybe hundredths of participants. This expresses the conceptual

distinction between a “small group”, having less than a tenth, and a “very large group”,

which may have thousands or more participants. We prefer not operationalizing this

concept as a variable, since we are not considering measuring the effects of varying group

size, which would require using interval or absolute scales. Still, the “large group” concept

supports a clear operational definition of the concept being studied, can be easily

operationalized in an interpretive case study, and no less important, it can be easily

explained to the case participants.

We hypothesise that a large group may affect the way groups accomplish a task and

consider two main task constituents where the participants may perceive such effects:

discussion and consensus. This selection is not comprehensive but mainly pragmatic. In

particular, we are excluding the early stages of a decision process were the participants

typically gather information in a divergent way, or generate ideas like in brainstorming

sessions. Our intention is to delimit the study to the stages where groups must start

consolidating information and converging towards a solution, which we perceive as

potentially being more affected by a large group constituency. Furthermore, the research

literature on group decision-making also notes that convergence is usually more difficult to

achieve than divergence, manly because the participants must go beyond information

sharing and communication towards other cognitive phenomena such as persuasion and

conflict management. The adopted qualitative approach seems particularly adequate to such

a complex scenario.

The influence of a large group on the task may be exerted along multiple factors. In our

study we consider four factors: information overload, awareness, task involvement, and

collaboration support. We define information overload as the participants’ perceived

inability to handle the flows of information generated by the group. Information overload is

a natural candidate for this study. Since group decisions necessarily involve information

exchange, not only with the purpose to share knowledge and ideas but also to coordinate

individual contributions and to build consensus, we expect that a large functional group will

generate more information and hypothesise that the group participants may feel overloaded

with so much information. We are excluding dysfunctional groups from the study, where

the participants may not collaborate, may engage in conflicting behaviours, or may not be

able to accomplish the assigned tasks.

We define awareness as the perception of the group contributions to the task, including

awareness of who belongs to the group, what contributions to the task are submitted by the

group members, what messages supporting discussion and consensus are shared by the

group, and also awareness of the group dynamics as the task enfolds. We selected

awareness as a factor to consider in the study because research on collaboration support has

been reporting its impact on group behaviour. Especially in remote collaboration, where the

communication technology may accommodate fewer cues about who is participating in the

group and what they are doing, building collaboration awareness may be challenging. For

obvious logistic reasons, we expect large group collaborations to mainly rely on

communication technology and hypothesise that large groups may have significant issues

finding what, how and when contributions are brought to the group.

Task involvement refers to commitment to the tasks goals, perceived attachment to the

group, and willingness to contribute to the group. Prior research refers that large groups

tend to contribute less to the group, since the participants perceive their contributions to be

diluted and can downgrade their efforts while being unnoticed. We hypothesise that this

phenomenon will also be present in our study and so it is natural to include it in the

conceptual framework.

The last factor that we consider is collaboration support. In our research context, we define

it as the perceived contribution of the several tools used by the group when accomplishing

the task. Often collaboration support is operationalized as a variable measuring different

levels of support, for instance using an ordinal scale with two categories, lack of support

and tool support. However, as with the other considered factors, we explicitly avoided

using variables and scales, since the study is not focussed on measuring the impact of

different values assigned to the task outcomes. Instead, we seek to obtain qualitative

insights about the tools adopted by the group and their perceived impact on the group

process.

Of course more factors could have been considered for this study. Other possible factors,

which have already been mentioned in the literature review, would be synergy, production

blocking, motivation, social loafing, coordination costs, etc. Many qualitative studies adopt

a grounded approach and let researchers find evidence in the rich data acquired from the

study participants. This is usually possible because the process of inquiry is open to

surprises and flexible enough to capture different types of insights, for instance using the

interview as a data collection instrument. But this is usually possible because the study

designs tend to trade off data richness with a small number of contacts. In our case, the

number of contacts suggests a research design with less information richness but capturing

insights from a larger number of contacts, for instance using the survey as a data collection

instrument. This explains why only four factors have been considered in the conceptual

framework (Figure 1).

The research questions are directly derived from the elements discussed above. We

consider one initial research question about the relationship between large groups and the

group task:

RQ1: What was the perceived impact of group size on the task?

Besides establishing the relationship between the two main concepts under analysis, large

groups and group task, we will also use this question to validate the applicability of the

conceptual framework: If the gathered data does not reveal the participants’ perceptions

about information overload, awareness, task involvement, and collaboration support, then

the framework should be rejected for being inadequate to guide data gathering process.

The following questions are directly related with the various framework elements

previously described. Regarding the tasks constituents, we have:

RQ2 (discussion): How the participants discussed ideas?

RQ3 (consensus): How the participants reached consensus?

Regarding the factors affecting the group tasks, we have:

RQ4 (information overload): The participants felt information overload during the task?

RQ5 (awareness): The participants were aware of the group members’ contributions to the

task?

RQ6: (task involvement): The participants felt involved in the task?

RQ7 (collaboration support): What was the perceived contribution of collaboration support

to the task?

Figure 1. Conceptual framework

3.2 Data Collection Instrument

The adopted instrument is the survey of participants based on a questionnaire administered

electronically two days after the group has completed the task. Considering the adopted

interpretive approach, all selected questions were open-ended. The questions were

formulated to address the research questions in a contextualised way and using simple

language. The list of questions was then randomised:

Q1: Did you feel information overload during the task?

Q2: What was the impact of group size on the task?

Q3: Could you perceive the others’ comments?

Q4: Did you feel involved in the task?

Q5: What do you think about the collaboration support?

Q6: What did you do to discuss ideas?

Q7: How did you reach consensus?

To the list above, we added some more specific questions with the purpose to capture more

detailed information from the participants:

Q3.1: How were you aware of the others’ work?

Q5.1: What did you do to collaborate?

Q6.1: What ideas have you discussed most?

Q7.1: How did you select the best ideas?

The data collected from this questionnaire was complemented with snapshots of the data

shared by the participants using the adopted software tools. These snapshots allow

corroborating the participants’ comments about collaboration support (Q5) and the group

task (Q6 and Q7).

3.3 Sample

The study involved students from an undergraduate course in information systems divided

in two samples. Both samples were subject to a standardized set of research conditions. The

same task was assigned to both groups. There were no shared members between groups.

The samples had approximately the same group size, gender composition and age: One

group had 48 students (26 male; average age was 22.8); another group had 44 students (25

male, average age was 22.3); and another group had 42 students (XXXXXX). The students

were taking the same undergraduate course at the same faculty. All participants were

proficient with computers and knowledgeable about using collaborative tools, and therefore

the question regarding collaboration support was not ambiguous to them. These conditions

were checked with pre-test questionnaires.

However, the samples were different in one particular contextual factor: the primary

collaboration tool that was used. We decided to recommend each group to adopt different

tools as the primary collaboration mechanism. Our objective was not studying tool use as

an independent variable, or even comparing which tool performs better. Instead, the goal is

to increase data richness through diversity while avoiding analytical bias caused by

anchoring the study on one single collaborative tool.

3.4. Group Task

The research literature suggests that the type of task should be made explicit in studies

involving group decision making, since it can be considered an intervening factor [29]. The

McGrath’s typology of tasks [30] defines four main types of tasks, generate, choose,

negotiate, and execute. Within the choose type, McGrath makes the further distinction

between tasks with correct answers and tasks with no right answer. For this study we

selected the choose task with no right answer. The main reason is that the lack of a correct

answer stimulates discussion, while choosing also requires the participants to reach

consensus. This ensures that both group task constituents defined in the conceptual

framework are addressed by the study.

The task was set up as a combination of remote asynchronous activities and face-to-face

synchronous activities, the former done at home or on the move, and the latter done in

assembly rooms. This combination aims to avoid biases caused by restricting the study to a

synchronous or asynchronous collaboration mode.

We arbitrarily specified that the task should be completed in a period of one week. The

main reason was giving the participants sufficient time to discuss the task, to settle rules

and procedures, and to develop strategies for converging towards a group choice.

The assigned task was to collaboratively identify community problems in a delimited urban

area, proposing innovative solutions based on information technology and assigning

priorities. The task was not restricted to any particular type of ideas. Examples were given

to the groups before the study, when the groups were instructed about the task. Given

examples included improving local businesses, community services, and improving social

interaction and responsibility.

The task instructions specified minimum levels of participation. Each student should

suggest two innovative solutions as a minimum requirement for participating in the group.

Group participation was also a minimum requirement for being approved in a course

assignment.

The students were encouraged to suggest problems and solutions, and to discuss and give

their opinions about the others’ suggestions and comments. The groups were also instructed

to generate a consensus list with the ten best solutions. The specific task instructions

indicated:

“1) you must work on a specific urban area of Santiago; 2) you must use tool [A / B] in collaborative

mode to share and select problems and ideas; 3) the problems and ideas may be commented and

supplemented with text, sketches, and photos, as much as necessary to emphasize their importance; 4)

the whole work must be accomplished collaboratively; 5) you may take a picture of the place or

context where you identify a problem; 6) the list of the 10 best ideas must be accepted by consensus;

and 7) you have one week to accomplish the task.”

No instructions were given about what type of technology to use (e.g. mobile phones of

laptops). Neither a certain type of convergence mechanism for selecting the best ideas was

indicated or recommended. The definition of coordination and consensus mechanisms had

to be resolved by the groups.

3.5 Data Analysis

To analyse the collected data, we adopted the traditional qualitative coding process [4].

This is a three-stage process where, initially, commonalities in the data are captured in

“descriptive” codes to clearer capture the essential attributes of the phenomenon. Next, as

more data and codes are available, “interpretive” codes are abstracted from the idiographic

confines of the concrete incidents to help understand what is going on “behind” the data.

And lastly, inferential “pattern” codes, now abstract of space and time and etic to the

substantive range of the research, are conceptualized; they are explanatory and often

predictive. Since the study has an exploratory nature, only the two first stages of qualitative

coding were done. The Nvivo tool supported the coding process.

4. Results

4.1 Initial comments

In all groups, the activities started with a face-to-face meeting. The participants relied on

these meetings to discuss the given instructions and devise collaboration strategies.

After the initial meeting, the participants started to asynchronously work on the task,

walking around the designated city area in their spare times, identifying problems and

opportunities, while sharing ideas and comments with the others, and jointly creating the

required list of priorities. Most participants took notes and pictures while on the move using

mobile phones.

During the time assigned to accomplish the task, the groups organized sessions where they

all met face-to-face again. During these sessions, they redefined the collaboration strategies

devised on the initial meeting, changed work conventions, and made several changes on the

way they used the collaboration support tools (more on that later). In particular, the

convergence procedures necessary to successfully complete the task were discussed and

settled face-to-face.

All 134 students that participated in the study answered the questionnaire. The open

questions generated a corpus with about 56.022 words and 272.435 characters (without

spaces), which indicates that the students carefully considered the questions. In the

following, we present the results according to the research questions.

4.2 Perceived impact of group size

Overall, the major impact of group size perceived by the participants was related to

coordination problems, since 54 participants (about 40% of the cohort) mentioned them.

Within the list of coordination problems, the participants emphasized in particular the lack

of experience handling large groups and problems with time management. As some noted,

“the majority was accustomed to work in much smaller groups” and “never before I have

faced collective work involving so many people”.

We also found evidence of impact of group size on the discussion and consensus

components of the task. The participants noted multiple problems discussing ideas, in

particular, confusing information brought in from many sources, lost information because

of deficient access control, difficulties communicating, lack of control over the decision

process, lack of moderation, lack of motivation mechanisms, and lack of rules. Regarding

reaching consensus, it was noted that consensus requires improved communication, is a

slow process, there were too many items to converge to, and there were initial difficulties

establishing the work process.

On a more positive note, the participants found multiple benefits of group size on task

involvement, noting in particular the good communication/interaction, ample

brainstorming, and especially improved quality through consensus. In particular, a large

number of participants (14) referred that the discussion was improved with critical

comments, new points of view, and not only more ideas but also better integration of

others’ comments in suggested ideas. As one participant explicitly noted, “the results are

much richer”. One participant noted the participants felt some social pressure towards

completing the task.

Some participants established the relationship between group size and awareness: The

decision process benefited from improved feedback and contributed to develop a holistic

view of the problem. In the words of one of them, “the opinions from the other participants

allowed improving the proposals made by each one of us, using some sort of feedback”.

However, a negative impact was found on information overload. The participants noted the

diminishing returns, caused by repetition and information losses, having to process too

many contributions, difficulties following all contributions, and also difficulties following

rules of conduct. More than one participant mentioned each one of these contributors to

information overload. One of them noted in particular that “it becomes more complex to

maintain a certain order and conventions about how to do things, a moderator is necessary

with more capacity to bring cohesion and unity to the work”.

Few impacts on collaboration support were identified by the participants, which were

divided between the positive and negative sides. On the positive side, the participants

referred to having a faster decision process and better information sharing. On the negative

side, they referred to difficulties managing the collaboration tools.

Considering that this question was fundamentally asked to check if the participants would

spontaneously bring forward the set of factors we considered as potentially affecting the

group task, we were content that all elements specified in the conceptual framework were

raised. The main surprise was the significant importance given by the participants to

coordination, a factor that is subdued under collaboration support in the conceptual

framework. Future research should definitely promote this factor and make it explicit in the

framework. In Table 2 we provide a quantitative summary of the responses related to the

perceived impact of group size on the task.

Table 2. Perceived impact of group size.

Benefits Drawbacks

Coordination 54

Discussion 37

Task involvement 19 1

Information overload 2 15

Collaboration support 4 1

Awareness 3

4.3 Information overload

Our inquiry about the participants’ perception of information overload raised four

considerations. The first one is that the participants had multiple problems processing

information, a category that was mentioned by 36 participants. They found it hard to

analyse so much information in detail, commented about having stress while receiving so

many comments, the time spent reading them, missing new comments, forgetting

information, dealing with repetition, and losing interest on the task. One of the participants

noted, “the owner of an idea collapsed with so much information and comments and would

forget to add all suggestions in [her/his] own idea”. Other noted there was “too much

information, very unclear, and unstructured”. And another made an interesting comment

that the group was “complexifying the problem”.

Our second consideration concerns the group interaction. The participants raised several

interaction problems, such as the dependence on others’ feedback, dealing with peak

activity times, and dealing with coordination issues. One participant noted “the ideas had to

receive feedback with other opinions, since this was the only way to really measure the real

impact of the idea and its social acceptance”.

It was particularly interesting to find out that the participants adopted several strategies to

avoid information overload. Some of the mentioned strategies were just scanning the first

words of an idea or comment, using alternative communication channels (e.g. chat and

mail) to avoid saturating the primary one, and relying on an emergent leader to coordinate

the contributions.

Still, the participants perceived several positive factors from large group collaborations. It

was noted the process had democratic value, the communication was immediate, and there

was value in receiving diverse opinions and explanations, as well as imagination.

4.4. Awareness

Feedback was mainly perceived as constructive. Of the twenty participants pointing out

constructive feedback, nine explicitly referred that feedback promoted quality through

“truthful comments”, which either “helped completing ideas” or “raised valid

interrogations”, and five referred that feedback promoted collaboration.

However, the participants also noted they often could not follow all feedback. In particular,

two participants said, “it was difficult to follow the order of the discussion”, and “although

it was possible to follow all comments, there was too much information, which did not

favour total attention to all comments”.

Some participants also complained about the quality of feedback. Information in messages

was often repeated, noisy and without significant contents. As one participant commented,

“there were many comments made with the sole objective to be present, that, essentially

saying ‘I like it’, and which would did not contribute to the discussion”. On the other hand,

repetition was also perceived as contributing to reinforce ideas, which allowed the

participants to naturally converge towards certain ideas.

4.5 Task involvement

The participants reported increased task involvement caused by increased participation and

fluid collaboration. The increased participation was mainly caused by the process of

elaborating ones’ and others’ contributions, and also interest in the others’ propositions. As

mentioned by one participant, “many ideas got my interest and I felt that I wanted to be

involved more in them”. Interestingly, three participants noted they felt peer pressure

towards contributing to the task: “the constant participation of my colleagues and the

actualisation of ideas generated, in a certain way, a pressure to keep a proposal up to date to

the colleagues’ opinions”.

The perceived fluid collaboration was mainly caused by the constant revising of

contributions. As one participant noted, “the communication becomes fluid, since everyone

sees what you are doing and you feel well regarded by the group”.

Some participants reported they felt special responsibility for the shared task and therefore

assumed more central roles, such as facilitating the discussion or configuring tools for

specific goals. Though many participants indicated they did not like the format of their

participation: they lacked time to process information, noted the lack of commitment of

some colleagues, lacked feedback to some of their ideas, and the feedback was indirect.

Some participants explicitly indicated that having others deleting their contributions

reduced their commitment to the task. Finally, some participants explained their marginal

participation to the task completion as caused by the large number of participants. As one

participant said, “one feels aside when our idea is moved to the end”.

4.6 Collaboration support

The participants perceived the hybrid collaborative approach as better than face-to-face

meetings, mainly because of better access to shared information and better time/space

management when off mettings. However, the initially suggested collaboration support

tools were considered inadequate for collaboration. The participants complained especially

about the lack of control over individual contributions, which could be freely deleted by

any participant. Other negative comments hint that the tools made it difficult to associate

comments to their authors, did not log data modifications, did not support the rules agreed

on the first meeting, and were not democratic. In both groups, some time after starting to

work on the task, the participants decided to have another face-to-face meeting where they

discussed the collaboration support and eventually decided to adopt other tools and new

participation rules. An example of a rule that was settled in one these meetings was

structuring the participants’ contributions in two different categories: ideas and comments.

Another example was the adopted principle of always respecting the person taking the

initiative for proposing an idea.

4.7 Group task: Discussion and consensus

The participants ended up adopting multiple channels and tools to discuss ideas, which

included opportunistic encounters, face-to-face group meetings, one-to-one phone calls, and

also using chat tools for multi party discussions.

The face-to-face meetings were primarily focussed on defining the discussion rules,

including how to behave in the group and what tools to use and how to use them.

The participants structured the discussion by separating ideas from comments and agreed

on having a primary communication channel for sharing them, while using other channels

for chatting about the ideas and clarifying the others’ views. The main reason for separating

the discussion in ideas and comments was that it allowed easily deciding in favour or

against an idea, and it allowed to focus the discussion on the original idea.

The participants adopted multiple data reduction strategies, such as only focussing on the

most interesting or the most important ideas (those with more comments), and constraining

the discussion to pros/cons and to agreements/disagreements. One participant referred that

“I’ve gone to the extremes, commenting all ideas that seemed to me as either very good or

very bad”.

The participants also adopted several strategic attitudes towards preserving the discussion,

which included “being constructive by only submitting relevant contributions”, focussing

on rational criteria, identifying flaws in ideas and commenting on them, and showing

mutual respect and avoiding personal attacks.

Regarding consensus building, both groups initially adopted the voting strategy to converge

on the ten best ideas. However, this strategy was later on perceived as erroneous because,

even though it was fast and could be automated with software tools, it did not allow

justifying the selections. One participant referred that “consensus should go beyond

voting”. The consensus strategy that was lately adopted by one group consisted in ordering

the list of ideas according to the number of received comments, and relying on a moderator

to maintain the list (using a colour scheme). The other group adopted a similar strategy that

consisted in prioritising ideas according to comments in favour or against made by all

participants. In both groups, the later strategies were perceived as much more effective, as

they allowed elaborating ideas and justifying the participants’ preferences.

6. Discussion

The collected data provides some interesting insights on how the participants perceived the

task in the context of large group collaboration. In particular, the participants seemed well

aware of the trade-offs brought by information overload, which in the one hand bring too

much information to process but, in the other hand, bring diverse opinions, imagination and

also immediacy to the group interaction. Considering these trade-offs, the groups were able

to develop successful strategies to avoid information overload, which emphasised scanning

information for relevant contributions and avoiding saturating the primary communication

channel with secondary interactions. It is interesting to note that the strategies operate at

both channel ends, at the origin by refraining from sending data and at the destination by

aggressively filtering out irrelevant data. Although having a leader/facilitator to coordinate

contributions may also reduce information overflow, from the collected data it seems that

the leader/facilitator was mainly responsible for implementing the collaboration rules that

were decided by the group in the face-to-face meeting, and not to constrain the participants’

contributions.

The collected data also indicates that the participants perceived the trade-offs between

having too much or too few feedback, and promoted some balance between the two

extremes. For instance, repetition contributed to increase awareness about the participants’

preferences for certain ideas. However, the participants also complained that the quality of

feedback could be improved, notably by decreasing the number of repeated comments and

ideas. All in all, there seems to be a significant relationship between information overflow

and awareness, although information overflow appears as a negative factor and awareness

appears as a positive factor affecting the task.

Task involvement and awareness also seem to be highly related. The participants noted that

increased task involvement was caused by increased participation and fluid collaboration,

which in turn are promoted by awareness information regarding what ideas the participants

prefer and their opinions about them. This emphasis on collaboration fluidity is quite

interesting because once again it suggests that the different factors that were investigated,

information overload, awareness and task involvement, affect each other and indeed have to

be carefully balanced by the participants.

The gathered data provide some insights about the checks and balances done by the groups

to develop collaboration fluidity. An interesting one was the special responsibility felt by

some participants, and recognised by the others, to the shared task. This lead them to work

for the benefit of the others, e.g. reconfiguring the collaboration tools to better serve the

group or reminding them about the deadlines. Though it did not lead them to lead the group

or to monopolise the discussion. As one participant noted, “it was not my intention, but

found myself in the role of task coordinator, and therefore 100% involved in the

discussion”.

Another contributor to task involvement, which has been already mentioned, was the

overall constructive attitude. The collected data provide some glimpses on how that attitude

was constructed. One aspect was the perceived peer pressure towards contributing to the

task, even though some of them perceived that their contributions were marginal. One

participant noted, “part of being involved was communicating and collaborating with the

others”. Another noted, “being able to implement my own idea and having rapid feedback

creates a felling of involvement and commitment to the idea”. Though it was also

interesting to note that the participants were committed to change their ideas based on the

received feedback: “after uploading my idea, I received many comments, which helped me

to see things that I had not though before, or which I did not thought were relevant, and

because of that feedback cycle my idea took more sense”.

Analysing the collected data on collaboration support, the most interesting observation is

the flexibility the groups revealed adopting new collaboration strategies and tools. Though

the decisions had to be made face-to-face.

Regarding the group task, we note that the discussion of ideas involved multiple data

reduction strategies, which in particular constrained the discussion to focus on the most

important ideas and to divide the contributions to binomial categories, e.g. in favour and

against, or agree and disagree. The consensus processes were interesting because of the

need to change the initial strategies. The groups started the task with the perspective that

voting was the best strategy. Actually, both groups implemented different voting

procedures, one by voting on the best 3 ideas and the other by prioritising the ideas with

more positions in favour. However, after going through the process, the groups decided to

require the participants to justify their positions. As one participant noted, “majority voting

is not very good if you have a real project but yes, it is very efficient”. We coded comments

about the change in the convergence strategy from 24 participants, and the data suggests

that the decisions were made considering that relevance and feasibility could not be

assessed with the initial procedure.

Looking more generally to the conceptual framework, we emphasise again the strong

interrelationships and delicate balance between information overload, awareness and task

involvement. It seems that such balance may be easily disrupted by small changes in the

task or the technology. Perhaps another factor could be identified to unify these three

factors. The study hints that collaboration fluidity could be such a factor. Furthermore, the

relationships between these three factors and the task also seem complex. In particular, it

seems that the type of task affects the participants’ attitudes and behaviours that are

strongly related with information overload, awareness and task involvement. Some

contributors like the perception of value brought by diverse opinions, attitudes towards

preserving the discussion, and responsibility for the shared task seem to depend on the task

characteristics, which emphasises discussion and consensus building. But on the other

hand, we could not establish clear relationships between the studied factors and the two

task components, discussion and consensus, since the factors seem to indifferently apply to

them.

Regarding collaboration support, what stood out from the case was the groups’ flexibility

adopting different tools and the dependence on face-to-face meetings to make significant

strategic decisions.

Another issue that could be raised is the relationship between the task and time. We gave

one week to accomplish the task, which gave them plenty of time to experiment different

collaboration strategies, to fail, and finally to adopt winning solutions. Besides, this

extended period of time allowed some participants to emerge as group facilitators. It is

unclear if in a shorter period of time the groups would be willing to adapt, or instead would

go with the first strategy, which was inadequate.

This study contributes to understand various trade-offs that large collaborating groups face,

some of them with implications to collaboration support. Information overload is as much

as a threat as an opportunity, so the challenge is to find the right balance. Information has to

be organised in a way that keeps the participants constantly involved, aware and focussed

on the most important pieces of information. Updates have to be carefully done to avoid

stress and to preserve context and respect for the others’ opinions. For instance, a constant

flow of new comments keeps the discussion in pace and increases peer pressure. However,

often the participants allude to lacking time to process information or, the other side of the

coin, feeling aside by the lack of comments.

The participants’ understanding of awareness seems to be more global than local. For

instance, repetition can be globally understood as consensus and locally understood as

noise. The participants use multiple communication channels with different goals, often

with the intention to avoid cluttering the primary communication channel.

Collaboration support should allow groups to flexibly define how information is aggregated

and prioritised, and possibly should allow experimenting different arrangements. The

technology should foster refining the initial contributions based on incremental feedback.

Often the most obvious or efficient collaboration processes are not the ones preferred by the

participants, which seem to favour other criteria such as democratic participation. Though

the biggest challenge is that it seems that strategic changes still have to be negotiated face-

to-face.

7. Conclusions

.

References

[1] P. Antunes, V. Herskovic, S. Ochoa, and J. Pino, "Structuring Dimensions for

Collaborative Systems Evaluation," ACM Computing Surveys, vol. 44, 2012.

[2] G. Mark, "Large-scale distributed collaboration: Tension in a new interaction order," in

Computerization movements and technology diffusion: From Mainframes to

Ubiquitous Computing, M. Elliot and K. Kraemer, Eds., ed Medford, NJ: Information

Today, Inc., 2008, pp. 289-310.

[3] M. Bernstein, J. Brandt, R. Miller, and D. Karger, "Crowds in Two Seconds: Enabling

Realtime Crowd-Powered Interfaces," in Proceedings of the 24th annual ACM

symposium on User interface software and technology, Santa Barbara, California,

USA, 2011.

[4] M. Miles and A. Huberman, Qualitative Data Analysis. Thousand Oaks, California:

SAGE, 1994.

[5] J. Fowler and N. Christakis, "Cooperative behavior cascades in human social

networks," Proceedings of the National Academy of Sciences, vol. 107, pp. 5334-5338,

2010.

[6] D. Siegel, "Social Networks and Collective Action," American Journal of Political

Science, vol. 53, pp. 122-138, 2009.

[7] R. Shaw, "The relationships among group size, participation, and performance of

programming language learning supported with online forums," Computers &

Education, vol. 62, pp. 196–207, 2013.

[8] E. Schenk, "consultezTowards a characterization of crowdsourcing practices," Journal

of Innovation Economics, vol. 1, pp. 93-107, 2011.

[9] M. Goodchild and A. Glennon, "Crowdsourcing geographic information for disaster

response: a research frontier," International Journal of Digital Earth, vol. 3, pp. 231-

241, 2010.

[10]A. Kittur and R. Kraut, "Harnessing the wisdom of crowds in wikipedia: quality

through coordination," in Proceedings of the 2008 ACM conference on Computer

supported cooperative work, San Diego, CA, USA, 2008, pp. 37-46.

[11]R. Gallupe, A. Dennis, W. Cooper, J. Valacich, L. Bastianutti, and J. Nunamaker,

"Electronic Brainstorming and Group Size," Academy of Management Journal, vol. 35,

pp. 350-369, 1992.

[12]A. Dennis and M. Williams, "A meta-analysis of group side effects in electronic

brainstorming: More heads are better than one," International Journal of e-

Collaboration, vol. 1, pp. 24-42, 2007.

[13]L. Chidambaram and L. Tung, "Is Out of Sight, Out of Mind? An Empirical Study of

Social Loafing in Technology-Supported Groups," Information Systems Research, vol.

16, pp. 149-168, 2005.

[14]E. Bradner, "Team size and technology fit: participation, Awareness, and rapport in

distributed teams," IEEE Transactions on Professional Communication, vol. 48, pp.

68-77, 2005.

[15]O. Alnuaimi, L. Robert, and L. Maruping, "Team Size, Dispersion, and Social Loafing

in Technology-Supported Teams:

A Perspective on the Theory of

Moral Disengagement," Journal of Management Information Systems, vol. 27, pp. 203-230,

2010.

[16]T. Sando, M. Tory, and P. Irani, "Impact of Group Size on Spatial Structure

Understanding Tasks," in IEEE Pacific Visualisation Symposium, Hong Kong, China,

2011.

[17]K. Ryall, C. Forlines, and C. Shen, "Exploring the Effects of Group Size and Table

Size on Interactions with Tabletop Shared-Display Groupware," in Proceedings of the

2004 ACM conference on Computer supported cooperative work, Chicago, Illinois,

USA, 2004, pp. 284-293.

[18]R. White, M. Richardson, and Y. Liu, "Effects of community size and contact rate in

synchronous social Q&A," in Proceedings of the SIGCHI Conference on Human

Factors in Computing Systems, Vancouver, BC, Canada, 2011.

[19]T. Lindblom, M. Aiken, and M. Vanjani, "Electronic Facilitation of Large Meetings,"

Communications of the IIMA, vol. 9, p. 3, 2014.

[20]P. B. Paulus, N. W. Kohn, L. E. Arditti, and R. M. Korde, "Understanding the group

size effect in electronic brainstorming," Small Group Research, p. 1046496413479674,

2013.

[21]D. Brabham, "Crowdsourcing as a Model for Problem Solving," The International

Journal of Research into New Media Technologies, vol. 14, pp. 75–90, 2008.

[22]I. Steiner, Group Process and Productivity: Academic Press, 1972.

[23]D. Bawden and L. Robinson, "The dark side of information: overload, anxiety and

other paradoxes and pathologies," Journal of Information Science,, vol. 35, pp. 180-

191, 2009.

[24]A. Ferreira and P. Antunes "Tackling information overload in electronic

brainstorming," in Proceedings of the 2008 Conference on Group Decision and

Negotiation Coimbra, Portugal, 2008.

[25]C. Gutwin and S. Greenberg, "A Descriptive Framework of Workspace Awareness for

Real-Time Groupware," Computer Supported Cooperative Work, vol. 11, pp. 411-446,

2002.

[26]J. Bardram and T. Hansen, "Context-Based Workplace Awareness," Computer

Supported Cooperative Work, vol. 19, pp. 105-138, 2010.

[27]J. Carroll, M. Rosson, G. Convertino, and C. Ganoe, "Awareness and teamwork in

computer-supported collaborations," Interacting with Computers, vol. 18, pp. 21-46,

2006.

[28]H. Klein and M. Myers, "A Set of Principles for Conducting and Evaluating

Interpretive Field Studies in Information Systems," MIS Quarterly, vol. 23, pp. 67-93,

1999.

[29]J. Fjermestad and S. Hiltz, "An assessment of group support systems experimental

research: Methodology and results," Journal of Management Information Systems, vol.

15, pp. 7-149, 1999.

[30]J. McGrath, Groups: Interaction and performance. Englewood Cliffs, NJ: Prentice-

Hall, 1984.