Crowd Science: Measurement, Models, and Methods

John Prpić & Prashant ShuklaHICSS 2016

Overview of the Field

Research Goals

Theoretical Grounding

Toward Crowd Science

Discussion

The Grand aim of science is to cover the greatest number of experimental facts by logical deduction from the smallest number of hypotheses or actions. - Albert Einstein.

Agenda

2

Crowdsourcing - Crowdflower, Wikipedia, TopCoder,

Citizen Science - GalaxyZoo, Foldit, Zooniverse,

Crowdfunding - Kickstarter, Indiegogo, Kiva,

Open Innovation Platforms - Kaggle, Innocentive, Challenge.gov,

Sharing Economy - AirBnB, Uber, Lyft,

Public Sector - IPaidaBribe, FixMyStreet, Open Ministry,

IT-Mediated Crowds – Practice

3

Human Computation - (Ipeirotis, Michelucci, von Ahn)

Open Collaboration - (Benkler, Majchrzak, Mako-Hill)

Ideas Competitions - (Afuah, Boudreau, Lakhani)

Citizen Science - (Cooper, Crowston, Meier)

Crowdfunding - (Agrawal, Burtch, Mollick)

Public Sector - (Aitamurto, Brabham, Noveck)

IT-Mediated Crowds – Research

4

We’re observing increased research and practice on organizations using IT to connect with dispersed individuals for explicit resource creation purposes.

This state of affairs precipitates the need to precisely measure the processes and benefits of these activities over myriad different implementations.

Research Motivation

5

We seek to address these salient and non-trivial considerations by laying a foundation of:

Theory, Measures, Research methods,

That allow us to test Crowd-engagement efficacy across organizations, industries, technologies, and geographies.

Research Goals

6

The Theory of Crowd Capital (Prpić & Shukla 2013; 2014)

Dispersed Knowledge (Hayek 1945)

Every individual has private knowledge that is useful, but cannot be accessed.

Crowd CapabilityThe IT structure, form of content, and internal processes through which an organization engages a Crowd.

Crowd CapitalA heterogeneous organizational resource generated from IT-mediated Crowds.

Theoretical Grounding

7

Dispersed Knowledge

Crowd Capability

Crowd Capital

IT Structure Crowd-engaging IT is found in Episodic or Collaborative forms, distinguished by whether the individuals in a Crowd interact with one another or not, through the IT (Prpić & Shukla 2013; 2014).

Theoretical Grounding

8

Theoretical Grounding

9

Typology of Crowd-derived content & Organizational processing

methods (Prpić, Shukla, Kietzmann & McCarthy 2015)

Theoretical Grounding

10

Comparison of Common Characteristics of Crowdsourcing Techniques (Prpić, Taeihagh & Melton 2015)

An empirical apparatus that considers counterfactuals in ascertaining the benefits of various implementations of IT-mediated Crowds.

Currently, to test hypotheses about the benefits of using IT-mediated Crowds, researchers use data from a single Crowdsourcing, Crowdfunding, Open innovation platform.

Need to consider counterfactuals. Can’t quantify the benefits of using Crowds otherwise. Can’t generalize, can’t predict. Can’t move toward a science of IT-mediated Crowds.

Toward Crowd Science

11

Counterfactuals

If the use of IT-mediated Crowds is the treatment, we need to measure the difference between the treatment and control group, before and after implementing a Crowd.

Toward Crowd Science: Counterfactuals

12

Operationalization Measures of processes and benefits across a variety of IT-mediated Crowds

implemented.

Toward Crowd Science: Operationalizations

13

Experiments

Randomly select organizations/units seeking specific and similar resources from IT-mediated Crowds.

Observe how they do with respect to Crowd Capital generation relative to the control group over a period of time.

Toward Crowd Science: Methods

14

Meta-Analysis

Focus on quantitative meta analysis that accumulates the evidence from the extant hypothesis testing endeavors in the field.

Revolves around collection of effect sizes/coefficients and applications of procedures such as meta analytic regression analysis (MARA) and homogeneity analysis (HOMA).

Toward Crowd Science: Methods

15

Natural Experiments

Difference-in-differences techniques.

Toward Crowd Science: Methods

16

Unprecedented shocks to knowledge production function Unprecedented on-demand scale of human participation. Unprecedented on-demand speed and aggregation of human effort. Unprecedented on-demand access to human knowledge. New outcomes & new configurations of socio-technical systems.

Why we need Crowd Science

17

The Grand aim of science is to cover the greatest number of experimental facts by logical deduction from the smallest number of hypotheses or actions.

- Albert Einstein.

Are we moving toward more perfect information through Crowd Science?

Can Crowd Science optimize stewardship of common-pool resources?

Crowd vs. Market vs. Firm?

AI, IoT, Machine Learning, with Crowds?

Crowd Science: Open Questions

18

Twitter: @JPnuggets @Prashshukla

Blogs: phdinstrategicmanagement.wordpress.com creativecommodum.com

Thank You!

19