cs 5306 info 5306: crowdsourcing and human computation hirsh/5306/ آ  2017-09-05آ  upcoming

Download CS 5306 INFO 5306: Crowdsourcing and Human Computation hirsh/5306/ آ  2017-09-05آ  Upcoming

Post on 11-Jul-2020

0 views

Category:

Documents

0 download

Embed Size (px)

TRANSCRIPT

  • CS 5306 INFO 5306:

    Crowdsourcing and Human Computation

    Lecture 4 8/31/17

    Haym Hirsh

  • Upcoming Speakers

    • Thursday, Aug 31, 4:15 (after class) Louis Hyman. The Return of The Independent Workforce: The History and The Future of Work (extra credit)

    • Thursday, Sep 7, 4:15 (after class) Henry Kautz, Mining Social Media to Improve Public Health (extra credit)

    • Tuesday, Sep 12 (in class) Serge Belongie

    • Thursday, Sep 28, 4:15 (after class) Michael Bernstein (extra credit)

  • “Prehistory” of Human Computation

  • Characterizing Crowdsourcing and Human Computation Systems

    • “Overt” vs “Covert”: Are human participants explicitly participating to achieve the collective outcomes, or is some form of mining of human activity achieving the collective outcomes

    – Overt: Amazon reviews, Wikipedia

    – Covert (“Crowd Mining”): Google, Amazon recommendations

  • Characterizing Crowdsourcing and Human Computation Systems

    • What are they doing? – Collecting

    – Collaborative Creation

    – Smartest in the Crowd

    – Collaborative Decisions

    – Micro-Labor

    – Mining User Behavior • Search logs

    • Social media

  • Characterizing Crowdsourcing and Human Computation Systems

    • What are they doing? – Collecting

    – Collaborative Creation

    – Smartest in the Crowd

    – Collaborative Decisions

    – Micro-Labor

    – Mining User Behavior • Search logs

    • Social media

  • Characterizing Crowdsourcing and Human Computation Systems

    • What are they doing? – Collecting

    – Collaborative Creation

    – Smartest in the Crowd

    – Collaborative Decisions

    – Micro-Labor

    – Mining User Behavior • Search logs

    • Social media

  • http://upload.wikimedia.org/wikipedia/commons/6/63/Wikipedia-logo.png

  • Characterizing Crowdsourcing and Human Computation Systems

    • What are they doing? – Collecting

    – Collaborative Creation

    – Smartest in the Crowd

    – Collaborative Decisions

    – Micro-Labor

    – Mining User Behavior • Search logs

    • Social media

  • Characterizing Crowdsourcing and Human Computation Systems

    • What are they doing? – Collecting

    – Collaborative Creation

    – Smartest in the Crowd

    – Collaborative Decisions

    – Micro-Labor

    – Mining User Behavior • Search logs

    • Social media

  • “Prediction Markets”

  • Characterizing Crowdsourcing and Human Computation Systems

    • What are they doing? – Collecting

    – Collaborative Creation

    – Smartest in the Crowd

    – Collaborative Decisions

    – Micro-Labor

    – Mining User Behavior • Search logs

    • Social media

  • Copyright 2011 Haym Hirsh

  • “Games with a Purpose”

  • • Introduction: Broad overview of collective intelligence, a framework for understanding it, and its various connections to machine learning – Animals to humans images/videos

    – CI mosaic (icons, then list of keywords)

    – CI, crowdsourcing, human computation

    – Overt vs covert • Collaborative creation

    • Collaborative decision making

    • Smartest in the crowd / contests

    • HC and micro-labor markets

    • Crowd mining

    – Roles of Machine Learning

  • Characterizing Crowdsourcing and Human Computation Systems

    • What are they doing? – Collecting

    – Collaborative Creation

    – Smartest in the Crowd

    – Collaborative Decisions

    – Micro-Labor

    – Mining User Behavior • Search logs

    • Social media

  • 5 September 2017 Copyright 2011 Haym Hirsh 41

  • 5 September 2017 Copyright 2011 Haym Hirsh 42

  • 5 September 2017 Copyright 2011 Haym Hirsh 43

  • 5 September 2017 Copyright 2011 Haym Hirsh 44

  • 5 September 2017 Copyright 2011 Haym Hirsh 45

  • “Games with a Purpose” Luis von Ahn

    IEEE Computer, June 2006

  • “A game-theoretic analysis of games with a purpose” S. Jain and D. Parkes

    Internet and Network Economics, pp.342-350, 2008

  • Important Game Theory Concepts

    • Nash Equilibrium: A strategy for each player to take actions wherein if either player changes his/her strategy, the outcome will be worse for that actor

    • Bayes Game: Each player has information that is unknown to the other player, but the other player has a probability distribution over what that information might be

    • Bayes Nash Equilibrium: If either player changes his/her strategy, the expected value of the outcome will be worse, given the probability distribution

  • Result

    • The ESPGame incentivizes players to take a strategy of playing the most common words first • It’s a Bayes Nash Equilibrium

    • It yields the best response even for playing otherwise

    • This might not match up with the goals of the game designer

  • Readings for Next Time

    • Yu, L., André, P., Kittur, A. and Kraut, R., 2014, February. A comparison of social, learning, and financial strategies on crowd engagement and output quality. In Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing (pp. 967- 978). ACM.

    • Mason, Winter, and Duncan J. Watts. "Financial incentives and the performance of crowds." Proceedings HComp 2009. http://crowdsourcing-class.org/readings/downloads/econ/financial- incentives-and-the-performance-of-crowds.pdf

    http://kraut.hciresearch.org/sites/kraut.hciresearch.org/files/open/yu14-ComparisonOfSocialLearning&FinancialStrategies.pdf http://crowdsourcing-class.org/readings/downloads/econ/financial-incentives-and-the-performance-of-crowds.pdf

Recommended

View more >