collaborative human computing
DESCRIPTION
Collaborative Human Computing. Zack Zhu March 31, 2010 Seminar for Distributed Computing. Distributed Computing. ...redefined: Distributed Thinking. “ Crowdsourcing ”. + Internet. Human Resource = $$$$$!!. + Web 2.0. $. $. Crowdsourcing. Search for Extraterrestrial Intelligence - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Collaborative Human Computing](https://reader035.vdocuments.mx/reader035/viewer/2022062310/56816384550346895dd467fb/html5/thumbnails/1.jpg)
1
Collaborative Human Computing
Zack ZhuMarch 31, 2010
Seminar for Distributed Computing
![Page 2: Collaborative Human Computing](https://reader035.vdocuments.mx/reader035/viewer/2022062310/56816384550346895dd467fb/html5/thumbnails/2.jpg)
2
Distributed Computing...
![Page 3: Collaborative Human Computing](https://reader035.vdocuments.mx/reader035/viewer/2022062310/56816384550346895dd467fb/html5/thumbnails/3.jpg)
3
...redefined: Distributed Thinking
![Page 4: Collaborative Human Computing](https://reader035.vdocuments.mx/reader035/viewer/2022062310/56816384550346895dd467fb/html5/thumbnails/4.jpg)
4
“Crowdsourcing”Human Resource = $$$$$!!+
Internet+
Web 2.0 $ $
![Page 5: Collaborative Human Computing](https://reader035.vdocuments.mx/reader035/viewer/2022062310/56816384550346895dd467fb/html5/thumbnails/5.jpg)
5
Crowdsourcing• Search for Extraterrestrial Intelligence • Earliest project utilizing the idea
(launched in May 1999)• Voluntary distributed computing
![Page 6: Collaborative Human Computing](https://reader035.vdocuments.mx/reader035/viewer/2022062310/56816384550346895dd467fb/html5/thumbnails/6.jpg)
6
Distributed Thinking +
Crowdsourcing
Collaborative Human Computing
![Page 7: Collaborative Human Computing](https://reader035.vdocuments.mx/reader035/viewer/2022062310/56816384550346895dd467fb/html5/thumbnails/7.jpg)
7
Collaborative Human Computing
![Page 8: Collaborative Human Computing](https://reader035.vdocuments.mx/reader035/viewer/2022062310/56816384550346895dd467fb/html5/thumbnails/8.jpg)
• Crowdsourced R&D
10
![Page 9: Collaborative Human Computing](https://reader035.vdocuments.mx/reader035/viewer/2022062310/56816384550346895dd467fb/html5/thumbnails/9.jpg)
11
• Why it works:– Solver Diversity– Workforce Mentality– Vetted Input
![Page 10: Collaborative Human Computing](https://reader035.vdocuments.mx/reader035/viewer/2022062310/56816384550346895dd467fb/html5/thumbnails/10.jpg)
12
![Page 11: Collaborative Human Computing](https://reader035.vdocuments.mx/reader035/viewer/2022062310/56816384550346895dd467fb/html5/thumbnails/11.jpg)
13
![Page 12: Collaborative Human Computing](https://reader035.vdocuments.mx/reader035/viewer/2022062310/56816384550346895dd467fb/html5/thumbnails/12.jpg)
14
Mechanical Turk
Human Intelligence Tasks (HIT)
– Relatively trivial for users– Difficult to automate– Low payout: $0.01-$5/HIT
For example:– Image tagging– Write a review (movies, CDs)– Rank a series of pictures
Virtual Sweatshop????
![Page 13: Collaborative Human Computing](https://reader035.vdocuments.mx/reader035/viewer/2022062310/56816384550346895dd467fb/html5/thumbnails/13.jpg)
15
How about harnessing the power of masses for
FREE and Get Paid?
![Page 14: Collaborative Human Computing](https://reader035.vdocuments.mx/reader035/viewer/2022062310/56816384550346895dd467fb/html5/thumbnails/14.jpg)
16
![Page 15: Collaborative Human Computing](https://reader035.vdocuments.mx/reader035/viewer/2022062310/56816384550346895dd467fb/html5/thumbnails/15.jpg)
17
6,969,696,969 votes / 85%
![Page 16: Collaborative Human Computing](https://reader035.vdocuments.mx/reader035/viewer/2022062310/56816384550346895dd467fb/html5/thumbnails/16.jpg)
18
To see the next picture…
Lesson: Give the crowd something they need...
![Page 17: Collaborative Human Computing](https://reader035.vdocuments.mx/reader035/viewer/2022062310/56816384550346895dd467fb/html5/thumbnails/17.jpg)
19
• Initiative to digitize typeset text– Today: OCR fails to recognize 20% of scanned text
• How?1. Scanned page2. Decipher with 2 independent OCR programs3. List suspicious words (no consensus)4. Distort and send out as reCaptcha
![Page 18: Collaborative Human Computing](https://reader035.vdocuments.mx/reader035/viewer/2022062310/56816384550346895dd467fb/html5/thumbnails/18.jpg)
20
Unrecognized WordControl Word (known from previous reCaptchas)
6. Enter unrecognized word into database (consensus established between n people)
![Page 19: Collaborative Human Computing](https://reader035.vdocuments.mx/reader035/viewer/2022062310/56816384550346895dd467fb/html5/thumbnails/19.jpg)
21
3. Natural Fading
1. Scanning Noise
2. Artificial Transformation
Is it secure?
• More secure than conventional Captchas– Anti-captcha algorithms– 100% Successful in failing
anti-captcha algorithms– Computer-generated
Captcha 90% successful
![Page 20: Collaborative Human Computing](https://reader035.vdocuments.mx/reader035/viewer/2022062310/56816384550346895dd467fb/html5/thumbnails/20.jpg)
22
Is it successful?
– Accuracy of 99.1% • Human: 99%• Standard OCR: 83.5%
– 440 Million words deciphered in the 1st year (~17,600 books)
– 35 Million words/day (March, 2009)
![Page 21: Collaborative Human Computing](https://reader035.vdocuments.mx/reader035/viewer/2022062310/56816384550346895dd467fb/html5/thumbnails/21.jpg)
23
9 BILLION human-hours/year
![Page 22: Collaborative Human Computing](https://reader035.vdocuments.mx/reader035/viewer/2022062310/56816384550346895dd467fb/html5/thumbnails/22.jpg)
24
gwap
![Page 23: Collaborative Human Computing](https://reader035.vdocuments.mx/reader035/viewer/2022062310/56816384550346895dd467fb/html5/thumbnails/23.jpg)
25
gwap
Image Tagging
![Page 24: Collaborative Human Computing](https://reader035.vdocuments.mx/reader035/viewer/2022062310/56816384550346895dd467fb/html5/thumbnails/24.jpg)
26
• Is it fun?
– 15 million agreements (tags) from 75,000 players– 200,000 regular players– Many people play >20 hours a week– Playing streaks of >15 hours
![Page 25: Collaborative Human Computing](https://reader035.vdocuments.mx/reader035/viewer/2022062310/56816384550346895dd467fb/html5/thumbnails/25.jpg)
27
• Why?– Sense of connection with your partner
“...the two of you are bringing your minds together in ways lovers would envy.”
• Bush• President• Man• Yuck
![Page 26: Collaborative Human Computing](https://reader035.vdocuments.mx/reader035/viewer/2022062310/56816384550346895dd467fb/html5/thumbnails/26.jpg)
28
Single Player Version?
• Record moves of players with time stamps• Play pre-recorded moves
• ESN Game– Moves recorded (Player A): (0:02) goddess; (0:03) ziyi(0:04) thoughtful; (0:08) hot
Taboo Words Time Player 1 Bot (Player A)
Woman 0:01 ziyi
Beautiful 0:02 asian goddess
Gorgeous 0:03 model ziyi
![Page 27: Collaborative Human Computing](https://reader035.vdocuments.mx/reader035/viewer/2022062310/56816384550346895dd467fb/html5/thumbnails/27.jpg)
29
…0 Player?
Moves recordedBot 1: (0:02) goddess; (0:04) face; (0:08) hot (0:14) flowersBot 2: (0:01) flowers; (0:02) model; (0:03) asian; (0:09) girl
![Page 28: Collaborative Human Computing](https://reader035.vdocuments.mx/reader035/viewer/2022062310/56816384550346895dd467fb/html5/thumbnails/28.jpg)
30
Generalization
• Game <-> algorithm: Input-Output
• Symmetric/Parallel: n player completing the same task
Consensus (e.g. ESN Game)
Store: applePlayer 1: “pear, orange, apple”
Player 2: “…apple…”
![Page 29: Collaborative Human Computing](https://reader035.vdocuments.mx/reader035/viewer/2022062310/56816384550346895dd467fb/html5/thumbnails/29.jpg)
31
![Page 30: Collaborative Human Computing](https://reader035.vdocuments.mx/reader035/viewer/2022062310/56816384550346895dd467fb/html5/thumbnails/30.jpg)
32
User-Created Pings
Trunk
Trunk/Tusk/Ear EarTusk
![Page 31: Collaborative Human Computing](https://reader035.vdocuments.mx/reader035/viewer/2022062310/56816384550346895dd467fb/html5/thumbnails/31.jpg)
33
Hints:
![Page 32: Collaborative Human Computing](https://reader035.vdocuments.mx/reader035/viewer/2022062310/56816384550346895dd467fb/html5/thumbnails/32.jpg)
34
Generalization
Asymmetric/Sequential: Player 1’s output fed to Player 2’s input
“Object”
Player 1’s Task
Player 2’s Guess “Object”
![Page 33: Collaborative Human Computing](https://reader035.vdocuments.mx/reader035/viewer/2022062310/56816384550346895dd467fb/html5/thumbnails/33.jpg)
35
Security Measures
Pretty standard …
• Player queue• IP Check (location proximity)
![Page 34: Collaborative Human Computing](https://reader035.vdocuments.mx/reader035/viewer/2022062310/56816384550346895dd467fb/html5/thumbnails/34.jpg)
36
More interesting…
• Test image/behaviour matching• Aggregated consensus• reCaptcha the gwap games?
Security Measures
![Page 35: Collaborative Human Computing](https://reader035.vdocuments.mx/reader035/viewer/2022062310/56816384550346895dd467fb/html5/thumbnails/35.jpg)
37
References• L. von Ahn, M. Blum (2006). Peekaboom: A game for locating objects in
images. In ACM CHI.• L. von Ahn, B. Maurer, C. McMillen, D. Abraham, and M. Blum.
“reCAPTCHA: Human-Based Character Recognition via Web Security Measures.” Science, September 2008.
• J. Howe. “The Rise of Crowd Surfing”, Wired, June 2006.• D. P. Anderson , J. Cobb , E. Korpela , M. Lebofsky , D. Werthimer,
“SETI@home: an experiment in public-resource computing,” Communications of the ACM, v.45 n.11, p.56-61, November 2002 .
• gwap, http://www.gwap.com• Amazon Mechanical Turk, https://www.mturk.com/mturk/welcome• Google Tech Talk, http://www.cs.cmu.edu/~biglou/
![Page 36: Collaborative Human Computing](https://reader035.vdocuments.mx/reader035/viewer/2022062310/56816384550346895dd467fb/html5/thumbnails/36.jpg)
38
Discussion
• Net productivity?
• Declining popularity with time, repackagable?
• …your input?