Introduction to Mechanical TurkArtificial Artificial Intelligence

AWS User Group BerlinThomas Metschke25.03.2010Peritor GmbH

Amazon Mechanical Turkis a marketplace for work.

2

Mechanical Turk Marketplace

400,000+ Workers

In 100+ Countries

Available 24/7

ProgrammaticallyAccessible

3

http://www.flickr.com/photos/diamond_rain/2543837414/

Workers Requesters

4

So there are basically

http://www.flickr.com/photos/saad/1968774 http://www.flickr.com/photos/chicagobart/4181533461

Workers

5

Mechanical Turk as a Worker

http://www.flickr.com/photos/saad/1968774

Make money by working on Human Intelligence Tasks

Workers can work from home and choose their own work hours

Your Dashboard

6

Your Dashboard

7

The number of available tasks.

Your Dashboard

8

Total Earnings and Bonuses.

Your Dashboard

9

HIT Status and Totals.

How do I get the money?

U.S. Bank account

Amazon Gift

Certificate

Bank Checks in Rupees

10

Have access to a global,on-demand, 24 x 7 workforce

Can get thousands of HITs completed in minutes

Pay only when they are satisfied with the results

Requesters

11

Mechanical Turk as a Requester

http://www.flickr.com/photos/chicagobart/4181533461

Requesting HITs

Requesters

• define and create your HITs

• load HITs to Mechanical Turk

Workers

• work on your HITs

• submit results

Requesters

• approve and pay for completed HITs

• use the results

12

Design HITs

13

Enter Properties Design Layout

Design HITs - faster

Take developer and useCSV filesSOAP / REST orAmazon Mechanical Turk developer tools

14

What would it look like

http://mechanicalturk.amazonaws.com/?Service=AWSMechanicalTurkRequester&AWSAccessKeyId=[the Requester's Access Key ID]&Version=2008-08-02&Operation=CreateHIT&Signature=[signature for this request]&Timestamp=[your system's local time]&Title=Location%20and%20Photograph%20Identification&Description=Select%20the%20image%20that%20best%20represents&Reward.1.Amount=5 &Reward.1.CurrencyCode=USD&Question=[URL-encoded question data]&AssignmentDurationInSeconds=30&LifetimeInSeconds=604800 &Keywords=location,%20photograph,%20image,%20identification,%20opinion

15

Publish HITs

credit card debit card

Amazon Payments account

U.S. bank account

HITs have to be paid in advance

Amazon takes 10% on top

16

Use Mechanical Turk for

Work that requires Human Judgment

Work that algorithms cannot completely solve

Work that has unpredictable or spiky volume

17

Improving Data Quality

Are these two businesses the same?

Peritor GmbH

Blücherstraße 2210961 Berlin

http://peritor.com

YES

Peritor Consulting

Blücherstraße 22Hof III Aufgang 6

10961 Berlin

NO

Background Data is the company’s business

Accuracy and breadth are key to differentiation

Process 1 MM data points to ingest each day

200 data sources

Problem Data needs to be normalized,

enhanced and de-dupped

Algorithms could get data about 70% clean

18

YES NO

Moderating UserGenerated Content

Background User generated content is a key part

of a web 2.0 experience

Process Millions of photos uploaded every

day

Problem Need to ensure user generated

content meets site guidelines

19

Is this image explicit?

http://www.flickr.com/photos/cmak/1521356521/

Categorization

What kind of dress is this?

Cocktail

Bridal dress

Background Consumers need to be able to

quickly find a product when shopping online

The Business Process Millions of new products are

introduced everyday

Products are sourced from hundreds of merchants and manufacturers, each with their own taxonomy

Problem Need to properly categorize new

products quickly in order to monetize

20

http://www.flickr.com/photos/34801476@N00/296743627/

Optimizing your HITs for

Price

SpeedAccuracy

21

Check it out!

http://mturk.comhttp://turkers.proboards.com

22

© Peritor GmbH - Alle Rechte vorbehalten

Peritor GmbHBlücherstr. 22, Hof III Aufgang 610961 Berlin

Tel.: +49 (0)30 69 20 09 84 0Fax: +49 (0)30 69 20 09 84 9

Internet: www.peritor.comE-Mail: info@peritor.com

Thank you for your attention