Guylhem's most recent funny hacks & thoughts

Mechanical turk experiment with extreme narcissism

A few weeks ago, I discovered Amazon Mechanical Turk and decided to try to play with it, to see what it could used for.

Digging for stupid ideas, I found my inspiration after reading a post on narcissism: basically, I uploaded a HIT asking workers to write a compliment on my official facebook page in exchange for money. How vain is that? 

The HIT description was :

 YOU MUST HAVE A FACEBOOK ACCOUNT AND BE LOGGED IN TO DO THIS !

1) Click on the facebook page link

2) Click on the “Like” button

3) Write a compliment of at least one phrase of good english, ex: “He is very helpful and good looking”.

4) Do you see anything already close to what you want to post ? Then edit your compliment to avoid duplicate data !

5) Press enter

6) Copy/paste the compliment you wrote in the following box

In order to avoid turning this lame experiment into an abuse of facebook, the scope was limited to 20 request. Also, to be fair with the workers who were not told in any way that this was an experiment, I offered $0.10 for each task since it required multiple actions, creativity and a final copy/paste of their results in a form for subsequent analysis.

The results are very interesting : every single worker *did* what was requested and wrote a compliment. The compliments are also at the bottom of this message. Since it was work for hire, I guess I’m clear with the copyrights!

I laughed my ass out after reading the comments - some were just so funny, and at least one was very cleverly worded and genuine enough to pass for what would be an actual client feedback (other applications: if you feel down, for less than $3 bucks you can have a good laugh!)

Of all these comments, only 4 i.e. less than 20% were caught by facebook antispam filter - the grammar was not perfect in these 4 comments, but what seemed the most important factor was that the worker had not uploaded a picture.

Also, every single worker did like the facebook page as requested in step 2, which is more shocking. Not a single one did unlike it after the experiment.

The implications are very simple : when playing a “popularity contest”, as some companies do, all it takes is $10 for every 100 votes, with 0% loss on the likes, and 20% loss if you care about having compliments published on a facebook without any action on your part.

I didn’t want to be a cheapskate, hence my “generous” offer by Mechanical Turk standards. However, due to the very short time it took - 1 day before being completed, the price could obviously be reduced - even more if the task was simplified into just pushing “like”.

Let’s take a conservative hypothesis and estimate that Amazon default price of $0.05 will do. To get a million like would require around $250k. Try to reduce the default price by half, say by only asking for the press of “like” and you could reach $100k per million like.

How many companies have more than a million likes on facebook? Do you think a mere $100k in advertising budget would give you that? Can criminals operating spambots match the price?

Besides the fact that many people will find such an offer good enough or need the money bad enough, I’m appalled by the conclusions for any fair player of the social game.

I wonder how prevalent is such an abuse in the social media world, following the rumors of widespread pay-for-like abuse on Apple iTunes, where the popularity of an applications can make the difference between success and failure.

The compliments were:

• He is looking smart and very clever.
• It should be helped for the future.
• Really nice information is given in english on the website. Thanks.
• I found this page very useful. it provides me lot of  good information.
• he is friendly and looking attractive.
• This guy is looking smart and has a lot of positive energy. Looks very confident
• Very handsome and working in a good company
• If you are facing any problem starting from medical field to embedded programming - your one stop solution is Guylhem, a helpful person by all means!
• he is good and concerning indeed…
• He has good personality and interesting.
• He is very innovative about himself.
• He is intelligent, charming and really good looking besides having a likeable personality.
• He is very handsome and also educated well
• NICE GUY WITH GOOD LOOKING
• this would be very helpful
• He seems smart and very talented
• He looks beautiful in sky blue shirt.
• Hai Guylhem Your very hansom
• He is a very kind hearted and honest person.
• He is handsome and good looking person
• I’m a fan of yours Guylhem :) Thanks for all that you do! :)

Take note on how one took some time to do a google search, found out I was in health-IT and wrote the comment around that.

This was disclosed to facebook, along with the following suggestions in an email exchange where they pointed out proxies:

Of course they could ask user to set up proxies, but this would create
another step and add a risk of failure. More steps raise the price per
HIT, which would require a much bigger budget and reduce the potential
number of abusers.

In this case, either they would ask the user to set up a given proxy
or a machine they control, or setup a full range of IP with proxies -
you could then catch the abuser by IP match or IP range.

Alternatively, they could ask users to set up a random proxy from a
public list, but most of these proxies are busy and finding a good one
usually requires trying 5 to 10 candidates - also increasing the
number of actions and thus reducing user compliance for a given price,
or raising the price.

The silver bullet would be monitoring Mechanical turk HIT jobs for
keywords like facebook, or match facebook URL to trigger or weight the
spam filter.

There is no perfect solution unless you can strike a deal with Amazon
to serve a cookie to HIT workers. You can only increase the price of
the abuse, try to detect it by IP match or monitor the HIT jobs to
feed the spam filter.

I don’t know your internal structure so I can’t advise you on the most
cost-effective way. By default, I would go to HIT jobs monitoring.

Guylhem