Social Turing Tests: Crowdsourcing Sybil Detection

ABSTRACT

As popular tools for spreading spam and malware, Sybils (or fake accounts) pose a serious threat to online communities such as Online Social Networks (OSNs). Today, sophisti- cated attackers are creating realistic Sybils that effectively befriend legitimate users, rendering most automated Sybil detection techniques ineffective. In this paper, we explore the feasibility of a crowdsourced Sybil detection system for OSNs. We conduct a large user study on the ability of hu- mans to detect today's Sybil accounts, using a large corpus of ground-truth Sybil accounts from the Facebook and Ren- ren networks. We analyze detection accuracy by both "ex- perts" and "turkers" under a variety of conditions, and find that while turkers vary significantly in their effectiveness, experts consistently produce near-optimal results. We use these results to drive the design of a multi-tier crowdsourc- ing Sybil detection system. Using our user-study data, we show that this system is scalable, and can be highly effective either as a standalone system or as a complementary tech- nique to current tools.

Award ID: 0916307