Text Captcha Is Dead? A Large Scale Deployment and Empirical Study

Abstract

advCAPTCHA can be adapted/fine-tuned to accommodate the attack model evolution. Overall, advCAPTCHA can serve as a key enabler for generating robust captchas in practice and providing useful guidelines for captcha developers and practitioners.

Robust Text CAPTCHAs Using Adversarial Examples

“…The accuracy of these workers is 91.25%, which is comparable to human accuracy on ordinary CAPTCHAs reported in previous works, such as 89.8% in [32]. Besides, in terms of time cost, the average time the MTurk workers spend on each CAPTCHA is 8.5s, which is also comparable to 7.5s on ordinary (nonadversarial) CAPTCHAs reported in [32]. This verifies the usability of our CAPTCHAs -they are similarly easy for human users compared to ordinary CAPTCHAs while they have the ability to defend against automatic CAPTCHA solvers.…”

“…Specifically, we sample 400 our adversarial CAPTCHAs consisting of four characters each that is generated under the default setting described in Section 4.1 , and present them to MTurk workers as simulated users of websites using our CAPTCHAs. The accuracy of these workers is 91.25%, which is comparable to human accuracy on ordinary CAPTCHAs reported in previous works, such as 89.8% in [32]. Besides, in terms of time cost, the average time the MTurk workers spend on each CAPTCHA is 8.5s, which is also comparable to 7.5s on ordinary (nonadversarial) CAPTCHAs reported in [32].…”

“…However, [23] applies white-box attacks and requires access to CAPTCHA solvers of attackers, which is an impractical assumption instead of making the generation solver-agnostic. [33,32] propose to apply black-box attack to MNIST letters [11] or existing CAPTCHA images, but [32] still have a high recognition rate by CAPTCHA solvers (higher than 30% as they show in the paper) and rely on queries to the inferred attack models to adapt their attacking strategies. Moreover, none of them considered a more practical situation when attackers collect a training dataset from deployed CAPTCHAs with human annotation, to train a stronger CAPTCHA solver.…”

“…The first row lists three traditional text-based CAPTCHAs generated by the BotDetect CAPTCHA Generator (https://captcha.com). The second row shows adversarial CAPTCHAs from prior works [33,32]. The third row presents some samples generated by our RTC method, which is more user-friendly and can be more diverse.…”

“…On the other hand, despite the high capacity of deep neural networks (DNN), research shows DNNs are vulnerable to small input perturbations [41,16], which casts light on CAPTCHA design. To defend against deep neural network CAPTCHA solvers, some works constructed adversarial examples to fool deep models [23,33,32]. However, [23] applies white-box attacks and requires access to CAPTCHA solvers of attackers, which is an impractical assumption instead of making the generation solver-agnostic.…”