Adversarial Ranking Attack and Defense

Zhou, Mo; Niu, Zhenxing; Wang, Le; Zhang, Qilin; Hua, Gang

doi:10.48550/arxiv.2002.11293

Cited by 3 publications

(5 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A similar pairwise robustness definition is given and they further define pairwise stability in the sense that the absolute difference of the score differences of two instances is bounded by the norm of the perturbation multiplied by some constant. Adversarial attacks for ranking are proposed for example in Zhou et al [2020], Raval and Verma [2020], He et al [2018] and Li et al [2019]. These approaches are inherently different to our setting since they assume an already trained ranking/recommender model.…”

Section: Relation To Adversarial Attacks and Other Ranking Robustness...mentioning

confidence: 99%

Quantitative robustness of instance ranking problems

Werner¹

2021

Preprint

View full text Add to dashboard Cite

Instance ranking problems intend to recover the true ordering of the instances in a data set with a variety of applications in for example scientific, social and financial contexts. Robust statistics studies the behaviour of estimators in the presence of perturbations of the data resp. the underlying distribution and provides different concepts to characterize local and global robustness. In this work, we concentrate on the global robustness of parametric ranking problems in terms of the breakdown point which measures the fraction of samples that need to be perturbed in order to let the estimator take unreasonable values. However, existing breakdown point notions do not cover ranking problems so far. We propose to define a breakdown of the estimator as a sign-reversal of all components which causes the predicted ranking to be inverted, therefore we call our concept the order-inversal breakdown point (OIBDP). We will study the OIBDP, based on a linear model, for several different ranking problems that we carefully distinguish and provide least favorable outlier configurations, characterizations of the order-inversal breakdown point as well as sharp asymptotic upper bounds. We also outline the case of SVM-type ranking estimators.

show abstract

Section: Relation To Adversarial Attacks and Other Ranking Robustness...mentioning

confidence: 99%

Quantitative robustness of instance ranking problems

Werner¹

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Furthermore, in adversarial attacks, the gradients for guiding the attack process are usually calculated based on the knowledge of the target model, e.g., the model structure and parameters. Existing studies on adversarial attacks against IR systems mainly focus on the white-box attacks in which attackers are assumed to have complete knowledge about the target model [43,26,38,35], so the gradients can be directly acquired. However, the underlying white-box assumption does not hold in reality.…”

Section: Corresponding Authormentioning

confidence: 99%

QAIR: Practical Query-efficient Black-Box Attacks for Image Retrieval

Li¹,

Li²,

Chen³

et al. 2021

Preprint

View full text Add to dashboard Cite

We study the query-based attack against image retrieval to evaluate its robustness against adversarial examples under the black-box setting, where the adversary only has query access to the top-k ranked unlabeled images from the database. Compared with query attacks in image classification, which produce adversaries according to the returned labels or confidence score, the challenge becomes even more prominent due to the difficulty in quantifying the attack effectiveness on the partial retrieved list. In this paper, we make the first attempt in Query-based Attack against Image Retrieval (QAIR), to completely subvert the top-k retrieval results. Specifically, a new relevance-based loss is designed to quantify the attack effects by measuring the set similarity on the top-k retrieval results before and after attacks and guide the gradient optimization. To further boost the attack efficiency, a recursive model stealing method is proposed to acquire transferable priors on the target model and generate the prior-guided gradients. Comprehensive experiments show that the proposed attack achieves a high attack success rate with few queries against the image retrieval systems under the black-box setting. The attack evaluations on the real-world visual search engine show that it successfully deceives a commercial system such as Bing Visual Search with 98% attack success rate by only 33 queries on average.

show abstract

“…I chose this case study because image retrieval based on text queries is a popular, real-world use case for neural models (e.g., Google Image Search, iStock, Getty Images, etc. ), and because prior work has shown that these models can potentially be fooled using adversarial perturbations [221] (although not in the context of fairness). To strengthen my case study, I adopt a strict threat model under which the adversary cannot poison training data [99] for the ranking model, and has no knowledge of the ranking model or fairness algorithm used by the victim search engine.…”

Section: Subverting Fair Image Search With Generative Adversarial Per...mentioning

confidence: 99%

“…My work considers adversarial ML attacks on IR systems. Previous work has demonstrated successful attacks on image-to-image search systems [132,221], allowing an adversary to control the results of targeted image queries using localized patches [34] or universal adversarial perturbations 5 [125]. Other work has demonstrated attacks on text-to-text retrieval systems [167] and personalized ranking systems [92].…”

Section: Adversarial Attacksmentioning

confidence: 99%

“…Other work has demonstrated attacks on text-to-text retrieval systems [167] and personalized ranking systems [92]. Work by [221] hypothesized that targeted attacks on selected items in a ranked list might be possible using universal adversarial perturbations. None of these works consider compromising text-to-image models or group fairness objectives, as I do in this study.…”

Section: Adversarial Attacksmentioning

confidence: 99%

See 1 more Smart Citation

Algorithmic fairness in the real world: challenges and considerations

Ghosh

View full text Add to dashboard Cite

Happy moments and sad moments. Many, many cups of hot chocolate and late nights. The work I've done, the papers I've written, and the people and organizations I've collaborated with have all helped to get me through this process, and I am grateful for every experience that I have had on this journey.Foremost among those to whom I owe my thanks is my advisor, Christo Wilson. His guidance, support, and mentorship have given me the tools necessary to achieve most of the things I had listed in my Ph.D. statement of purpose. I've also been privileged enough to work with amazing individuals in the industry. Aalok, Mary, Lea, and Josh at Fiddler; Kristian, Tomo, and Rumman at Twitter; Your guidance, support and friendship have helped me jump start my career in the field, and I count myself incredibly lucky to have met and collaborated with you all. These acknowledgments would be incomplete without mentioning my friends and lab-mates in academia, both within and outside Northeastern. In no particular order, Ritam, Rajarshi, Geno,

show abstract

Adversarial Ranking Attack and Defense

Cited by 3 publications

References 50 publications

Quantitative robustness of instance ranking problems

Quantitative robustness of instance ranking problems

QAIR: Practical Query-efficient Black-Box Attacks for Image Retrieval

Algorithmic fairness in the real world: challenges and considerations

Contact Info

Product

Resources

About