Ranking Robustness Under Adversarial Document Manipulations

Goren, Gregory; Kurland, Oren; Tennenholtz, Moshe; Raiber, Fiana

doi:10.1145/3209978.3210012

Cited by 23 publications

(29 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As for evaluation measure, we employ Top Change (TC) and Kendall's-𝜏 distance (KT) following the previous work [25]. Suppose that 𝐿 (𝑞 𝑡 ) is a ranked document list with respect to a given query 𝑞 𝑡 achieved by a ranking model.…”

Section: Metric Of Defensive Ability Against Document Manipulationmentioning

confidence: 99%

“…To evaluate the defensive ability against document manipulation, we follow the previous work [25] to conduct experiments on the ASRC and ClueWeb09-B dataset. The detailed statistics of these datasets are shown in Table 7.…”

Section: Datasetsmentioning

confidence: 99%

“…• Implementation of Document Manipulation. We use the public dataset ASRC that was created as a result of an on-going ranking competition [59], to measure the document manipulation [25]. Specifically, the ranking competition involved 31 repeated matches that lasted for 8 rounds, where each match was with respect to a different query.…”

Section: Datasetsmentioning

confidence: 99%

“…Then, we propose the drop rate metric to measure the defensive ability against query attack. For document manipulation, we conduct experiments on the ASRC dataset [59] and follow the previous work [25] to measure the robustness of different ranking models.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Are Neural Ranking Models Robust?

Chen¹,

Zhang²,

Guo³

et al. 2021

Preprint

View full text Add to dashboard Cite

Recently, we have witnessed the bloom of neural ranking models in the information retrieval (IR) field. So far, much effort has been devoted to developing effective neural ranking models that can generalize well on new data. There has been less attention paid to the robustness perspective. Unlike the effectiveness which is about the average performance of a system under normal purpose, robustness cares more about the system performance in the worst case or under malicious operations instead. When a new technique enters into the real-world application, it is critical to know not only how it works in average, but also how would it behave in abnormal situations. So we raise the question in this work: Are neural ranking models robust? To answer this question, firstly, we need to clarify what we refer to when we talk about the robustness of ranking models in IR. We show that robustness is actually a multi-dimensional concept and there are three ways to define it in IR: 1) The performance variance under the independent and identically distributed (I.I.D.) setting; 2) The out-of-distribution (OOD) generalizability; and 3) The defensive ability against adversarial operations. The latter two definitions can be further specified into two different perspectives respectively, leading to 5 robustness tasks in total. Based on this taxonomy, we build corresponding benchmark datasets, design empirical experiments, and systematically analyze the robustness of several representative neural ranking models against traditional probabilistic ranking models and learning-to-rank (LTR) models. The empirical results show that there is no simple answer to our question. While neural ranking models are less robust against other IR models in most cases, some of them can still win 1 out of 5 tasks. This is the first comprehensive study on the robustness of neural ranking models. We believe the way we study the robustness as well as our findings would be beneficial to the IR community. We will also release all the data and codes to facilitate the future research in this direction.CCS Concepts: • Information systems → Retrieval models and ranking.

show abstract

Section: Metric Of Defensive Ability Against Document Manipulationmentioning

confidence: 99%

Section: Datasetsmentioning

confidence: 99%

Section: Datasetsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Are Neural Ranking Models Robust?

Chen¹,

Zhang²,

Guo³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Large original distances carry over to a higher robustness. Note that this relates our work to Goren et al [2018] where point-wise and pair-wise robustness for ranking problems have been defined where similar distances are considered.…”

Section: Cell-wise Scenariosmentioning

confidence: 97%

Trimming Stability Selection increases variable selection robustness

Werner¹

2021

Preprint

View full text Add to dashboard Cite

Contamination can severely distort an estimator unless the estimation procedure is suitably robust. This is a well-known issue and has been addressed in Robust Statistics, however, the relation of contamination and distorted variable selection has been rarely considered in literature. As for variable selection, many methods for sparse model selection have been proposed, including Stability Selection which is a meta-algorithm based on some variable selection algorithm in order to immunize against particular data configurations. We introduce the variable selection breakdown point that quantifies the number of cases resp. cells that have to be contaminated in order to let no relevant variable be detected. We show that particular outlier configurations can completely mislead model selection and argue why even cell-wise robust methods cannot fix this problem. We combine the variable selection breakdown point with resampling, resulting in the Stability Selection breakdown point that quantifies the robustness of Stability Selection. We propose a trimmed Stability Selection which only aggregates the models with the lowest in-sample losses so that, heuristically, models computed on heavily contaminated resamples should be trimmed away. We provide a short simulation study that reveals both the potential of our approach as well as the fragility of variable selection, even for an extremely small cell-wise contamination rate.

show abstract

Adversarial Ranking Attack and Defense

Zhou

Niu²,

Wang³

et al. 2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Deep Neural Network classifiers are vulnerable to adversarial attack, where an imperceptible perturbation could result in misclassification. However, the vulnerability of DNN-based image ranking systems remains under-explored. In this paper, we propose two attacks against deep ranking systems, i.e., Candidate Attack and Query Attack, that can raise or lower the rank of chosen candidates by adversarial perturbations. Specifically, the expected ranking order is first represented as a set of inequalities, and then a triplet-like objective function is designed to obtain the optimal perturbation. Conversely, an anti-collapse triplet defense is proposed to improve the ranking model robustness against all proposed attacks, where the model learns to prevent the positive and negative samples being pulled close to each other by adversarial attack. To comprehensively measure the empirical adversarial robustness of a ranking model with our defense, we propose an empirical robustness score, which involves a set of representative attacks against ranking models. Our adversarial ranking attacks and defenses are evaluated on MNIST, Fashion-MNIST, CUB200-2011, CARS196 and Stanford Online Products datasets. Experimental results demonstrate that a typical deep ranking system can be effectively compromised by our attacks. Nevertheless, our defense can significantly improve the ranking system robustness, and simultaneously mitigate a wide range of attacks.

show abstract

Ranking Robustness Under Adversarial Document Manipulations

Cited by 23 publications

References 20 publications

Are Neural Ranking Models Robust?

Are Neural Ranking Models Robust?

Trimming Stability Selection increases variable selection robustness

Adversarial Ranking Attack and Defense

Contact Info

Product

Resources

About