CamBench -- Cryptographic API Misuse Detection Tool Benchmark Suite

Schlichtig, Michael; Wickert, Anna-Katharina; Krüger, Stefan; Bodden, Eric; Mezini, Mira

doi:10.48550/arxiv.2204.06447

Cited by 1 publication

(2 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Apart from devising detectors, researchers have also prepared two benchmarks, CryptoAPI-Bench [16] and CamBench [17].…”

Section: Related Workmentioning

confidence: 99%

“…Instead, the condition is whether the targeted method (e.g., verify()) is empty, which is checked by verifying that there are no statements of invocation or field loading 17 . In this case, a simple verify() method that always returns false, which rejects any hostnames, would be identified as an insecure implementation of HostnameVerifier.…”

Section: Generalizability Of False Alarm Patternsmentioning

confidence: 99%

See 1 more Smart Citation

Towards Precise Reporting of Cryptographic Misuses

Chen,

Liu,

et al. 2024

Proceedings 2024 Network and Distributed System Security Symposium

View full text Add to dashboard Cite

In the last decade, a series of papers were published on using static analysis to detect cryptographic API misuse. In each paper, apps are checked against a set of rules to see if violations exist. A common theme among these papers is that rule violations are plentiful, often at the scale of thousands. Interestingly, while much effort went into tackling false negatives, curiously, not much has been said on (1) whether the misuse alarms are indeed correct and meaningful, and ( 2) what can future work improve upon apart from finding more misuses.In this paper, we take a deep dive into the rule violations reported by various academic papers as well as the rules, models and implementations of their detectors, in an attempt to (1) explain the gap between their misuse alarms and actual vulnerabilities, and (2) shed light on possible directions for improving the precision and usability of misuse detectors. Results of our analysis suggest that the small-scale inspections done by previous work had some unfortunate blind-spots, leaving problems in their rules, models, and implementations unnoticed, which in turn led to unnecessary overestimation of misuses (and vulnerabilities). To facilitate future research on the topic, we distill these avoidable false alarms into high-level patterns that capture their root causes, and discuss design, evaluation and reporting strategies that can improve the precision of misuse findings. Furthermore, to demonstrate the generalizability of these false alarm patterns and improvement directions, we also investigate a popular industry detector and a dynamic detector, and discuss how some of the false alarm patterns do and do not apply to them. Our findings suggest that the problem of precisely reporting cryptographic misuses still has much room for future work to improve upon.

show abstract

“…Apart from devising detectors, researchers have also prepared two benchmarks, CryptoAPI-Bench [16] and CamBench [17].…”

Section: Related Workmentioning

confidence: 99%

Section: Generalizability Of False Alarm Patternsmentioning

confidence: 99%