On the recall of static call graph construction in practice

Sui, Li; Dietrich, Jens; Tahir, Amjed; Fourtounis, George

doi:10.1145/3377811.3380441

Cited by 33 publications

(13 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Static analysis, such as call graph construction for Java, is known to have limitations [44]. The effectiveness of dynamic analysis, such as Steady's, is also dependent on having a good test-suite and test coverage.…”

Section: Code Analysis-based Metricsmentioning

confidence: 99%

A comparative study of vulnerability reporting by software composition analysis tools

Imtiaz

Thorne

Williams

2021

Proceedings of the 15th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM)

View full text Add to dashboard Cite

Background: Modern software uses many third-party libraries and frameworks as dependencies. Known vulnerabilities in these dependencies are a potential security risk. Software composition analysis (SCA) tools, therefore, are being increasingly adopted by practitioners to keep track of vulnerable dependencies. Aim: The goal of this study is to understand the difference in vulnerability reporting by various SCA tools. Understanding if and how existing SCA tools differ in their analysis may help security practitioners to choose the right tooling and identify future research needs. Method: We present an in-depth case study by comparing the analysis reports of 9 industry-leading SCA tools on a large web application, OpenMRS, composed of Maven (Java) and npm (JavaScript) projects. Results: We find that the tools vary in their vulnerability reporting. The count of reported vulnerable dependencies ranges from 17 to 332 for Maven and from 32 to 239 for npm projects across the studied tools. Similarly, the count of unique known vulnerabilities reported by the tools ranges from 36 to 313 for Maven and from 45 to 234 for npm projects. Our manual analysis of the tools' results suggest that accuracy of the vulnerability database is a key differentiator for SCA tools. Conclusion: We recommend that practitioners should not rely on any single tool at the present, as that can result in missing known vulnerabilities. We point out two research directions in the SCA space: i) establishing frameworks and metrics to identify false positives for dependency vulnerabilities; and ii) building automation technologies for continuous monitoring of vulnerability data from open source package ecosystems.

show abstract

Section: Code Analysis-based Metricsmentioning

confidence: 99%

A comparative study of vulnerability reporting by software composition analysis tools

Imtiaz

Thorne

Williams

2021

Proceedings of the 15th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM)

View full text Add to dashboard Cite

show abstract

“…Sui et al [33] evaluated the recall of call-graph construction and present how it impacts the algorithms in practice. Their evaluation expose the problems in the state-of-the-art esp.…”

Section: Related Workmentioning

confidence: 99%

Effects of Program Representation on Pointer Analyses — An Empirical Study

Prakash

Tiwari

Hammer

2021

Fundamental Approaches to Software Engineering

View full text Add to dashboard Cite

Static analysis frameworks, such as Soot and Wala, are used by researchers to prototype and compare program analyses. These frameworks vary on heap abstraction, modeling library classes, and underlying intermediate program representation (IR). Often, these variations pose a threat to the validity of the results as the implications of comparing the same analysis implementation in different frameworks are still unexplored. Earlier studies have focused on the precision, soundness, and recall of the algorithms implemented in these frameworks; however, little to no work has been done to evaluate the effects of program representation. In this work, we fill this gap and study the impact of program representation on pointer analysis. Unfortunately, existing metrics are insufficient for such a comparison due to their inability to isolate each aspect of the program representation. Therefore, we define two novel metrics that measure these analyses’ precision after isolating the influence of class-hierarchy and intermediate representation. Our results establish that the minor differences in the class hierarchy and IR do not impact program analysis significantly. Besides, they reveal the sources of unsoundness that aid researchers in developing program analysis.

show abstract

“…Second, static analysis lacks important information available only at runtime, and making conservative assumptions introduces false negatives, i.e. reduces recall [Livshits et al 2015;Sui et al 2020].…”

Section: Recent Trends In Static Analysismentioning

confidence: 99%

“…addressing issues around false negatives [Livshits et al 2015]. Sui et al [2020] have demonstrated the importance of this-standard call graph construction algorithms suffer from significant numbers of false negatives, undermining the utility of graph-and points-to-based analyses. Moreover, there are multiple sources of unsoundness, each having a significant impact, while research has focused on only one category (reflection) [Livshits et al 2005;.…”

Section: Recent Trends In Static Analysismentioning

confidence: 99%

Putting the Semantics into Semantic Versioning

Lam¹,

Dietrich²,

Pearce³

2020

Preprint

Self Cite

View full text Add to dashboard Cite

The long-standing aspiration for software reuse has made astonishing strides in the past few years. Many modern software development ecosystems now come with rich sets of publicly-available components contributed by the community. Downstream developers can leverage these upstream components, boosting their productivity.However, components evolve at their own pace. This imposes obligations on and yields benefits for downstream developers, especially since changes can be breaking, requiring additional downstream work to adapt to. Upgrading too late leaves downstream vulnerable to security issues and missing out on useful improvements; upgrading too early results in excess work. Semantic versioning has been proposed as an elegant mechanism to communicate levels of compatibility, enabling downstream developers to automate dependency upgrades.While it is questionable whether a version number can adequately characterize version compatibility in general, we argue that developers would greatly benefit from tools such as semantic version calculators to help them upgrade safely. The time is now for the research community to develop such tools: large component ecosystems exist and are accessible, component interactions have become observable through automated builds, and recent advances in program analysis make the development of relevant tools feasible. In particular, contracts (both traditional and lightweight) are a promising input to semantic versioning calculators, which can suggest whether an upgrade is likely to be safe.

show abstract

On the recall of static call graph construction in practice

Cited by 33 publications

References 46 publications

A comparative study of vulnerability reporting by software composition analysis tools

A comparative study of vulnerability reporting by software composition analysis tools

Effects of Program Representation on Pointer Analyses — An Empirical Study

Putting the Semantics into Semantic Versioning

Contact Info

Product

Resources

About