Towards an Ecosystem for Reproducible Research in Computer Networking

Scheitle, Quirin; Wählisch, Matthias; Gasser, Oliver; Schmidt, Thomas C.; Carle, Georg

doi:10.1145/3097766.3097768

Cited by 52 publications

(33 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this paper, therefore, we extend RIOT [20], [21], a common IoT operating system. By selecting an open source platform and making our software publicly available we enable reproducible research [22], [23]. Based on our extensions, we gain detailed insights into system and network performance.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

A Lesson in Scaling 6LoWPAN – Minimal Fragment Forwarding in Lossy Networks

Lenders

Schmidt

Wählisch

2019

2019 IEEE 44th Conference on Local Computer Networks (LCN)

Self Cite

View full text Add to dashboard Cite

This paper evaluates two forwarding strategies for fragmented datagrams in the IoT: hop-wise reassembly and a minimal approach to direct forwarding of fragments. Direct fragment forwarding is challenged by the lack of forwarding information at subsequent fragments in 6LoWPAN and thus requires additional data at nodes. We compare the two approaches in extensive experiments evaluating reliability, end-to-end latency, and memory consumption. In contrast to previous work and due to our real-world testbed setup, we obtained different results and conclusions. Our findings indicate that direct fragment forwarding should be deployed with care, since higher packet transmission rates on the link layer can significantly reduce its reliability, which in turn can even further reduce end-to-end latency because of highly increased link layer retransmissions.

show abstract

Section: Methodsmentioning

confidence: 99%

“…We explicitly support reproducible research [22], [23]. Our experiments have been conducted in an open testbed.…”

Section: A Note On Reproducibilitymentioning

confidence: 99%

A Lesson in Scaling 6LoWPAN – Minimal Fragment Forwarding in Lossy Networks

Lenders

Schmidt

Wählisch

2019

2019 IEEE 44th Conference on Local Computer Networks (LCN)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Data Release: We publish our curated ground truth data set, acquired raw data, and developed source code for both reproducibility (cf. [5], [28]) and use by other researchers under:…”

Section: Discussionmentioning

confidence: 99%

Large-scale classification of IPv6-IPv4 siblings with variable clock skew

Scheitle

Gasser

Rouhi

et al. 2017

2017 Network Traffic Measurement and Analysis Conference (TMA)

Self Cite

View full text Add to dashboard Cite

Linking the growing IPv6 deployment to existing IPv4 addresses is an interesting field of research, be it for network forensics, structural analysis, or reconnaissance. In this work, we focus on classifying pairs of server IPv6 and IPv4 addresses as siblings, i.e., running on the same machine. Our methodology leverages active measurements of TCP timestamps and other network characteristics, which we measure against a diverse ground truth of 682 hosts. We define and extract a set of features, including estimation of variable (opposed to constant) remote clock skew. On these features, we train a manually crafted algorithm as well as a machine-learned decision tree. By conducting several measurement runs and training in cross-validation rounds, we aim to create models that generalize well and do not overfit our training data. We find both models to exceed 99% precision in train and test performance. We validate scalability by classifying 149k siblings in a large-scale measurement of 371k sibling candidates. We argue that this methodology, thoroughly cross-validated and likely to generalize well, can aid comparative studies of IPv6 and IPv4 behavior in the Internet. Striving for applicability and replicability, we release ready-to-use source code and raw data from our study.

show abstract

“…Repeatability, replicability, and reproducibility are ongoing concerns in Computer Networks [85,86] and Internet Measurement [87]. While specifying the date of when a top list was downloaded, and the date when measurements where conducted, are not necessarily sufficient to reproduce studies, they are important first steps.…”

Section: Are Studies Replicable?mentioning

confidence: 99%

A Long Way to the Top

Scheitle

Hohlfeld

Gamba

et al. 2018

Proceedings of the Internet Measurement Conference 2018

Self Cite

View full text Add to dashboard Cite

A broad range of research areas including Internet measurement, privacy, and network security rely on lists of target domains to be analysed; researchers make use of target lists for reasons of necessity or efficiency. The popular Alexa list of one million domains is a widely used example. Despite their prevalence in research papers, the soundness of top lists has seldom been questioned by the community: little is known about the lists' creation, representativity, potential biases, stability, or overlap between lists.In this study we survey the extent, nature, and evolution of top lists used by research communities. We assess the structure and stability of these lists, and show that rank manipulation is possible for some lists. We also reproduce the results of several scientific studies to assess the impact of using a top list at all, which list specifically, and the date of list creation. We find that (i) top lists generally overestimate results compared to the general population by a significant margin, often even an order of magnitude, and (ii) some top lists have surprising change characteristics, causing high day-to-day fluctuation and leading to result instability. We conclude our paper with specific recommendations on the use of top lists, and how to interpret results based on top lists with caution.

show abstract

Towards an Ecosystem for Reproducible Research in Computer Networking

Cited by 52 publications

References 4 publications

A Lesson in Scaling 6LoWPAN – Minimal Fragment Forwarding in Lossy Networks

A Lesson in Scaling 6LoWPAN – Minimal Fragment Forwarding in Lossy Networks

Large-scale classification of IPv6-IPv4 siblings with variable clock skew

A Long Way to the Top

Contact Info

Product

Resources

About