Identifying, analyzing, and modeling flashcrowds in BitTorrent

Zhang, Boxun; Iosup, Alexandru; Pouwelse, Johan; Epema, Dick

doi:10.1109/p2p.2011.6038742

Cited by 27 publications

(16 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Figure 8(b) plots the percentage of peers with higher progress among all the peers interesting to a peer, across all combinations of piece and peer selection policies. We have used an arrival rate of 0.05peers/s, which is the average value for popular content [31]; the results for other peer arrival rates showed similar results. In each scenario, the majority of the peers that are interesting to a peer have higher progress than itself, confirming our hypothesis.…”

Section: Analyzing Interestsmentioning

confidence: 75%

BitTorrent-like P2P approaches for VoD: A comparative study

et al. 2013

View full text Add to dashboard Cite

The enormous popularity of Video on Demand (VoD) has attracted substantial research attention into the effective use of peer-to-peer (P2P) architectures to provide solutions at large-scale. In particular, the high efficiency of BitTorrent has inspired many P2P protocols for VoD. However, these protocols use different approaches to adapt the design of Bittorrent to VoD, and in most cases their performance has been evaluated separately and in limited scenarios. As a consequence, the research community still lacks a clear understanding of how these protocols compare against each other and how well each of them would work in real world conditions, where, for instance, peers have heterogeneous bandwidths, may freeride or may be located behind NAT/firewall.In this paper, we propose a simulation based methodology which aims at putting forward a common base for comparing the performance of these different protocols under a wide range of conditions. We show that, despite their considerable differences, (i) existing BitTorrent-like VoD approaches all share some characteristics, such as that their bandwidth reciprocity based methods to incentivize cooperation do not always yield an optimal overall performance. Furthermore, we demonstrate that (ii) in these protocols there is a trade-off between QoS and resilience to freeriding and malicious attacks. We also discover that, (iii) when peers doing streaming coexist with peers doing traditional file transfer, the latter actually benefit from this coexistence, at the expenses of the former. Finally, we show that (iv) early departures of peers from the system do not significantly affect the QoS de- livered, while jumping to a different position in the file has a bigger negative impact. Overall, our findings provide important implications for both VoD service providers and future system designers. On the one hand, our results can guide VoD service providers in selecting the most appropriate protocol for a given environment. On the other hand, exposing the flaws of current approaches will help researchers in improving them and/or designing better ones.

show abstract

Section: Analyzing Interestsmentioning

confidence: 75%

BitTorrent-like P2P approaches for VoD: A comparative study

et al. 2013

View full text Add to dashboard Cite

show abstract

“…The P2P analyst can extract information about the service level provided by BitTorrent to its users by examining the ratio between seeders and leechers in swarms, which is known to be correlated with download speed [23] and with severe degradation of performance during flashcrowds [24]. The life time of swarms is an indicator of reliability [25]: for how long are files available in the system?…”

Section: P2p Analyst Questionsmentioning

confidence: 99%

“…These larger subsets include months of data for several BitTorrent trackers, and are thus representative for the types of measurement studies already published about BitTorrent [22], [24]. Table IV summarizes the characteristics of the 100 GB BTWorld subset used in this work.…”

Section: A Experimental Setupmentioning

confidence: 99%

The BTWorld use case for big data analytics: Description, MapReduce logical workflow, and empirical evaluation

Hegeman

Ghit

Capotă

et al. 2013

2013 IEEE International Conference on Big Data

Self Cite

View full text Add to dashboard Cite

Abstract-The commoditization of big data analytics, that is, the deployment, tuning, and future development of big data processing platforms such as MapReduce, relies on a thorough understanding of relevant use cases and workloads. In this work we propose BTWorld, a use case for time-based big data analytics that is representative for processing data collected periodically from a global-scale distributed system. BTWorld enables a datadriven approach to understanding the evolution of BitTorrent, a global file-sharing network that has over 100 million users and accounts for a third of today's upstream traffic. We describe for this use case the analyst questions and the structure of a multi-terabyte data set. We design a MapReduce-based logical workflow, which includes three levels of data dependencyinter-query, inter-job, and intra-job-and a query diversity that make the BTWorld use case challenging for today's big data processing tools; the workflow can be instantiated in various ways in the MapReduce stack. Last, we instantiate this complex workflow using Pig-Hadoop-HDFS and evaluate the use case empirically. Our MapReduce use case has challenging features: small (kilobytes) to large (250 MB) data sizes per observed item, excellent (10 −6 ) and very poor (10 2 ) selectivity, and short (seconds) to long (hours) job duration.

show abstract

“…Intuitively, because in peer-to-peer systems the users provide additional service capacity while being online, a (long) flashcrowd can lead to a beneficial accumulation of (bandwidth) capacity rather than to poor performance. However, even through 2009 and 2010 the performance of BitTorrent users during flashcrowds could be up to an order of magnitude lower than the performance observed in normal conditions [25].…”

Section: Failures In Real-world Distributed Systemsmentioning

confidence: 83%

The Failure Trace Archive: Enabling the comparison of failure measurements and models of distributed systems

Javadi

Kondo

Iosup

et al. 2013

Journal of Parallel and Distributed Computing

Self Cite

View full text Add to dashboard Cite

With the increasing presence, scale, and complexity of distributed systems, resource failures are becoming an important and practical topic of computer science research. While numerous failure models and failure-aware algorithms exist, their comparison has been hampered by the lack of public failure data sets and data processing tools. To facilitate the design, validation, and comparison of fault-tolerant models and algorithms, we have created the Failure Trace Archive (FTA)-an online, public repository of failure traces collected from diverse parallel and distributed systems. In this work, we first describe the design of the archive, in particular of the standard FTA data format, and the design of a toolbox that facilitates automated analysis of trace data sets. We also discuss the use of the FTA for various current and future purposes. Second, after applying the toolbox to nine failure traces collected from distributed systems used in various application domains (e.g., HPC, Internet operation, and various online applications), we present a comparative analysis of failures in various distributed systems. underline the need for public availability of trace data from different distributed systems. Last, we show how different interpretations of the meaning of failure data can result in different conclusions for failure modeling and job scheduling in distributed systems. Our results for different interpretations show evidence that there may be a need for further revisiting existing failureaware algorithms, when applied for general rather than for domain-specific distributed systems.

show abstract

Identifying, analyzing, and modeling flashcrowds in BitTorrent

Cited by 27 publications

References 24 publications

BitTorrent-like P2P approaches for VoD: A comparative study

BitTorrent-like P2P approaches for VoD: A comparative study

The BTWorld use case for big data analytics: Description, MapReduce logical workflow, and empirical evaluation

The Failure Trace Archive: Enabling the comparison of failure measurements and models of distributed systems

Contact Info

Product

Resources

About