Unconstrained endpoint profiling (googling the internet)

Abstract

Understanding Internet access trends at a global scale, i.e., what do people do on the Internet, is a challenging problem that is typically addressed by analyzing network traces. However, obtaining such traces presents its own set of challenges owing to either privacy concerns or to other operational difficulties. The key hypothesis of our work here is that most of the information needed to profile the Internet endpoints is already available around us -on the web.In this paper, we introduce a novel approach for profiling and classifying endpoints. We implement and deploy a Google-based profiling tool, which accurately characterizes endpoint behavior by collecting and strategically combining information freely available on the web. Our 'unconstrained endpoint profiling' approach shows remarkable advances in the following scenarios: (i) Even when no packet traces are available, it can accurately predict application and protocol usage trends at arbitrary networks; (ii) When network traces are available, it dramatically outperforms state-ofthe-art classification tools; (iii) When sampled flow-level traces are available, it retains high classification capabilities when other schemes literally fall apart. Using this approach, we perform unconstrained endpoint profiling at a global scale: for clients in four different world regions (Asia, South and North America and Europe). We provide the first-of-its-kind endpoint analysis which reveals fascinating similarities and differences among these regions.

Link Homophily in the Application Layer and its Usage in Traffic Classification

“…This is particularly important for applications such as P2P, which can be more challenging [7], especially at the backbone [4]. Figure 4 summarizes our results.…”

“…For example, in the case of classification conflicts, we could incorporate the classifier with the better accuracy or precision, especially since some methods are better for different applications. (c) We can query search engines for IP addresses that we want to classify, in essence, "Googling the Internet" [7], thus harnessing the power of the Web. (d) We can use active and passive measurement techniques for the seeding process.…”

Uncovering Relations between Traffic Classifiers and Anomaly Detectors via Graph Theory

“…-Whereas, BLINC [18] decides a label for a source (IP address, source port) based on its connection pattern. Also, a recent method [19] labels directly hosts without any traffic information by collecting and analyzing information freely available on the web. Thus, researchers faced difficulties in comparing events standing for flows with events representing hosts.…”

GTVS: Boosting the Collection of Application Traffic Ground Truth

“…The assumption is that if a host is an SMTP server, all the flows generated from this host toward port 25 are mail traffic. In general, this heuristic is applicable for P2P traffic as long as the information about the port number can be utilized 8 and the assumption of the heuristic can be validated. In our experience, for example, there is a large number of eDonkey flows which can be identified using port 4662 and for BitTorrent on port 6881.…”

“…On the technical aspects, our work can be seen as a cumulative progress, with lots of inspirations from previous traffic classification works, including [2,3,[6][7][8][9].…”