Unconstrained endpoint profiling (googling the internet)

Abstract

Understanding Internet access trends at a global scale, i.e., what do people do on the Internet, is a challenging problem that is typically addressed by analyzing network traces. However, obtaining such traces presents its own set of challenges owing to either privacy concerns or to other operational difficulties. The key hypothesis of our work here is that most of the information needed to profile the Internet endpoints is already available around us -on the web.In this paper, we introduce a novel approach for profiling and classifying endpoints. We implement and deploy a Google-based profiling tool, which accurately characterizes endpoint behavior by collecting and strategically combining information freely available on the web. Our 'unconstrained endpoint profiling' approach shows remarkable advances in the following scenarios: (i) Even when no packet traces are available, it can accurately predict application and protocol usage trends at arbitrary networks; (ii) When network traces are available, it dramatically outperforms state-ofthe-art classification tools; (iii) When sampled flow-level traces are available, it retains high classification capabilities when other schemes literally fall apart. Using this approach, we perform unconstrained endpoint profiling at a global scale: for clients in four different world regions (Asia, South and North America and Europe). We provide the first-of-its-kind endpoint analysis which reveals fascinating similarities and differences among these regions.

AUTOVAC: Automatically Extracting System Resource Constraints and Generating Vaccines for Malware Immunization

“…Our basic idea is inspired by a Googling approach used in previous studies [29]. Essentially we use Google query APIs to search resourceidentifiers.…”

“…Based on the return results and their context, we infer whether these resources are already associated with benign software. We refer readers to [29] for more details. In short, from our search query, if the resource-identifiers does not conflict with benign software or there is no any matching search result, then we proceed with further analysis.…”

MUCM: Multilevel User Cluster Mining Based on Behavior Profiles for Network Monitoring

“…Many researchers focused on profiling network behavior of individual end users [14], [15] or on classifying the roles and communities of end users based on their traffic patterns [16], [17]. By using information available on the web, a traffic classification technique is developed based on the Google search engine in [14].…”

“…Many researchers focused on profiling network behavior of individual end users [14], [15] or on classifying the roles and communities of end users based on their traffic patterns [16], [17]. By using information available on the web, a traffic classification technique is developed based on the Google search engine in [14]. In [16], two algorithms are implemented to group users of enterprise networks into different roles based on their observed connection patterns; similarly, in [17], the communities of interest (COI) for user communications are studied using traffic data collected from a large enterprise network.…”

Uses and Challenges for Network Datasets

“…While a number of groups have maintained manual lists of dynamicallyassigned addresses, Xie et al have inferred this information for accesses to e-mail provider logs [74]. Trestian et al developed a classification method for addresses based on their presence on the web, as shown through the Google search engine [70]. We have been conducting census of all Internet addresses for several years [26].…”