STINGER: High performance data structure for streaming graphs

Ediger, David; McColl, Rob; Riedy, Jason; Bader, David A.

doi:10.1109/hpec.2012.6408680

Cited by 176 publications

(84 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Adjacencies are stored in semi-dense lists of blocks of edges. The data structure's basic operations are thread-safe, which enables the insertion and deletion of edges and vertices at rates of millions of updates per second on a modern multicore sharedmemory x86 platform [9]. Combining this with a convenient parallel filtering and traversal allows algorithms to easily process and understand the structural changes in the graph over time to ` identify anomalous relationships, actors, and groups.…”

Section: Stingermentioning

confidence: 99%

Detecting insider threats in a real corporate database of computer usage activity

Senator

Goldberg

Memory

et al. 2013

Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

Self Cite

102

View full text Add to dashboard Cite

This paper reports on methods and results of an applied research project by a team consisting of SAIC and four universities to develop, integrate, and evaluate new approaches to detect the weak signals characteristic of insider threats on organizations' information systems. Our system combines structural and semantic information from a real corporate database of monitored activity on their users' computers to detect independently developed red team inserts of malicious insider activities. We have developed and applied multiple algorithms for anomaly detection based on suspected scenarios of malicious insider behavior, indicators of unusual activities, high-dimensional statistical patterns, temporal sequences, and normal graph evolution. Algorithms and representations for dynamic graph processing provide the ability to scale as needed for enterpriselevel deployments on real-time data streams. We have also developed a visual language for specifying combinations of features, baselines, peer groups, time periods, and algorithms to detect anomalies suggestive of instances of insider threat behavior. We defined over 100 data features in seven categories based on approximately 5.5 million actions per day from approximately 5,500 users. We have achieved area under the ROC curve values of up to 0.979 and lift values of 65 on the top 50 user-days identified on two months of real data.

show abstract

Section: Stingermentioning

confidence: 99%

Detecting insider threats in a real corporate database of computer usage activity

Senator

Goldberg

Memory

et al. 2013

Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

Self Cite

102

View full text Add to dashboard Cite

show abstract

“…where our work focus on distributed cluster environments. It provides a shared memory data structure [13] for large scale dynamic graph processing. A series of dynamic graph algorithms have been developed using this data structure [18], [19].…”

Section: Related Workmentioning

confidence: 99%

“…Existing research has so far focused on static graph processing, While some solutions have explored computation over a sequence of updates condensed into a set of snapshots of static graphs [11], [12], [5], We argue that naively adopting a system designed for static graph processing for real-time incremental computation over dynamic graphs is inefficient. Even though research has been performed on large scale dynamic graph processing [13], [11], developing incremental algorithms for large-scale graphs can be a daunting programming task.…”

Section: Introductionmentioning

confidence: 99%

Empowering Fast Incremental Computation over Large Scale Dynamic Graphs

Wickramaarachchi

Chelmis

Prasanna

2015

2015 IEEE International Parallel and Distributed Processing Symposium Workshop

View full text Add to dashboard Cite

Unprecedented growth of online social networks, communication networks and internet of things have given birth to large volume, fast changing datasets. Data generated from such systems have an inherent graph structure in it. Updates in staggering frequencies (e.g. edges created by message exchanges in online social media) impose a fundamental requirement for real-time processing of unruly yet highly interconnected data. As a result, large-scale dynamic graph processing has become a new research frontier in computer science.In this paper, we present a new vertex-centric hierarchical bulk synchronous parallel model for distributed processing of dynamic graphs. Our model allows users to easily compose static graph algorithms similar to the widely used vertex-centric model. It also enables incremental processing of dynamic graphs by automatically executing user composed static graph algorithms in an incremental manner. We map widely used single source shortest path and connected component algorithms to this model and empirically analyze the performance on real-world large scale graphs. Experimental results show that our model improves the performance of both static and dynamic graph computation compared to the vertex-centric model by reducing the global synchronization overhead.

show abstract

“…Our distributed architecture and the implementation are for a well established, generic algorithm for clustering which can be applied to any domain, including the Twitter domain as we have demonstrated in this paper. [33] [34] present work on processing streaming graphs in parallel as opposed to the clustering of streaming feature vectors that our work focuses on. Further, [35] presents a simplified clustering algorithm which does not deal with synchronization among the parallel clustering workers.…”

Section: Distributed Clustering Algorithmsmentioning

confidence: 99%