HDFS memory usage analysis

Rao, B. P. C.; Rao, N. Nagamalleswara

doi:10.1109/icici.2017.8365298

Cited by 1 publication

(2 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This platform is one of the 60 components of the Apache Hadoop ecosystem, having the ability to store large files in a distributed way (split in chunks across multiple nodes), while offering reliability and extreme fault-tolerance. Being based on the Google File System [17], HDFS is tailored to a write-once-read-many [18] usage pattern.…”

Section: Persistent Storagementioning

confidence: 99%

See 1 more Smart Citation

Parameterization and Performance Analysis of a Scalable, near Real-Time Packet Capturing Platform

Oliveira,

Pedrosa,

Rufino

et al. 2024

Systems

View full text Add to dashboard Cite

The rapid evolution of technology has fostered an exponential rise in the number of individuals and devices interconnected via the Internet. This interconnectedness has prompted companies to expand their computing and communication infrastructures significantly to accommodate the escalating demands. However, this proliferation of connectivity has also opened new avenues for cyber threats, emphasizing the critical need for Intrusion Detection Systems (IDSs) to adapt and operate efficiently in this evolving landscape. In response, companies are increasingly seeking IDSs characterized by horizontal, modular, and elastic attributes, capable of dynamically scaling with the fluctuating volume of network data flows deemed essential for effective monitoring and threat detection. Yet, the task extends beyond mere data capture and storage; robust IDSs must integrate sophisticated components for data analysis and anomaly detection, ideally functioning in real-time or near real-time. While Machine Learning (ML) techniques present promising avenues for detecting and mitigating malicious activities, their efficacy hinges on the availability of high-quality training datasets, which in turn poses a significant challenge. This paper proposes a comprehensive solution in the form of an architecture and reference implementation for (near) real-time capture, storage, and analysis of network data within a 1 Gbps network environment. Performance benchmarks provided offer valuable insights for prototype optimization, demonstrating the capability of the proposed IDS architecture to meet objectives even under realistic operational scenarios.

show abstract

Section: Persistent Storagementioning

confidence: 99%

“…• 128 KB (2 17 or 131,072 bytes) ; • 256 KB (2 18 or 262,144 bytes); • 512 KB (2 19 or 524,288 bytes).…”

Section: Impact Of the Kafka Message Sizeunclassified