ActiveSort: Efficient external sorting using active SSDs in the MapReduce framework

Lee, Young-Sik; Quero, Luis Cavazos; Kim, Sang-Hoon; Maeng, Seungryoul

doi:10.1016/j.future.2016.03.003

Cited by 24 publications

(19 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In-storage NDP. In-storage NDP has been widely studied in various use-cases, such as databases [17,19,31,37], map-reduce [23,24,29], regex [8,28], searching [19,35,36], etc. A common theme of such works is to explore the idea of pushing computation to the storage devices for improved performance.…”

Section: Background and Related Workmentioning

confidence: 99%

blockNDP

Barbalace

Decky

Picorel

et al. 2020

Proceedings of the 1st International Middleware Conference Industrial Track

View full text Add to dashboard Cite

Section: Background and Related Workmentioning

confidence: 99%

blockNDP

Barbalace

Decky

Picorel

et al. 2020

Proceedings of the 1st International Middleware Conference Industrial Track

View full text Add to dashboard Cite

“…Of the eleven papers in this issue, four address searching and processing of Big Data, three are concerned with performing workflows and event modelling in the Cloud with four more about modelling real world systems and social media in the context of the Cloud. The first of the four papers about data, ''ActiveSort: Efficient External Sorting using Active SSDs in the MapReduce Framework'' by Young-Sik Lee et al [1], looks gaining an advantage in data intensive applications by using the capabilities of solid state drives (SSDs) to perform part of the work more usually carried out by the host. The paper presents an improved external sorting algorithm, ActiveSort which uses this concept of active SSDs and reports results obtained when applying this algorithm to a real active SSD platform that outperform the original Hadoop implementation by more than 35%.…”

Section: Content Of This Issuementioning

confidence: 99%

Editorial for FGCS special issue: Big Data in the cloud

Chang

Ramachandran

Wills

et al. 2016

Future Generation Computer Systems

View full text Add to dashboard Cite

Research associated with Big Data in the Cloud will be important topic over the next few years. The topic includes work on demonstrating architectures, applications, services, experiments and simulations in the Cloud to support the cases related to adoption of Big Data. A common approach to Big Data in the Cloud to allow better access, performance and efficiency when analysing and understanding the data is to deliver Everything as a Service. Organisations adopting Big Data this way find the boundaries between private clouds, public clouds and Internet of Things (IoT) can be very thin. Volume, variety, velocity, veracity and value are the major factors in Big Data systems but there are other challenges to be resolved. The papers of this special issue address a variety of issues and concerns in Big Data, including: searching and processing Big Data, implementing and modelling event and workflow systems, visualisation modelling and simulation and aspects of social media.

show abstract

“…Currently, the growth of data consumption by internet users has exponentially increased (Laga et al, 2017;Bey Ahmed Khernache, Laga & Boukhobza, 2018), and a massive storage size is required to store all incoming data to avoid any data loss in case of storage overflow (Thusoo et al, 2010;Katal, Wazid & Goudar, 2013;Witayangkurn, Horanont & Shibasaki, 2012;Mehmood et al, 2016). However, many applications such as data management, finance, sensor networks, security-relevant data, and web search possibly face this unexpected situation of a storage overload issue (Lee et al, 2016;Babcock et al, 2002;Keim, Qu & Ma, 2013;Cardenas, Manadhata & Rajan, 2013;Dave & Gianey, 2016). This issue induces the problem of representing big data with a limited storage size.…”

Section: Introductionsmentioning

confidence: 99%

Correct and stable sorting for overflow streaming data with a limited storage size and a uniprocessor

Chaikhan¹,

Phimoltares²,

Lursinsap³

2021

PeerJ Computer Science

View full text Add to dashboard Cite

Tremendous quantities of numeric data have been generated as streams in various cyber ecosystems. Sorting is one of the most fundamental operations to gain knowledge from data. However, due to size restrictions of data storage which includes storage inside and outside CPU with respect to the massive streaming data sources, data can obviously overflow the storage. Consequently, all classic sorting algorithms of the past are incapable of obtaining a correct sorted sequence because data to be sorted cannot be totally stored in the data storage. This paper proposes a new sorting algorithm called streaming data sort for streaming data on a uniprocessor constrained by a limited storage size and the correctness of the sorted order. Data continuously flow into the storage as consecutive chunks with chunk sizes less than the storage size. A theoretical analysis of the space bound and the time complexity is provided. The sorting time complexity is O (n), where n is the number of incoming data. The space complexity is O (M), where M is the storage size. The experimental results show that streaming data sort can handle a million permuted data by using a storage whose size is set as low as 35% of the data size. This proposed concept can be practically applied to various applications in different fields where the data always overflow the working storage and sorting process is needed.

show abstract

ActiveSort: Efficient external sorting using active SSDs in the MapReduce framework

Cited by 24 publications

References 21 publications

blockNDP

blockNDP

Editorial for FGCS special issue: Big Data in the cloud

Correct and stable sorting for overflow streaming data with a limited storage size and a uniprocessor

Contact Info

Product

Resources

About