Big Data Redux: New Issues and Challenges Moving Forward

Abstract

As of the time of this writing, our HICSS-46 proceedings article has enjoyed over 520 Google Scholar citations. We have published several HICSS proceedings, articles and a book on this subject, but none of them have generated this level of interest. In an effort to update our findings six years later, and to understand what is driving this interest, we have downloaded the first 500 citations to our article and the corresponding citing article, when available. We conducted an in-depth literature review of the articles published in top journals and leading conference proceedings, along with articles with a high volume of citations. This paper provides a brief summary of the key concepts in our original paper and reports on the key aspects of interest we found in our review, and also updates our original paper with new directions for future practice and research in big data and analytics.

Towards Bridging the Gap Between BDA Challenges and BDA Capability: A Conceptual Synthesis Based on a Systematic Literature Review

“…These include problems with the bandwidth required for instant data transmission allowing for real-time processing [37, 38, Table 1. Systematization of BDA implementation-related challenges (* selected theme f. mapping) 39], and a lack of scalability and integration of data storage units for large datasets [18,19,37,38,39,40]. As a whole, BDA requires a powerful infrastructure that enables an organization to gain insights from the available datasets and extract value through the application of data analysis [2,19,41,42,43].…”

“…A frequently observed phenomenon is the insufficient data quality, recognizable through a lack of data standardization, a high degree of data heterogeneity, and data inconsistencies as well as incompleteness [18,19,21,37,38,43,44,46]. Observable consequences of insufficient data quality encompass interpretability, reliability issues as well as lower trustworthiness of derived insights [11,37,39,40,45,47]. The unique characteristics of BD furthermore affect the utilization of data along the entire analytical lifecycle.…”

“…The unique characteristics of BD furthermore affect the utilization of data along the entire analytical lifecycle. Exemplary challenges within this lifecycle include data transmission, data integration and (pre-) processing, data mining and analysis, data modeling, and data accessibility [19,20,37,38,39,40,43,45,46,47]. Further issues arise from the inherent security and privacy concerns associated with BD [18,19,21,37,38,40,42,43,47].…”

“…Exemplary challenges within this lifecycle include data transmission, data integration and (pre-) processing, data mining and analysis, data modeling, and data accessibility [19,20,37,38,39,40,43,45,46,47]. Further issues arise from the inherent security and privacy concerns associated with BD [18,19,21,37,38,40,42,43,47]. The deficiencies of analytical techniques, which can be applied to different datasets, are closely related to the above-mentioned lack of BDA tools.…”

Big Data in Smart City: Management Challenges

“…• data are generated in real time, which leads to the tremendous amount of data to be dealt with-volume (identified by [13,14]); • data are collected from different sources (smart phones, computers, GNSS, enviromental sensors, cameras)-variety [6]; • big data are catalogued and stored in different platforms that often cause a problem in that some data are unused-velocity [15]; • smart city demands change in the approach of performing the complex analytics operations over such huge amount of data-velocity [16]; • big data systems must provide the data that are included in analytical processes with a focus on enhancing smart city applications-variability [14]; • a need to transform a huge amount of data into business based on big data collection, management, and analysis-value [12,17,18].…”