Statistical Perspectives on “Big Data”

Abstract

As our information infrastructure evolves, our ability to store, extract, and analyze data is rapidly changing. Big data is a popular term that is used to describe the large, diverse, complex and/or longitudinal datasets generated from a variety of instruments, sensors and/or computer-based transactions. The term big data refers not only to the size or volume of data, but also to the variety of data and the velocity or speed of data accrual. As the volume, variety, and velocity of data increase, our existing analytical methodologies are stretched to new limits. These changes pose new opportunities for researchers in statistical methodology, including those interested in surveillance and statistical process control methods. Although it is well documented that harnessing big data to make better decisions can serve as a basis for innovative solutions in industry, healthcare, and science, these solutions can be found more easily with sound statistical methodologies. In this paper, we discuss several big data applications to highlight the opportunities and challenges for applied statisticians interested in surveillance and statistical process control. Our goal is to bring the research issues into better focus and encourage methodological developments for big data analysis in these areas.

Knowledge, Skills, and Abilities for Entry‐Level Business Analytics Positions: A Multi‐Method Study

“…Although we agree that veracity and value‐added are important aspects of data, we do not see them as factors that distinguish big data from other forms of data. As stated by Megahed and Jones‐Farmer (), “It is important to evaluate the veracity and value of all data, both big and small.” We thus see veracity and value‐added as components of data quality rather than determining factors of what constitutes big data. For more information on research in data quality readers are referred to Wang and Strong (), Huang, Lee, and Wang (), Batini, Cappiello, Francalanci, and Maurino (), and Jones‐Farmer, Ezell, and Hazen ().…”

“…Although we agree that veracity and value-added are important aspects of data, we do not see them as factors that distinguish big data from other forms of data. As stated by Megahed and Jones-Farmer (2013), "It is important to evaluate the veracity and value of all data, both big and small." We thus see veracity and value-added as components of data quality rather than determining factors of what constitutes big data.…”

Big Data Driven Supply Chain Management and Business Administration

“…They are: Volume, Velocity, Variety, Value, Variability and Veracity (Russom, 2011;Eaton et al, 2012;O'Reilly Radar Team, 2012;Zikopoulos and Eaton, 2012;Bellini et al, 2013;Demchenko et al, 2013;Megahed and Jones-Farmer, 2013;Rajpathak and Narsingpurkar, 2013).…”

Using Visual Data Mining to Enhance the Simple Tools in Statistical Process Control: A Case Study

“…Emerging measurement technologies (such as coordinate measuring machines, machine vision systems, and 3D surface scanners) diversifies the types of data being collected, pushing data collection away from the historically low‐dimensional data. The use of computerized data acquisition systems has transformed the nature of the process monitoring and control problem, as real‐time process data are now available on hundreds of processes and product quality characteristics …”

“…The use of computerized data acquisition systems has transformed the nature of the process monitoring and control problem, as real-time process data are now available on hundreds of processes and product quality characteristics. 5,6,8 Despite the changes in the nature and volume of process data, the seven basic quality tools remain the most widely used methods in industry. These basic tools constitute the basis for much of the work in six sigma 9 and lean manufacturing.…”