Big Data and Fog Computing

Abstract

Cloudlets DefinitionsFog Computing A model of distributed computing comprising of virtualized, heterogeneous, commodity computing and storage resources for hosting applications, analytics, content and services, that are accessible with low latency from the edge of wide area networks where clients are present, while also having back-end connectivity to cloud computing resources. BackgroundEver since computers could connect over a network, computing paradigms have under-gone cyclical phases on where within the network the computation is performed. While the original mainframes till the 1970's were large, centralized time-sharing machines accessed by multiple users through remote terminals, the Personal Computers (PCs) of the 1980's heralded local processing for individuals [35]. The growth of Local-Area Networks (LAN), the Internet, and the World Wide Web (WWW) brought about client-server models in the 1990's where many clients could access content hosted on individual servers, though most of the processing was still done on the PC [48]. Web services and eCommerce of the 2000's lead to the growth of cloud computing, where computation once again skewed to centralized data centers, but with server-farms rather than single servers hosting services that were consumed by PCs [27]. A complementary phenomenon in that decade was Peer-to-Peer (P2P) systems where PCs distributed across the Internet would work collaboratively for content sharing, to tackle bandwidth limitations of the Internet [32]. Both cloud computing and P2P signaled the arrival of the Big Data age, where the ability to collect large enterprise, scientific and web datasets and media content put an emphasis on being able to share and process them at large scales. The decade of 2010 is seeing a similar cyclical shift, but at a faster pace due to several technology advances. Starting with a more centralized cloud computing model hosting thousands of Virtual Machines (VMs), we have seen the roll-out of pervasive broadband Internet and cellular network communication combined with the rapid growth of smart phones as general-purpose computing platforms backed by cloud computing. Internet of Things (IoT) is yet another paradigm, enabled by the convergence of these other technologies [21]. Sensors and constrained devices connected to the Internet are being deployed to support vertical IoT domains such as personal fitness using wearables, smart utilities using metering infrastructure, and even self-driving cars. Both smart phones and IoT mark the advent of Edge Computing (or mobile cloud computing). Here, ubiquitous devices numbering in the billions are present at the edge of the Wide-Area Network (WAN) that is the Internet, and host applications that

Characterizing application scheduling on edge, fog, and cloud computing resources

“…If edge and fog are deployed by the same provider or are a part of the same private network, such as a Wi‐Fi access point, then there may be no pricing costs for the network usage . The data transfer within a layer and between these layers will be free.…”

“…∘ Multitenancy Multitenancy allows different users to run their applications on the same host resource . The tenants may be separated from each other by VMs, containers, platforms, or not at all.…”

“…These can be leveraged to execute IoT applications at the edge of the network . Further, there has been heightened interest in fog resources, which are between the edge and cloud in the network hierarchy, with compute, storage, and memory capacities that fall between these layers as well . These edge and fog resources provide the opportunity for low‐latency processing of the generated data, closer to its source, and on the wide‐area network .…”

“…However, given the nascency of edge and fog resources, there is a lack of a systematic review of distributing and scheduling applications on these resource classes individually, and together with clouds . Existing literature has proposed the conceptual foundations of edge and fog computing . Others, including us, have discussed the benefits and challenges involved in the coordination among edge, fog, and cloud layers in a hierarchical model, but have failed to examine in detail their impact of the applications and their schedule.…”

“…12 Further, there has been heightened interest in fog resources, which are between the edge and cloud in the network hierarchy, with compute, storage, and memory capacities that fall between these layers as well. 13 These edge and fog resources provide the opportunity for low-latency processing of the generated data, closer to its source, and on the wide-area network. 14 As a result, there is critical need to understand how this diverse ecosystem of edge, fog and cloud resources can be effectively used by large-scale distributed applications.…”