NetML: An NFV Platform with Efficient Support for Machine Learning Applications

Dhakal, Aditya; Ramakrishnan, K. K.

doi:10.1109/netsoft.2019.8806698

Cited by 14 publications

(8 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The authors use Apache Storm [34], to ensure the communication of the video streams and to orchestrate the transcoding tasks in a group of networked physical machines. Work in [35] presents NetML, an NFV platform that allows the execution of machine learning algorithms in the edge server. NetML is built on top of OpenNetVM [36] NFV platform and uses GPUs to perform intensive ML tasks.…”

Section: Related Workmentioning

confidence: 99%

A measurement study of offloading virtual network functions to the edge

et al. 2021

View full text Add to dashboard Cite

The deployment of virtual network functions (VNFs) at edge servers potentially impairs the performance of latency-sensitive applications due to their computational cost. This work considers a new approach to addressing this problem that provides line rate acceleration of VNFs by employing field-programmable gate array (FPGA) equipped edge servers. This approach has been validated by practical use cases with both TCP and UDP as underlying protocol on a physical testbed environment. We examine the performance implications of executing a security VNF at an FPGA-equipped edge server. We experimentally demonstrate reduced VNF execution latency and energy consumption for a real-time video streaming application in comparison with a software-only baseline. In particular, the results show that the approach lowers VNF execution latency and power consumption at the edge by up to 44% and 76%, respectively, in our experiments while satisfying time constraints and maintaining confidentiality with high scalability. We also highlight the potential research challenges to make this approach viable in practice.

show abstract

Section: Related Workmentioning

confidence: 99%

A measurement study of offloading virtual network functions to the edge

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Data transfer to GPU: To eciently transfer data without taxing the host CPU, we pin the DPDK hugepages (used for network packets) with CUDA. Then, using NetML's approach [23] the GPU's DMA performs zero-copy scattergather of the packet data using a GPU kernel. An alternative is a NIC to GPU transfer using GPUDirect.…”

Section: Data Aggregation and Transfer To Gpu Dnn Appli-mentioning

confidence: 99%

“…This has many limitations, such as inecient packet processing in GPU [55]. Using NetML's approach [23], we only transfer data for ML processing to the GPU, leaving packet processing to the CPU. This avoids the CPU copy overhead.…”

Section: Data Aggregation and Transfer To Gpu Dnn Appli-mentioning

confidence: 99%

Gslice

Dhakal

Kulkarni

Ramakrishnan

2020

Proceedings of the 11th ACM Symposium on Cloud Computing

Self Cite

View full text Add to dashboard Cite

The increasing demand for cloud-based inference services requires the use of Graphics Processing Unit (GPU). It is highly desirable to utilize GPU eciently by multiplexing dierent inference tasks on the GPU. Batched processing, CUDA streams and Multi-process-service (MPS) help. However, we nd that these are not adequate for achieving scalability by eciently utilizing GPUs, and do not guarantee predictable performance. GSLICE addresses these challenges by incorporating a dynamic GPU resource allocation and management framework to maximize performance and resource utilization. We virtualize the GPU by apportioning the GPU resources across dierent Inference Functions (IFs), thus providing isolation and guaranteeing performance. We develop self-learning and adaptive GPU resource allocation and batching schemes that account for network trac characteristics, while also keeping inference latencies below service level objectives. GSLICE adapts quickly to the streaming data's workload intensity and the variability of GPU processing costs. GSLICE provides scalability of the GPU for IF processing through ecient and controlled spatial multiplexing, coupled with a GPU resource reallocation scheme with near-zero (< 100`s) downtime. Compared to default MPS and TensorRT, GSLICE improves GPU utilization eciency by 60-800% and achieves 2-13⇥ improvement in aggregate throughput. CCS CONCEPTS • Computer systems organization ! Cloud computing; • Computing methodologies ! Machine learning.

show abstract

“…It also achieves more flexible traffic steering as both the VNFs and management entities can make routing decisions. NetML [106] is built based on OpenNetVM and runs machine learning applications as VNFs. To unburden CPU and accelerate traffic processing, NetML further extends CUDA library to exploit GPU for computation offloading.…”

Section: Single Nfvi-pop Optimizationmentioning

confidence: 99%

NFV Platform Design: A Survey

Zhang

2020

Preprint

View full text Add to dashboard Cite

Due to the intrinsically inefficient service provisioning in traditional networks, Network Function Virtualization (NFV) keeps gaining attention from both industry and academia. By replacing the purposebuilt, expensive, proprietary network equipments with software network functions consolidated on commodity hardware, NFV envisions a shift towards a more agile and open service provisioning paradigm with much lower capital expenditure (CapEx) and operational expenditure (OpEx). Nonetheless, just as any complex system, NFV platforms commonly consist of abounding software and hardware components, and usually incorporate disparate design choices based on distinct motivations or use cases. This broad collection of convoluted alternatives makes it extremely arduous for network operators to make proper choices. Although numerous efforts have been devoted to investigating different aspects of NFV, none of them specifically focused on NFV platforms or attempted to explore the design space. In this paper, we present a comprehensive survey on NFV platform design. Our study solely targets existing NFV platform implementations. We begin with an architectural view of the standard reference NFV platform and present our taxonomy of existing NFV platforms based on principal purpose of design. Then we thoroughly explore the design space and elaborate the implementation choices each platform opts for. We believe that our study gives a detailed guideline for network operators or service providers to choose the most appropriate NFV platform based on their respective requirements.

show abstract

NetML: An NFV Platform with Efficient Support for Machine Learning Applications

Cited by 14 publications

References 16 publications

A measurement study of offloading virtual network functions to the edge

A measurement study of offloading virtual network functions to the edge

Gslice

NFV Platform Design: A Survey

Contact Info

Product

Resources

About