A computational model for TensorFlow: an introduction

Abadi, Martı́n; Isard, Michael; Murray, Derek G.

doi:10.1145/3088525.3088527

Cited by 72 publications

(60 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…All algorithms are implemented using TensorFlow [12] and fully connected neural networks (NNets) employing ReLU activation functions. In the DBS and DBT algorithm variants, a NNet architecture of 784 → 400 → 400 → 20 is used to implement f θ (·).…”

Section: Resultsmentioning

confidence: 99%

Efficient learning of neighbor representations for boundary trees and forests

Adikari

Draper

2019

2019 53rd Annual Conference on Information Sciences and Systems (CISS)

View full text Add to dashboard Cite

We introduce a semiparametric approach to neighbor-based classification. We build off the recently proposed Boundary Trees algorithm by Mathy et al. (2015) which enables fast neighbor-based classification, regression and retrieval in large datasets. While boundary trees use an Euclidean measure of similarity, the Differentiable Boundary Tree algorithm by Zoran et al. (2017) was introduced to learn low-dimensional representations of complex input data, on which semantic similarity can be calculated to train boundary trees. As is pointed out by its authors, the differentiable boundary tree approach contains a few limitations that prevents it from scaling to large datasets. In this paper, we introduce Differentiable Boundary Sets, an algorithm that overcomes the computational issues of the differentiable boundary tree scheme and also improves its classification accuracy and data representability. Our algorithm is efficiently implementable with existing tools and offers a significant reduction in training time. We test and compare the algorithms on the well known MNIST handwritten digits dataset and the newer Fashion-MNIST dataset by Xiao et al. (2017).

show abstract

Section: Resultsmentioning

confidence: 99%

Efficient learning of neighbor representations for boundary trees and forests

Adikari

Draper

2019

2019 53rd Annual Conference on Information Sciences and Systems (CISS)

View full text Add to dashboard Cite

show abstract

“…The experiment in this paper was completed by TensorFlow framework [ 20 ], which was implemented in the form of single machine multi-card. The Parameter Server (PS) node and Worker nodes in distributed training were simulated by three port addresses of the local machine.…”

Section: Methodsmentioning

confidence: 99%

“…The datasets used in this experiment were MNIST and Cifar10, and the parameter information of the neural network models used in training is shown in Table 1. The experiment in this paper was completed by TensorFlow framework [20], which was implemented in the form of single machine multi-card. The Parameter Server (PS) node and Worker nodes in distributed training were simulated by three port addresses of the local machine.…”

Section: Methodsmentioning

confidence: 99%

A Partition Based Gradient Compression Algorithm for Distributed Training in AIoT

Guo

Liu

Zhang

2021

Sensors

View full text Add to dashboard Cite

Running Deep Neural Networks (DNNs) in distributed Internet of Things (IoT) nodes is a promising scheme to enhance the performance of IoT systems. However, due to the limited computing and communication resources of the IoT nodes, the communication efficiency of the distributed DNN training strategy is a problem demanding a prompt solution. In this paper, an adaptive compression strategy based on gradient partition is proposed to solve the problem of high communication overhead between nodes during the distributed training procedure. Firstly, a neural network is trained to predict the gradient distribution of its parameters. According to the distribution characteristics of the gradient, the gradient is divided into the key region and the sparse region. At the same time, combined with the information entropy of gradient distribution, a reasonable threshold is selected to filter the gradient value in the partition, and only the gradient value greater than the threshold is transmitted and updated, to reduce the traffic and improve the distributed training efficiency. The strategy uses gradient sparsity to achieve the maximum compression ratio of 37.1 times, which improves the training efficiency to a certain extent.

show abstract

“…These frameworks have evolved out of dataflow programming paradigms in which the abstractions are operators with input and output connections. The semantics provided by these languages have been sketched in previous work [5].…”

Section: Background and Related Workmentioning

confidence: 99%

Relay: a new IR for machine learning frameworks

Roesch

Lyubomirsky

Weber

et al. 2018

Proceedings of the 2nd ACM SIGPLAN International Workshop on Machine Learning and Programming Languages

View full text Add to dashboard Cite

Machine learning powers diverse services in industry including search, translation, recommendation systems, and security. The scale and importance of these models require that they be efficient, expressive, and portable across an array of heterogeneous hardware devices. These constraints are often at odds; in order to better accommodate them we propose a new high-level intermediate representation (IR) called Relay. Relay is being designed as a purely-functional, statically-typed language with the goal of balancing efficient compilation, expressiveness, and portability. We discuss the goals of Relay and highlight its important design constraints. Our prototype is part of the open source NNVM compiler framework, which powers Amazon's deep learning framework MxNet.

show abstract

A computational model for TensorFlow: an introduction

Cited by 72 publications

References 8 publications

Efficient learning of neighbor representations for boundary trees and forests

Efficient learning of neighbor representations for boundary trees and forests

A Partition Based Gradient Compression Algorithm for Distributed Training in AIoT

Relay: a new IR for machine learning frameworks

Contact Info

Product

Resources

About