Generating a function for network delay

Sukhov, Andrei M.; Astrakhantseva, M. A.; Pervitsky, A. K.; Boldyrev, Sergey; Bukatov, Alexander Alexeyevich

doi:10.3233/jhs-160552

Cited by 11 publications

(6 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…After the tracking information is converted into the packet delay sequence according to the watermark encoding scheme, the delay of the information is affected, and the statistical law is different from that of the normal data flow. The interval arrival delay of packets in the network is a random variable in accordance with normal distribution [ 40 ]. Distributed simulation methods obtain the approximate reversible transformation function by detecting the statistical distribution of normal interval arrival delay in the network.…”

Section: Materials and Methodsmentioning

confidence: 99%

SACT: A New Model of Covert Communication Based on SDN

Shi

Song

Xue

et al. 2020

Sensors

View full text Add to dashboard Cite

Anonymous tracking technology of network watermarking is limited by the deployment of tracking devices in traditional network structure, resulting in poor scalability and reusability. Software Defined Network (SDN) boasts more freedom thanks to its separation of the control plane from the data plane. In this paper, a new anonymous communication tracking model SDN-based Anonymous Communication Tracking (SACT) is proposed, which introduces network watermarking into SDN and combines IP time hidden channel and symbol expansion technology. In addition, we introduce a hopping protection mechanism to improve the anti detection ability of the watermark as well. The experimental results show that in a variety of simulated network environments, SACT achieves excellent detection rate and bit error rate, thus it is sufficient to determine the communication relationship between the two parties. Meanwhile, SACT solves the deployment problem of anonymous tracking and improves the availability and scalability of covert communication.

show abstract

Section: Materials and Methodsmentioning

confidence: 99%

SACT: A New Model of Covert Communication Based on SDN

Shi

Song

Xue

et al. 2020

Sensors

View full text Add to dashboard Cite

show abstract

“…The communication latency is 1-6ms depending on the location. To simulate shared usage, at the beginning of each communication round we inject additional latency sampled from the exponential distribution (Sukhov et al, 2016) with the mean of 100ms.…”

Section: Imagenet Trainingmentioning

confidence: 99%

Moshpit SGD: Communication-Efficient Decentralized Training on Heterogeneous Unreliable Devices

Ryabinin¹,

Gorbunov²,

Plokhotnyuk³

et al. 2021

Preprint

View full text Add to dashboard Cite

Training deep neural networks on large datasets can often be accelerated by using multiple compute nodes. This approach, known as distributed training, can utilize hundreds of computers via specialized message-passing protocols such as Ring All-Reduce. However, running these protocols at scale requires reliable high-speed networking that is only available in dedicated clusters. In contrast, many real-world applications, such as federated learning and cloud-based distributed training, operate on unreliable devices with unstable network bandwidth. As a result, these applications are restricted to using parameter servers or gossip-based averaging protocols. In this work, we lift that restriction by proposing Moshpit All-Reduce -an iterative averaging protocol that exponentially converges to the global average. We demonstrate the efficiency of our protocol for distributed optimization with strong theoretical guarantees. The experiments show 1.3× speedup for ResNet-50 training on ImageNet compared to competitive gossip-based strategies and 1.5× speedup when training ALBERT-large from scratch using preemptible compute nodes.Running large-scale distributed training in these circumstances requires fault-and latency-tolerant algorithms (Lian et al., 2017;Assran et al., 2019b). Most of these algorithms replace all-reduce averaging with gossip: each participant periodically downloads the latest parameters from his neigh-

show abstract

“…We simulate network latency by adding an artificial delay after computation of each block. The delay time is sampled from the exponential distribution, which was shown to model latency well in Sukhov et al (2016).…”

Section: Model Throughputmentioning

confidence: 99%

Towards Crowdsourced Training of Large Neural Networks using Decentralized Mixture-of-Experts

Ryabinin,

Gusev

2020

Preprint

View full text Add to dashboard Cite

Many recent breakthroughs in deep learning were achieved by training increasingly larger models on massive datasets. However, training such models can be prohibitively expensive. For instance, Megatron Language Model with 8.3B parameters was trained on a GPU cluster worth $25 million. As a result, most researchers cannot afford to train state of the art models and contribute to their development. Hypothetically, a researcher could crowdsource the training of large neural networks with thousands of regular PCs provided by volunteers. The raw computing power of ten thousand $2500 desktops dwarfs that of a $25M server pod, but one cannot utilize that power efficiently with conventional distributed training methods. In this work, we propose Learning@home: a neural network training paradigm designed to handle millions of poorly connected participants. We analyze the performance, reliability, and architectural constraints of this paradigm and compare it against existing distributed training techniques.

show abstract

Generating a function for network delay

Cited by 11 publications

References 26 publications

SACT: A New Model of Covert Communication Based on SDN

SACT: A New Model of Covert Communication Based on SDN

Moshpit SGD: Communication-Efficient Decentralized Training on Heterogeneous Unreliable Devices

Towards Crowdsourced Training of Large Neural Networks using Decentralized Mixture-of-Experts

Contact Info

Product

Resources

About