On a Meta Learning-Based Scheduler for Deep Learning Clusters

Yang, Jin; Bao, Liang; Liu, Wenjing; Yang, Rong; Wu, Chase Q.

doi:10.1109/tcc.2023.3308161

Cited by 2 publications

(2 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The meta-learning approach is based on previously known combinations of devices and the necessary parameters for building a virtual cluster [30]. At the same time, the selection of virtual computing nodes is based on the templates of tested devices in the cluster for a predetermined algorithm.…”

Section: The Study Materials and Methodsmentioning

confidence: 99%

Devising a method for the virtual clustering of the Internet of Things edge environment

Kuchuk,

Mozhaiev,

Kuchuk

et al. 2024

EEJET

View full text Add to dashboard Cite

The object of research is the process of load distribution in the edge environment of the Internet of Things. The task to improve the efficiency of the functioning of the network of computing devices in the Internet of Things edge environment has been solved. Free resources of heterogeneous single-board computers were used to this end. In the process of conducting research, an approach to the construction of an architecture for a virtual cluster of computers with limited resources was devised. The design took into account specific features of the edge environment on the Internet of Things. This has made it possible to propose a four-layer architecture instead of the standard seven-layer architecture of IoT sensor information processing device networks. Stages in the virtual cluster construction in the edge environment on the Internet of Things were also defined. A three-stage procedure to form a virtual cluster was justified. This procedure made it possible to devise a method for the virtual clustering in the Internet of Things edge environment based on the proposed virtual cluster architecture. The proposed method for building a virtual cluster in the Internet of Things edge environment was investigated. With a small network load, a virtual cluster has no advantage over a classic cluster. But with the growth of the network load, the virtual cluster prevails over the classic cluster in total performance; the advantage in total performance can exceed 10 %. It was also proven that for a heterogeneous environment, performance changes at full network load significantly depend on the number of virtual node groups. The research results on the method for building a virtual cluster in the Internet of Things edge environment can be explained by improving the balance of the network load at virtual clustering

show abstract

Section: The Study Materials and Methodsmentioning

confidence: 99%

Devising a method for the virtual clustering of the Internet of Things edge environment

Kuchuk,

Mozhaiev,

Kuchuk

et al. 2024

EEJET

View full text Add to dashboard Cite

show abstract

“…Traditional cluster analysis methods work with objects specified as vectors signs [1][2][3][4]. When working with texts, the first step of the algorithm is clustering is definition space signs and construction in it vectors available texts [5,6]. Typically received vectors have big dimensions and when working with them traditional cluster analysis methods do not provide sufficient efficiency [7][8][9][10].…”

Section: Introductionmentioning

confidence: 99%

Fuzzy Interactive Clustering Method

Steblyanko,

Ni,

Kuchuk

et al. 2024

CNCS

View full text Add to dashboard Cite

The article examines an example of a system in which a large number of short texts are generated. In it, participants create strategic planning documents, within which key performance indicators are determined. The formulations of key performance indicators form a data set consisting of short texts. Within the framework of this system, there is an urgent task of forming and updating a classifier based on this set. A solution to this problem is presented using the fuzzy interactive clustering method. This method allows expert to perform clustering sets of short texts, issuing reverse communication based on the results of each step interactive clustering. Collection procedure reverse does not imply any connection availability of an expert special knowledge about work neural network and is assembled in human-readable form matrices reverse communications. Such an approach has advantages over clustering methods requiring adjustments metaparameters algorithm not related directly with the clustering results. Also important advantage the proposed method is opportunity realize clustering sets data related to various language domains that do not match the domain on which was produced education language models, due to proposed extension method dictionary language models This property allows use the proposed algorithm in a narrow way specialized domains, as well as in domains that do not allow you to obtain a full-fledged corpus of texts for yourself training language models.

show abstract

On a Meta Learning-Based Scheduler for Deep Learning Clusters

Cited by 2 publications

References 26 publications

Devising a method for the virtual clustering of the Internet of Things edge environment

Devising a method for the virtual clustering of the Internet of Things edge environment

Fuzzy Interactive Clustering Method

Contact Info

Product

Resources

About