The Platform for Creation of Event-Driven Applications Based on Wolfram Mathematica and Apache Kafka

Zolotariov, Denis

doi:10.30837/itssi.2021.16.012

Cited by 6 publications

(7 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Mathematica document had its own kernel of the mathematical processor, which was allocated its own CPU core for guaranteed independent processing. The connection to the Apache Kafka cluster was performed as described in [14][15][16]. Cloud technologies of DigitalOcean were chosen as a platform for deployment, where servers based on Ubuntu 20.04 LTS x64 OS were located.…”

Section: Methodsmentioning

confidence: 99%

The New Modification of the Approximating Functions Method for Cloud Computing

Zolotariov¹,

Dr.Math.²

2021

IJMCR

Self Cite

View full text Add to dashboard Cite

The approach for building cloud-ready fault-tolerant calculations by approximating functions method, which is an analytical-numerical part of Volterra integral equation method for solving 1D+T nonlinear electromagnetic problems, is presented. The solving process of the original algorithm of the method is modified: it is broken down into the sequential chain of stages with a fixed number of sequential or parallel steps, each of which is built in a fault-tolerant manner and saves execution results in fault-tolerant storage for high availability. This economizes RAM and other computer resources and does not damage the calculated results in the case of a failure, and allows stopping and starting the calculations easily after manual or accidental shutdown. Also, the proposed algorithm has self-healing and data deduplication for cases of corrupted saved results. The presented approach is universal and does not depend on the type of medium or the initial signal. Also, it does not violate the natural description of non-stationary and nonlinear features, the unified definition of the inner and outer problems, as well as the inclusion of the initial and boundary conditions in the same equation as the original approximating functions method. The developed approach stress-tested on the known problems, stability checked and errors compared.

show abstract

Section: Methodsmentioning

confidence: 99%

The New Modification of the Approximating Functions Method for Cloud Computing

Zolotariov¹,

Dr.Math.²

2021

IJMCR

Self Cite

View full text Add to dashboard Cite

show abstract

“…As shown in [19], the Wolfram Mathematica processor does not have built-in mechanisms to connect to the Kafka queue cluster. This opportunity to work in two directions: publishing messages in the queue and consuming messages from it -provides the use of a package with [19] with the extension [20] for use on a computing server. To run it on the computing server, Apache Kafka is additionally installed as a queue client and third-party Kafkacat software [21].…”

Section: Microservice Of Calculationsmentioning

confidence: 99%

“…For the Wolfram Mathematica package, such a package was developed in [20], which allows a group of similar clients to connect to the queue through a single proxy server. For individual connection to the queue without additional implementation of the proxy server, you can use the method described in [19].…”

Section: Microservices Based On Mathematical Processorsmentioning

confidence: 99%

“…-Kafka queue cluster servers in the amount of 3 pieces based on Apache Kafka 2.6.0, -computing server with Wolfram Mathematica 11.3.0 for Linux x86 (64-bit), Kafkacat 1.5.0 and Apache Kafka 2.6.0, built according to [8,20] for a computing microservice, -intermediary server for queuing with Kafkacat 1.5.0 and Apache Kafka 2.6.0, built according to [20] for microservices "Mathematica microservices", -server based on Lumen 8.x framework and MySQL 8.0.20 Community Server database for DB microservice, -server based on nginx 1.18.0 web server and Lumen 8.x framework for REST-microservice "Telegram Bot microservice", -server based on the Lumen 8.x framework for Dropbox microservice and Telegram Graph microservice, the latter using the high-performance Amenadiel \ JpGraph php library, which is responsible for plotting near real time.…”

Section: Demonstration Of the Workmentioning

confidence: 99%

“…Transfer the proxy server from the mechanism of interaction with clients described in [20] to RESTfull interaction according to the HTTPS protocol. A similar mechanism has already been developed in [24][25], although at the moment none of them has built-in capabilities for queuing -only publishing or receiving messages.…”

Section: Prospects For Further Development Of the Systemmentioning

confidence: 99%

See 2 more Smart Citations

Microservice Architecture for Building High-Availability Distributed Automated Computing System in a Cloud Infrastructure

Zolotariov¹

2021

ITSSI

Self Cite

View full text Add to dashboard Cite

The article is devoted to the research and development of a highly available distributed automated computing system by iterative algorithms based on the microservice architecture in a cloud infrastructure. The subject of the research is the practical foundations of building high-availability automated computing systems based on microservice architecture in a cloud-based distributed infrastructure. The purpose of the article is to develop and to substantiate practical recommendations for the formation of the infrastructure of a high-availability automated computing system based on the microservice architecture, the choice of its constituent elements and their components. The task of the work: to identify the necessary structural elements of a microservice automated computing system and to analyze the constituent components and functional load for each of them, set specific tasks for building each of them and justify the choice of tools for their creation. In the course of the research, methods of system analysis were used to decompose a complex system into elements and each element into functional components, and tools: information technologies Apache Kafka, Kafkacat, Wolfram Mathematica, nginx, Lumen, Telegram, Dropbox, and MySQL. As a result of the study, it was found that the system infrastructure should consist of: fault-tolerant interservice transport, a high-availability computing microservice, and communication microservices with end customers, which save or process the results. For each of them, recommendations are provided regarding the formation and selection of implementation tools. According to the recommendations, one variant of implementation of such system has been developed, the principles of its operation are shown and the results are presented. It has been proven that when using a Kafka queue it is efficient to publish batches of results rather than one at a time, which results to significant overhead on queue servers and data latency for its clients. Recommendations are given on the implementation of the CI/CD system to build a continuous cycle of adding and improving microservices. Conclusions. Practical foundations have been developed for the implementation of high availability distributed automated computing systems based on microservice architecture in a cloud infrastructure. The flexibility in processing the results of such a system is shown due to the possibility of adding microservices and using third-party analytical applications that support connection to the Kafka queue. The economic benefit of using the described system is shown. Future ways of its improvement are given.

show abstract