Quantitative Analysis of Apache Storm Applications: The NewsAsset Case Study

Abstract

The development of Information Systems today faces the era of Big Data. Large volumes of information need to be processed in realtime, for example, for Facebook or Twiter analysis. This paper addresses the redesign of NewsAsset, a commercial product that helps journalists by providing services, which analyze millions of media items from the social network in realtime. Technologies like Apache Storm can help enormously in this context. We have quantitatively analyzed the new design of NewsAsset to assess whether the introduction of Apache Storm can meet the demanding performance requirements of this media product. Our assessment approach, guided by the Unified Modeling Language (UML), takes advantage, for performance analysis, of the software designs already used for development. In addition, we converted UML into a domain-specific modeling language (DSML) for Apache Storm, thus creating a profile for Storm. Later, we transformed said DSML into an appropriate language for performance evaluation, specifically, stochastic Petri nets. The assessment ended with a successful software design that certainly met the scalability requirements of NewsAsset.

A UML Profile for the Design, Quality Assessment and Deployment of Data-intensive Applications

“…Obviously, for space reasons, we could not show up all the material we produced, but all the technical work is available at [60,62]. While, the reader can find published the Apache Storm profile, not here reported, in [29].…”

“…This experience has allowed us to identify some of the initial requirements for the profile. A fullfledged example of DTSM modeling and related analysis is, instead, presented in [47,29] where we have shown how the profile is suitable for capturing the characteristics of Apache Storm [58] applications. Finally, [22] focuses on DDSM modeling and on the generation of the corresponding blueprint.…”

“…Other technologies could have been selected as well. In [29] we describe how a different Stormbased case study can be modeled at the DPIM and DTSM level. In [22] we have shown how we can model at the DDSM level a Storm-based and a Spark-based deployment of WikiStats that exploits Cassandra as a storage node.…”

“…In this validation, we rely on the Hadoop technology since it has been the profile used to illustrate the DTSM level. Another satisfactory experience was carried out with the DTSM profile in [47], which has been extended in [29], but using the Apache Storm technology. Finally, Subsection 8.5 discusses choices of suitable performance languages for assessment.…”

Guest Editorial: Recent Trends in Reuse and Integration

“…The problem with these abstractions is that they reportedly don't scale well. Thus, researchers are attempting to incorporate Apache Storm and stochastic (colored) Petri Nets to enhance scalability (Requeno et al 2019). An open question is if it would not be more cost effective to pursue Fourth or Fifth generation (functional) languages as modeling tools in preference to UML.…”