Evaluation of a Semi-automated Semantic Annotation Approach for Bootstrapping the Analysis of Large-Scale Web Service Networks

Abstract

Abstract-In recent years many methods have been proposed, which require semantic annotations of Web services as an input. Such methods include discovery, match-making, composition and execution of Web services in dynamic settings, just to mention few. At the same time automated Web service annotation approaches have been proposed for supporting application of former methods in settings where it is not feasible to provide the annotations manually. However, lack of effective automated evaluation frameworks has seriously limited proper evaluation of the constructed annotations in practical settings where the overall annotation quality of millions of Web services needs to be evaluated. This paper describes an evaluation framework for measuring the quality of semantic annotations of large number of Web services descriptions provided in form of WSDL and XSD documents. The evaluation framework is based on analyzing network properties, namely scale-free and smallworld properties, of Web service networks, which in turn have been constructed from semantic annotations of Web services. The evaluation approach is demonstrated through evaluation of a semi-automated annotation approach, which was applied to a set of publicly available WSDL documents describing altogether ca 200 000 Web service operations.

Analyzing Web Services Networks: Theory and Practice

“…The effectiveness of the weighted network model in capturing and preserving the structural properties of underlying Web services network is already studied in [7,12]. The following definition defines how a weighted Web services network is transformed into a weighted label network.…”

“…They formalized the interaction between these two domains in terms of overlapping concepts used in annotation of input and output parameter of relevant Web services. Finally, Mokarizadeh et al [12] suggested a method for analyzing information diffusion between categories of data-centric Web services. The method operates on a Web services network constructed by linking interface descriptions of categorized Web services.…”

Information Diffusion in Web Services Networks

“…Since the annotated elements are in fact instances in the generated reference ontology, the instance matching process is used to find ontological relationships between those annotated elements. We employ a rule-based instance matching method that has already been described and evaluated in our previous work [13]. The matching method takes as input a pair of instances and produces a correspondence element.…”

“…Every semantic node is further associated with category vector − → Q denoting the weight of the semantic node in different categories wrt its relative occurrence in the categories. The effectiveness of the semantic network model in capturing and preserving the structural properties of underlying Web services network is evaluated and proved in our previous work [13].…”

“…However, there are some publications addressing Web services network analysis. These papers mainly target determination of structural characteristics of Web services networks [9], [6], [13] and/or propose service composition methods operating on Web services networks [6], [14], [5]. Anyway, further advances are required to take full advantage of Web services networks.…”