Capacity Planning for Vertical Search Engines: An Approach Based on Coloured Petri Nets

Gil-Costa, Verónica; Lobos, Jair; Inostrosa-Psijas, Alonso

doi:10.1007/978-3-642-31131-4_16

Cited by 12 publications

(8 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Minimize the total number of processors P = p × r provided that no processor utilization gets above a target utilization bound U ≤ 1. In practice, search engines are over-provisioned so that they are prepared to cope with sudden peaks in query traffic [6,18]. This means that they maintain processor utilization well below 100% at steady state operation.…”

Section: Case Study: a Distributed Text Search Enginementioning

confidence: 99%

Distributed text search using suffix arrays

Arroyuelo¹,

Bonacic²,

Gil-Costa³

et al. 2014

Parallel Computing

Self Cite

View full text Add to dashboard Cite

Text search is a classical problem in Computer Science, with many data-intensive applications. For this problem, suffix arrays are among the most widely known and used data structures, enabling fast searches for phrases, terms, substrings and regular expressions in large texts. Potential application domains for these operations include large-scale search services, such as Web search engines, where it is necessary to efficiently process intensive-traffic streams of on-line queries. This paper proposes strategies to enable such services by means of suffix arrays. We introduce techniques for deploying suffix arrays on clusters of distributed-memory processors and then study the processing of multiple queries on the distributed data structure. Even though the cost of individual search operations in sequential (non-distributed) suffix arrays is low in practice, the problem of processing multiple queries on distributed-memory systems, so that hardware resources are used efficiently, is relevant to services aimed at achieving high query throughput at low operational costs. Our theoretical and experimental performance studies show that our proposals are suitable solutions for building efficient and scalable on-line search services based on suffix arrays. IntroductionIn the last decade, the design of efficient data structures and algorithms for textual databases and related applications has received a great deal of attention, due to the rapid growth of the amount of text data available from different sources. Typical applications support text searches over big text collections in a client-server fashion, where the user queries are answered by a dedicated server [15]. The server efficiency-in terms of running time-is of paramount importance in cases where the services demanded by clients generate a heavy work load. A feasible way to overcome the limitations of sequential computers is to resort to the use of several computers, or processors, which work together to serve the ever increasing client demands [19].One such approach to efficient parallelization is to distribute the data onto the processors, in such a way that it becomes feasible to exploit locality via parallel processing of user requests, each on a subset of the data. As opposed to shared-memory models, this distributed-memory model provides the benefit of better * Corresponding author. Address: Av. España 1680, Valparaíso, Chile. Phone: +56 2 432 6722. Fax: +56 2 432 6702. , in distributed memory systems, and describes strategies to reduce the inter-processor communication and to improve the load balance at search time. Indexed Text SearchingThe advent of powerful processors and cheap storage has enabled alternative models for information retrieval, other than the traditional one of a collection of documents indexed by a fixed set of keywords. One is the full text model, in which the user expresses its information need via words, phrases or patterns to be matched for, and the information system retrieves those documents containing the user-specified pa...

show abstract

Section: Case Study: a Distributed Text Search Enginementioning

confidence: 99%

Distributed text search using suffix arrays

Arroyuelo¹,

Bonacic²,

Gil-Costa³

et al. 2014

Parallel Computing

Self Cite

View full text Add to dashboard Cite

show abstract

“…To the best of our knowledge, there is no work on using CPNs to predict performance of large search engines. We have presented a preliminary work on this topic in [15] which in the present paper has been substantially expanded with more precise CPN models and a tool for efficient simulation.…”

Section: Coloured Petri Nets (Cpns)mentioning

confidence: 99%

“…With almost 140 partitions the IS cluster cannot improve performance any further because the inverted index becomes very small. Further details on these experiments can be found in [15].…”

Section: Benchmarksmentioning

confidence: 99%

“…We also compare one of our CPN models against a simple CPN model of the same system that we have developed with CPN Tools (cpntools.org) and presented in [15]. In this case, because of memory and running time limitations of CPN tools, the simple model assumes one partition for the IS assuming that the remaining partitions perform the same work in parallel and thereby they finish together.…”

Section: Model Resolutionmentioning

confidence: 99%

See 1 more Smart Citation

Modelling Search Engines Performance Using Coloured Petri Nets

Gil-Costa

Inostrosa-Psijas

Lobos³

et al. 2014

Fundamenta Informaticae

Self Cite

View full text Add to dashboard Cite

This paper proposes using Coloured Petri Nets to model performance of vertical search engines for Web search. In such systems, queries submitted by users or client systems are handled by different components implemented as services deployed on large clusters of dedicated processors. We propose models that represent key features of components running time cost at a suitable level of detail. A comprehensive evaluation study is presented to reveal good precision of models when compared against actual implementations and complex process-oriented simulators of the same search engine instances. A C++ class library is proposed to enable rapid model construction by using a hierarchical and scalable approach, and to enable transparent generation and efficient execution of respective simulation programs either sequentially or in parallel.

show abstract

“…Performance evaluation by means of discrete-event simulation of large scale social software applications is a topic that has deserved little attention even in wellestablished research areas such as Web search [1,2], whereas has deserved none in much less developed areas such as emergency management of natural disasters. Yet performance is a relevant topic to be taken into consideration when designing social applications intended to scale to thousands/millions of users.…”

Section: Introductionmentioning

confidence: 99%

Experimental Framework to Simulate Rescue Operations after a Natural Disaster

Veas-Castillo

Ovando-Leon

Astudillo

et al. 2020

JC&ST

Self Cite

View full text Add to dashboard Cite

Computational simulation is a powerful tool for performance evaluation of computational systems. It is useful to make capacity planning of data center clusters, to obtain profiling reports of software applications and to detect bottlenecks. It has been used in different research areas like large scale Web search engines, natural disaster evacuations, computational biology, human behavior and tendency, among many others. However, properly tuning the parameters of the simulators, defining the scenarios to be simulated and collecting the data traces is not an easy task. It is an incremental process which requires constantly comparing the estimated metrics and the flow of simulated actions against real data. In this work, we present an experimental framework designed for the development of large scale simulations of two applications used upon the occurrence of a natural disaster strikes. The first one is a social application aimed to register volunteers and manage emergency campaigns and tasks. The second one is a benchmark application a data repository named MongoDB. The applications are deployed in a distributed platform which combines different technologies like a Proxy, a Containers Orchestrator, Containers and a NoSQL Database. We simulate both applications and the architecture platform. We validate our simulators using real traces collected during simulacrums of emergency situations.

show abstract

Capacity Planning for Vertical Search Engines: An Approach Based on Coloured Petri Nets

Cited by 12 publications

References 24 publications

Distributed text search using suffix arrays

Distributed text search using suffix arrays

Modelling Search Engines Performance Using Coloured Petri Nets

Experimental Framework to Simulate Rescue Operations after a Natural Disaster

Contact Info

Product

Resources

About