Highly scalable algorithm for distributed real-time text indexing

Narang, Ankur; Agarwal, Vikas; Kedia, Monu; Garg, Vijay K.

doi:10.1109/hipc.2009.5433193

Cited by 5 publications

(3 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…With proper application of indexing techniques and implementation of indexing algorithms, the searching cost can be considerably reduced in terms of the decreased response time. As suggested by literature survey in [26,27,28,29,30,31,32] there are several techniques applied for indexing mechanism.…”

Section: Indexing Techniquesmentioning

confidence: 99%

Least Cost Techniques for Hidden Web

RANJAN¹

2022

Fundamentals of Engineering and Biotechnology

View full text Add to dashboard Cite

Section: Indexing Techniquesmentioning

confidence: 99%

Least Cost Techniques for Hidden Web

RANJAN¹

2022

Fundamentals of Engineering and Biotechnology

View full text Add to dashboard Cite

“…A. Narang, et al . [12], uses a distributed indexing algorithm on IBM BlueGene/L supercomputer platform. Their system focused on high-throughput data handling, real-time scalability with increased data size, indexing latency, and distributed search across 8 to 512 nodes with 2GB-8GB size data.…”

Section: Related Workmentioning

confidence: 99%

Large scale document inversion using a multi-threaded computing system

Jung

Chang

Park

2017

SIGAPP Appl. Comput. Rev.

View full text Add to dashboard Cite

Current microprocessor architecture is moving towards multi-core/multi-threaded systems. This trend has led to a surge of interest in using multi-threaded computing devices, such as the Graphics Processing Unit (GPU), for general purpose computing. We can utilize the GPU in computation as a massive parallel coprocessor because the GPU consists of multiple cores. The GPU is also an affordable, attractive, and user-programmable commodity. Nowadays a lot of information has been flooded into the digital domain around the world. Huge volume of data, such as digital libraries, social networking services, e-commerce product data, and reviews, etc., is produced or collected every moment with dramatic growth in size. Although the inverted index is a useful data structure that can be used for full text searches or document retrieval, a large number of documents will require a tremendous amount of time to create the index. The performance of document inversion can be improved by multi-thread or multi-core GPU. Our approach is to implement a linear-time, hash-based, single program multiple data (SPMD), document inversion algorithm on the NVIDIA GPU/CUDA programming platform utilizing the huge computational power of the GPU, to develop high performance solutions for document indexing. Our proposed parallel document inversion system shows 2-3 times faster performance than a sequential system on two different test datasets from PubMed abstract and e-commerce product reviews. CCS Concepts •Information systems➝Information retrieval • Computing methodologies➝Massively parallel and high-performance simulations.

show abstract

“…The above table shows execution time of the parallel algorithm and the serial algorithm respectively with different size of the collection of Web page texts. For distributed algorithm dataset is divided into four parts and index is created on the different machine of equal capacity and then they are merged in to the single index [9] on a central machine. The above table illustrates the indexing times in minutes for Centralized and Distributed Indexing approach for UCI Datasets of size 740MB.…”

Section: For Performance Evaluation Of Distributed Indexingmentioning

confidence: 99%

A highly efficient distributed indexing system based on large cluster of commodity machines

Pole¹,

Potey²

2012

2012 Ninth International Conference on Wireless and Optical Communications Networks (WOCN)

View full text Add to dashboard Cite

An Information Retrieval System using centralized approach demands long time to update the web index. A highly efficient distributed indexing system operates on large & diverse datasets with optimum time consumption compared to centralized approach to update web index. In this paper, a prototype model of highly efficient distributed indexing system deployed to run on cluster of commodity machines for the creation of large index using functionality of Apache Lucene. Experimental results showed efficiency of distributed indexing process. This distributed approach helps to reduce time interval for index creation and updation, in turn keeps the index content more fresh.

show abstract

Highly scalable algorithm for distributed real-time text indexing

Abstract: Abstract

Cited by 5 publications

References 8 publications

Least Cost Techniques for Hidden Web

Least Cost Techniques for Hidden Web

Large scale document inversion using a multi-threaded computing system

A highly efficient distributed indexing system based on large cluster of commodity machines

Contact Info

Product

Resources

About