A web search engine model based on index-query bit-level compression

Abstract

In this paper, we propose a new web search engine model based on index-query bit-level compression. The model incorporates two bit-level compression layers both implemented at the backend processor (server) side, one layer resides after the indexer acting as a second compression layer to generate a double compressed index, and the second layer be located after the query parser for query compression to enable bit-level compressed index-query search. This contributes to reducing the size of the index file as well as reducing disk I/O overheads, and consequently yielding higher retrieval rate and performance. The data compression scheme used in this model is the adaptive character wordlength (ACW(n,s)) scheme, which is an asymmetric, lossless, bit-level scheme that permits compressed index-query search. Results investigating the performance of the ACW(n,s) scheme is presented and discussed.

Development of a Novel Compressed Index-Query Web Search Engine Model

“…al. [15] and Al-Bahadili & Al-Saab [14] proposed an investigated the performance of a new Web search engine model based on index-query bit-level compression. The model incorporates two bitlevel compression layers both implemented at the back-end processor (server) side, one layer resides after the indexer acting as a second compression layer to generate a double compressed index, and the second layer be located after the query parser for query compression to enable bit-level compressed index-query search.…”

“…The HCDC algorithm [14,15] is implemented in three different sub procedures. Two of them for compression, in particular for index and query compression, and one for retrieved data decompression to be used at the different compression/ decompression layers as shown in Figure 2.…”

“…The HCDC algorithm [14,15] is implemented in three different subprocedures. Two of them for compression, in particular for index and rieved data decompression to be decompression layers as shown in It reads the index file, finds the to binary sequence using the characters ASCII code, creates and stores the compression header, compresses the binary sequence, and converts the compressed binary sequence to characters and stores them in a reads the keywords, converts characters to binary sequence using the characters ASCII code, reads the compression header created by INXCOM, compresses the binary sequence, converts the compressed binary sequence back to compressed index.…”

“…al. [14] and Al-Bahadili and Al-Saab [15] proposed a compressed index-query Web search engine model that enables performing the searching process at a compressed index-query level. The data compression algorithm used was the adaptive character wordlength (ACW) algorithm [16,17].…”

“…The different components of the new Web model are implemented and integrated to build a prototype CIQ Web search engine, which also used as a test bench to validate the accuracy and evaluate and compare the performance of the CIQ model, namely, the CIQ test tool (CIQTT) (Al-Saab, 2011). The CIQTT was used to collect a test corpus of 104000 documents from 30 well-known Websites; process and analyze the test corpus; generate five inverted indexes of different sizes (1000, 10000, 25000, 50000, and 75000 documents), compress the indexes and measure the compression ratio and the storage reduction factor; search the indexes for 29 different keywords in both compressed and uncompressed forms; and finally, compare the outcomes of the different search processes and estimate the speedup factor and the time reduction factor.…”