Indexing compressed text

Ferragina, Paolo; Manzini, Giovanni

doi:10.1145/1082036.1082039

Cited by 583 publications

(541 citation statements)

References 29 publications

Supporting

Mentioning

539

Contrasting

Unclassified

Order By: Relevance

“…Node: is a mapping from phrase identifiers to their node in LZTrie. 4. Range: is a data structure for two-dimensional searching in the space [0 .…”

Section: The Lz-index Data Structurementioning

confidence: 99%

“…When we replace Range by RNode structure, the result is actually a "navigation" scheme that permits us moving back and forth from trie nodes to the corresponding preorder positions 4 , both in LZTrie and RevTrie. The phrase identifiers are common to both tries and permit moving from one trie to the other.…”

Section: Lz-index As a Navigation Schemementioning

confidence: 99%

“…Then a compressed full-text self-index replaces the text with a more spaceefficient representation of it, which at the same time provides indexed access to the text. Some works on compressed self-indexes are [19,4,6,9,5]. Taking space proportional to the compressed text, replacing it, and providing efficient indexed access to it is an unprecedented breakthrough in text indexing and compression.…”

Section: Introduction and Previous Workmentioning

confidence: 99%

“…If the text is parsed into n phrases by the LZ78 algorithm, then the LZindex takes 4n log 2 n(1 + o(1)) bits of space, which is 4 times the size of the compressed text, i.e. 4uH k (T ) + o(u log σ ) bits, for any k = o(log σ u) [8,4]. The LZ-index answers queries in O(m 3 log σ + (m + occ) log n) worst case time.…”

Section: Introduction and Previous Workmentioning

confidence: 99%

“…However, in practice the space requirement of LZ-index is relatively large compared with competing schemes, such as CS-Array [19] and FM-index [4], which require 0.6-0.7 and 0.3-0.8 times the text size respectively, versus 1.2-1.6 times the text size of LZ-index. Yet, the LZ-index is faster to report and to display the context of an occurrence.…”

Section: Introduction and Previous Workmentioning

confidence: 99%

See 4 more Smart Citations

Reducing the Space Requirement of LZ-Index

Arroyuelo

Navarro

Sadakane

2006

Combinatorial Pattern Matching

View full text Add to dashboard Cite

Abstract. The LZ-index is a compressed full-text self-index able to represent a text T 1...u , over an alphabet of size σ and with k-th order empirical entropy H k (T ), using 4uH k (T ) + o(u log σ ) bits for any k = o(log σ u). It can report all the occ occurrences of a pattern P 1...m in T in O(m 3 log σ + (m + occ) log u) worst case time. This is the only existing data structure of size O(uH k (T )) able of spending O(log u) time per occurrence reported. Its main drawback is the factor 4 in its space complexity, which makes it larger than other stateof-the-art alternatives. In this paper we present two different approaches to reduce the space requirement of LZ-index. In both cases we achieve (2 + ε)uH k (T ) + o(u log σ ) bits of space, for any constant ε > 0, and we simultaneously improve the search time to O(m 2 log m + (m + occ) log u). Both indexes support displaying any subtext of length ℓ in optimal O(ℓ/ log σ u) time. In addition, we show how the space can be squeezed to (1 + ε)uH k (T ) + o(u log σ ) to obtain a structure with O(m 2 ) average search time for m 2 log σ u.

show abstract

“…Node: is a mapping from phrase identifiers to their node in LZTrie. 4. Range: is a data structure for two-dimensional searching in the space [0 .…”

Section: The Lz-index Data Structurementioning

confidence: 99%

Section: Lz-index As a Navigation Schemementioning

confidence: 99%

Section: Introduction and Previous Workmentioning

confidence: 99%

Section: Introduction and Previous Workmentioning

confidence: 99%

Section: Introduction and Previous Workmentioning

confidence: 99%

See 3 more Smart Citations

Reducing the Space Requirement of LZ-Index

Arroyuelo

Navarro

Sadakane

2006

Combinatorial Pattern Matching

View full text Add to dashboard Cite

show abstract

Efficiently compressing 3D medical images for teleinterventions via CNNs and anisotropic diffusion

et al. 2021

View full text Add to dashboard Cite

Purpose Efficient compression of images while preserving image quality has the potential to be a major enabler of effective remote clinical diagnosis and treatment, since poor Internet connection conditions are often the primary constraint in such services. This paper presents a framework for organ‐specific image compression for teleinterventions based on a deep learning approach and anisotropic diffusion filter. Methods The proposed method, deep learning and anisotropic diffusion (DLAD), uses a convolutional neural network architecture to extract a probability map for the organ of interest; this probability map guides an anisotropic diffusion filter that smooths the image except at the location of the organ of interest. Subsequently, a compression method, such as BZ2 and HEVC‐visually lossless, is applied to compress the image. We demonstrate the proposed method on three‐dimensional (3D) CT images acquired for radio frequency ablation (RFA) of liver lesions. We quantitatively evaluate the proposed method on 151 CT images using peak‐signal‐to‐noise ratio (normalPSNR), structural similarity (italicSSIM), and compression ratio (italicCR) metrics. Finally, we compare the assessments of two radiologists on the liver lesion detection and the liver lesion center annotation using 33 sets of the original images and the compressed images. Results The results show that the method can significantly improve italicCR of most well‐known compression methods. DLAD combined with HEVC‐visually lossless achieves the highest average italicCR of 6.45, which is 36% higher than that of the original HEVC and outperforms other state‐of‐the‐art lossless medical image compression methods. The means of italicPSNR and italicSSIM are 70 dB and 0.95, respectively. In addition, the compression effects do not statistically significantly affect the assessments of the radiologists on the liver lesion detection and the lesion center annotation. Conclusions We thus conclude that the method has a high potential to be applied in teleintervention applications.

show abstract

Discussion on common data analysis strategies used in MS‐based proteomics

et al. 2011

View full text Add to dashboard Cite

Current proteomics technology is limited in resolving the proteome complexity of biological systems. The main issue at stake is to increase throughput and spectra quality so that spatiotemporal dimensions, population parameters and the complexity of protein modifications on a quantitative scale can be considered. MS-based proteomics and protein arrays are the main players in large-scale proteome analysis and an integration of these two methodologies is powerful but presently not sufficient for detailed quantitative and spatiotemporal proteome characterization. Improvements of instrumentation for MS-based proteomics have been achieved recently resulting in data sets of approximately one million spectra which is a large step in the right direction. The corresponding raw data range from 50 to 100 Gb and are frequently made available. Multidimensional LC-MS data sets have been demonstrated to identify and quantitate 2000-8000 proteins from whole cell extracts. The analysis of the resulting data sets requires several steps from raw data processing, to database-dependent search, statistical evaluation of the search result, quantitative algorithms and statistical analysis of quantitative data. A large number of software tools have been proposed for the above-mentioned tasks. However, it is not the aim of this review to cover all software tools, but rather discuss common data analysis strategies used by various algorithms for each of the above-mentioned steps in a non-redundant approach and to argue that there are still some areas which need improvements.

show abstract

Indexing compressed text

Cited by 583 publications

References 29 publications

Reducing the Space Requirement of LZ-Index

Reducing the Space Requirement of LZ-Index

Efficiently compressing 3D medical images for teleinterventions via CNNs and anisotropic diffusion

Discussion on common data analysis strategies used in MS‐based proteomics

Contact Info

Product

Resources

About