Improving Accessibility of Archived Raster Dictionaries of Complex Script Languages

Abstract

We propose an approach to index raster images of dictionary pages which in turn would require very little manual effort to enable direct access to the appropriate pages of the dictionary for lookup. Accessibility is further improved by feedback and crowdsourcing that enables highlighting of the specific location on the page where the lookup word is found, annotation, digitization, and fielded searching. This approach is equally applicable on simple scripts as well as complex writing systems. Using our proposed approach, we have built a Web application called "Dictionary Explorer" which supports word indexes in various languages and every language can have multiple dictionaries associated with it. Word lookup gives direct access to appropriate pages of all the dictionaries of that language simultaneously. The application has exploration features like searching, pagination, and navigating the word index through a tree-like interface. The application also supports feedback, annotation, and digitization features. Apart from the scanned images, "Dictionary Explorer" aggregates results from various sources and user contributions in Unicode. We have evaluated the time required for indexing dictionaries of different sizes and complexities in the Urdu language and examined various tradeoffs in our implementation. Using our approach, a single person can make a dictionary of 1,000 pages searchable in less than an hour.

CLEU ‐ A Cross‐language english‐urdu corpus and benchmark for text reuse experiments

“…Urdu belongs to the Indo-Aryan family, widely spoken in Pakistan and the northern parts of India (Alam, Mehmood, & Nelson, 2015). Urdu belongs to the Indo-Aryan family, widely spoken in Pakistan and the northern parts of India (Alam, Mehmood, & Nelson, 2015).…”

“…This study aims to develop a publicly available large‐scale benchmark corpus that contains real examples of cross‐language text reuse at sentence/passage level for the English‐Urdu language pair. Urdu belongs to the Indo‐Aryan family, widely spoken in Pakistan and the northern parts of India (Alam, Mehmood, & Nelson, ). Moreover, it has a strong Perso‐Arabic influence in its vocabulary and is written in a Perso‐Arabic script from right to left.…”

Identifying Documents In-Scope of a Collection from Web Archives

“…Our CNNs comprise of mainly two layers (as shown in Figure 1): a convolutional layer followed by max pooling and a fully connected layer for the classification. For the CNN input, we consider a document (partial) as a sequence of words and use pre-trained word embeddings 1 https://scikit-learn.org/stable/ for each word. These pre-trained word embeddings are trained on the Google News dataset using the Word2Vec 2 [35] algorithm.…”

“…Nwala et al [36] studied bootstrapping of the web archive collections from the social media and showed that sources such as Reddit, Twitter, and Wikipedia can produce collections that are similar to expert generated collections (i.e., Archive-It collections). Alam et al [1] proposed an approach to index raster images of dictionary pages and built a Web application that supports word indexes in various languages with multiple dictionaries. Alam et al [2] used CDX summarization for web archive profiling, whereas AlNoamany et al [3] proposed the Dark and Stormy Archive (DSA) framework for summarizing the holdings of these collections and arranging them into a chronological order.…”