Mathematical Symbol Indexing Using Topologically Ordered Clusters of Shape Contexts

Abstract

This paper addresses the indexing and retrieval of mathematical symbols from digitized documents. The proposed approach exploits Shape Contexts (SC) to describe the shape of mathematical symbols. Starting from the vector space method, that is based on SC clustering, we explore the use of topological ordered clusters to improve the retrieval performance. The clustering is computed by means of SelfOrganizing Maps that organize the clusters in two dimensional topologically ordered feature maps. The retrieval performance are compared with those obtained using the K-means clustering on a large collection of mathematical symbols gathered from the widely used INFTY database.

Recognition and retrieval of mathematical expressions

“…Retrieval is based on the similarity of math notation images, without recognizing their math content. For example, Marinai et al propose a method based on shape contexts for retrieving mathematical symbols [96], while Yu and Zanibbi propose a retrieval method in which handwritten queries are matched to document images using a combination of X-Y cutting and word shape matching [161,167].…”

“…One interesting approach compared retrieval using the Active-Math system [91] with retrieval from the ActiveMath web pages using the Google search engine, as well as a human-centered evaluation using a 'talk aloud' protocol, where participants are asked to speak their thoughts as they completed search tasks involving keywords and/or small expressions. Marinai et al [96] provide precisionrecall plots for their method for image-based math symbol retrieval using a bag-of-visual-words produced from clustered shape contexts [15]. Precision at 0% recall is presented, with precision values as high as 87% reported.…”

“…Example queries are shown in Figure 5. Recently, images of symbols [96] and complete expressions (handwritten [161,167] and typeset [168]) have been used for query-by-expression.…”

Keyword and image-based retrieval of mathematical expressions

“…Previously Marinai et al 3 devised a method for image-based retrieval of individual mathematical symbols using shape contexts. We present a new technique for retrieving whole expressions using images as queries, employing a simple Content-Based Image Retrieval (CBIR 4 ) approach.…”

Rotation-robust math symbol recognition and retrieval using outer contours and image subsampling

“…Content-based math symbol image retrieval is a fairly new area of research and it can allow people to search a math symbol just using the image of itself. Marinai et al 15 proposed an image-based math symbol retrieval method. They use Self Organizing Map (SOM) to cluster Shape Contexts (SC) extracted from each symbol into 30 classes.…”