Interactive handwriting recognition with limited user effort

Serrano, Nicolás; Giménez, Adrià; Civera, Jorge; Sanchís, Alberto; Juan, Alfons

doi:10.1007/s10032-013-0204-5

Cited by 16 publications

(18 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As a novelty, we further extend in this work the approach presented in to include the supervision of words that need to be deleted (Serrano et al, 2013), i.e. the fourth supervision case described above (e.g.…”

Section: User Supervisionmentioning

confidence: 99%

Interactive Transcription of Old Text Documents

Martínez-Santos¹

View full text Add to dashboard Cite

Nowadays, there are huge collections of handwritten text documents in libraries all over the world. The high demand for these resources has led to the creation of digital libraries in order to facilitate the preservation and provide electronic access to these documents. However text transcription of these documents images are not always available to allow users to quickly search information, or computers to process the information, search patterns or draw out statistics. The problem is that manual transcription of these documents is an expensive task from both economical and time viewpoints. This thesis presents a novel approach for efficient Computer Assisted Transcription (CAT) of handwritten text documents using state-of-the-art Handwriting Text Recognition (HTR) systems.The objective of CAT approaches is to efficiently complete a transcription task through human-machine collaboration, as the effort required to generate a manual transcription is high, and automatically generated transcriptions from state-of-the-art systems still do not reach the accuracy required. This thesis is centered on a special application of CAT, that is, the transcription of old text document when the quantity of user effort available is limited, and thus, the entire document cannot be revised. In this approach, the objective is to generate the best possible transcription by means of the user effort available. This thesis provides a comprehensive view of the CAT process from feature extraction to user interaction.First, a statistical approach to generalise interactive transcription is proposed. As its direct application is unfeasible, some assumptions are made to apply it to two different tasks. First, on the interactive transcription of handwritten text documents, and next, on the interactive detection of the document layout.Next, the digitisation and annotation process of two real old text documents is described. This process was carried out because of the scarcity of similar resources and the need of annotated data to thoroughly test all the developed tools and techniques in this thesis. These two documents were carefully selected to represent the general difficulties that are encountered when dealing with HTR. Baseline results are presented on these two documents to settle down a benchmark with a standard HTR system. Finally, these annotated documents were made freely available to the community. It must be noted that, all the techniques and methods vii developed in this thesis have been assessed on these two real old text documents.Then, a CAT approach for HTR when user effort is limited is studied and extensively tested. The ultimate goal of applying CAT is achieved by putting together three processes. Given a recognised transcription from an HTR system. The first process consists in locating (possibly) incorrect words and employs the user effort available to supervise them (if necessary). As most words are not expected to be supervised due to the limited user effort available, only a few are selected to be revised. The s...

show abstract

Section: User Supervisionmentioning

confidence: 99%

Interactive Transcription of Old Text Documents

Martínez-Santos¹

View full text Add to dashboard Cite

show abstract

“…This second phase incorporates a new interaction strategy called intelligent interaction [23] in order to study if review times could be further improved. This strategy is based on the application of Active Learning (AL) techniques to ASR [3].…”

Section: Second Phase: Intelligent Interactionmentioning

confidence: 99%

“…The RNNLM is trained using the RNNLM toolkit [23] on a small quantity of text related to the topic. Once this RNNLM is estimated, it is employed to search better transcriptions or translations.…”

Section: Recurrent Neural Network Lmmentioning

confidence: 99%

“…Specifically, we adapted the acoustic models to the speaker with the Maximum Likelihood Linear Regression (MLLR) technique [5], and the language models using a linear interpolation between the language model trained on the reviewed transcriptions and the large language model previously trained [13]. Then, the automatic transcriptions were regenerated, preserving those segments already reviewed by lecturers, and using them to improve the recognition of the context words using a constrained search [9,23]. This two-step review process was successfully completed by 15 lecturers on a total of 26 video lectures with 3.7 hours of video.…”

Section: Third Phase: Two-step Supervisionmentioning

confidence: 99%

“…The ASR system was trained on a novel large speech corpus, known as poliMedia [32], that was manually transcribed for the European project transLectures [34]. This speech recognition system was developed within the framework of the European transLectures project [34], along the lines of other systems, such as KALDI [23], JANUS-II [36], UPC RAMSES [19] or SPHINX-II [13]. Initial results on the poliMedia corpus are also reported to compare the performance of different ASR systems based on speaker and topic adaptation.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Evaluation of innovative computer-assisted transcription and translation strategies for video lecture repositories

Miró¹,

Daniel²

View full text Add to dashboard Cite

Nowadays, the technology enhanced learning area has experienced a strong growth with many new learning approaches like blended learning, flip teaching, massive open online courses, and open educational resources to complement face-to-face lectures. Specifically, video lectures are fast becoming an everyday educational resource in higher education for all of these new learning approaches, and they are being incorporated into existing university curricula around the world.Transcriptions and translations can improve the utility of these audiovisual assets, but rarely are present due to a lack of cost-effective solutions to do so. Lecture searchability, accessibility to people with impairments, translatability for foreign students, plagiarism detection, content recommendation, note-taking, and discovery of content-related videos are examples of advantages of the presence of transcriptions.For this reason, the aim of this thesis is to test in real-life case studies ways to obtain multilingual captions for video lectures in a cost-effective way by using state-of-the-art automatic speech recognition and machine translation techniques. Also, we explore interaction protocols to review these automatic transcriptions and translations, because unfortunately automatic subtitles are not error-free. In addition, we take a step further into multilingualism by extending our findings and evaluation to several languages. Finally, the outcomes of this thesis have been applied to thousands of video lectures in European universities and institutions. vii RESUMENHoy en día, el área del aprendizaje mejorado por la tecnología ha experimentado un fuerte crecimiento con muchos nuevos enfoques de aprendizaje como el aprendizaje combinado, la clase inversa, los cursos masivos abiertos en línea, y nuevos recursos educativos abiertos para complementar las clases presenciales. En concreto, los videos docentes se están convirtiendo rápidamente en un recurso educativo cotidiano en la educación superior para todos estos nuevos enfoques de aprendizaje, y se están incorporando a los planes de estudios universitarios existentes en todo el mundo.Las transcripciones y las traducciones pueden mejorar la utilidad de estos recursos audiovisuales, pero rara vez están presentes debido a la falta de soluciones rentables para hacerlo. La búsqueda de y en los videos, la accesibilidad a personas con impedimentos, la traducción para estudiantes extranjeros, la detección de plagios, la recomendación de contenido, la toma de notas y el descubrimiento de videos relacionados son ejemplos de las ventajas de la presencia de transcripciones.Por esta razón, el objetivo de esta tesis es probar en casos de estudio de la vida real las formas de obtener subtítulos multilingües para videos docentes de una manera rentable, mediante el uso de técnicas avanzadas de reconocimiento automático de voz y de traducción automática. Además, exploramos diferentes modelos de interacción para revisar estas transcripciones y traducciones automáticas, pues desafortunadamente los su...

show abstract

Lexicon-based probabilistic indexing of handwritten text images

Vidal

Toselli

Puigcerver

2023

Neural Comput & Applic

View full text Add to dashboard Cite

Keyword Spotting (KWS) is here considered as a basic technology for Probabilistic Indexing (PrIx) of large collections of handwritten text images to allow fast textual access to the contents of these collections. Under this perspective, a probabilistic framework for lexicon-based KWS in text images is presented. The presentation aims at providing formal insights which help understanding classical statements of KWS (from which PrIx borrows fundamental concepts), as well as the relative challenges entailed by these statements. The development of the proposed framework makes it clear that word recognition or classification implicitly or explicitly underlies any formulation of KWS. Moreover, it suggests that the same statistical models and training methods successfully used for handwriting text recognition can advantageously be used also for PrIx, even though PrIx does not generally require or rely on any kind of previously produced image transcripts. Experiments carried out using these approaches support the consistency and the general interest of the proposed framework. Results on three datasets traditionally used for KWS benchmarking are significantly better than those previously published for these datasets. In addition, good results are also reported on two new, larger handwritten text image datasets (Bentham and Plantas), showing the great potential of the methods proposed in this paper for indexing and textual search in large collections of untranscribed handwritten documents. Specifically, we achieved the following Average Precision values: IAMDB: 0.89, George Washington: 0.91, Parzival: 0.95, Bentham: 0.91 and Plantas: 0.92.

show abstract

Interactive handwriting recognition with limited user effort

Cited by 16 publications

References 27 publications

Interactive Transcription of Old Text Documents

Interactive Transcription of Old Text Documents

Evaluation of innovative computer-assisted transcription and translation strategies for video lecture repositories

Lexicon-based probabilistic indexing of handwritten text images

Contact Info

Product

Resources

About