Abstract

A brain-computer interface (BCI) based on code modulated visual evoked potentials (c-VEP) is among the fastest BCIs that have ever been reported, but it has not yet been given a thorough study. In this study, a pseudorandom binary M sequence and its time lag sequences are utilized for modulation of different stimuli and template matching is adopted as the method for target recognition. Five experiments were devised to investigate the effect of stimulus specificity on target recognition and we made an effort to find the optimal stimulus parameters for size, color and proximity of the stimuli, length of modulation sequence and its lag between two adjacent stimuli. By changing the values of these parameters and measuring classification accuracy of the c-VEP BCI, an optimal value of each parameter can be attained. Experimental results of ten subjects showed that stimulus size of visual angle 3.8°, white, spatial proximity of visual angle 4.8° center to center apart, modulation sequence of length 63 bits and the lag of 4 bits between adjacent stimuli yield individually superior performance. These findings provide a basis for determining stimulus presentation of a high-performance c-VEP based BCI system.

We apply a continuous relevance model (CRM) to the problem of directly retrieving the visual content of videos using text queries. The model computes a joint probability model for image features and words using a training set of annotated images. The model may then be used to annotate unseen test images. The probabilistic annotations are used for retrieval using text queries. We also propose a modified model -the normalized CRM -which substantially improves performance on a subset of the TREC Video dataset.

A number of projects are creating searchable digital libraries of printed books. These include the Million Book Project, the Google Book project and similar efforts from Yahoo and Microsoft. Content-based on line book retrieval usually requires first converting printed text into machine readable (e.g. ASCII) text using an optical character recognition (OCR) engine and then doing full text search on the results. Many of these books are old and there are a variety of processing steps that are required to create an end to end system. Changing any step (including the scanning process) can affect OCR performance and hence a good automatic statistical evaluation of OCR performance on book length material is needed. Evaluating OCR performance on the entire book is non-trivial. The only easily obtainable ground truth (the Gutenberg e-texts) must be automatically aligned with the OCR output over the entire length of a book. This may be viewed as equivalent to the problem of aligning two large (easily a million long) sequences. The problem is further complicated by OCR errors as well as the possibility of large chunks of missing material in one of the sequences. We propose a Hidden Markov Model (HMM) based hierarchical alignment algorithm to align OCR output and the ground truth for books. We believe this is the first work to automatically align a whole book without using any book structure information. The alignment process works by breaking up the problem of aligning two long sequences into the problem of aligning many smaller subsequences. This can be rapidly and effectively done. Experimental results show that our hierarchical alignment approach works very well even if OCR output has a high recognition error rate. Finally, we evaluate the performance of a commercial OCR engine over a large dataset of books based on the alignment results.

In this paper we describe a novel approach for jointly modeling the text and the visual components of multimedia documents for the purpose of information retrieval(IR). We propose a novel framework where individual components are developed to model different relationships between documents and queries and then combined into a joint retrieval framework. In the state-of-the-art systems, a late combination between two independent systems, one analyzing just the text part of such documents, and the other analyzing the visual part without leveraging any knowledge acquired in the text processing, is the norm. Such systems rarely exceed the performance of any single modality (i.e. text or video) in information retrieval tasks. Our experiments indicate that allowing a rich interaction between the modalities results in significant improvement in performance over any single modality. We demonstrate these results using the TRECVID03 corpus, which comprises 120 hours of broadcast news videos. Our results demonstrate over 14% improvement in IR performance over the best reported textonly baseline and ranks amongst the best results reported on this corpus.