Determining the Type of Document Read Using Eye Movement Properties by Hybrid CNN Method

Abstract

By examining the change of eye movements during reading, it is possible to determine the type of document read. Returning to the previous position (negative saccade), blink, fixation, and position are important indicators in determining the type of document being read. In this paper, a hybrid deep learning model is proposed to determine the type of document read. The MPIIDPEye dataset, which includes eye movement data of 10-minute comic, newspaper and text document readings from 20 participants, was used. First, the eye movements obtained over time were augmented by the non-linear interpolation technique. In order to process the data of each class with convolutional neural network, spectrogram images of the signals were created. Spectrogram images were given as input to Resnet architectures and the features in the Fc1000 layer were combined. Concatenated feature vectors were given as an input to feature selection algorithms. The most effective features in classification accuracy were determined and classified using the SVM algorithm. The classification was carried out for 3 different cases, and the highest accuracy of 98.41% was obtained for case-2, where Fixation, Position, and Blink properties were used.

YOLOv5 ile Topluluk Öğrenmesine Dayalı Olarak Ray Yüzeyindeki Kusurların Tespiti

“…Yapay zekâ alanının alt dallarından biri olan derin öğrenme yöntemleri, görüntülerden nesne tespiti yapma alanında kullanılan görüntü işleme algoritmaları birlikte kullanılmaktadır. Derin öğrenme sadece demiryolu alanında değil inşaat, sağlık gibi pek çok alanda kullanılmaktadır [4]. Literatürdeki derin öğrenme ağlarını kullanan çözümler demiryollarındaki kusurları tespit etmek için video ve resimlerden oluşan veri kümelerine ihtiyaç duymaktadır.…”

A convolutional neural network‐based, quantitative complete blood count scattergram‐mapping framework promptly screens acute promyelocytic leukemia with high sensitivity

“…[12][13][14][15][16][17][18] In addition, there have been studies applying CNN to CBC scattergrams for other classification tasks. [19][20][21] Specifically, deep learning models have been applied to predict APL from cell images. 17,22 However, patients who have potential APL and normal CBC counts would not be flagged and potentially would be missed under the current review rules recommended by the International Society of Laboratory Hematology (ISLH).…”

“…Recently, progress has been made in developing convolutional neural networks (CNNs) to recognize cell subtypes from microscopic images of peripheral blood or bone marrow smears 12–18 . In addition, there have been studies applying CNN to CBC scattergrams for other classification tasks 19–21 . Specifically, deep learning models have been applied to predict APL from cell images 17,22 .…”