Using of Transformers Models for Text Classification to Mobile Educational Applications

Pilicita, Anabel; Barra, Enrique

doi:10.1109/tla.2023.10172138

Cited by 4 publications

(2 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The paper categorizes TB models, compares their architectures, and discusses limitations, offering insights to boost innovation in NLP applications and AI-powered products. Pilicita et al [29] investigate the utility of five BERT-based pre-trained models in classifying mobile educational applications. Leveraging a dataset enriched with descriptions and categories from the Google Play Store, the study demonstrates the effectiveness of these models, achieving notable accuracy rates ranging from 76% to 81%.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

VisFormers—Combining Vision and Transformers for Enhanced Complex Document Classification

Dutta,

Adhikary,

Dwivedi

2024

MAKE

View full text Add to dashboard Cite

Complex documents have text, figures, tables, and other elements. The classification of scanned copies of different categories of complex documents like memos, newspapers, letters, and more is essential for rapid digitization. However, this task is very challenging as most scanned complex documents look similar. This is because all documents have similar colors of the page and letters, similar textures for all papers, and very few contrasting features. Several attempts have been made in the state of the art to classify complex documents; however, only a few of these works have addressed the classification of complex documents with similar features, and among these, the performances could be more satisfactory. To overcome this, this paper presents a method to use an optical character reader to extract the texts. It proposes a multi-headed model to combine vision-based transfer learning and natural-language-based Transformers within the same network for simultaneous training for different inputs and optimizers in specific parts of the network. A subset of the Ryers Vision Lab Complex Document Information Processing dataset containing 16 different document classes was used to evaluate the performances. The proposed multi-headed VisFormers network classified the documents with up to 94.2% accuracy, while a regular natural-language-processing-based Transformer network achieved 83%, and vision-based VGG19 transfer learning could achieve only up to 90% accuracy. The model deployment can help sort the scanned copies of various documents into different categories.

show abstract

Section: Related Workmentioning

confidence: 99%

“…In contrast, our model obtained an accuracy of over 96% and a training time of 951 s, surpassing all other models. In the paper [29], the author used BERT-based pre-trained models for classifying documents. Their model achieved an accuracy of 81%, which is very low compared to our proposed vision Transformer.…”

Section: Comparison With the State Of The Artmentioning

confidence: 99%