End-to-End Text Classification via Image-based Embedding using Character-level Networks

Abstract

For analysing and/or understanding languages having no word boundaries based on morphological analysis such as Japanese, Chinese, and Thai, it is desirable to perform appropriate word segmentation before word embeddings. But it is inherently difficult in these languages. In recent years, various language models based on deep learning have made remarkable progress, and some of these methodologies utilizing characterlevel features have successfully avoided such a difficult problem. However, when a model is fed character-level features of the above languages, it often causes overfitting due to a large number of character types. In this paper, we propose a CE-CLCNN, character-level convolutional neural networks using a character encoder to tackle these problems. The proposed CE-CLCNN is an end-to-end learning model and has an image-based character encoder, i.e. the CE-CLCNN handles each character in the target document as an image. Through various experiments, we found and confirmed that our CE-CLCNN captured closely embedded features for visually and semantically similar characters and achieves state-of-the-art results on several open document classification tasks. In this paper we report the performance of our CE-CLCNN with the Wikipedia title estimation task and analyse the internal behaviour.

AraDIC: Arabic Document Classification Using Image-Based Character Embeddings and Class-Balanced Loss

“…We choose two classifiers for our framework. A character CNN (CLCNN) similar to Kitada et al (2018), but tuned to Arabic language, and a bidirectional gated recurrent unit (BiGRU) (Chung et al, 2014) based classifier. The outline of our framework is shown in Figure 2.…”

“…To solve these problems, character-based approaches utilizing deep learning methods mainly used in image processing have been proposed (Zhang et al, 2015;Shimada et al, 2016;Kitada et al, 2018). Zhang et al (2015) introduced a character-level CNN (CLCNN) that treats text as a raw signal at character level.…”

“…Later, Liu et al (2017) used image-based character embeddings learned through a character en-coder (CE) to train a gated recurrent unit (GRU) for Japanese, Chinese, and Korean text classification. Kitada et al (2018) proposed CE-CLCNN that concatenated Liu et al (2017)'s CE with CLCNN as an end-to-end system and introduced random erasing on image domain as a data augmentation method. These models using character-level features learn language morphology eliminating the need for complex morphological analysis and word segmentation.…”

Text Classification through Glyph-aware Disentangled Character Embedding and Semantic Sub-character Augmentation

“…Embedding methods based on character images have been proposed with some excellent success (Chen et al, 2015;Sun et al, 2016;Yu et al, 2017;Sun et al, 2019;Dai and Cai, 2017;Shimada et al, 2016;Kitada et al, 2018;Ke and Hagiwara, 2017;Aldón Mínguez et al, 2016). These methods are also called glyphaware embedding as they generate embeddings that take into account the shape of the characters or subcharacters.…”

“…For example, the following Japanese characters have a common form of "辶," which is a sub-character meaning of the related word road: "迫" (approach: come near the destination by road) and "追" (follow: track the road). In consideration of these characteristics of the language, several glyph-aware natural language processing (NLP) models have been proposed (Shimada et al, 2016;Kitada et al, 2018;Sun et al, 2019). These deep-learning-based models train input text as a sequence of character images and learn character embeddings from the images.…”