Deep multi-modal networks for book genre classification based on its cover

Abstract

Book covers are usually the very first impression to its readers and they often convey important information about the content of the book. Book genre classification based on its cover would be utterly beneficial to many modern retrieval systems, considering that the complete digitization of books is an extremely expensive task. At the same time, it is also an extremely challenging task due to the following reasons: First, there exists a wide variety of book genres, many of which are not concretely defined. Second, book covers, as graphic designs, vary in many different ways such as colors, styles, textual information, etc, even for books of the same genre. Third, book cover designs may vary due to many external factors such as country, culture, target reader populations, etc. With the growing competitiveness in the book industry, the book cover designers and typographers push the cover designs to its limit in the hope of attracting sales. The cover-based book classification systems become a particularly exciting research topic in recent years. In this paper, we propose a multi-modal deep learning framework to solve this problem. The contribution of this paper is four-fold. First, our method adds an extra modality by extracting texts automatically from the book covers. Second, image-based and text-based, state-of-the-art models are evaluated thoroughly for the task of book cover classification. Third, we develop an efficient and salable multi-modal framework based on the images and

Deep learning for video game genre classification

“…Their approach consists of three stages: image feature extraction using the Xception model [9], title feature extraction using the GloVe Model [40], and classification based on the combined extracted features. Kundu and Zheng [29] proposed a multimodal deep learning framework based on book covers and text directly extracted from the book covers, which resembles the way people obtain both visual and textual information from the cover. They provided a detailed evaluation of the state-of-the-art models for the task of book genre classification.…”