With massive explosion of social media such as Twitter and Instagram, people daily share billions of multimedia posts, containing images and text. Typically, text in these posts is short, informal and noisy, leading to ambiguities which can be resolved using images. In this paper we explore text-centric Named Entity Recognition task on these multimedia posts. We propose an end to end model which learns a joint representation of a text and an image. Our model extends multi-dimensional self attention technique, where now image help to enhance relationship between words. Experiments show that our model is capable of capturing both textual and visual contexts with greater accuracy, achieving state-of-the-art results on Twitter multimodal Named Entity Recognition dataset.• Unrelated image : Text information do not match with an image, as we can see in Fig. 8(a), "Reddit" belongs to

0000−0002−4355−0366] , Ignazio Gallo 1[0000−0002−7076−8328] , Alessandro Calefati 1[0000−0003−3860−4785] , and Dimitri Ognibene 2[0000−0002−9454−680X]Abstract. One-class classifiers are trained only with target class samples. Intuitively, their conservative modeling of the class description may benefit classical classification tasks where classes are difficult to separate due to overlapping and data imbalance. In this work, three methods leveraging on the combination of one-class classifiers based on nonparametric models, N-ary Trees and Minimum Spanning Trees class descriptors (MST CD) are proposed. These methods deal with inconsistencies arising from combining multiple classifiers and with spurious connections that MST-CD creates in multimodal class distributions. Experiments on several datasets show that the proposed approach obtains comparable and, in some cases, state-of-theart results.

No abstract

No abstract

No abstract