AdaTag: Multi-Attribute Value Extraction from Product Profiles with Adaptive Decoding

Abstract

Automatic extraction of product attribute values is an important enabling technology in e-Commerce platforms. This task is usually modeled using sequence labeling architectures, with several extensions to handle multi-attribute extraction. One line of previous work constructs attribute-specific models, through separate decoders or entirely separate models. However, this approach constrains knowledge sharing across different attributes. Other contributions use a single multiattribute model, with different techniques to embed attribute information. But sharing the entire network parameters across all attributes can limit the model's capacity to capture attribute-specific characteristics. In this paper we present AdaTag, which uses adaptive decoding to handle extraction. We parameterize the decoder with pretrained attribute embeddings, through a hypernetwork and a Mixture-of-Experts (MoE) module. This allows for separate, but semantically correlated, decoders to be generated on the fly for different attributes. This approach facilitates knowledge sharing, while maintaining the specificity of each attribute. Our experiments on a realworld e-Commerce dataset show marked improvements over previous methods. * Most of the work was done during an internship at Amazon.

WebFormer: The Web-page Transformer for Structure Information Extraction

“…Aghajanyan et al [3] train a hyper-text language model based on BART [24] on a largescale web crawl for various downstream tasks. More recently, several attribute extraction approaches [47,49,53] have been proposed, which treat each field as an attribute of interest and extract its corresponding value from clean object context such as web title. Chen et al [9] formulate the web information extraction problem as structural reading comprehension and build a BERT [15] based model to extract structured fields from the web documents.…”

“…Most previous sequence modeling approaches [2,53] only encode the text sequence of the web document without utilizing the HTML layout structure. In this work, we jointly model the text sequence with the HTML layout in a unified Transformer model.…”

OA-Mine: Open-World Attribute Mining for E-Commerce Products with Weak Supervision

“…With a limited budget, the development set would allow us to evaluate the performance of different models on 100 product types. Note that for most prior works on product attribute mining [14,18,19], the authors use the same method for gold-standard evaluation. While in this paper, the development set serves the purpose of relative performance comparison.…”

“…Inspired by named entity recognition models, earlier work leverage statistical models [9] for extraction. With the advancement of deep learning, the most prominent systems designed in recent years adopt BiLSTM-CRF [19,22] or BERT-BiLSTM-CRF [18] architectures for attribute value extraction. Supervised method combined with active learning [22] was explored in OpenTag [22], while follow up works typically settle on distant supervision [14,18,19].…”

“…The model is trained to find more values for the given attributes, but does not expand to unseen attributes without additional training data. Moreover, they usually rely on extensive human labeling [22] or noisy distantly supervised data [14,18,19].…”