Multi-label Patent Classification using Attention-Aware Deep Learning Model

Roudsari, Arousha Haghighian; Afshar, Jafar; Lee, Charles Cheolgi; Lee, Wookey

doi:10.1109/bigcomp48618.2020.000-2

Cited by 16 publications

(8 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Methods for the second task captured temporal dependencies from the patent citation sequences by a point process [5] or an attention mechanism [6]. Another branch aimed to make patents analysis more automatic, such as patent classification [10], [11], patent retrieval [12], and patent text generation [13]. Specifically, these tasks mostly focused on capturing semantic relationships from patent descriptions with text representation learning methods in Natural Language Processing (NLP) field.…”

Section: Related Workmentioning

confidence: 99%

“…Specifically, these tasks mostly focused on capturing semantic relationships from patent descriptions with text representation learning methods in Natural Language Processing (NLP) field. For example, [10] utilized Bert [14] pre-trained model to capture semantic dependencies in patent documents while [11] combined GCN and attention mechanism to embed patent representations for classification.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Event-Based Dynamic Graph Representation Learning for Patent Application Trend Prediction

Zou,

Yu,

Sun

et al. 2024

IEEE Trans. Knowl. Data Eng.

View full text Add to dashboard Cite

Accurate prediction of what types of patents that companies will apply for in the next period of time can figure out their development strategies and help them discover potential partners or competitors in advance. Although important, this problem has been rarely studied in previous research due to the challenges in modeling companies' continuously evolving preferences and capturing the semantic correlations of classification codes. To fill this gap, we propose an event-based dynamic graph learning framework for patent application trend prediction. In particular, our method is founded on the memorable representations of both companies and patent classification codes. When a new patent is observed, the representations of the related companies and classification codes are updated according to the historical memories and the currently encoded messages. Moreover, a hierarchical message passing mechanism is provided to capture the semantic proximities of patent classification codes by updating their representations along the hierarchical taxonomy. Finally, the patent application trend is predicted by aggregating the representations of the target company and classification codes from static, dynamic and hierarchical perspectives. Experiments on real-world data demonstrate the effectiveness of our approach under various experimental conditions, and also reveal the abilities of our method in learning semantics of classification codes and tracking technology developing trajectories of companies.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Event-Based Dynamic Graph Representation Learning for Patent Application Trend Prediction

Zou,

Yu,

Sun

et al. 2024

IEEE Trans. Knowl. Data Eng.

View full text Add to dashboard Cite

show abstract

“…The dynamic word vector pre-trained by ALBERT is used to replace the traditional static word vector, and the BiGRU neural network model is used for training, which preserves the semantic association between long-distance words in the patent text to the maximum extent. Roudsari et al [23]. Studied the effect of applying the DistilBERT pre-training model and finetuned it to complete the important task of multi-label patent classification.…”

Section: )mentioning

confidence: 99%

Patent Text Classification based on Deep Learning and Vocabulary Network

Li¹,

Yu²,

Huang³

et al. 2023

IJACSA

View full text Add to dashboard Cite

Patent documents are a special long text format, and traditional deep learning methods have insufficient feature extraction ability, which results in a weaker classification effect than ordinary text. Based on this, this paper constructs a text feature extraction method based on the lexical network, according to the inner relation between words and classification. Firstly, the inner relationship between words and classification was obtained from linear and probability dimensions and the lexical network were constructed. Secondly, the lexical network is fused with the features extracted from the deep learning model. Finally, the fusion features are trained in the original model to get the final classification result. T This method is a classification enhancement method that can classify patent text alone or enhance the accuracy of various types of neural networks in patent text classification. Experimental results demonstrate that the accuracy of BERT combined with lexical network method is as high as 82.73%, and the accuracy of lexical network method combined with CNN and LSTM is increased by 2.19% and 2.25% respectively. In addition, it was demonstrated that the lexical network feature extraction method accelerated the convergence speed of the model during training and improved the classification ability of the model in Chinese patent texts.

show abstract

“…Following this trend, Lee et al [15] leveraged and fine-tuned the BERT-Base model and applied it to patent classification getting better accuracy than other recent DL approaches. This method was evaluated on a multi-label classification and achieved 54.33 recall at 1 Similarly, Roudsari et al [16] presented a short work applying and fine-tuning Distil-BERT model for patent classification.…”

Section: Prior Workmentioning

confidence: 99%

Automated Single-Label Patent Classification using Ensemble Classifiers

Kamateri

Stamatis

Diamantaras

et al. 2022

2022 14th International Conference on Machine Learning and Computing (ICMLC)

View full text Add to dashboard Cite

Many thousands of patent applications arrive at patent offices around the world every day. One important task when a patent application is submitted is to assign one or more classification codes from the complex and hierarchical patent classification schemes that will enable routing of the patent application to a patent examiner who is knowledgeable about the specific technical field. This task is typically undertaken by patent professionals, however due to the large number of applications and the potential complexity of an invention, they are usually overwhelmed. Therefore, there is a need for this code assignment manual task to be supported or even fully automated by classification systems that will classify patent applications, hopefully with an accuracy close to patent professionals. Like in many other text analysis problems, in the last years, this intellectually demanding task has been studied using word embeddings and deep learning techniques. In this paper these research efforts are shortly reviewed and re-produced with similar deep learning techniques using different feature representations on automatic patent classification in the level of sub-classes. On top of that, an innovative method of ensemble classifiers trained with different parts of the patent document is proposed. To the best of our knowledge, this is the first time that an ensemble method was proposed for the patent classification problem. Our first results are quite promising showing that an ensemble architecture of classifiers significantly outperforms current state-of-the-art techniques using the same classifiers as standalone solutions.

show abstract

Multi-label Patent Classification using Attention-Aware Deep Learning Model

Cited by 16 publications

References 7 publications

Event-Based Dynamic Graph Representation Learning for Patent Application Trend Prediction

Event-Based Dynamic Graph Representation Learning for Patent Application Trend Prediction

Patent Text Classification based on Deep Learning and Vocabulary Network

Automated Single-Label Patent Classification using Ensemble Classifiers

Contact Info

Product

Resources

About