Paying Deep Attention to Both Neighbors and Multiple Tasks

Liang, Guoxi; Mo, Haoran; Qiao, Ying; Wang, Chuxin; Wang, Jingyan

doi:10.1007/978-3-030-60799-9_12

Cited by 2 publications

(2 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the ground distance is a critical component of the OT, and the quality of the ground distance directly affects the quality of OT-based distance methods. In this paper, we proposed a novel ground distance metric learning method, which employs the cross-attention mechanism [27][28][29][30][31]. To calculate the ground distance between two items from two sequences, respectively, we firstly represent each item by paying attention from itself to the neighbors of the other item and then paying attention back.…”

Section: Existing Workmentioning

confidence: 99%

The Novel Sequence Distance Measuring Algorithm Based on Optimal Transport and Cross‐Attention Mechanism

Lai

Yan³

et al. 2021

Shock and Vibration

View full text Add to dashboard Cite

In this paper, we propose a novel sequence distance measuring algorithm based on optimal transport (OT) and cross-attention mechanism. Given a source sequence and a target sequence, we first calculate the ground distance between each pair of source and target terms of the two sequences. The ground distance is calculated over the subsequences around the two terms. We firstly pay attention from each the source terms to each target terms with attention weights, so that we have a representative source subsequence vector regarding each term in the target subsequence. Then, we pay attention from each representative vector of the term of the target subsequence to the entire source subsequence. In this way, we construct the cross-attention weights and use them to calculate the pairwise ground distances. With the ground distances, we derive the OT distance between the two sequences and train the attention parameters and ground distance metric parameters together. The training process is conducted with training triplets of sequences, where each triplet is composed of an anchor sequence, a must-link sequence, and a cannot-link sequence. The corresponding hinge loss function of each triplet is minimized, and we develop an iterative algorithm to solve the optimal transport problem and the attention/ground distance metric parameters in an alternate way. The experiments over sequence similarity search benchmark datasets, including text, video, and rice smut protein sequence data, are conducted. The experimental results show the algorithm is effective.

show abstract

Section: Existing Workmentioning

confidence: 99%

The Novel Sequence Distance Measuring Algorithm Based on Optimal Transport and Cross‐Attention Mechanism

Lai

Yan³

et al. 2021

Shock and Vibration

View full text Add to dashboard Cite

show abstract

“…Table 1 summarizes the advantages and disadvantages of machine learning models in ITR prediction when compared to traditional AMM, DMM, and SMAMM models, and MD simulations. In general, machine learning models are more reliable when trained on large datasets [18,19]. However, the available ITR dataset from the literature is relatively small (692 instances) [7,8].…”

Section: Introductionmentioning

confidence: 99%

Predicting Interfacial Thermal Resistance by Ensemble Learning

Chen

Tian

et al. 2021

Computation

View full text Add to dashboard Cite

Interfacial thermal resistance (ITR) plays a critical role in the thermal properties of a variety of material systems. Accurate and reliable ITR prediction is vital in the structure design and thermal management of nanodevices, aircraft, buildings, etc. However, because ITR is affected by dozens of factors, traditional models have difficulty predicting it. To address this high-dimensional problem, we employ machine learning and deep learning algorithms in this work. First, exploratory data analysis and data visualization were performed on the raw data to obtain a comprehensive picture of the objects. Second, XGBoost was chosen to demonstrate the significance of various descriptors in ITR prediction. Following that, the top 20 descriptors with the highest importance scores were chosen except for fdensity, fmass, and smass, to build concise models based on XGBoost, Kernel Ridge Regression, and deep neural network algorithms. Finally, ensemble learning was used to combine all three models and predict high melting points, high ITR material systems for spacecraft, automotive, building insulation, etc. The predicted ITR of the Pb/diamond high melting point material system was consistent with the experimental value reported in the literature, while the other predicted material systems provide valuable guidelines for experimentalists and engineers searching for high melting point, high ITR material systems.

show abstract

Paying Deep Attention to Both Neighbors and Multiple Tasks

Cited by 2 publications

References 24 publications

The Novel Sequence Distance Measuring Algorithm Based on Optimal Transport and Cross‐Attention Mechanism

The Novel Sequence Distance Measuring Algorithm Based on Optimal Transport and Cross‐Attention Mechanism

Predicting Interfacial Thermal Resistance by Ensemble Learning

Contact Info

Product

Resources

About