₁ Sparsity-Regularized Attention Multiple-Instance Network for Hyperspectral Target Detection<i />

Jiao, Changzhe; Chen, Chao; Gou, Shuiping; Wang, Xiuxiu; Yang, Bo; Chen, Xiaoying; Jiao, Licheng

doi:10.1109/tcyb.2021.3087662

Cited by 17 publications
(8 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To solve this problem, we use attention thresholding after Equation (4). [30] A a,g ¼ & A a,g if A a > γ 0 else , g ∈ 1, : : : , G; a ∈ 1, : : : , A…”

Section: Attentional Filtering Blockmentioning
confidence: 99%

“…To solve this problem, we use attention thresholding after Equation (). [ 30 ] $A_{a , g} = \left{\right. \begin{matrix}A_{a , g} i f A_{a} &#x00026;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;gt; \gamma \\ 0 e l s e\end{matrix} g \in 1 , \hdots , G , a \in 1 , \hdots , A$ where γ is the threshold value, and g is the index of packets.…”

Section: Proposed Deep Learning‐based Ble 51 Aoa Ipsmentioning
confidence: 99%

See 1 more Smart Citation
Deep Learning‐Based Bluetooth Low‐Energy 5.1 Multianchor Indoor Positioning with Attentional Data Filtering
Lyu,
Chan,
Hung
et al. 2023
Advanced Intelligent Systems
2
0
0
0
View full text Add to dashboard Cite
Indoor positioning system (IPS) technologies have widespread applications in logistics, intelligent manufacturing, healthcare monitoring, etc. The recently released Bluetooth low‐energy (BLE) 5.1 specification enables in‐phase and quadrature‐phase (I/Q) data measurements. It allows angle of arrival estimation and becomes a natural choice for IPS implementation. Conventional BLE 5.1 IPSs use multiple anchors to provide massive redundancy to improve system robustness. It however demands effective approaches to leverage redundancy. Besides, interference due to various environmental factors can introduce severe errors to I/Q data and affect positioning accuracy. Facing these challenges, herein, a novel deep learning‐based multianchor BLE 5.1 IPS is proposed. The system aggregates measurements from multiple anchors and makes them available at regular time steps. Then, a novel attentional filtering network tailored to infer high‐quality I/Q sample data is developed and a spatial regularization loss incorporating spatial location relationships to strengthen the feature embedding discrimination is proposed. Two multianchor BLE 5.1 I/Q sample datasets are developed and released for public download. Numerical experiments are carried out to compare the proposed method with previous BLE 5.1 IPS methods and methods utilizing other radio frequency data. Results indicate that the proposed method consistently achieves submeter accuracy and significantly outperforms the state‐of‐the‐art approaches.
show abstract

“…To solve this problem, we use attention thresholding after Equation (4). [30] A a,g ¼ & A a,g if A a > γ 0 else , g ∈ 1, : : : , G; a ∈ 1, : : : , A…”

Section: Attentional Filtering Blockmentioning
confidence: 99%

Section: Proposed Deep Learning‐based Ble 51 Aoa Ipsmentioning
confidence: 99%

Deep Learning‐Based Bluetooth Low‐Energy 5.1 Multianchor Indoor Positioning with Attentional Data Filtering
Lyu,
Chan,
Hung
et al. 2023
Advanced Intelligent Systems
2
0
0
0
View full text Add to dashboard Cite

Indoor positioning system (IPS) technologies have widespread applications in logistics, intelligent manufacturing, healthcare monitoring, etc. The recently released Bluetooth low‐energy (BLE) 5.1 specification enables in‐phase and quadrature‐phase (I/Q) data measurements. It allows angle of arrival estimation and becomes a natural choice for IPS implementation. Conventional BLE 5.1 IPSs use multiple anchors to provide massive redundancy to improve system robustness. It however demands effective approaches to leverage redundancy. Besides, interference due to various environmental factors can introduce severe errors to I/Q data and affect positioning accuracy. Facing these challenges, herein, a novel deep learning‐based multianchor BLE 5.1 IPS is proposed. The system aggregates measurements from multiple anchors and makes them available at regular time steps. Then, a novel attentional filtering network tailored to infer high‐quality I/Q sample data is developed and a spatial regularization loss incorporating spatial location relationships to strengthen the feature embedding discrimination is proposed. Two multianchor BLE 5.1 I/Q sample datasets are developed and released for public download. Numerical experiments are carried out to compare the proposed method with previous BLE 5.1 IPS methods and methods utilizing other radio frequency data. Results indicate that the proposed method consistently achieves submeter accuracy and significantly outperforms the state‐of‐the‐art approaches.

show abstract

“…However, the localness limits the ability to model long-term and non-consecutive dependencies. Subsequently, non-localized sparsity mechanisms were proposed, including DropAttention [17], Cluster-Former [18] and L 1 sparsity-regularized attention (L1-attention) [19], [20]. DropAttention randomly sets attention weights to zero, interpreted as dropping a set of neurons along different dimensions.…”

Section: Introductionmentioning
confidence: 99%

“…The second type (Cluster-Former Layer) encodes global information beyond the initial chunked sequences [18]. The L 1 sparsityregularized attention introduced a L 1 sparse prior, which minimizes contributions of the irrelevant connections in the feature learning process [19]. These methods encourage the model to make decisions relying on the full context of the input sequences rather than a few pieces of input.…”

Section: Introductionmentioning
confidence: 99%

BaSFormer: A Balanced Sparsity Regularized Attention Network for Transformer
Jiang,
Chen,
Xiang
et al. 2024
IEEE/ACM Trans. Audio Speech Lang. Process.
0
0
0
0
View full text Add to dashboard Cite

Attention networks often make decisions relying solely on a few pieces of tokens, even if those reliances are not truly indicative of the underlying meaning or intention of the full context. That can lead to over-fitting in Transformers and hinder their ability to generalize. Attention regularization and sparsity-based methods have been used to overcome this issue. However, these methods cannot guarantee that all tokens have sufficient receptive fields for global information inference. Thus, the impact of individual biases cannot be effectively reduced. As a result, the generalization of these approaches improved slightly from the training data to new data. To address these limitations, we proposed a balanced sparsity (BaS) regularized attention network on top of the Transformers, called BaSFormer. BaS regularization introduces the K-regular graph constraint on self-attention connections, which replaces SoftMax with SparseMax in the attention transformation. In BaS-regularized self-attentions, SparseMax assigns zero attention scores to low-scoring connections, highlighting influential and meaningful contexts. The K-regular graph constraint ensures that all tokens have an equal-sized receptive field to aggregate information, which facilitates the involvement of global tokens in the feature update of each layer and reduces the impact of individual biases. As no continuous loss can be used as the K-regular graph regularization, we proposed an exponential extremum loss with augmented Lagrangian. Experimental results show that BaSFormer improves debiasing effectiveness compared to the newest large language models, such as the chatGPT, GPT-4 and LLaMA. In addition, BaSFormer achieves new state-of-the-art results in text generation tasks. Interestingly, this paper also evaluates that BaSFormer can learn hierarchically linguistic dependencies in gradient attributions, which improves interpretability and adversarial robustness. Our implementation is anonymously available at Google Drive.

show abstract

“…In summary, to overcome the obstacles in table tennis ball detection tasks, this study designed a dedicated detection network, which was able to reduce the use of network layers by efficiently reusing the feature information to achieve the lightweight requirements. The implementation of the network mainly relies on the feature reuse module, which stores the feature information extracted in the previous iteration and passes it to the corresponding network layer in the next iteration, so that the feature information can be fully extracted with only a few network layers [ 13 , 14 , 15 ]. At the same time, the Transformer module was added to the network to utilize its excellent capability of global feature information extraction, while combining with the local feature extraction capability of the convolutional network, in order to improve the networks’ capacity of precepting small targets [ 16 , 17 , 18 , 19 ].…”

Section: Introductionmentioning
confidence: 99%

Table Tennis Track Detection Based on Temporal Feature Multiplexing Network
Li
¹
,
Liu
²
,
An
³

et al. 2023
Sensors
9
0
1
0
View full text Add to dashboard Cite

Recording the trajectory of table tennis balls in real-time enables the analysis of the opponent’s attacking characteristics and weaknesses. The current analysis of the ball paths mainly relied on human viewing, which lacked certain theoretical data support. In order to solve the problem of the lack of objective data analysis in the research of table tennis competition, a target detection algorithm-based table tennis trajectory extraction network was proposed to record the trajectory of the table tennis movement in video. The network improved the feature reuse rate in order to achieve a lightweight network and enhance the detection accuracy. The core of the network was the “feature store & return” module, which could store the output of the current network layer and pass the features to the input of the network layer at the next moment to achieve efficient reuse of the features. In this module, the Transformer model was used to secondarily process the features, build the global association information, and enhance the feature richness of the feature map. According to the designed experiments, the detection accuracy of the network was 96.8% for table tennis and 89.1% for target localization. Moreover, the parameter size of the model was only 7.68 MB, and the detection frame rate could reach 634.19 FPS using the hardware for the tests. In summary, the network designed in this paper has the characteristics of both lightweight and high precision in table tennis detection, and the performance of the proposed model significantly outperforms that of the existing models.

show abstract

₁ Sparsity-Regularized Attention Multiple-Instance Network for Hyperspectral Target Detection

Cited by 17 publications

References 52 publications

Deep Learning‐Based Bluetooth Low‐Energy 5.1 Multianchor Indoor Positioning with Attentional Data Filtering

Deep Learning‐Based Bluetooth Low‐Energy 5.1 Multianchor Indoor Positioning with Attentional Data Filtering

BaSFormer: A Balanced Sparsity Regularized Attention Network for Transformer

Table Tennis Track Detection Based on Temporal Feature Multiplexing Network

Contact Info

Product

Resources

About