The Structure Transfer Machine Theory and Applications

Zhang, Baochang; Yang, Wankou; Wang, Ze; Zhuo, Li’an; Han, Jungong; Zhen, Xiantong

doi:10.1109/tip.2019.2954178

Cited by 6 publications

(3 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Experimental results demonstrate that the proposed RGB-T salient object detection method performs better than the stateof-the-art methods, especially for those challenging scenes with poor illumination, complex background or low contrast. One possible future work is to ap-ply our saliency detector to industrial applications, such as image classification [72], object tracking [73], [74] and instance-level object retrieval [75], [76].…”

Section: Discussionmentioning

confidence: 99%

RGB-T Salient Object Detection via Fusing Multi-Level CNN Features

Zhang

Huang

Yang

et al. 2020

IEEE Trans. on Image Process.

Self Cite

180

View full text Add to dashboard Cite

RGB-induced salient object detection has recently witnessed substantial progress, which is attributed to the superior feature learning capability of deep convolutional neural networks (CNNs). However, such detections suffer from challenging scenarios characterized by cluttered backgrounds, low-light conditions and variations in illumination. Instead of improving RGB based saliency detection, this paper takes advantage of the complementary benefits of RGB and thermal infrared images. Specifically, we propose a novel end-to-end network for multimodal salient object detection, which turns the challenge of RGB-T saliency detection to a CNN feature fusion problem. To this end, a backbone network (e.g., VGG-16) is first adopted to extract the coarse features from each RGB or thermal infrared image individually, and then several adjacent-depth feature combination (ADFC) modules are designed to extract multi-level refined features for each single-modal input image, considering that features captured at different depths differ in semantic information and visual details. Subsequently, a multi-branch group fusion (MGF) module is employed to capture the crossmodal features by fusing those features from ADFC modules for a RGB-T image pair at each level. Finally, a joint attention guided bi-directional message passing (JABMP) module undertakes the task of saliency prediction via integrating the multi-level fused features from MGF modules. Experimental results on several public RGB-T salient object detection datasets demonstrate the superiorities of our proposed algorithm over the state-of-the-art approaches, especially under challenging conditions, such as poor illumination, complex background and low contrast.

show abstract

Section: Discussionmentioning

confidence: 99%

RGB-T Salient Object Detection via Fusing Multi-Level CNN Features

Zhang

Huang

Yang

et al. 2020

IEEE Trans. on Image Process.

Self Cite

180

View full text Add to dashboard Cite

show abstract

“…Luan et al (20) proposed Gabor convolutional networks (GCNs), which utilize Gabor filters as the convolutional filters, such that the robustness of learned features against the orientation and scale changes can be reinforced. Zhang et al (21) developed a new representation learning method, named Structure Transfer Machine (STM), which enables the feature learning process to converge at the representation expectation in a probabilistic way.…”

Section: Related Workmentioning

confidence: 99%

Camera-based discomfort detection using multi-channel attention 3D-CNN for hospitalized infants

Sun

Wang

et al. 2021

Quant Imaging Med Surg

View full text Add to dashboard Cite

Background: Detecting discomfort in infants is an important topic for their well-being and development.In this paper, we present an automatic and continuous video-based system for monitoring and detecting discomfort in infants.Methods: The proposed system employs a novel and efficient 3D convolutional neural network (CNN), which achieves an end-to-end solution without the conventional face detection and tracking steps. In the scheme of this study, we thoroughly investigate the video characteristics (e.g., intensity images and motion images) and CNN architectures (e.g., 2D and 3D) for infant discomfort detection. The realized improvements of the 3D-CNN are based on capturing both the motion and the facial expression information of the infants.Results: The performance of the system is assessed using videos recorded from 24 hospitalized infants by visualizing receiver operating characteristic (ROC) curves and measuring the values of area under the ROC curve (AUC). Additional performance metrics (labeling accuracy) are also calculated. Experimental resultsshow that the proposed system achieves an AUC of 0.99, while the overall labeling accuracy is 0.98. Conclusions:These results confirms the robustness by using the 3D-CNN for infant discomfort monitoring and capturing both motion and facial expressions simultaneously.

show abstract

“…i-vectors), or by feeding the neural network with a fixed length of extracted vectors multiple times until the audio file ends, while computing statistics across the timeline. The mean or the mode is typically used, but it is also possible to have an additional classifier, such as the extreme learning machine (ELM) [23], adopted in this work.…”

Section: Proposed Neural Network Modelsmentioning

confidence: 99%

Non-intrusive Speech Quality Assessment Using Neural Networks

Avila

Gamper

Reddy

et al. 2019

ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Estimating the perceived quality of an audio signal is critical for many multimedia and audio processing systems. Providers strive to offer optimal and reliable services in order to increase the user quality of experience (QoE). In this work, we present an investigation of the applicability of neural networks for non-intrusive audio quality assessment. We propose three neural network-based approaches for mean opinion score (MOS) estimation. We compare our results to three instrumental measures: the perceptual evaluation of speech quality (PESQ), the ITU-T Recommendation P.563, and the speech-to-reverberation energy ratio. Our evaluation uses a speech dataset contaminated with convolutive and additive noise, labeled using a crowd-based QoE evaluation, evaluated with Pearson correlation with MOS labels, and mean-squared-error of the estimated MOS. Our proposed approaches outperform the aforementioned instrumental measures, with a fully connected deep neural network using Mel-frequency features providing the best correlation (0.87) and the lowest mean squared error (0.15).Index Terms-Audio quality assessment, speech quality assessment, deep neural network * Work on this project performed as an intern at Microsoft Research Labs, Redmond, WA.

show abstract

The Structure Transfer Machine Theory and Applications

Abstract: How to cite:Please refer to published version for the most recent bibliographic citation information. If a published version is known of, the repository item page linked to above, will contain details on accessing it.

Cited by 6 publications

References 46 publications

RGB-T Salient Object Detection via Fusing Multi-Level CNN Features

RGB-T Salient Object Detection via Fusing Multi-Level CNN Features

Camera-based discomfort detection using multi-channel attention 3D-CNN for hospitalized infants

Non-intrusive Speech Quality Assessment Using Neural Networks

Contact Info

Product

Resources

About