Convolutional analysis operator learning for multifocus image fusion

Zhang, Chengfang; Feng, Ziliang

doi:10.1016/j.image.2022.116632

Cited by 7 publications

(6 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Objects at a specific distance from the camera will be clearly captured, while other objects tend to be blurred. Multi-focus image fusion is one of the approaches to solve this problem, and it can extract the focused areas from multiple source images of the same scene to form a clear all-in-focus image [7][8][9].…”

Section: Introductionmentioning

confidence: 99%

“…Thus, the pixel-level image fusion method has been most widely researched [10][11][12][13]. The pixel-level image fusion method of a scene can be divided into three categories: multi-focus image fusion methods in spatial domain, transform domain and deep learning [1,8,9].…”

Section: Introductionmentioning

confidence: 99%

“…The development of multi-focus image fusion based on deep learning is a transition process from supervised learning to unsupervised learning [8,9,34,36]. Compared with traditional methods, CNN, as a key branch of deep learning, is more capable of feature extraction [8,9]. Fusion methods based on CNN are mainly divided into two categories: end-to-end methods and non-end-to-end methods.…”

Section: Introductionmentioning

confidence: 99%

“…With the rise of computer hardware and theory, deep learning has achieved unprecedented success in many fields. Thus, the multi‐focus image fusion using deep learning has also become a hot topic [8–10, 12, 13]. The development of multi‐focus image fusion based on deep learning is a transition process from supervised learning to unsupervised learning [8, 9, 34, 36].…”

Section: Introductionmentioning

confidence: 99%

“…Thus, the multi‐focus image fusion using deep learning has also become a hot topic [8–10, 12, 13]. The development of multi‐focus image fusion based on deep learning is a transition process from supervised learning to unsupervised learning [8, 9, 34, 36]. Compared with traditional methods, CNN, as a key branch of deep learning, is more capable of feature extraction [8, 9].…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

An unsupervised multi‐focus image fusion method based on Transformer and U‐Net

Jin

Zhou

et al. 2022

IET Image Processing

View full text Add to dashboard Cite

This work presents a multi‐focus image fusion method based on Transformer and U‐Net with an unsupervised training fashion. In this work, the authors introduce Transformer into image fusion because it has great ability to capture the global dependencies and low‐frequency features. In image processing, convolutional neural network (CNN) has good performance of detailed feature extraction but a weakness for global feature extraction, and Transformer has limited power in local or detailed information extraction but a strong capacity for global feature extraction. Thus, this work combines the advantages of CNN and Transformer to propose an unsupervised decision map making model for image fusion joint U‐Net. The authors construct a model including feature extraction and feature reconstruction modules which correspond to the encoder and decoder network of U‐Net, respectively. In addition, perceptual loss is introduced on the basis of structural similarity loss because the combination of these two loss functions can achieve better performance with lower training cost. Experiments show that the proposed image fusion method performs better fusion performance compared with the existing methods.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

An unsupervised multi‐focus image fusion method based on Transformer and U‐Net

Jin

Zhou

et al. 2022

IET Image Processing

View full text Add to dashboard Cite

show abstract

PCGC: a performance compact graph compiler based on multilevel fusion-splitting rules

et al. 2023

View full text Add to dashboard Cite

The existing deep learning compilers are unable to perform e cient hardware performance-related graph fusion when both time and power consumption are considered. The operator optimization cost is too high because of excessive fusion or skipping fusion. In addition, the compilers optimize the computational graph of Deep Neural Networks (DNN) by performing static graph transformation based on the greedy algorithm, only considering the runtime performance and ignoring the cost of the tuning process. To solve these problems, this paper proposes PCGC, a DNN computational graph optimization compiler. Through the performance feedback at runtime, PCGC designs a computational graph fusion and splitting optimization strategy based on multilevel operator layer fusion-splitting rules. First, PCGC uses a rule-guided graph segmentation algorithm to recursively segment the computational graph into smaller subgraph to achieve an e cient and detailed search. Then, PCGC uses the cost model to receive the feedback of hardware performance information, uses the cost model and operator fusion rules to guide the partial fusion and split of the nodes and edges of the computational graph, and exibly generates the optimal subgraph according to different hardware. Finally, in the fusion process, the operator computing attributes are considered. The graph-level node and operator-level cyclic fusion are closely combined to optimize the search space of partial fusion. Compared with other advanced accelerators, PCGC optimizes the overall power consumption on an embedded GPU by an average of 130.5% when the time consumption on each hardware is not lower than the average time consumption.On Domain Speci c Architecture (DSA), PCGC optimizes power consumption by an average of 66.5%. On FPGA, PCGC optimizes power consumption by 66.1%. In a sense, PCGC can achieve high-speed inference in speci c power supply scenarios, reducing the carbon emissions of edge computing.

show abstract

Standard simplex induced clustering with hierarchical deep dictionary learning

Rodríguez-Domínguez

Dalmau

2023

Signal Processing: Image Communication

View full text Add to dashboard Cite

Convolutional analysis operator learning for multifocus image fusion

Cited by 7 publications

References 33 publications

An unsupervised multi‐focus image fusion method based on Transformer and U‐Net

An unsupervised multi‐focus image fusion method based on Transformer and U‐Net

PCGC: a performance compact graph compiler based on multilevel fusion-splitting rules

Standard simplex induced clustering with hierarchical deep dictionary learning

Contact Info

Product

Resources

About