Masked Neural Style Transfer using Convolutional Neural Networks

Handa, Arushi; Garg, Prerna; Khare, Vijay

doi:10.1109/icrieece44171.2018.9008937

Cited by 10 publications

(5 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Results indicate that their method can generate plausible results for multiple objects within an image as well as for multiple artistic styles. Hand et al [43] proposed a methodology for local style transfer using masks. These masks were generated either with a histogram or based on the neighborhood of each pixel.…”

Section: Related Workmentioning

confidence: 99%

SAMStyler: Enhancing Visual Creativity With Neural Style Transfer and Segment Anything Model (SAM)

Psychogyios,

Leligou,

Melissari

et al. 2023

IEEE Access

View full text Add to dashboard Cite

Neural Style Transfer (NST) is a popular technique of computer vision where the content of an image is blended with the style of another, which results in a fused image with certain properties of both original images. This approach has practical applications in various domains and has garnered significant attention in both industry and academia. An interesting application of this technique is segmented style transfer where a segmentation algorithm is used to locate objects within an image and then the style transfer method is performed locally, producing images with different styles for different objects. This approach opens up possibilities for creating visually striking compositions by seamlessly blending various artistic styles onto specific objects within an image, allowing for a new level of creative expression. This paper proposes a novel method that combines Segment Anything Model (SAM), a state-of-the-art vision transformer-based image segmentation model developed by Facebook, with style transfer. Our approach includes performing localized style transfer in selected segmentation regions of an image using classical style transfer algorithms. To ensure smooth transitions between the stylized and non-stylized border we also develop our loss function with a border smoothing technique. Experimental results demonstrate the robustness and effectiveness of the proposed methodology, including the ability to infuse multiple artistic styles into different objects within an image. The contributions of this work include integrating SAM with style transfer, proposing a novel loss function, evaluating the segmented style transfer in multiple content regions, comparing with state-of-the-art approaches, and experimenting with multiple style images for diverse stylization. Our primary focus centers on creating a model that serves as a digital painter across a wide range of image genres and artistic styles.

show abstract

Section: Related Workmentioning

confidence: 99%

SAMStyler: Enhancing Visual Creativity With Neural Style Transfer and Segment Anything Model (SAM)

Psychogyios,

Leligou,

Melissari

et al. 2023

IEEE Access

View full text Add to dashboard Cite

show abstract

Section: Non-subsampled Shearlet Transformmentioning

confidence: 99%

“…The pooling layer, which is sandwiched between subsequent convolutional layers, is mostly in charge of compressing images. Overfitting is avoided while the feature dimension is decreased by the pooling layer[33,35].The final layer of the CNN is the fully connected output layer, which maps the features extracted from the network's end to the output category tags.Meanwhile, the Dual Branch CNN architecture has recently received significant attention for its effectiveness in feature extraction and classification tasks. It consists of two branches, one for processing spatial information and the other for processing channel information.…”

mentioning

confidence: 99%

Detection of Brain Tumor Based on Multimodality Brain Image Fusion Using Dual Branch Convolution Neural Network

Khare¹,

Kumari²

2023

ISI

View full text Add to dashboard Cite

Computed tomography scan (CT-scan) images show structural features of brain, while magnetic resonance imaging (MRI) images show brain tissue anatomy but do not comprise any functional information. Now it become a research challenge that how we successfully combine the images of the two modes. In this paper, CT-scan and MRI images are used for detection of brain tumor. The acquired images were pre-processed with the help of median filter and the mathematical morphological operations. These pre-processed images were registered. After registration CT-scan and MRI images fusion has been done using nonsubsampled shearlet transform (NSST) and dual branch convolutional neural networks (CNNs). This method successfully retains the functional information of the CT-scan image and brain structure information and spatial distortion of the MRI image loss will reduce. Brain tumor detection is performed using cuckoo search algorithm with different fitness functions.

show abstract

“…These approaches can also be used for sketch-photo recognition problems. Face to localize facial images using both facial sketches [8], [64] or semantic label masks [65], [66]. The semantic [44], [46], [68], [69] was to balance the targeted and generated images, making the generated image look more realistic and natural.…”

Section: Related Workmentioning

confidence: 99%

Face Recognition via Multi-Level 3D-GAN Colorization

et al. 2022

View full text Add to dashboard Cite

Rapid development in sketch-to-image translation methods boosts the investigation procedure in law enforcement agencies. But, the large modality gap between manually generated sketches makes this task challenging.Generative adversarial network (GAN) and encoder-decoder approach are usually incorporated to accomplish sketchto-image generation with promising results. This paper targets the sketch-to-image translation with heterogeneous face angles and lighting effects using a multi-level conditional generative adversarial network. The proposed multilevel cGAN work in four different phases. Three independent cGANs' networks are incorporated separately into each stage, followed by a CNN classifier. The Adam stochastic gradient descent mechanism was used for training with a learning rate of 0.0002 and momentum estimates β1 and β2 as 0.5 and 0.999, respectively. The multi-level 3Dconvolutional architecture help to preserve spatial facial attributes and pixel-level details. The 3D convolution and deconvolution guide the G1, G2 and G3 to use additional features and attributes for encoding and decoding. This helps to preserve the direction, postures of targeted image attributes and special relationships among the whole image's features. The proposed framework process the 3D-Convolution and 3D-Deconvolution using vectorization. This process takes the same time as 2D convolution but extracts more features and facial attributes. We used pre-trained ResNet-50, ResNet-101, and Mobile-Net to classify generated high-resolution images from sketches. We have also developed, and state-of-the-art Pakistani Politicians Face-sketch Dataset (PPFD) for experimental purposes. Result reveals that the proposed cGAN model's framework outperforms with respect to Accuracy, Structural similarity index measure (SSIM), Signal to noise ratio (SNR), and Peak signal-to-noise ratio (PSNR).

show abstract

Masked Neural Style Transfer using Convolutional Neural Networks

Cited by 10 publications

References 5 publications

SAMStyler: Enhancing Visual Creativity With Neural Style Transfer and Segment Anything Model (SAM)

SAMStyler: Enhancing Visual Creativity With Neural Style Transfer and Segment Anything Model (SAM)

Detection of Brain Tumor Based on Multimodality Brain Image Fusion Using Dual Branch Convolution Neural Network

Face Recognition via Multi-Level 3D-GAN Colorization

Contact Info

Product

Resources

About