Regionwise Generative Adversarial Image Inpainting for Large Missing Areas

Ma, Yuqing; Liu, Xianglong; Bai, Shihao; Wang, Lei; Liu, Aishan; Tao, Dacheng; Hancock, Edwin R.

doi:10.1109/tcyb.2022.3194149

Cited by 23 publications

(6 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Images of persons seated behind the table were included in our proprietary dataset. LaMa [105], MADF [48], Regionwise [113], and AOTGAN [114] were chosen for comparison since they were based on a large mask inpainting method, suitable for human-like forms and thus were similar in this regard to our approach. We have also included a state of the art method E2FGVI [115], operating on semantic masks, as it claims the best accuracy of most recent methods.…”

Section: B Experimental Settingsmentioning

confidence: 99%

See 1 more Smart Citation

Pareto Optimized Large Mask Approach for Efficient and Background Humanoid Shape Removal

et al. 2023

View full text Add to dashboard Cite

We introduce deep learning-based methodology for removing unwanted human-like shapes in videos. The method uses Pareto-optimized Generative Adversarial Networks (GANs) technology, which is a novel contribution. The system automatically selects the Region of Interest (ROI) for each humanoid shape and uses a skeleton detection module to determine which humanoid shape to retain. The semantic masks of human like shapes are created using a semantic-aware occlusion-robust model that has four primary components: feature extraction, and local, global, and semantic branches. The global branch encodes occlusion-aware information to make the extracted features resistant to occlusion, while the local branch retrieves fine-grained local characteristics. A modified big mask inpainting approach is employed to eliminate a person from the image, leveraging Fast Fourier convolutions and utilizing polygonal chains and rectangles with unpredictable aspect ratios. The inpainter network takes the input image and the mask to create an output image excluding the background humanoid shapes. The generator uses an encoder-decoder structure with included skip connections to recover spatial information and dilated convolution and squeeze and excitation blocks to make the regions behind the humanoid shapes consistent with their surroundings. The discriminator avoids dissimilar structure at the patch scale, and the refiner network catches features around the boundaries of each background humanoid shape. The method is evaluated on two video object segmentation datasets (DAVIS and YouTube-VOS) and a database of 66 distinct video sequences of people behind a desk in an office environment. The efficiency was assessed using the Structural Similarity Index Measure (SSIM), Frechet Inception Distance (FID), and Learned Perceptual Image Patch Similarity (LPIPS) metrics and showed promising results in fully automated background person removal task.

show abstract

Section: B Experimental Settingsmentioning

confidence: 99%

“…where Q ⊂ R n and F is a vector of the objective functions CP U 4208 1.03 MADF [48] GP U 405 4.03 RegionWise [113] CP U 4586 0.66 RegionWise [113] GP U 312 3.66 AOTGAN [114] CP U 6248 1.25 AOTGAN [114] GP U 784 4.84…”

Section: E Performance Evaluationmentioning

confidence: 99%

Pareto Optimized Large Mask Approach for Efficient and Background Humanoid Shape Removal

et al. 2023

View full text Add to dashboard Cite

show abstract

“…Phutke et al [29] suggested computationally efficient, lightweight networks for image restoration with minimal parameters and without guidance information. Ma et al [30] presented a versatile restoration framework capable of addressing incomplete images exhibiting considerable missing regions, encompassing both continuous and discontinuous areas. Region operations are implemented in the generator and discriminator, catering to distinct region types specifically, existing and missing regions.…”

Section: Advanced Deep Learning Approaches For Image Restoration and ...mentioning

confidence: 99%

DenMerD: A Feature Enhanced Approach to Radar Beam Blockage Correction with Edge-Cloud Computing

Liu,

Sun,

Zhang

et al. 2023

Preprint

View full text Add to dashboard Cite

In the field of meteorological, the global radar network is indispensable for detecting weather phenomena and offering early warning services. Nevertheless, radar data frequently exhibit anomalies, including gaps and clutter, arising from atmospheric refraction, equipment malfunctions, and other factors, resulting in diminished data quality. Traditional radar blockage correction methods, such as employing approximate radial information interpolation and supplementing missing data, often fail to effectively exploit potential patterns in massive radar data, for the large volume of data precludes a thorough analysis and understanding of the inherent complex patterns and dependencies through simple interpolation or supplementation techniques. Fortunately, edge computing possesses certain data processing capabilities and cloud center boasts substantial computational power, which together can collaboratively offer timely computation and storage for the correction of radar beam blockage. To this end, an edge-cloud collaborative driven deep learning model named DenMerD is proposed in this paper, which includes dense connection module and merge distribution (MD) unit. The model improves the key metrics such as CSI and POD by 30.7% and 30.1%, respectively, and performs well in SSIM metrics compared to the existing models. These findings underscore the efficacy of the design in improving feature propagation and beam blockage accuracy, and also highlights the potential and value of mobile edge computing in processing large-scale meteorological data.

show abstract

“…With the advancement of deep learning, object classification and detection techniques have found application in various domains [28,29]. Existing object detection approaches in campus scenarios often involve the direct application of traditional closed-set detection models to the classroom environment.…”

Section: Object Detection In Campus Scenariosmentioning

confidence: 99%

A Parallel Open-World Object Detection Framework with Uncertainty Mitigation for Campus Monitoring

Dong,

Zhang,

et al. 2023

Applied Sciences

Self Cite

View full text Add to dashboard Cite

The recent advancements in artificial intelligence have brought about significant changes in education. In the context of intelligent campus development, target detection technology plays a pivotal role in applications such as campus environment monitoring and the facilitation of classroom behavior surveillance. However, traditional object detection methods face challenges in open and dynamic campus scenarios where unexpected objects and behaviors arise. Open-World Object Detection (OWOD) addresses this issue by enabling detectors to gradually learn and recognize unknown objects. Nevertheless, existing OWOD methods introduce two major uncertainties that limit the detection performance: the unknown discovery uncertainty from the manual generation of pseudo-labels for unknown objects and the known discrimination uncertainty from perturbations that unknown training introduces to the known class features. In this paper, we introduce a Parallel OWOD Framework with Uncertainty Mitigation to alleviate the unknown discovery uncertainty and the known discrimination uncertainty within the OWOD task. To address the unknown discovery uncertainty, we propose an objectness-driven discovery module to focus on capturing the generalized objectness shared among various known classes, driving the framework to discover more potential objects that are distinct from the background, including unknown objects. To mitigate the discrimination uncertainty, we decouple the learning processes for known and unknown classes through a parallel structure to reduce the mutual influence at the feature level and design a collaborative open-world classifier to achieve high-performance collaborative detection of both known and unknown classes. Our framework provides educators with a powerful tool for effective campus monitoring and classroom management. Experimental results on standard benchmarks demonstrate the framework’s superior performance compared to state-of-the-art methods, showcasing its transformative potential in intelligent educational environments.

show abstract

Regionwise Generative Adversarial Image Inpainting for Large Missing Areas

Cited by 23 publications

References 46 publications

Pareto Optimized Large Mask Approach for Efficient and Background Humanoid Shape Removal

Pareto Optimized Large Mask Approach for Efficient and Background Humanoid Shape Removal

DenMerD: A Feature Enhanced Approach to Radar Beam Blockage Correction with Edge-Cloud Computing

A Parallel Open-World Object Detection Framework with Uncertainty Mitigation for Campus Monitoring

Contact Info

Product

Resources

About