Automatic detection, localization and segmentation of nano-particles with deep learning in microscopy images

Oktay, Ayşe Betül; Gurses, Anil

doi:10.1016/j.micron.2019.02.009

Cited by 71 publications

(26 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The main complication of this approach is the mandatory selection of empirical parameters, which leads to a loss of the universality of the approach. In 2019, simultaneously our first work [ 23 ], our colleagues applied the MO-CNN neural network [ 24 ] and the Mask-RCNN [ 25 ] to find the localization of nanoparticles on TEM-images, the size of round particles was finally determined by fitting by circles. Meanwhile, TEM-images analyzed in the cited papers are characterized by uniform and homogenous noise, the particles are clearly visualized and have a rounded shape.…”

Section: Introductionmentioning

confidence: 99%

Nanoparticle Recognition on Scanning Probe Microscopy Images Using Computer Vision and Deep Learning

et al. 2020

View full text Add to dashboard Cite

Identifying, counting and measuring particles is an important component of many research studies. Images with particles are usually processed by hand using a software ruler. Automated processing, based on conventional image processing methods (edge detection, segmentation, etc.) are not universal, can only be used on good-quality images and need to set a number of parameters empirically. In this paper, we present results from the application of deep learning to automated recognition of metal nanoparticles deposited on highly oriented pyrolytic graphite on images obtained by scanning tunneling microscopy (STM). We used the Cascade Mask-RCNN neural network. Training was performed on a dataset containing 23 STM images with 5157 nanoparticles. Three images containing 695 nanoparticles were used for verification. As a result, the trained neural network recognized nanoparticles in the verification set with 0.93 precision and 0.78 recall. Predicted contour refining with 2D Gaussian function was a proposed option. The accuracies for mean particle size calculated from predicted contours compared with ground truth were in the range of 0.87–0.99. The results were compared with outcomes from other generally available software, based on conventional image processing methods. The advantages of deep learning methods for automatic particle recognition were clearly demonstrated. We developed a free open-access web service “ParticlesNN” based on the trained neural network, which can be used by any researcher in the world.

show abstract

Section: Introductionmentioning

confidence: 99%

Nanoparticle Recognition on Scanning Probe Microscopy Images Using Computer Vision and Deep Learning

et al. 2020

View full text Add to dashboard Cite

show abstract

“…[ 24 ] Applications of deep learning methodologies in nanoinformatics are very rare. Güven and Oktay applied CNN to distinguish Fe 3 O 4 ENMs from background [ 25 ] and in a follow‐up study, Oktay and Gurses [ 26 ] applied multiple output CNNs (MO‐CNN) to detect the locations of Fe 3 O 4 ENMs in electronic images, to provide their boundaries, and to define their size and shape based on the segmentation output.…”

Section: Introductionmentioning

confidence: 99%

Development of Deep Learning Models for Predicting the Effects of Exposure to Engineered Nanomaterials on Daphnia magna

Karatzas

Melagraki

Ellis

et al. 2020

Small

View full text Add to dashboard Cite

This study presents the results of applying deep learning methodologies within the ecotoxicology field, with the objective of training predictive models that can support hazard assessment and eventually the design of safer engineered nanomaterials (ENMs). A workflow applying two different deep learning architectures on microscopic images of Daphnia magna is proposed that can automatically detect possible malformations, such as effects on the length of the tail, and the overall size, and uncommon lipid concentrations and lipid deposit shapes, which are due to direct or parental exposure to ENMs. Next, classification models assign specific objects (heart, abdomen/claw) to classes that depend on lipid densities and compare the results with controls. The models are statistically validated in terms of their prediction accuracy on external D. magna images and illustrate that deep learning technologies can be useful in the nanoinformatics field, because they can automate time‐consuming manual procedures, accelerate the investigation of adverse effects of ENMs, and facilitate the process of designing safer nanostructures. It may even be possible in the future to predict impacts on subsequent generations from images of parental exposure, reducing the time and cost involved in long‐term reproductive toxicity assays over multiple generations.

show abstract

“…In contrast with traditional object detection methods, which have found broad application in bioimage analysis for spotting intracellular particles [16] , [18] , [210] , [211] , cell nuclei [17] , [26] , and cellular events such as mitosis [212] , [213] , [214] , deep learning approaches for these tasks have been explored since only recently. First results are promising [196] , [215] , [216] , [217] , [218] ( Fig. 4 B) but more extensive evaluations are needed to assess their general superiority.…”

Section: Deep Learning For Bioimage Analysismentioning

confidence: 99%

A bird’s-eye view of deep learning in bioimage analysis

Meijering

2020

Computational and Structural Biotechnology Journal

115

View full text Add to dashboard Cite

Deep learning of artificial neural networks has become the de facto standard approach to solving data analysis problems in virtually all fields of science and engineering. Also in biology and medicine, deep learning technologies are fundamentally transforming how we acquire, process, analyze, and interpret data, with potentially far-reaching consequences for healthcare. In this mini-review, we take a bird’s-eye view at the past, present, and future developments of deep learning, starting from science at large, to biomedical imaging, and bioimage analysis in particular.

show abstract

Automatic detection, localization and segmentation of nano-particles with deep learning in microscopy images

Cited by 71 publications

References 15 publications

Nanoparticle Recognition on Scanning Probe Microscopy Images Using Computer Vision and Deep Learning

Nanoparticle Recognition on Scanning Probe Microscopy Images Using Computer Vision and Deep Learning

Development of Deep Learning Models for Predicting the Effects of Exposure to Engineered Nanomaterials on Daphnia magna

A bird’s-eye view of deep learning in bioimage analysis

Contact Info

Product

Resources

About