The recent outbreak of the COVID-19 pandemic has shown the importance of medical testing for an early detection of regional disease hot spots. The PAMONO sensor (Plasmon-Assisted Microscopy of Nano-Objects) can detect virus-like particles. It utilizes the surface plasmon resonance of particles attaching to an antibody coated gold plate to generate a stream of noisy images containing characteristic spatiotemporal signals. These signals are then analyzed via a deep neural network based architecture which targets a parameter-free usage on-site. It is able to detect individual (virus) particles within ten seconds and to reach a count exactness of higher than 80% in a minute.

Deep learning methods have become increasingly popular for optical sensor image analysis. They are adaptable to specific tasks and simultaneously demonstrate a high degree of generalization capability. However, applying deep neural networks to problems with low availability of labeled training data can lead to a model being incapable of generalizing to possible scenarios that may occur in test data, especially with the occurrence of dominant imaging artifacts. We propose a data-centric augmentation approach based on generative adversarial networks that overlays the existing labeled data with synthetic artifacts that are generated from data not present in the training set. This augmentation leads to a more robust generalization capability in semantic segmentation. Our method does not need any additional labeling and does not lead to additional memory or time consumption during inference. Further, we find it to be more effective than comparable approaches that are based on procedurally generated disturbances and the direct use of real disturbances. Building upon the improved segmentation results, we observe that our approach leads to improvements of 22% in the F1-score for an evaluated detection problem, which promises significant robustness towards future disturbances. In the context of sensor-based data analysis, the compensation of image artifacts is a challenge. When the structures of interest are not clearly visible in an image, algorithms that can cope with artifacts are crucial for obtaining the desired information. Thereby, the high variation of artifacts, the combination of different types of artifacts, and their similarity to signals of interest are specific issues that have to be considered in the analysis. Despite the high generalization capability of deep learning-based approaches, their recent success was driven by the availability of large amounts of labeled data. Therefore, the provision of comprehensive labeled image data with different characteristics of image artifacts is of importance. At the same time, applying deep neural networks to problems with low availability of labeled data remains a challenge. This work presents a data-centric augmentation approach based on generative adversarial networks that augments the existing labeled data with synthetic artifacts generated from data not present in the training set. In our experiments, this augmentation leads to a more robust generalization in segmentation. Our method does not need additional labeling and does not lead to additional memory or time consumption during inference. Further, we find it to be more effective than comparable augmentations based on procedurally generated artifacts and the direct use of real artifacts. Building upon the improved segmentation results, we observe that our approach leads to improvements of 22% in the F1-score for an evaluated detection problem. Having achieved these results with an example sensor, we expect increased robustness against artifacts in future applications.

No abstract

A wide-field surface plasmon resonance (SPR) microscopy sensor employs the surface plasmon resonance phenomenon to detect individual biological and non-biological nanoparticles. This sensor enables the detection, sizing, and quantification of biological nanoparticles (bioNPs), such as extracellular vesicles (EVs), viruses, and virus-like particles. The selectivity of bioNP detection does not require biological particle labeling, and it is achieved via the functionalization of the gold sensor surface by target-bioNP-specific antibodies. In the current work, we demonstrate the ability of SPR microscopy sensors to detect, simultaneously, silica NPs that differ by four times in size. Employed silica particles are close in their refractive index to bioNPs. The literature reports the ability of SPR microscopy sensors to detect the binding of lymphocytes (around 10 μm objects) to the sensor surface. Taken together, our findings and the results reported in the literature indicate the power of SPR microscopy sensors to detect bioNPs that differ by at least two orders in size. Modifications of the optical sensor scheme, such as mounting a concave lens, help to achieve homogeneous illumination of a gold sensor chip surface. In the current work, we also characterize the improved magnification factor of the modified SPR instrument. We evaluate the effectiveness of the modified and the primary version of the SPR microscopy sensors in detecting EVs isolated via different approaches. In addition, we demonstrate the possibility of employing translation and rotation stepper motors for precise adjustments of the positions of sensor optical elements—prism and objective—in the primary version of the SPR microscopy sensor instrument, and we present an algorithm to establish effective sensor–actuator coupling.