Robust deep learning optical autofocus system applied to automated multiwell plate single molecule localization microscopy

Abstract

We presenta robust, long‐range optical autofocus system for microscopy utilizing machine learning. This can be useful for experiments with long image data acquisition times that may be impacted by defocusing resulting from drift of components, for example due to changes in temperature or mechanical drift. It is also useful for automated slide scanning or multiwell plate imaging where the sample(s) to be imaged may not be in the same horizontal plane throughout the image data acquisition. To address the impact of (thermal or mechanical) fluctuations over time in the optical autofocus system itself, we utilize a convolutional neural network (CNN) that is trained over multiple days to account for such fluctuations. To address the trade‐off between axial precision and range of the autofocus, we implement orthogonal optical readouts with separate CNN training data, thereby achieving an accuracy well within the 600 nm depth of field of our 1.3 numerical aperture objective lens over a defocus range of up to approximately +/–100 μm. We characterize the performance of this autofocus system and demonstrate its application to automated multiwell plate single molecule localization microscopy.

Robust, single-shot, optical autofocus system utilizing cylindrical lenses to provide high precision and long range of operation

“…This approach also provides single-shot autofocusing capability by determining the sign of defocus within the range of the shorter autofocus confocal parameter (±37.5 µm), thereby enabling closed loop "real-time" focus lock. This new approach does not require machine learning but could still benefit from the approach outlined in reference [2], which may extend the operating range as the machine learning can work with lower signal to noise ratio.…”

“…Figure 1 shows the first configuration of this new optical autofocus system 9 , which was implemented on an openFrame 10 microscope configured for easySTORM 11 using a 100x objective lens of 1.4 NA (Olympus UplanSApo 100x 1.40 oil). It is similar to our previously published configuration 2 except that the autofocus laser beam is being collimated after emerging from a single mode optical fibre by a pair of cylindrical lenses of different focal length orientated at 90° to each other to produce the desired elliptical collimated beam (instead of using a spherical collimating lens and a slit 2 ). By using cylindrical lenses with different focal lengths, we can adjust the autofocus confocal parameter in orthogonal directions and thus the precision and operating range of the autofocus.…”

“…Because this unwanted background contribution to the autofocus camera image varies with defocus, it cannot be easily subtracted. To mitigate this, we replaced the single mode diode laser used in our previous autofocus system 2 with a superluminescent diode (SLD, Superlum S-930 nm) that provides an autofocus laser beam that is spatially coherent but has a short (∼7 µm) temporal coherence length. Thus, only background signals that are back reflected from components within half the coherence length of the coverslip will produce an interference pattern at the autofocus camera and any back reflections from optical components in the autofocus system or from the other side of the coverslip will only contribute to a fixed background, which can be subtracted from the autofocus camera images acquired.…”

“…Normally this leads to a compromise between precision and range of the optical autofocus system. In a recent paper 2 , however, we relaxed this compromise by using a slit to shape the autofocus laser beam such that it exhibited a different confocal parameter in the planes parallel and perpendicular to the slit. This enabled us to realise an accuracy better than the depth of field of the objective lens (∼600 nm) over an operating range of ∼+/-100 µm.…”

“…The range of this autofocus approach typically scales with the confocal parameter of the objective lens and rarely exceeds 10 µm. This can be a problem when moving between FOV in multiwell plates 2 or slide-scanning.…”