1Advances in microscopy, microfluidics and optogenetics enable single cell monitoring and 2 environmental regulation and offer the means to control cellular phenotypes. The development 3 of such systems is challenging and often results in bespoke setups that hinder reproducibility. To 4 address this, we introduce Cheetah -a flexible computational toolkit that simplifies the integration 5 of real-time microscopy analysis with algorithms for cellular control. Central to the platform is an 6 image segmentation system based on the versatile U-Net convolutional neural network. This is 7 supplemented with functionality to robustly count, characterise and control cells over time. We 8 demonstrate Cheetah's core capabilities by analysing long-term bacterial and mammalian cell 9 growth and by dynamically controlling protein expression in mammalian cells. In all cases, 10 Cheetah's segmentation accuracy exceeds that of a commonly used thresholding-based method, 11 allowing for more accurate control signals to be generated. Availability of this easy-to-use 12 platform will make control engineering techniques more accessible and offer new ways to probe 13 and manipulate living cells.
14
Introduction
15Modern automated microscopy techniques enable researchers to collect vast amounts of single-16 cell imaging data at high temporal resolutions. This has resulted in time-lapse microscopy 17 becoming the go to method for studying cellular dynamics, enabling the quantification of 18 processes such as stochastic fluctuations during gene expression 1-3 , emerging oscillatory 19 patterns in protein concentrations 4 , lineage selection 5,6 , and many more 7 .
20To make sense of microscopy images, segmentation is performed whereby an image is 21 broken up into regions corresponding to specific features of interest (e.g. cells and the 22 background). Image segmentation allows for the accurate quantification of cellular phenotypes 23 encoded by visual cues (e.g. fluorescence) by ensuring only those pixels corresponding to a cell 24 are considered. A range of segmentation algorithms have been proposed to automatically 25 analyse images of various organisms and tissues 3,8-11 . The most common of these are 26 thresholding 12 and seeded watershed 13 methods, which are available in many scientific image 27 processing toolkits. Commercial software packages also implement this type of functionality, 28 enabling both automated image acquisition and analysis (e.g. NIS-Elements, Nikon). While these 29 proprietary systems are user-friendly requiring no programming skills to be used, they are often 30 difficult to tailor for specific needs and cannot be easily extended to new forms of analysis.
31More recently, deep learning-based approaches to image segmentation have emerged 32 7,14-17 . Compared to the more common thresholding-based approaches 12 , deep learning methods 33 tend to require more significant computational resources when running on traditional computer 34 architectures and often require the time-consuming manual step of generating...