A major challenge in the analysis of tissue imaging data is cell segmentation, the task of identifying the precise boundary of every cell in an image. To address this problem we constructed TissueNet, a dataset for training segmentation models that contains more than 1 million manually labeled cells, an order of magnitude more than all previously published segmentation training datasets. We used TissueNet to train Mesmer, a deep learning-enabled segmentation algorithm. We demonstrated that Mesmer is more accurate than previous methods, generalizes to the full diversity of tissue types and imaging platforms in TissueNet, and achieves human-level performance. Mesmer enabled the automated extraction of key cellular features, such as subcellular localization of protein signal, which was challenging with previous approaches. We then adapted Mesmer to harness cell lineage information in highly multiplexed datasets and used this enhanced version to quantify cell morphology changes during human gestation. All code, data, and models are released as a community resource.

SUMMARY Ductal carcinoma in situ (DCIS) is a pre-invasive lesion that is thought to be a precursor to invasive breast cancer (IBC). To understand the changes in the tumor microenvironment (TME) accompanying transition to IBC, we used multiplexed ion beam imaging by time of flight (MIBI-TOF) and a 37-plex antibody staining panel to interrogate 79 clinically annotated surgical resections using machine learning tools for cell segmentation, pixel-based clustering, and object morphometrics. Comparison of normal breast with patient-matched DCIS and IBC revealed coordinated transitions between four TME states that were delineated based on the location and function of myoepithelium, fibroblasts, and immune cells. Surprisingly, myoepithelial disruption was more advanced in DCIS patients that did not develop IBC, suggesting this process could be protective against recurrence. Taken together, this HTAN Breast PreCancer Atlas study offers insight into drivers of IBC relapse and emphasizes the importance of the TME in regulating these processes.

Understanding the spatial organization of tissues is of critical importance for both basic and translational research. While recent advances in tissue imaging are opening an exciting new window into the biology of human tissues, interpreting the data that they create is a significant computational challenge. Cell segmentation, the task of uniquely identifying each cell in an image, remains a substantial barrier for tissue imaging, as existing approaches are inaccurate or require a substantial amount of manual curation to yield useful results. Here, we addressed the problem of cell segmentation in tissue imaging data through large-scale data annotation and deep learning. We constructed TissueNet, an image dataset containing >1 million paired whole-cell and nuclear annotations for tissue images from nine organs and six imaging platforms. We created Mesmer, a deep learning-enabled segmentation algorithm trained on TissueNet that performs nuclear and whole-cell segmentation in tissue imaging data. We demonstrated that Mesmer has better speed and accuracy than previous methods, generalizes to the full diversity of tissue types and imaging platforms in TissueNet, and achieves human-level performance for whole-cell segmentation. Mesmer enabled the automated extraction of key cellular features, such as subcellular localization of protein signal, which was challenging with previous approaches. We further showed that Mesmer could be adapted to harness cell lineage information present in highly multiplexed datasets. We used this enhanced version to quantify cell morphology changes during human gestation. All underlying code and models are released with permissive licenses as a community resource.

While technologies for multiplexed imaging have provided an unprecedented understanding of tissue composition in health and disease, interpreting this data remains a significant computational challenge. To understand the spatial organization of tissue and how it relates to disease processes, imaging studies typically focus on cell-level phenotypes. However, images can capture biologically important objects that are outside of cells, such as the extracellular matrix. Here, we developed a pipeline, Pixie, that achieves robust and quantitative annotation of pixel-level features using unsupervised clustering and show its application across a variety of biological contexts and multiplexed imaging platforms. Furthermore, current cell phenotyping strategies that rely on unsupervised clustering can be labor intensive and require large amounts of manual adjustments. We demonstrate how pixel clusters that lie within cells can be used to improve cell annotations and decrease the amount of manual fine-tuning needed. We comprehensively evaluate pre-processing steps and parameter choices to optimize clustering performance and quantify the reproducibility of our method. Importantly, Pixie is open source and easily customizable through a user-friendly interface.

Background The long-term consequences of COVID-19 remain unclear. There is concern a proportion of patients will progress to develop pulmonary fibrosis. We aimed to assess the temporal change in CXR infiltrates in a cohort of patients following hospitalisation for COVID-19. Methods We conducted a single-centre prospective cohort study of patients admitted to University Hospital Southampton with confirmed SARS-CoV2 infection between 20th March and 3rd June 2020. Patients were approached for standard-of-care follow-up 12-weeks after hospitalisation. Inpatient and follow-up CXRs were scored by the assessing clinician for extent of pulmonary infiltrates; 0–4 per lung (Nil = 0, < 25% = 1, 25–50% = 2, 51–75% = 3, > 75% = 4). Results 101 patients with paired CXRs were included. Demographics: 53% male with a median (IQR) age 53.0 (45–63) years and length of stay 9 (5–17.5) days. The median CXR follow-up interval was 82 (77–86) days with median baseline and follow-up CXR scores of 4.0 (3–5) and 0.0 (0–1) respectively. 32% of patients had persistent CXR abnormality at 12-weeks. In multivariate analysis length of stay (LOS), smoking-status and obesity were identified as independent risk factors for persistent CXR abnormality. Serum LDH was significantly higher at baseline and at follow-up in patients with CXR abnormalities compared to those with resolution. A 5-point composite risk score (1-point each; LOS ≥ 15 days, Level 2/3 admission, LDH > 750 U/L, obesity and smoking-status) strongly predicted risk of persistent radiograph abnormality (0.81). Conclusion Persistent CXR abnormality 12-weeks post COVID-19 was common in this cohort. LOS, obesity, increased serum LDH, and smoking-status were risk factors for radiograph abnormality. These findings require further prospective validation.