Motivation Single-cell time-lapse microscopy is a ubiquitous tool for studying the dynamics of complex cellular processes. While imaging can be automated to generate very large volumes of data, the processing of the resulting movies to extract high-quality single-cell information remains a challenging task. The development of software tools that automatically identify and track cells is essential for realizing the full potential of time-lapse microscopy data. Convolutional neural networks (CNNs) are ideally suited for such applications, but require great amounts of manually annotated data for training, a time-consuming and tedious process. Results We developed a new approach to CNN training for yeast cell segmentation based on synthetic data, and present i) a software tool for the generation of synthetic images mimicking brightfield images of budding yeast cells and ii) a convolutional neural network (Mask-RCNN) for yeast segmentation that was trained on a fully synthetic dataset. The Mask-RCNN performed excellently on segmenting actual microscopy images of budding yeast cells, and a density-based clustering algorithm (DBSCAN) was able to track the detected cells across the frames of microscopy movies. Our synthetic data creation tool completely bypassed the laborious generation of manually annotated training datasets, and can be easily adjusted to produce images with many different features. The incorporation of synthetic data creation into the development pipeline of CNN-based tools for budding yeast microscopy is a critical step towards the generation of more powerful, widely applicable and user-friendly image processing tools for this microorganism. Availability The synthetic data generation code can be found at https://github.com/prhbrt/synthetic-yeast-cells. The Mask R-CNN, as well as the tuning and benchmarking scripts can be found at https://github.com/ymzayek/yeastcells-detection-maskrcnn We also provide Google Colab scripts that reproduce all the results of this work. Supplementary information Supplementary material is available at Bioinformatics online.