Human pluripotent stem cells (hPSCs) attract tremendous attention due to their unique properties. Manual extraction of trajectories of cell colonies in experimental image time series is labor intensive and subjective, thus the aim of the work was to develop a computer semi-automated protocol for colony tracking. The developed procedure consists of three major stages, namely, image registration, object detection and tracking. Registration using discrete Fourier transform and tracking based on the solution of a linear assignment problem was implemented as console programs in the Python 3 programming language using a variety of packages. Object detection was implemented as a multistep procedure in the ProStack in-house software package. The procedure consists of more than 40 elementary operations that include setting of several biologically relevant parameters, image segmentation and performing of quantitative measurements. The developed procedure was applied to the dataset containing bright-field images from time-lapse recording of the human embryonic cell line H9. The detection step took about 6 h for one image time series with a resolution of 2560 by 2160; about 1 min was required for image registration and trajectories extraction. The developed procedure was effective in detecting and analyzing the time series of images with “good” and “bad” phenotypes. The differences between phenotypes in the distance in pixels between the starting and finishing positions of trajectories, in the path length along the trajectory, and the mean instant speed and mean instant angle of the trajectories were identified as statistically significant by Mann–Whitney and Student’s tests. The measured area and perimeter of the detected colonies differed, on average, for different phenotypes throughout the entire time period under consideration. This result confirms previous findings obtained by analyzing static images.