Single-cell methods are beginning to reveal the intrinsic heterogeneity in cell populations, which arises from the interplay or deterministic and stochastic processes. For example, the molecular mechanisms of cell cycle control are well characterised, yet the observed distribution of cell cycle durations in a population of cells is heterogenous. This variability may be governed either by stochastic processes, inherited in a deterministic fashion, or some combination of both. Previous studies have shown poor correlations within lineages when observing direct ancestral relationships but remain correlated with immediate relatives. However, assessing longer-range dependencies amid noisy data requires significantly more observations, and demands the development of automated procedures for lineage tree reconstruction. Here, we developed an open-source Python library, btrack, to facilitate retrieval of deep lineage information from live-cell imaging data. We acquired 3,500 hours of time-lapse microscopy data of epithelial cells in culture and used our software to extract 22,519 fully annotated single-cell trajectories. Benchmarking tests, including lineage tree reconstruction assessments, demonstrate that our approach yields high-fidelity results and achieves state-of-the-art performance without the requirement for manual curation of the tracker output data. To demonstrate the robustness of our supervision-free cell tracking pipeline, we retrieve cell cycle durations and their extended inter- and intra-generational family relationships, for up to eight generations, and up to fourth cousin relationships. The extracted lineage tree dataset represents approximately two orders of magnitude more data, and longer-range dependencies, than in previous studies of cell cycle heritability. Our results extend the range of observed correlations and suggest that strong heritable cell cycling is present. We envisage that our approach could be extended with additional live-cell reporters to provide a detailed quantitative characterisation of biochemical and mechanical origins to cycling heterogeneity in cell populations.