When studying physical cellular response observed by light microscopy, variations in cell behavior are difficult to quantitatively measure and are often only discussed on a subjective level. Hence, cell properties are described qualitatively based on a researcher’s impressions. In this study, we aim to define a comprehensive approach to estimate the physical cell activity based on migration and morphology based on statistical analysis of a cell population within a predefined field of view and timespan. We present quantitative measurements of the influence of drugs such as cytochalasin D and taxol on human neuroblastoma, SH-SY5Y cell populations. Both chemicals are well known to interact with the cytoskeleton and affect the cell morphology and motility. Being able to compute the physical properties of each cell for a given observation time, requires precise localization of each cell even when in an adhesive state. Also, the risk of confusion through contaminants is desired to be minimized. In relation to the cell detection process, we have developed a customized encoder-decoder based deep learning cell detection and tracking procedure. Further, we discuss the accuracy of our approach to quantify cell activity and its viability in regard to the cell detection accuracy.