The importance of T cells in immunotherapy has motivated developing technologies to better characterize T cells and improve therapeutic efficacy. One specific objective is assessing antigen-induced T cell activation because only functionally active T cells are capable of killing the desired targets. Autofluorescence imaging can distinguish T cell activity states of individual cells in a non-destructive manner by detecting endogenous changes in metabolic co-enzymes such as NAD(P)H. However, recognizing robust patterns of T cell activity is computationally challenging in the absence of exogenous labels or information-rich autofluorescence lifetime measurements. We demonstrate that advanced machine learning can accurately classify T cell activity from NAD(P)H intensity images and that those image-based signatures transfer across human donors. Using a dataset of 8,260 cropped single-cell images from six donors, we meticulously evaluate multiple machine learning models. These range from traditional models that represent images using summary statistics or extract image features with CellProfiler to deep convolutional neural networks (CNNs) pre-trained on general non-biological images. Adapting pre-trained CNNs for the T cell activity classification task provides substantially better performance than traditional models or a simple CNN trained with the autofluorescence images alone. Visualizing the images with dimension reduction provides intuition into why the CNNs achieve higher accuracy than other approaches. However, we observe that fine-tuning all layers of the pre-trained CNN does not provide a classification performance boost commensurate with the additional computational cost. Our software detailing our image processing and model training pipeline is available as Jupyter notebooks at https: //github.com/gitter-lab/t-cell-classification. lifetime imaging to identify macrophages within the tumor microenvironment in vivo 7 and classified activation state of T cells in vitro 6 . The fluorescence lifetime of NAD(P)H and FAD is highly sensitive to the microenvironment and binding of NAD(P)H and FAD. Thus, this fluorescence lifetime can be used to resolve metabolic differences between functional states of immune cells. However, fluorescence lifetime imaging requires specialized and expensive microscope components, limiting its use to particular labs. Although autofluorescence intensity images lack the depth of information provided by the fluorescence lifetime, intensity images can easily and quickly be acquired on almost any commercial fluorescence microscope, allowing widespread adoption and seamless integration of a new technique into existing protocols of live cell assessment. Here, we develop a computational framework that uses autofluorescence intensity images to assess T cell activation state at the single-cell level.Machine learning is promising for classifying cell subtypes from label-free images. For example, Pavillon et al. used regularized logistic regression to predict macrophage activation state 8 , and Yoo...