Autonomous vehicles (AVs) have moved from hype to reality as the penetration and acceptance rate continues to increase. As they are slowly integrated into traffic with human-driven vehicles (HDVs), it is necessary to predict the car-following behaviors of AVs and HDVs for better control of AV–HDV mixed traffic. This study extends a data-driven car-following model to incorporate drivers’ memory, and cooperation with the lead vehicle. The model predicts the following vehicle’s speed in AV–HDV mixed traffic. The effect of drivers’ cooperation on car-following behavior was modeled using prospect theory (PT), whereas the driver’s memory was incorporated using the memory cell of a long short-term memory (LSTM) neural network. This extended car-following model is called the “PT-LSTM model.” Real-world vehicle trajectories of HDVs and AVs in the Waymo AV Open Dataset were used to calibrate and validate the PT-LSTM model. The PT-LSTM model demonstrated higher accuracy compared with the LSTM model that did not consider drivers’ cooperation, the multiple layer perceptron model, Gipps’ model, and the intelligent driver model that incorporated PT. The importance of variables in different time steps in the PT-LSTM model was also evaluated using SHapley Additive exPlanations (SHAP). The SHAP results showed that AV followers were more likely to cooperate with the lead HDV, whereas HDV followers were more likely to cooperate with the lead AV than the lead HDV. Thus, this study underscores the importance of considering drivers’ memory and cooperation with the lead vehicle for the prediction of car-following behaviors in AV–HDV mixed traffic.