Permeability glycoprotein (P-gp) is involved in the pathology of various diseases including cancer and epilepsy, mainly through the translocation of some medicines across the cell membrane. Here, we employed image-based quantitative structure-activity relationship (QSAR) models to predict the P-gp inhibitory activity of some Tariquidar derivatives. The structures of 65 Tariquidar derivatives and their P-gp inhibition activities were collected from the literature. For each compound, the pixels of bidimensional images and their principal components (PCs) were calculated using MATLAB software. Various statistical methods including principal component regression, artificial neural networks, and support vector machines were employed to investigate the correlation between the PCs and the activity of the compounds. The predictability of the models was investigated using external validation and applicability domain analysis. An artificial neural network-based model demonstrated the best prediction results for the test set. Moreover, external validation analysis of the developed models supports the idea that R(2) cannot assure the validity of QSAR models and another criterion, i.e., the concordance correlation coefficient (CCC) parameter, should be involved to evaluate the validity of the QSAR models. The results of this study indicate that image analysis could be as suitable as descriptors calculated by commercial software to predict the activity of drug-like molecules.