Sexual stimuli processing is a key element in the repertoire of human affective and motivational states. Previous neuroimaging studies of sexual stimulus processing have revealed a complicated mosaic of activated regions, leaving unresolved questions about their sensitivity and specificity to sexual stimuli per se, generalizability across individuals, and potential utility as neuromarkers for sexual stimulus processing. In this study, data on sexual, negative, non-sexual positive, and neutral images from Wehrum et al. (2013) (N = 100) were re-analyzed with multivariate Support Vector Machine models to create the Brain Activation-based Sexual Image Classifier (BASIC) model. This model was tested for sensitivity, specificity, and generalizability in cross-validation (N = 100) and an independent test cohort (N = 18; Kragel et al. 2019). The BASIC model showed highly accurate performance (94-100%) in classifying sexual versus neutral or nonsexual affective images in both datasets. Virtual lesions and test of individual large-scale networks (e.g., ‘visual’ or ‘attention’ networks) show that these individual networks are neither necessary nor sufficient to capture sexual stimulus processing. These findings suggest that brain responses to sexual stimuli constitute a category of mental event that is distinct from general affect and involves multiple brain networks. It is, however, largely conserved across individuals, permitting the development of neuromarkers for sexual processing in individual persons. Future studies could assess performance of BASIC to a broader array of affective/motivational stimuli and link brain responses with physiological and subjective measures of sexual arousal.