Despite excellent average-case performance of many image classifiers, their performance can substantially deteriorate on semantically coherent subgroups of the data that were under-represented in the training data. These systematic errors can impact both fairness for demographic minority groups as well as robustness and safety under domain shift. A major challenge is to identify such subgroups with subpar performance when the subgroups are not annotated and their occurrence is very rare. We leverage recent advances in text-to-image models and search in the space of textual descriptions of subgroups ("prompts") for subgroups where the target model has low performance on the prompt-conditioned synthesized data. To tackle the exponentially growing number of subgroups, we employ combinatorial testing. We denote this procedure as PROMPTAT-TACK as it can be interpreted as an adversarial attack in a prompt space. We study subgroup coverage and identifiability with PROMPTATTACK in a controlled setting and find that it identifies systematic errors with high accuracy. Thereupon, we apply PROMPTATTACK to ImageNet classifiers and identify novel systematic errors on rare subgroups.