When individuals (observers) assess how appealing they find sexual stimuli (targets), which factors matter and to whom? The present study examined how observer and target characteristics interact and impact perceived sexual appeal. Participants were 302 men (206 heterosexual, 96 gay) and 289 women (196 heterosexual, 93 lesbian) between the ages of 18 and 67 years, who viewed 34 photographs of targets of their preferred gender and rated each target for sexual appeal, masculinity-femininity, and estimated age. Participants also rated their own masculinity-femininity. A baseline model indicated that roughly 30 % of the variance in sexual appeal ratings was at the observer level (between observers) and 70 % of the variance was at the target level (within observers). In the final model, five characteristics of the participant observers (gender, sexual orientation, age, race/ethnicity, and self-described masculinity-femininity) and six characteristics of the target photographs (gender, whether the photographs were taken from heterosexual versus gay/lesbian media, race/ethnicity, perceived masculinity-femininity, and estimated age) were independently and interactively related to observer ratings of target sexual appeal. Observers displayed preferences for similar targets in terms of race/ethnicity and masculinity-femininity, while also displaying a general preference for target youth. Variation in the strength of these preferences occurred according to observers' own gender, race/ethnicity, masculinity-femininity, and sexual orientation.