Introduction: Public willingness to collect personal family health history (FHH) assessments is integral to implement population screening to identify those at high cancer risk who could benefit most from lifesaving interventions. Yet, surprisingly little consideration has been given to factors associated with the public’s perceived importance of FHH in the context of cancer. Methods: Using data from the 2013 Health Information National Trends survey, we assessed the association of intrapersonal (e.g., cancer worry), sociodemographic (e.g., education), and interpersonal-level factors (e.g., family trust) associated with not perceiving FHH assessment to be very important for personal health. Associations were tested with bivariate analyses and hierarchical logistic regression. Results: Of the 3,007 respondents, 32.7% reported perceiving FHH as not very important to their health. Whites (p < 0.001), males (p = 0.003), and those born in the United States (p = 0.004) were most likely to perceive FHH as not very important. Those who were least worried about cancer and perceived that cancer risk could not be lowered also viewed FHH as not very important (p = 0.002, p = 0.018, respectively). In hierarchical regression analyses, the association of low cancer worry remained significant after accounting for sociodemographic and interpersonal factors. The addition of sociodemographic factors modestly improved the model; the addition of interpersonal factors did not improve the model. Conclusions: A sizable proportion of the public does not perceive FHH to be very important, may be hard to reach, and impede implementation of population screening guidelines for inherited cancers. Campaigns to increase the perceived value of FHH assessment may need to be tailored to demographic subgroups, emphasize cancer prevention, and encourage family communication.