Germline mutations in the PTEN gene, which cause Cowden syndrome (CS), are known to be one of the genetic factors for primary thyroid and breast cancers, however, PTEN mutations are found in only a small subset of research participants with non-syndrome breast and thyroid cancers. In this study, we aimed to identify germline variants that may be related to genetic risk of primary thyroid and breast cancers. Genomic DNAs extracted from peripheral blood of 14 PTEN-wild-type female research participants with primary thyroid and breast cancers were analyzed by whole-exome sequencing. Gene-based case control association analysis using the information of 406 Europeans obtained from the 1000 Genomes Project database identified 34 genes possibly associated with the phenotype with P<1.0×10−3. Among them, rare variants in the PARP4 gene were detected at significant high frequency (odds ratio = 5.2, P = 1.0×10−5). The variants, G496V and T1170I, were found in 6 of the 14 study participants (43%) while their frequencies were only 0.5% in controls. Functional analysis using HCC1143 cell line showed that knockdown of PARP4 with siRNA significantly enhanced the cell proliferation, compared with the cells transfected with siControl (P = 0.02). Kaplan-Meier analysis using GEO, EGA and TCGA datasets showed poor progression-free survival (P = 0.006, Hazard ratio 0.71) and overall survival (P < 0.0001, Hazard ratio 0.79) in a PARP4 low-expression group, suggesting that PARP4 may function as a tumor suppression. In conclusion, we identified PARP4 as a possible susceptibility gene of primary thyroid and breast cancer.