BackgroundIn a considerable number of patients with a suspected hereditary tumor syndrome (HTS), no underlying germline mutation is detected in the most likely affected genes. The present study aimed to establish and validate a large gene panel for HTS, and determine its diagnostic yield and clinical utility.MethodsThe study cohort comprised 173 patients with suspected, but unexplained, HTS (group U) and 64 HTS patients with a broad spectrum of known germline mutations (group K). All patients in group U presented with early age at onset, multiple tumors, and/or a familial clustering of various tumor types; no germline mutation had been identified during routine diagnostics. Sequencing of leukocyte DNA was performed for the 94 HTS genes of the Illumina TruSight™Cancer Panel and 54 additional HTS genes.ResultsThe sensitivity of the panel to identify known germline variants was 99.6%. In addition to known mutations, a total of 192 rare, potentially pathogenic germline variants in 86 genes were identified. Neither the proportion of rare variants per patient (group K: 0.9 variants; group U: 0.8 variants) nor the proportion of variants in the most frequently mutated, moderately penetrant genes CHEK2 and ATM showed significant inter-group difference. Four of the five patients from group U with a truncating CHEK2 mutation had thyroid cancer, pointing to a broader tumor spectrum in patients with pathogenic CHEK2 variants. In 22% of patients from group K, a further potential causative variant was identified. Here, the most interesting finding was an NF1 nonsense mutation in a child with a known TP53 frameshift mutation. In 17% of patients from group U, potential causative variants were identified. In three of these patients (2%), mutations in PMS2, PTEN, or POLD1 were considered to be causative. In both groups, incidental findings with presumptive predictive value were generated.ConclusionsThe gene panel identified the genetic cause in some prescreened, unexplained HTS patients and generated incidental findings. Some patients harbored predicted pathogenic mutations in more than one established HTS gene, rendering interpretation of the respective alterations challenging. Established moderate risk genes showed an almost equal distribution among patients with known and unexplained disease.Electronic supplementary materialThe online version of this article (10.1186/s13053-018-0102-4) contains supplementary material, which is available to authorized users.