BackgroundOver 500,000 women worldwide are diagnosed with ovarian or endometrial cancer each year. We have used a two-step strategy to identify plasma proteins that could be used to improve the diagnosis of women with an indication of gynecologic tumor and in population screening.MethodsIn the discovery step we screened 441 proteins in plasma using the proximity extension assay (PEA) and five Olink Multiplex assays (CVD II, CVD III, INF I, ONC II, NEU I) in women with ovarian cancer (n = 106), endometrial cancer (n = 74), benign ovarian tumors (n = 150) and healthy population controls (n = 399). Based on the discovery analyses a set of 27 proteins were selected and two focused multiplex PEA assays were developed. In a replication step the focused assays were used to study an independent set of cases with ovarian cancer (n = 280), endometrial cancer (n = 228), women with benign ovarian tumors (n = 76) and healthy controls (n = 57).ResultsIn the discovery step, 27 proteins that showed an association to cancer status were identified. In the replication analyses, the focused assays distinguished benign tumors from ovarian cancer stage III–IV with a sensitivity of 0.88 and specificity of 0.92 (AUC = 0.92). The assays had a significantly higher AUC for distinguishing benign tumors from late stage ovarian cancer than using CA125 and HE4 (p = 9.56e−22). Also, population controls could be distinguished from ovarian cancer stage III–IV with a sensitivity of 0.85 and a specificity of 0.92 (AUC = 0.89).ConclusionThe PEA assays represent useful tools for identification of new biomarkers for gynecologic cancers. The selected protein assays could be used to distinguish benign tumors from ovarian and endometrial cancer in women diagnosed with an unknown suspicious pelvic mass. The panels could also be used in population screening, for identification of women in need of specialized gynecologic transvaginal ultrasound examination.FundingThe Swedish Cancer Foundation, Vinnova (SWELIFE), The Foundation for Strategic Research (SSF), Assar Gabrielsson Foundation.Electronic supplementary materialThe online version of this article (10.1186/s12014-018-9216-y) contains supplementary material, which is available to authorized users.