Although several prognostic factors in nivolumab therapy have been reported in recurrent or metastatic head and neck cancer (RM-HNC) patients, these factors remain controversial. Here, we conducted a multicenter retrospective cohort study to investigate the impact of clinico-hematological factors on survival in RM-HNC patients treated with nivolumab. We reviewed 126 RM-HNC patients from seven institutes. We evaluated the prognostic effects of clinico-hematological factors on survival. The median overall survival (OS) was 12.3 months, and the 1 year-OS rate was 51.2%. Patients without immune-related adverse events, lower relative eosinophil count, worse best overall response, higher performance status, and higher modified Glasgow Prognostic Score had worse survival. The score, generated by combining these factors, was associated with survival. Patients with score of 4–5 had worse survival than those with score of 2–3 and 0–1 [adjusted HR for PFS: score of 4–5, 7.77 (3.98–15.15); score of 2–3, 3.44 (1.95–6.06), compared to score of 0–1], [adjusted HR for OS: score of 4–5, 14.66 (4.28–50.22); score of 2–3, 7.63 (2.29–25.37), compared to score of 0–1]. Our novel prognostic score utilizing clinico-hematological factors might be useful to establish an individual treatment strategy in RM-HNC patients treated with nivolumab therapy.