Soft tissue and bone sarcomas are a very heterogeneous group of tumors with many subtypes for which diagnosis and treatment remains a very challenging task. On top of that, the treatment choices are limited, and the prognosis of aggressive sarcomas remains poor. Immune checkpoint inhibitors (ICIs) have drawn a lot of attention last years because of their promising response rates and their durable effects. ICIs are currently widely used in the daily routine practice for the treatment of a different malignancies, such as melanoma, Hodgkin lymphoma, and non-small cell lung carcinoma. Still, ICIs are not included in the standard treatment protocols of the different sarcoma types. However, a plethora of clinical trials investigates the clinical benefit of ICIs in sarcomas. There is clear need to develop predictive biomarkers to determine which sarcoma patients are most likely to benefit from immune checkpoint blockade. This review will focus on (i) the clinical trial results on the use of ICIs in different sarcoma types; and on (ii) possible biomarkers predictive for the effectiveness of these drugs in sarcomas.