Immune checkpoint inhibitors (ICIs), such as anti-programmed cell death 1 (PD1) antibodies (Abs) and anti-cytotoxic T-lymphocyte associated protein 4 (CTLA4) Abs, have been widely administered for not only advanced melanoma, but also various non-melanoma skin cancers. Since profiles of tumor-infiltrating leukocytes (TILs) play important roles in immunotherapy using ICIs, it is important to evaluate cancer stromal cells such as tumor-associated macrophages (TAMs) and cancer-associated fibroblasts (CAFs), as well as stromal extracellular matrix protein, to predict the efficacy of ICIs. This review article focuses particularly on TAMs and related factors. Among TILs, TAMs and their related factors could be the optimal biomarkers for immunotherapy such as anti-PD1 Ab therapy. According to the studies presented, TAM-targeting therapies for advanced melanoma and non-melanoma skin cancer will develop in the future.