Cancers are the main cause of death worldwide. Assessment of tumors by imaging is key to formulating treatment plans. More noninvasive markers of imaging are needed to evaluate primary tumors and treatment responses to achieve individualized diagnosis and treatment. However, conventional image assessment has limitations based on differences in individual radiologists’ interpretation and inability to read hidden high-dimensional features. Artificial intelligence (AI) can provide quantitative, rather than qualitative, assessments based on radiographic characteristics, and has been frequently and successfully applied in the medical image analysis field. In this review, we summarize research progress in the use of AI for evaluating the diagnosis and treatment response of patients with common tumors, including gliomas, lung cancer, liver cancer and colorectal cancer. We further discuss several issues that must be solved in the future to make better use of AI in tumor diagnosis and assessment of treatment response.