In recent years, most Japanese universities have started to operate departments of institutional research (IR). In IR fields, Key Performance Indicator (KPI) is important to evaluate the quality of education. Eduinformatics, a novel interdisciplinary field proposed by us, provides good analyses that will help us to solve educational problems. This paper emphasizes that we have to consider whether the evaluation of KPIs is the best method to check the quality of education in universities. We discuss the problem of evaluating KPIs to check the quality of higher education and suggest that we have to recognize and understand the limitations of evaluating KPIs. We propose that it is better to evaluate the combination of both KPIs and signs of changes in students to improve the quality of education.