Machine learning has gained considerable attention in the material science domain and helped discover advanced materials for electrochemical applications. Numerous studies have demonstrated its potential to reduce the resources required for material screening. However, a significant proportion of these studies have adopted a supervised learning approach, which entails the laborious task of constructing random training databases and does not always ensure the model‘s reliability while screening unseen materials. Herein, we evaluate the limitations of supervised machine learning from the perspective of the applicability domain. The applicability domain of a model is the region in chemical space where the structure‐property relationship is covered by the training set so that the model can give reliable predictions. We review methods that have been developed to overcome such limitations, such as the active learning framework and self‐supervised learning.