The use of smartphone‐based analysis systems has been increasing over the past few decades. Among the important reasons for its popularity are its ubiquity, increasing computing power, relatively low cost, and capability to acquire and process data simultaneously in a point‐of‐need fashion. Furthermore, smartphones are equipped with various sensors, especially a complementary metal–oxide–semiconductor (CMOS) sensor. The high sensitivity of the CMOS sensor allows smartphones to be used as a colorimeter, fluorimeter, and spectrometer, constituting the essential part of point‐of‐care testing contributing to E‐health and beyond. However, despite its myriads of merits, smartphone‐based diagnostic devices still face many challenges, including high susceptibility to illumination conditions, difficulty in adapter uniformization, low interphone repeatability, and et al. These problems may hinder smartphone‐enabled diagnosis from passing the FDA regulations of medical devices. This review discusses the design and application of current smartphone‐based diagnostic devices and highlights challenges associated with existent methods and perspectives on how to deal with those challenges from engineering aspects on constant color signal acquisition, including smartphone adapter design, color space transformation, machine learning classification, and color correction.