Fluorescence immunochromatographic assay (FICA) is a rapid immunoassay technique that has the characteristics of high precision and sensitivity. Although image FICA strip readers have the advantages of high portability and easy operation, the use of high-precision complementary metal oxide semiconductor (CMOS) image sensors leads to an increase in overall cost. Considering the popularity of CMOS image sensors in smartphones and their powerful processing functions, this work developed a smartphone-based FICA strip reader. An optical module suitable for the test strips with different fluorescent markers was designed by replacing the excitation light source and the light filter. An android smartphone was used for image acquisition and image denoising. Then, the test and control lines of the test strip image were recognized by the sliding window algorithm. Finally, the characteristic value of the strip image was calculated. A linear detection range from 10 to 5000 mIU/mL (R2 = 0.95) was obtained for human chorionic gonadotrophin with the maximum relative error less than 9.41%, and a linear detection range from 5 to 4000 pg/mL (R2 = 0.99) was obtained for aflatoxin B1, with the maximum relative error less than 12.71%. Therefore, the smartphone-based FICA strip reader had high portability, versatility, and accuracy.