Carbohydrate antigen 199 (CA199) is a serum biomarker which has certain value and significance in the diagnosis, prognosis, treatment, and postoperative monitoring of cancer. In this study, a lateral flow immunoassay based on europium (III) polystyrene time-resolved fluorescence microspheres (TRFM-based LFIA), integrated with a portable fluorescence reader, has been successfully establish for rapid and quantitative analysis of CA199 in human serum. Briefly, time-resolved fluorescence microspheres (TRFMs) were conjugated with antibody I (Ab1) against CA199 as detection probes, and antibody II (Ab2) was coated as capture element, and a “TRFMs-Ab1-CA199-Ab2” sandwich format would form when CA199 was detected by the TRFM-based LFIA. Under the optimal parameters, the detection limit of the TRFM-based LFIA for visible quantitation with the help of an ultraviolet light was 4.125 U/mL, which was four times lower than that of LFIA based on gold nanoparticles. Additionally, the fluorescence ratio is well linearly correlated with the CA199 concentration (0.00–66.0 U/mL) and logarithmic concentration (66.0–264.0 U/mL) for quantitative detection. Serum samples from 10 healthy people and 10 liver cancer patients were tested to confirm the performances of the point-of-care application of the TRFM-based LFIA, 20.0 U/mL of CA199 in human serum was defined as the threshold for distinguishing healthy people from liver cancer patients with an accuracy of about 60%. The establishment of TRFM-based LFIA will provide a sensitive, convenient, and efficient technical support for rapid screening of CA199 in cancer diagnosis and prognosis.