This study investigated the chemical structures and probiotic potential of different polysaccharides (LPs) extracted from the litchi pulp that fermented with Lactobacillus fermentum for different times (i.e., 0–72 h corresponding to LP-0 through LP-72, respectively). Fermentation times affected the yields, total sugar contents, uronic acid contents, molecular weights, and monosaccharide compositions of LPs. The LPs yields and uronic acid contents exhibited irregular trends in association with fermentation time, while total sugar contents decreased, and the molecular weights increased. Particularly, LP-6 contained the highest extraction yields (2.67%), lowest uronic acid contents, and smallest average Mw (104 kDa) (p < 0.05). Moreover, analysis of the monosaccharide composition in the fermented LPs indicated that the proportions of glucose decreased, while arabinose and galacturonic acid proportions increased relative to unfermented LP-0. Further, LP-6 demonstrated the highest growth for Bifidobacterium compared to LP-0, while the other fermentation time led to comparable or worse probiotic promoting activities. These results suggest that lactic acid bacteria fermentation alters the physicochemical properties of litchi polysaccharides, such that suitable fermentation time can enhance their probiotic activities.