The desizing efficiencies of starches are different. However, the reasons and mechanism for this appearance are still unclear. Therefore, desizing efficiencies of starches were measured, and the reasons for their difference were analyzed. Swelling ability and degree of crystallinity of starch films were tested to conform to the relation with desizing. The size residue on the desized yarns was collected to detect the difference in the proportion of nonstarch components and the structure of hydrolyzed starch. In addition, correlation analyses were conducted between the factors affecting hydrolysis and desizing efficiency. It was found that corn starch has the lowest desizing efficiency, and the efficiency of cassava starch was the highest. The largest proportion of protein was the main reason for inhibiting the desizing. Furthermore, a higher proportion of lipid and the lower swelling ability of starch film could also be an obstacle to desizing. Removing the nonstarch components and improving the swelling ability of film may be effective methods to improve the desizing efficiency of starch.