The chemical, morphological, physical, and thermal properties of raw materials and single fibers extracted from different morphological parts of windmill palm were examined and comprehensively characterized after an alkali treatment. Leaf sheathes (LS) with the highest cellulose content (52.26%) achieved the most efficient extraction of fibers. Single fibers extracted from the vascular bundles of the windmill palm raw material had a slender shape with a tapering and sealing terminus, with each single fiber possessing a lumen in its cross-section. These windmill palm fibers displayed similar chemical compositions, but they exhibited significant differences in morphological parameters. Leaf blade fibers (LBFs) had the longest length (1240 μm ± 470 μm) and highest aspect ratio (121.39), which presented excellent potential as a reinforced fiber. After the alkali treatment, almost all of the hemicelluloses and lignin were removed, which resulted in increased crystallinity of extracted fibers. Thermogravimetric analysis confirmed LS stability up to 319 °C, which was higher than that of other materials from windmill palm.