Chinese fir (Cunninghamia lanceolata (Lamb.) Hook) was subjected to chemical extraction treatments with sodium chlorite (NaClO2) for delignification, as well as sodium hydroxide (NaOH) at various concentrations for extracting hemicelluloses gradually. Nanoindentation tests, X-Ray diffraction (XRD), and Fourier transform Raman (FT-Raman) spectroscopy studies revealed the changes in the mechanical properties and the nanostructure of the cell wall. The X-ray analysis indicated that delignification had only a moderate effect on the structure of the cell wall, while further alkali treatment led to major changes in the nanostructure. The nanoindentation tests showed that the indentation modulus and the hardness decreased after delignification and further alkali treatment, respectively. The indentation modulus of the cell wall with delignification decreased by 6.4% compared with the native cell wall, and the hardness decreased by 16.3%. After further alkali treatment, the indentation modulus and the hardness of the cell walls were 14.8% and 18.4% lower than that of the native cell walls, respectively. Additionally, the indentation modulus and the hardness of Chinese fir treated by NaOH decreased by 8.4%, and 2.1% in comparison with delignification, respectively. The results indicated that removal of hemicelluloses resulted in more damage to the mechanical properties of the cell wall compared with lignin.