Characterization of the cross-sectional morphologies of polymeric membranes are critical in understanding the relationship of structure and membrane separation performances. However, preparation of cross-sectional samples with flat surfaces for scanning electron microscopy (SEM) characterizations is challenging due to the toughness of the non-woven fabric support. In this work, a new frozen section technique was developed to prepare the cross-sectional membrane samples. A special mold was self-designed to embed membranes orientationally. The frozen section parameters, including the embedding medium, cryostat working temperature, and sectioning thickness were optimized. The SEM characterizations demonstrated that the frozen section technique, using ultrapure water as the embedding medium at a working temperature of −30 °C and a sectioning thickness of 0.5 µm, was efficient for the preparation of the membrane samples. Three methods of preparation for the cross-sectional polymeric membranes, including the conventional liquid nitrogen cryogenic fracture, the broad ion beam (BIB) polishing, and the frozen section technique were compared, which showed that the modified frozen section method was efficient and low cost. This developed method could not only accelerate the development of membrane technology but also has great potential for applications in preparation of other solid samples.