The synthesis and characterization of fibrous materials with a hierarchical structure are of great importance for materials sciences. Among this class of materials, microfibers of different natures coated with carbon nanofibers attract special interest. Such coating modifies the surface of microfibers, makes it rougher, and thus strengthens its interaction with matrices being reinforced by the addition of these microfibers. In the present work, a series of hierarchical materials based on carbon microfibers, basalt microfibers, and fiberglass cloth coated with up to 50 wt% of carbon nanofibers was synthesized via the catalytic chemical vapor deposition technique. The initial items were impregnated with an aqueous solution of nickel nitrate and reduced in a hydrogen flow. Then, the catalytic chemical vapor deposition process using C2H4 or C2H4Cl2 as a carbon source was carried out. A simple and cost-effective technique for the preparation of the samples of hierarchical materials for transmission electron microscopy examination was developed and applied for the first time. The proposed method of sample preparation for sequential TEM visualization implies an ultrasonic treatment of up to four samples simultaneously under the same conditions by using a special sample holder. As was found, the relative strength of carbon nanofibers coating the surface of microfibers decreases in the order of CNF/CMF > CNF/BMF > CNF/FGC. Two effects of the ultrasonic action on the carbon coating were revealed. First, strongly bonded carbon nanofibers undergo significant breakage. Such behavior is typical for carbon and basalt microfibers. Secondly, carbon nanofibers can be completely detached from the microfiber surface, as was observed in the case of fiberglass cloth. In the case of CNF/CMF material, the graphitized surface of carbon microfiber is coherent with the structure of carbon nanofiber fragments grown on it, which explains the highest adhesion strength of the carbon nanolayer coated on carbon microfibers.