Hydrothermally altered rhyolite rocks in the Permian Donnersberg Formation of a geothermal borehole in the Northern Upper Rhine Graben (Germany) were investigated to find out answers for the low hydraulic conductivity of the rocks. The composition of clay minerals and the temperature of smectite–illite transformation were carried out using X-ray diffraction, X-ray fluorescence, transmission electron microscopy, Fourier transform infrared spectroscopy, and polarized-light microscopy analyses. Clay mineral (CM) composition includes illite/muscovite (1M and 2M1 polytypes), illite–smectite interstratifications (IS-ml), smectite, and chlorite; and non-clay minerals such as quartz, feldspars, epidote, calcite, dolomite, and hematite were detected. The 2M1-polytype mica might be the only primary sheet silicates from the parent rocks, while the others occur as authigenic neo-formed CMs under heat flow and geothermal gradient. The development of CMs indicates different mechanisms of illitization and smectitization. Based on the texture, morphology, structure/polytype, and chemistry of rocks and minerals, in particular CMs, the study grouped the CM formation into three transformation processes: smectitization during magma cooling and possible contact metamorphisms with decreasing and low temperature, smectite illitization controlled by burial diagenesis and hydrothermal alteration, and illite smectitization followed exhumation and Cenozoic subsidence with decreasing temperature. The rhyolites were altered to all of the orders IS-R0, IS-R1, and IS-R3 by the dissolution-precipitation and layer-to-layer mechanisms. The first one supported small xenomorphic plates and flakes of 1Md, elongated particles of 1M, and pseudo-hexagonal forms of 2M1. The second one could lead to the platy particles of 1Md and 2M1 polytypes. The dominant temperature range for the transformation in the area has been 140–170 °C– ~ 230 °C.