A simple method to form ultra-thin (< 20 nm) semiconductor layers with a higher mobility on a 3D-structured insulating surface is required for next-generation nanoelectronics. We have investigated the solid-phase crystallization of amorphous Ge layers with thicknesses of 10−80 nm on insulators of SiO2 and Si3N4. We found that decreasing the Ge thickness reduces the grain size and increases the grain boundary barrier height, causing the carrier mobility degradation. We examined two methods, known effective to enhance the grain size in the thicker Ge (>100 nm). As a result, a relatively high Hall hole mobility (59 cm2/Vs) has been achieved with a 20-nm-thick polycrystalline Ge layer on Si3N4, which is the highest value among the previously reported works.