We demonstrate a straightforward and economical way to obtain smooth germanium layers of high quality on silicon. Thin amorphous germanium layers deposited by plasma enhanced chemical vapor deposition on Si(111) substrates are transformed into single crystalline and smooth layers by solid phase epitaxy in N2 atmosphere. The crystal orientation of the substrate has a clear influence on the crystal quality. This is most likely due to a different growth mode, namely, layer-by-layer for Si(111) and three-dimensional growth for Si(001). The amorphous germanium layer can roughen during annealing due to mobile atoms on the surface. This can be effectively suppressed by annealing in N2 ambient. Electrical measurements show high charge mobility.