This paper presents the recent progress on integrated second-order nonlinear waveguides on silicon substrates for second-harmonic generation. In particular, demonstrations of thin-film lithium niobate, III-V compound semiconductor and dielectric waveguides integrated on silicon substrates are reviewed. For completeness, the fundamentals of the nonlinear optical processes involved are briefly introduced. Methods demonstrated for phase matching, e.g., periodic poling and mode-shape modulation, in the compact integrated devices are discussed. Finally, an outlook for how integrated photonics may benefit from the progress in this field is provided.