Advanced light sources using relativistic electrons rely on coherent emission from high-density (compressed) beams. These beams, typically produced by photoinjected linear accelerators, can suffer from uncontrolled microbunching instabilities that are difficult to manage, since a complete understanding of their growth due to space charge and other wakefields is lacking. Here we present the first systematic measurements of microbunching instability using electron beams premodulated in a controlled fashion. By comparing beams having various modulation depths and wavelengths with unmodulated beams, we are able to benchmark, for the first time, the analytical calculations for the microbunching instability. In addition, our results give a proof of principle demonstration of a longitudinal space charge amplifier (LSCA), where a specific beam density pattern develops and grows. A potential application of this particular LSCA scheme is for controlling waveforms and enhancing the spectral content of linac-based sources of coherent terahertz radiation.