Aims. We identify optical counterparts, address uncertain identifications and measure previously unknown redshifts of the host galaxies of candidate GPS radio sources, and study their stellar populations. Methods. Long slit spectroscopy and deep optical imaging in the B, V and R bands, obtained with the Very Large Telescope. Results. We obtain new redshifts for B0316+161, B0407-658, B0904+039, B1433-040, and identify the optical counterparts of B0008-421 and B0742+103. We confirm the previous identification for B0316+161, B0407-658, B0554-026, and B0904+039, and find that the previous identification for B0914+114 is incorrect. Using updated published radio spectral information, we classify as non GPS the following sources: B0407-658, B0437-454, B1648+015. The optical colors of typical GPS sources are consistent with single instantaneous burst stellar population models but do not yield useful information on age or metallicity. A new master list of GPS sources is presented.