In a core-collapse supernova, a huge amount of energy is released in the Kelvin-Helmholtz phase subsequent to the explosion, when the proto-neutron star cools and deleptonizes as it loses neutrinos. Most of this energy is emitted through neutrinos, but a fraction of it can be released through gravitational waves. We model the evolution of a proto-neutron star in the Kelvin-Helmholtz phase using a general relativistic numerical code, and a recently proposed finite temperature, many-body equation of state; from this we consistently compute the diffusion coefficients driving the evolution. To include the many-body equation of state, we develop a new fitting formula for the high density baryon free energy at finite temperature and intermediate proton fraction. We estimate the emitted neutrino signal, assessing its detectability by present terrestrial detectors, and we determine the frequencies and damping times of the quasinormal modes which would characterize the gravitational wave signal emitted in this stage.