In this paper we present a novel method to estimate the time evolution of proto-neutron star (PNS) structure from the neutrino signal in core-collapse supernovae (CCSN). Employing recent results of multi-dimensional CCSN simulations, we delve into a relation between total emitted neutrino energy (TONE) and PNS mass/radius, and we find that they are strongly correlated with each other. We fit the relation by simple polynomial functions connecting TONE to PNS mass and radius as a function of time. By combining another fitting function representing the correlation between TONE and cumulative number of event at each neutrino observatory, PNS mass and radius can be retrieved from purely observed neutrino data. We demonstrate retrievals of PNS mass and radius from mock data of neutrino signal, and we assess the capability of our proposed method. While underlining the limitations of the method, we also discuss the importance of the joint analysis with gravitational wave signal. This would reduce uncertainties of parameter estimations in our method, and may narrow down the possible neutrino oscillation model. The proposed method is a very easy and inexpensive computation, which will be useful in real data analysis of CCSN neutrino signal.