While recent progress in the black hole information problem has shown that the entropy of Hawking radiation follows a unitary Page curve, the quantum state of Hawking radiation prior the Page time is still treated as purely thermal, containing no information about the microstructure of the black hole. We demonstrate that there is significant quantum information regarding the quantum state of the black hole in the Hawking radiation prior to the Page time. By computing of the quantum fidelity in a 2D boundary conformal field theory (BCFT) model of black hole evaporation, we demonstrate that an observer outside of an evaporating black hole may distinguish different black holes via measurements of the Hawking radiation at any time during the evaporation process, albeit with an exponentially large number of measurements. Furthermore, our results are universal, applicable to general BCFTs including those with large central charge and rational BCFTs. The techniques we develop for computing the fidelity are more generally applicable to excited states in CFT. As such, we are able to characterize more general aspects of thermalization in 2D conformal field theory.