Black holes were found to possess properties that mirror ordinary thermodynamical systems in the landmark paper by Bardeen, Carter and Hawking almost half a century ago. Since then much progress has been made, but many fundamental issues remain. For example, what are the underlying degrees of freedom of a black hole horizon that give rise to said thermodynamical properties? Furthermore, classical black holes also harbor a spacetime singularity. Although it is often believed that quantum gravity would "cure" the singularity, as emphasized by Penrose, this viewpoint requires a deeper examination. In this review, I will examine the possibility that singularities remain in quantum gravity, the roles they may play, and the possible links between singularity and black hole thermodynamics. I will also discuss how -inspired by Penrose's Weyl curvature hypothesis -gravitational entropy for a black hole can be defined using curvature invariants, and the surprising implication that the entropy of black holes in different theories of gravity are different manifestations of spacetime curvature, i.e., their underlying microstructures could be different. Finally, I review the "Hookean law" recently established for singly rotating Myers-Perry black holes (including 4-dimensional Kerr black holes) that connect black hole fragmentation -a consequence of the second law of black hole thermodynamicswith the maximum "Hookean force", as well as with the thermodynamic geometry of Ruppeiner. This also suggests a new way to study black hole microstructures, and hints at the possibility that some black holes are beyond the Hookean regime (and thus have different microstructures). While examining the remarkable connections between black hole thermodynamics, spacetime singularities and cosmic censorship, as well as gravitational entropy, I shall point out some subtleties, provide some new thoughts, and raise some hard but fundamental questions, including whether black hole thermodynamics is really just "ordinary thermodynamics" or something quite different.