After a major earthquake in a dense urban area, the spatial distribution of heavily damaged buildings is indicative of the impact of the event on public safety. Timely assessment of the locations of severely damaged buildings and their damage morphologies using remote sensing approaches is critical for search and rescue actions. Detection of damaged buildings that did not suffer collapse can be highly challenging from aerial or satellite optical imagery, especially those structures with height-reduction or inclination damage and apparently intact roofs. A key information cue can be provided by a comparison of predicted building shadows based on pre-event building models with shadow estimates extracted from post-event imagery. This paper addresses the detection of damaged buildings in dense urban areas using the information of building shadow changes based on shadow simulation, analysis, and image processing in order to improve real-time damage detection and analysis. A novel processing framework for the rapid detection of damaged buildings without collapse is presented, which includes (a) generation of building digital surface models (DSMs) from pre-event LiDAR data, (b) building shadow detection and extraction from imagery, (c) simulation of predicted building shadows utilizing building DSMs, and (d) detection and identification of shadow areas exhibiting significant pre- and post-event differences that can be attributed to building damage. The framework is demonstrated through two simulated case studies. The building damage types considered are those typically observed in earthquake events and include height-reduction, over-turn collapse, and inclination. Total collapse cases are not addressed as these are comparatively easy to detect using simpler algorithms. Key issues are discussed including the attributes of essential information layers and sources of error influencing the accuracy of building damage detection.