Commercial buildings as well as residential houses represent core structures of any modern day urban or semiurban areas. Consequently, 3-D models of urban buildings are of paramount importance to a majority of digital urban applications, such as city planning, 3-D mapping and navigation, video games and movies, and construction progress tracking, among others. However, current studies suggest that existing 3-D modeling approaches often involve high computational cost and large storage volumes for processing the geometric details of the buildings. Therefore, it is essential to generate concise digital representations of urban buildings from the 3-D measurements or images so that the acquired information can be efficiently utilized for various urban applications. Such concise representations, often referred to as "lightweight" models, strive to capture the details of the physical objects with less computational storage. Furthermore, lightweight models consume less bandwidth for online applications and facilitate accelerated visualizations. With many emerging digital urban infrastructure applications, lightweight reconstruction is poised to become a new area of research in the urban remote sensing community. We aim to provide a thorough review of data structures, representations, and state-of-the-art algorithms for lightweight 3-D urban reconstruction. We discuss the strengths and weaknesses of key lightweight urban reconstruction techniques, ultimately providing guidance on future research prospects to fulfill the pressing needs of urban applications.