Large panel system (LPS) buildings, during the 1950s–1990s, were the most widespread among precast systems and are still a huge part of some national housing stocks. Disadvantages in characteristics, poor quality of all construction stages, sudden disasters and poor retrofit and repair techniques can result in demolishing some LPS buildings and thus the creation of huge amounts of waste. There is urgent need of evaluating the possibility of reusing the elements of LPS buildings. The novelty of the article is based on the formulation of research gaps in terms of recycling these buildings and the justification of such need. The authors reviewed the current state of knowledge regarding characteristics of LPS buildings, directions of retrofitting, durability and their locations within cities. The possibility of recycling elements of LPS buildings was analysed. The findings are that concrete load bearing panels, proved to be in satisfactory conditions in terms of strength and durability, have the possibility to be recycled into recycled coarse aggregate (RCA), so such waste could be beneficial for concrete. However, there are research gaps of such an application highlighted in the text. The authors suggest that conducting comprehensive basic research in this area could have significant benefits for the next generation of researchers and engineers.