Uncontrolled built-up area expansion and building densification could bring some detrimental problems in social and economic aspects such as social inequality, urban heat islands, and disturbance in urban environments. This study monitored multi-decadal building density (1991–2019) in the Yogyakarta urban area, Indonesia consisting of two stages, i.e., built-up area classification and building density estimation, therefore, both built-up expansion and the densification were quantified. Multi sensors of the Landsat series including Landsat 5, 7, and 8 were utilized with some prior corrections to harmonize the reflectance values. A support vector machine (SVM) classifier was used to distinguish between built-up and non built-up areas. Regression algorithms, i.e., linear regression (LR), support vector regression (SVR), and random forest regression (RFR) were explored to obtain the best model to estimate building density using the inputs of built-up indices: Urban Index (UI), Normalized Difference Built-up Index (NDBI), Index-based Built-up Index (IBI), and NIR-based built-up index based on the red (VrNIR-BI) and green band (VgNIR-BI). The best models were revealed by SVR with the inputs of UI-NDBI-IBI and LR with a single predictor of UI, for Landsat 8 (2013–2019) and Landsat 5/7 (1991–2009), respectively, using separate training samples. We found that machine learning regressions (SVM and RF) could perform best when the sample size is abundant, whereas LR could predict better for a limited sample size if a linear positive relationship was identified between the predictor(s) and building density. We conclude that expansion in the study area occurred first, followed by rapid building development in the subsequent years leading to an increase in building density.