The received signal strength (RSS) based Wi-Fi fingerprinting method is one of the most potential and easily deployed approaches for a reliable indoor positioning system. However, due to the labor intensive and time-consuming radio map construction process, interpolation is often incorporated. To ensure the interpolated radio map is robust against environmental noise and RSS fluctuations, we propose two novel interpolation methods, termed as DimRed and DimRedClust, for an improved radio map construction. The former performs dimensionality reduction prior to the interpolation while the latter employs both the dimensionality reduction and clustering before interpolating the radio map. For dimensionality reduction, principal component analysis (PCA) or truncated singular value decomposition (TSVD) is adopted to profoundly extract essential features from the RSS data while the K-means algorithm is used to partition the reference points (RPs) into several clusters. Subsequently, the RSS for all virtual points are interpolated via inverse distance weighting (IDW). Numerical results based on the real-world multi-floor multi-building dataset confirm the supremacy of the proposed schemes over the baseline IDW interpolation. Compared to the baseline IDW, the proposed PCA-K-means-IDW, TSVD-K-means-IDW, PCA-IDW, and TSVD-IDW could attain a performance gain in terms of average positioning error of up to 30.17%, 30.93%, 19.33%, and 21.61%, respectively.