Dimension reduction is often used for several procedures of analysis of high dimensional biomedical data-sets such as classification or outlier detection. To improve performance of such data-mining steps, preserving both distance information and local topology among data-points could be more useful than giving priority to visualisation in low dimension. Therefore, we introduce topology preserving distance scaling (TPDS) to augment dimension reduction method meant to reproduce distance information in higher dimension. Our approach involves distance inflation to preserve local topology to avoid collapse during distance preservation based optimisation. Applying TPDS on diverse biomedical data-sets revealed that besides providing better visualisation than typical distance preserving methods, TPDS leads to better classification of data points in reduced dimension. For data-sets with outliers, the approach of TPDS also proves to be useful, even for purely distance-preserving method for achieving better convergence.