The paper presents a novel deep learning approach, which extracts latent information from trained Deep Neural Networks (DNNs) and derives concise representations that are analyzed in an effective, transparent way for prediction in medical imaging. A novel methodology is presented, in which deep neural architectures that have been trained to provide highly accurate predictions over existing datasets are adapted, in a consistent way, to make predictions over different contexts and datasets. Unified prediction is then achieved over the original and the new datasets. Successful application is illustrated through a large experimental study for prediction of Parkinson's disease from MRI and DaTScans, as well as for prediction of COVID-19 from CT scans and X-rays.