Motivated by diagnosing the COVID‐19 disease using two‐dimensional (2D) image biomarkers from computed tomography (CT) scans, we propose a novel latent matrix‐factor regression model to predict responses that may come from an exponential distribution family, where covariates include high‐dimensional matrix‐variate biomarkers. A latent generalized matrix regression (LaGMaR) is formulated, where the latent predictor is a low‐dimensional matrix factor score extracted from the low‐rank signal of the matrix variate through a cutting‐edge matrix factor model. Unlike the general spirit of penalizing vectorization plus the necessity of tuning parameters in the literature, instead, our prediction modeling in LaGMaR conducts dimension reduction that respects the geometric characteristic of intrinsic 2D structure of the matrix covariate and thus avoids iteration. This greatly relieves the computation burden, and meanwhile maintains structural information so that the latent matrix factor feature can perfectly replace the intractable matrix‐variate owing to high‐dimensionality. The estimation procedure of LaGMaR is subtly derived by transforming the bilinear form matrix factor model onto a high‐dimensional vector factor model, so that the method of principle components can be applied. We establish bilinear‐form consistency of the estimated matrix coefficient of the latent predictor and consistency of prediction. The proposed approach can be implemented conveniently. Through simulation experiments, the prediction capability of LaGMaR is shown to outperform some existing penalized methods under diverse scenarios of generalized matrix regressions. Through the application to a real COVID‐19 dataset, the proposed approach is shown to predict efficiently the COVID‐19.