Inelastic mechanical responses in solids, such as plasticity, damage and crack initiation, are typically modeled in constitutive ways that display microstructural and loading dependence. Nevertheless, linear elasticity at infinitesimal deformations is used for microstructural properties. We demonstrate a framework that builds on sequences of microstructural images to develop fingerprints of inelastic tendencies, and then use them for data-rich predictions of mechanical responses up to failure. In analogy to common fingerprints, we show that these two-dimensional instability-precursor signatures may be used to reconstruct the full mechanical response of unknown sample microstructures; this feat is achieved by reconstructing appropriate average behaviors with the assistance of a deep convolutional neural network that is fine-tuned for image recognition. We demonstrate basic aspects of microstructural fingerprinting in a toy model of dislocation plasticity and then, we illustrate the method's scalability and robustness in phase field simulations of model binary alloys under mode-I fracture loading.