The density and configurational changes of crystal dislocations during plastic deformation influence the mechanical properties of materials. These influences have become clearest in nanoscale experiments, in terms of strength, hardness and work hardening size effects in small volumes. The mechanical characterization of a model crystal may be cast as an inverse problem of deducing the defect population characteristics (density, correlations) in small volumes from the mechanical behavior. In this work, we demonstrate how a deep residual network can be used to deduce the dislocation characteristics of a sample of interest using only its surface strain profiles at small deformations, and then statistically predict the mechanical response of size-affected samples at larger deformations. As a testbed of our approach, we utilize high-throughput discrete dislocation simulations for systems of widths that range from nano-to micro-meters. We show that the proposed deep learning model significantly outperforms a traditional machine learning model, as well as accurately produces statistical predictions of the size effects in samples of various widths. By visualizing the filters in convolutional layers and saliency maps, we find that the proposed model is able to learn the significant features of sample strain profiles. Prediction of mechanical behavior up to failure is commonly achieved using constitutive laws written as equations in terms of phenomenological parameters in the cases where physical laws are not clear or for the purpose of simplification. The parameters are obtained experimentally (at the manufacturing stage) for individual classes of materials, thus classified by composition, prior processing and load history, all of which affect the material's micro-structure and thus its behavior 1. Beyond processing routes, materials have been known to also be highly sensitive to micro-structure changes, especially when used at extreme conditions such as small volume, high temperature, high pressure, and high strain rates 2-11. Such extreme conditions are experienced in numerous applications at the technological and industrial frontiers. Thus, constitutive laws are difficult to apply when the micro-structural changes brought by operating conditions are unknown. In order to assess the yield and failure strength values, current practice requires non-destructive characterization methods at the nanoscale that can swiftly assess mechanical properties. The case study in this work represents possibly one of the most challenging, but benchmarked 12 , applications of micromechanics. More specifically, we investigate, using synthetic data from discrete dislocation plasticity simulations 12-15 how such non-destructive characterization can effectively work in the realistic scenario of assessing and predicting the strength of small finite volumes by using digital image correlation (DIC) 16 techniques. Changes in the dislocation structure of a material caused by prior processing of a crystal may not be evident from a visual inspection o...