Large-scale coupled reservoir-geomechanical simulation is becoming a necessity for an in-depth assessment of subsurface energy developments such as hydrocarbon recovery and geological carbon storage, while a robust and efficient upscaling technique for the geomechanical constitutive behavior of heterogeneous reservoir is still missing to push forward the application of time-consuming coupled reservoir-geomechanical simulation. Here, we focus on the impact of lithological heterogeneity on the shear strength and stress-strain behavior and propose a deep learning-based upscaling technique that can provide the upscaled shear strength and stress-strain behavior from facies models and geomechanical parameters. The objectives of the proposed upscaling technique lie in the following two aspects: 1) bridge the gap between the fine-scale geological models and computationally efficient reservoir-geomechanical models used for large-scale subsurface energy development; 2) provide the upscaled realizations needed for geomechanical assessments considering geological uncertainties. The first step of the deep learning-based upscaling technique is generating a dataset that contains a sufficient number of data samples. Each sample consists of a randomly generated spatial correlated sand-shale realization (input) and the computed macroscopic shear strength and stress-strain behavior via finite element simulations (outputs). Using the assembled dataset, convolutional neural network (CNN) models are trained to build proxy models as an alternative for numerical upscaling. The trained CNN models can provide the upscaled shear strength (R2 > 0.95) and stress-strain behavior (R2 > 0.93) that highly agree with that from the computationally extensive numerical upscaling method in a much shorter time frame. The proposed deep learning-based upscaling technique can promote the application of large-scale reservoir-geomechanical simulation for geomechanical assessment and quantify the impact of geological uncertainties by conducting coupled simulations on a variety of reservoir realizations.