Crime prediction is crucial for sustainable urban development and protecting citizens’ quality of life. However, there exist some challenges in this regard. First, the spatio-temporal correlations in crime data are relatively complex and are heterogenous in time and space, hence it is difficult to model the spatio-temporal correlation in crime data adequately. Second, crime prediction at fine spatial temporal scales can be applied to micro patrol command; however, crime data are sparse in both time and space, making crime prediction very challenging. To overcome these challenges, based on the deep spatio-temporal 3D convolutional neural networks (ST-3DNet), we devise an improved ST-3DNet framework for crime prediction at fine spatial temporal scales (ST3DNetCrime). The framework utilizes diurnal periodic integral mapping to solve the problem of sparse and irregular crime data at fine spatial temporal scales. ST3DNetCrime can, respectively, capture the spatio-temporal correlations of recent crime data, near historical crime data and distant historical crime data as well as describe the difference in the correlations’ contributions in space. Extensive experiments on real-world datasets from Los Angeles demonstrated that the proposed ST3DNetCrime framework has better prediction performance and enhanced robustness compared with baseline methods. In additon, we verify that each component of ST3DNetCrime is helpful in improving prediction performance.