Solid form selection and design of crystalline small molecule active pharmaceutical ingredients (APIs) would benefit from computational prediction and rationalization of the mechanical properties. Before such practical applications can be considered, the robustness and reproducibility of the computed properties with respect to the chosen level of theory must be understood. In this work, elastic constants of eight molecular crystals, with an emphasis on APIs, have been calculated using dispersion-corrected density functional theory (DFT). The different DFT methods considered do not, in general, consistently predict the absolute magnitudes of the elastic moduli, which disagree by over 50% for some crystals. Relative properties such as elastic anisotropy are more robust, mostly consistent between models, and in qualitative agreement with experiment. Calculated anisotropies could also be rationalized in terms of the structural features of the crystal. Overall, this work reveals that DFT-computed elastic properties may not offer a ground truth in absolute terms. Future applications of DFT in the context of high-throughput material screening or training of machine learning models will therefore require judicious selection of target properties and evaluation metrics. Only after methodological limitations are properly identified can more in-depth investigations be undertaken to assess the feasibility of applying DFT in pharmaceutical development.