In this work, we advocate using Bayesian techniques for inversely identifying material parameters for multiscale crystal plasticity models. Multiscale approaches for modeling polycrystalline materials may significantly reduce the effort necessary for characterizing such material models experimentally, in particular when a large number of cycles is considered, as typical for fatigue applications. Even when appropriate microstructures and microscopic material models are identified, calibrating the individual parameters of the model to some experimental data is necessary for industrial use, and the task is formidable as even a single simulation run is time consuming (although less expensive than a corresponding experiment). For solving this problem, we investigate Gaussian process based Bayesian optimization, which iteratively builds up and improves a surrogate model of the objective function, at the same time accounting for uncertainties encountered during the optimization process. We describe the approach in detail, calibrating the material parameters of a high-strength steel as an application. We demonstrate that the proposed method improves upon comparable approaches based on an evolutionary algorithm and performing derivative-free methods.