As a non-imaging optical measurement technique, spectroscopic Mueller matrix polarimetry (MMP) has been introduced for critical dimension (CD) and overlay metrology with recent great success. Due to the additional information provided by the Mueller matrices when the most general conical diffraction configuration is considered, MMP has demonstrated a great potential in semiconductor manufacturing. In order to make full use of the additional information provided by the Mueller matrices, it is of great importance for MMP to optimize the measurement configuration. In this paper, we introduce the norm of a configuration error propagating matrix as the cost function to optimize the measurement configuration for spectroscopic MMP with the aim of finding an optimal combination of fixed incidence and azimuthal angles, which provides higher measurement accuracy. The optimal measurement configuration can be achieved by minimizing the norm of the configuration error propagating matrix in the available ranges of incidence and azimuthal angles. Experiments performed on a silicon grating with a dual-rotating compensator Mueller matrix polarimeter have demonstrated the validity of the proposed measurement configuration optimization method.