Calibration is a critical step for the phase measuring deflectometry system. Existing calibration methods are mainly optimizing the calibration parameters with respect to the 2D re-projection error criterion. However, such a procedure cannot reduce metric errors in the practical application. Therefore, an accurate and practical calibration method is proposed. In which, conventional calibration means is first applied for the primary calibration. Then, a precise square planar mirror is used for the optimization of system calibration parameters. All the intrinsic and extrinsic parameters are considered as a global multi-objective optimization problem. Three metric error criteria are introduced to evaluate the 3D reconstruction accuracy of the reference mirror. Compared with classical calibration means, which apply the parameter optimization in 2D image space to minimize the re-projection errors, the proposed optimization approach is executed in 3D space directly. An experiment and comparison are conducted to verify that the proposed optimal calibration approach can effectively reduce the system deviation and to improve the system measurement accuracy.