The camera’s intrinsic parameters calibration directly affects monocular pose measurement accuracy. To improve the calibration accuracy of the camera’s intrinsic parameters, the polarizing device is used to eliminate the high-brightness areas in the images collected for camera calibration. However, an unsuitable polarizing angle will reduce the image’s contrast. This causes the texture information of the image to be lost or makes the extracted feature change in the direction of grayscale value decline, ultimately affecting the feature extraction. Aiming at this issue, this paper proposed a camera calibration approach using polarized light based on high-frequency component variance weighted entropy. We determined the suitable polarizing angle according to the 2D homography matrix describing the mapping between the target plane and normalized imaging plane for each calibration location. Under the suitable polarizing angle, we determined the optimal exposure time according to the high-frequency component variance weighted entropy. The images collected under suitable polarizing angles and the optimal exposure time were used to finish the camera calibration. The suitable polarizing angles and optimal exposure time can ensure the image’s contrast and texture information. The experimental findings demonstrate that our approach can increase the camera calibration accuracy and thus pose measurement accuracy. In the -45° to +45° range, the average pose measurement error is less than 0.05° and lowered by 0.08° (62%) compared to that performed using the intrinsic parameters calibrated with the images collected under no polarizing device. In the 0 to 20 mm range, the average pose measurement error is less than 0.028 mm and lowered by 0.05 mm (64%) compared to that performed using the intrinsic parameters calibrated with the images collected under no polarizing device.