Integrating Infrared (IR) Focal Plane Array (FPA) detectors with micropolarizer array achieves viable IR polarimeters. This work systematically studies the extinction ratio (ER) variation tendencies of the polarimeter associated with the assembling process. A 3D Finite-Different Time Domain (FDTD) model was established to characterize the electrical field distributions and extract the concerned parameters. The ER variation caused by the distance between the plane of grating wires and the focal plane is significant, while the one caused by the tilting is moderate. Hereby reasonable assembling strategies can be determined. What’s more, optimization methods have been proposed, which enables preferrable ERs with no additional process difficulties.