The relative position between magnetic target and the device, which is installed on vehicle under water, is determined by measuring magnetic field magnitude and its gradient of target on condition that geomagnetic field anomaly can be viewed as a magnetic dipole object. The errors of measuring magnetic field and its gradient caused by nonorthogonality between axes, different sensitivity of each axis, and installation center departure of magnetometers are analyzed in theoretically. These errors of magnetic field magnitude and its gradient can act effect on the localization precision of underwater vehicle, which is analyzed theoretically and simulation numerically. These results show that magnetometer performances such as nonorthogonality between axes, and different sensitivity of each axis have the remarkable influence on the localization precision.