We have been researching a displacement measurement system for monitoring rock damage such as cracks in tunnel walls and pillars in an underground mine. The system consists of a displacement sensor unit with a pair of crossed optical fibers and a distributed fiber optic strain measuring device. The strain measuring device measures displacement-induced strains in the optical fibers and, from these measured strains, displacements in two directions are obtained. We analyzed the influence of the strain measurement error and sensor unit configuration on the displacement measurement. Moreover, we confirmed the usefulness of displacement measurement by the sensor unit through experiments using a prototype.