Object tracking is a very challenging branch in the field of computer vision and plays an extremely important role in different fields. However, most algorithms still have no good robustness when the target is out of view, fast motion and low resulution. Therefore, this paper proposes a method based on siamese architecture and optical flow to address these situations. The tracking framework is mainly divided into tracking network and target localization network based on optical flow. The tracking network uses SiamBAN and the target localization network uses LiteFlowNet3 to estimate the optical flow information of target. The proposed method in this paper expands the search area of SiamBAN, and uses the target localization network to reposition the target when tracking fails. The target localization network can be easily embed into the existing siamese network to provide more accurate location of target during the tracking process. Experiments results on VOT2016, VOT2018 show that this method has good robustness against extreme conditions.