Due to the influence of the shooting environment and inherent image characteristics, there is a large amount of interference in the process of image stitching a geological borehole video. To accurately match the acquired image sequences in the inner part of a borehole, this paper presents a new method of stitching an unfolded borehole image, which uses the image generated from the video to construct a large-scale panorama. Firstly, the speeded-up robust feathers (SURF) algorithm is used to extract the image feature points and complete the rough matching. Then, the M-estimator sample consensus (MSAC) algorithm is introduced to remove the mismatched point pairs and obtain the homography matrix. Subsequently, we propose a local homography matrix offset optimization (LHOO) algorithm to obtain the optimal offset. Finally, the above process is cycled frame by frame, and the image sequence is continuously stitched to complete the construction of a cylindrical borehole panorama. The experimental results show that compared with those of the SIFT, Harris, ORB and SURF algorithms, the matching accuracy of our algorithm has been greatly improved. The final test is carried out on 225 consecutive video frames, and the panorama has a good visual effect, and the average time of each frame is 100 ms, which basically meets the requirements of the project.