In recent years, image processing technology has been developing and maturing, but due to the influence of many interfering factors in the acquisition process, there is a large amount of redundant information in the images obtained. The line segment detection algorithm in image extraction needs to be improved. This study utilizes computer technology to improve the line segment detection technology, and designs a line segment detection algorithm based on the linear detection improvement. Firstly, based on the basic principle of straight line detection algorithm, for the problems of line segment breakage and missing in straight line detection, RGB three-channel grayscale map is applied to detect line segments. Then the detected line segments are connected, merged and deleted. The test results show that the line segment detection algorithm improved based on straight line detection has the highest accuracy rate of 94.50 %, and the average processing time per image is also the lowest at 0.2 s. The algorithm runs faster at 0.25 s and has a higher F-value. It is able to detect the boundaries of a variety of rectangular targets, using the improved line segment detection algorithm has a wide range of applicability, lower error rate, and strong anti-interference ability. The improved line segment detection algorithm has a greater advantage in rectangular target extraction for document, text and book type images.