This paper presents a fast and reliable approach for detecting 3D line segment from 3D point clouds. The main idea is to discover weak matching of line segments by re-projecting 3D point to 2D image plane and infer 3D line segment by spatial constraints. On the basis of 2D Line Segment Detector (LSD) and multi-view stereo, the proposed algorithm firstly reprojects the spatial point clouds into planar set on different camera matrices; then finds the best reprojected line from tentative matched points. Finally, 3D line segment is produced by back-projection in accordance with outlier removal based on plane clustering method. Experimental results show that the approach can obtain satisfactory visual effect as well as high efficiency. The fast line detection can also be extended in the application of 3D sketch for largescale scenes from multiple images.