In this paper a new horizon-based shipborne camera video stabilization algorithm is implemented. First, the edges are obtained by Canny operator, then effective lines are obtained by Hough transform. In order to get the horizon within the existing lines, a fast filter-based method detecting the lines location differences is proposed, and image stabilization is implemented by the difference with the current horizon and base horizon. Experiments show that the algorithm can meet the real-time requirements, and can achieve the better image stabilization effect even in the conditions of bad light and fogs.