This paper analyzes the problems of existing container positioning methods and proposed a vision-based container position measuring system to provide precise parameters for container lifting operations. This system uses camera to get container information, then detects container corners by the method that combined with convolutional neural network and traditional image processing algorithm. This system is used to provide specific parameters associated with container lifting operation. In the first of detection, it uses the modified SSD (Single Shot MultiBox Detector) neural network to detect the coarse position of container corners in the image, second stage detection uses the usage of rectangle fitting to detect the precise position of corner holes in the coarse position. In the last step the offset distance and deflection angle were calculated by precise corner position. The experiment shows the detection rate of the proposed system reach 94%. The positioning errors between 14.3 and 19.6 mm for a frame rate of 10 fps are obtained.