In the dairy farming industry, we can obtain the temperature, color, and location information of dairy cows by patrol inspection robot so as to monitor the health status and abnormal behaviors of dairy cows. We build and calibrate a heterogeneous binocular stereo vision (HBSV) system comprising a high-definition color camera and infrared thermal camera and mount it on a patrol inspection robot. First, based on the traditional chessboard, an easy-to-make calibration board for the HBSV system is designed. Second, an accurate locating and sorting algorithm for the calibration points of the calibration board is designed. Then, the cameras are calibrated and the HBSV system is stereo-calibrated. Finally, target locating is achieved based on the above calibration results and Yolo target detection technology. In this paper, several experiments are carried out from many aspects. The target locating average error of HBSV system is 3.11%, which satisfies the needs of the dairy farming environment. The video’s FPS captured by using HBSV is 7.3, which is 78% higher than that by using binocular stereo vision system and infrared thermal camera. The results show that the HBSV system has application value to a certain degree.

In the traditional direct visual odometry, it is difficult to satisfy the photometric invariant assumption due to the influence of illumination changes in the real environment, which will lead to errors and drift. This paper proposes an improved direct visual odometry system, which combines luminosity and depth information. The algorithm proposed in this paper uses Kinect 2 to collect RGB images with the corresponding depth information, and selects points with large changes of gray gradient to construct a luminosity error function and uses the corresponding depth information to construct a depth error function. The two error functions are merged into one function and converted into the least squares function of the pose of camera, the Levenberg-Marquardt algorithm is used to solve the camera pose. Finally, the Graph optimization theory and the g2o library are used to optimize the initial pose. Experiments show that the algorithm can reduce the error to a certain extent and reduce the drift caused by illumination changes.