This research aimed to solve the problem of fish position and pose recognition in the process of robot grasping fish. Crucian carp is the subject of this research and studies the position and pose recognition algorithm of crucian carp by photo-model-based stereo-vision. This research aims at the problem that the fitness algorithm has a poor effect in identifying photo models with more black and white information, this research proposes an improved fitness algorithm. The new fitness algorithm uses the improved HSV model and adds the Saturation and Value to the evaluation value of the original fitness algorithm. Then Real-Time Multi-step Genetic Algorithm (RM-GA) is used to verify the improvement effect of the fitness algorithm. We use the improved algorithm for experiments. The findings indicated that the peak fitness is significantly improved, with an average increase of about 95%. At the same time, the position and posture recognition error of crucian carp is reduced by more than 50%, and the real-time performance of the algorithm is improved.