It is extremely important for the performance and success of the system to know the cooling or heating surfaces of the thermoelectric modules (TEMs) as cooler or generator, according to the heat transfer direction, and to carry out their installation correctly, taking this into account. In TEMs, the direction of heat transfer between surfaces changes depending on the applied DC direction, and while one surface of the module cools, the other heats up. In this respect, the state of the positive and negative poles in the design and production of TEMs directly affects the heat transfer direction between the module surfaces. On the other hand, the contact direction of the surfaces of the TEMs used in a designed thermoelectric system (TES) is of great importance in order to perform the heat transfer correctly without loss of performance. In this study, a system is designed to detect the heat transfer direction of TEMs using image processing techniques. The basic principle of the system is to determine the positive and negative poles of the TEMs together with the ceramic plate, using the color-based image processing method, and to determine the heat transfer direction by utilizing their relative positions. With the designed system, the heat transfer direction of TEMs at different illuminance levels is tried to be determined and successful results were obtained. As a result, it is thought that the developed system will contribute to the automatic error control for the production and assembly of TEMs.