The three-chamber experiment, in which one test animal can choose between two animals placed in physically inaccessible compartments, is a widely adopted strategy for studying sexual preference in animals. Medaka, a small freshwater teleost, is an emerging model for dissecting the neurological/physiological mechanisms underlying mate choice for which intriguing findings have been accumulating. The three-chamber strategy has rarely been adopted in this species; therefore, here we investigated its validity using medaka colour variants that mate assortatively. First, a total of 551 movies, in which a test male and two choice females interacted for 30 min under a free-swimming condition, were manually analysed. The sexual preference of the males, calculated as a courtship ratio, was highly consistent between human observers (r > 0.96), supporting the objectivity of this manual-counting strategy. Second, we tested two types of three-chamber apparatuses, in which choice fish were presented in either a face-to-face or side-by-side location. Test fish (regardless of sex) spent most of the time associating with choice fish in the compartments. However, their sexual preference, calculated as an association ratio, was poorly reproduced when the locations of the choice fish were swapped. Third, the sexual preferences of males quantified using the manual-counting and either of the three-chamber strategies did not correlate (r = 0.147 or 0.297). Hence, we concluded that, even for individuals of a species like medaka, which spawn every day, sexual preference could not be reliably evaluated using the three-chamber strategy. Optimization of the protocol may solve this problem; however, the explanation for the observation that animals that are ready for spawning persist with never-accessible mating partners must be reconsidered.