Nowadays, non-toroidal shape primary pass-through current transformers are commonly used for large current machines with several cables per phase. As these transformers exhibit no radial symmetry, it is not clear if they can be tested using the indirect test described in the IEC 61869 standard. In order to answer this question, two non-toroidal shaped current transformers of different secondary winding designs have been tested and simulated. One transformer has a uniformly distributed secondary winding and the other has a partially distributed secondary winding. Both transformers have the same nameplate characteristics. Both perform correctly in the indirect test. However, only the transformer with the uniformly distributed secondary winding performs correctly in a direct test. A finite element simulation shows that the iron core of the partially distributed secondary winding transformer was saturated, while the iron core of the uniformly distributed one was not. This result explains their different performance. The main conclusion is that the indirect test is not sensitive enough to cover all cases and therefore under doubtful situations, the transformers should be tested using the direct test.