Chern-Simons (CS) theories with rank N and level k on Seifert manifold are discussed. The partition functions of such theories can be written as a function of modular transformation matrices summed over different integrable representations of affine Lie algebra u(N ) k associated with boundary Wess-Zumino-Witten (WZW) model. Using properties of modular transform matrices we express the partition functions of these theories as a unitary matrix model. We show that, the eigenvalues of unitary matrices are discrete and proportional to hook lengths of the corresponding integrable Young diagram. As a result, in the large N limit, the eigenvalue density develops an upper cap. We consider CS theory on S 2 × S 1 coupled with fundamental matters and express the partition functions in terms of modular transformation matrices. Solving this model at large N we find the dominant integrable representations and show how large N representations are related to each other by transposition of Young diagrams as a result of level rank duality. Next we consider U (N ) CS theory on S 3 and observed that in Seifert framing the dominant representation is no longer an integrable representation after a critical value of 't Hooft coupling. We also show that CS on S 3 admits multiple (two-gap phase) large N phases with the same free energy. * arghya@iiserb.ac.in † suvankar@iiserb.ac.in ‡ neetuj@iiserb.ac.in arXiv:1902.07538v2 [hep-th]