In this work the instability of the Taylor-Couette flow for Newtonian and non-Newtonian fluids (dilatant and pseudoplastic fluids) is investigated for cases of finite aspect ratios. The study is conducted numerically using the lattice Boltzmann method (LBM). In many industrial applications, the apparatuses and installations drift away from the idealized case of an annulus of infinite length, and thus the end caps effect can no longer be ignored. The inner cylinder is rotating while the outer one and the end walls are maintained at rest. The lattice two-dimensional nine-velocity (D2Q9) Boltzmann model developed from the Bhatnagar-Gross-Krook approximation is used to obtain the flow field for fluids obeying the power-law model. The combined effects of the Reynolds number, the radius ratio, and the power-law index n on the flow characteristics are analyzed for an annular space of finite aspect ratio. Two flow modes are obtained: a primary Couette flow (CF) mode and a secondary Taylor vortex flow (TVF) mode. The flow structures so obtained are different from one mode to another. The critical Reynolds number Re(c) for the passage from the primary to the secondary mode exhibits the lowest value for the pseudoplastic fluids and the highest value for the dilatant fluids. The findings are useful for studies of the swirling flow of non-Newtonians fluids in axisymmetric geometries using LBM. The flow changes from the CF to TVF and its structure switches from the two-cells to four-cells regime for both Newtonian and dilatant fluids. Contrariwise for pseudoplastic fluids, the flow exhibits 2-4-2 structure passing from two-cells to four cells and switches again to the two-cells configuration. Furthermore, the critical Reynolds number presents a monotonic increase with the power-law index n of the non-Newtonian fluid, and as the radius ratio grows, the transition flow regimes tend to appear for higher critical Reynolds numbers.