We present a comprehensive investigation of quantum transport in silicon nanowire field-effect transistors (Si NWFETs) at the scaling limit. The Si bulk $\bm{k}\cdot\bm{p}$ Hamiltonian parameters are rendered invalid at smaller scales due to pronounced quantum confinement effects. Consequently, nanowire $\bm{k}\cdot\bm{p}$ Hamiltonian parameters are meticulously calibrated using first-principle HSE06 band structures through the fast least square method. Based on the Non-Equilibrium Green's functon(NEGF) formalism, we systematically analyze the performance limits of \textrm{Si} gate-all-around NWFETs under varying gate lengths. With diminishing gate lengths, quantum tunneling from source to drain intensifies, leading to a degradation in subthreshold swing. Our findings reveal that the gate length scaling limit for N-type devices surpasses that of P-type devices, and distinct gate scaling limits are elucidated for varying cross-sectional sizes, for instance, the ultimate scaling limit of a 4 nm $\times$ 4 nm cross-section Si nanowires is approximately 10 nm.