Polarization-maintaining terahertz (THz) fibers have been considered as fundamental elements in polarization sensitive THz systems. Based on the anti-resonant waveguiding mechanism, we propose a low-loss polarization-maintaining hollow-core THz fiber with four half-elliptical cladding membranes. Numerical simulations indicate that birefringence of the fundamental mode group could be introduced by modifying the thickness of a pair of membranes while sacrificing available low-loss bandwidth. We observe that the core size, the dimension and shape of cladding membranes, have significant impacts on modal properties. For the half-circular membranes based fiber with optimized parameters, the attenuation coefficients of fundamental modes are below 10 dB m−1, the birefringence is above 10−4 and the group velocity dispersion values are lower than 60 ps THz−1 m−1 within the frequency bandwidths of 0.46–0.6 THz and 0.74–1.1 THz. At 0.9 THz, the lowest attenuation coefficient is 0.5 dB m−1 with birefringence of 1.52 × 10−4. Moreover, this THz fiber is insensitive to neither x- nor y-direction bend with a bend radius no less than 2 cm.