This work investigated the wear behavior of ultrafine-grained Ti65Nb23.33Zr5Ta1.67Fe5 (at.%, TNZTF) and Ti65Nb23.33Zr5Ta1.67Si5 (at.%, TNZTS) alloys fabricated by high-energy ball milling and spark plasma sintering. Wear tests were conducted in a simulated physiological solution under both reciprocating sliding and fretting wear conditions with different loads, frequencies, and stroke lengths. The microstructures, mechanical properties, and anti-wear properties of the investigated alloys were characterized. The results showed that the TNZTF and TNZTS alloys had much less wear volume than the commonly used Ti-6Al-4V (TC4) alloy and commercially pure titanium (CP-Ti). The TNZTF and TNZTS alloys exhibited much more smooth wear surfaces and shallower wear scars compared with TC4 and CP-Ti. The investigated alloys exhibited different wear mechanisms under the reciprocating sliding wear conditions, while they were similar under the fretting wear conditions. Compared with TC4 and CP-Ti, the fabricated TNZTF and TNZTS alloys showed a substantially higher wear resistance, owing to their ultrafine-grained microstructure and superior hardness. Additionally, the addition of Nb and Zr further enhanced the wear resistance by forming a protective Nb2O5 and ZrO2 oxide film. This work provides guidance for designing new biomedical titanium alloys with excellent wear resistance.