There has been intense recent interest in the two-dimensional electron gases (2DEGs) that form at the surfaces and interfaces of KTaO3 (KTO), with the discovery of superconductivity at temperatures significantly higher than those of similar 2DEGs based on SrTiO3 (STO). Like STO heterostructures, these KTO 2DEGs are formed by depositing an overlayer on top of appropriately prepared KTO surfaces. Some of these overlayers are magnetic, and the resulting 2DEGs show signatures of this magnetism, including hysteresis in the magnetoresistance (MR). Here, we show that KTO 2DEGs fabricated by depositing AlOx on top of KTO also show hysteretic MR, indicative of long-range magnetic order, even though the samples nominally contain no intrinsic magnetic elements. The hysteresis appears in both the transverse and longitudinal resistance in magnetic fields both perpendicular to and in the plane of the 2DEG. The hysteretic MR has different characteristic fields and shapes for surfaces of different crystal orientations and vanishes above a few Kelvin. Density functional theory (DFT) calculations indicate that the magnetism likely arises from Ta4+ local moments created in the presence of oxygen vacancies.