Exploring the intricate interplay between magnetism and superconductivity is crucial for unveiling the underlying mechanisms of unconventional superconductivity. Here, we report on the magnetotransport evidence for the coexistence of a two-dimensional (2D) superconducting state and a 2D ferromagnetic state at the interface between amorphous CaZrO3 film and (111)-oriented KTaO3 single crystal. Remarkably, the fingerprint of ferromagnetism, i.e., hysteretic magnetoresistance loops, is observed in the superconducting state. The butterfly-shaped hysteresis with twin peaks emerges against the background of superconducting zero resistance, and the peak amplitude increases with the sweep rate of the magnetic field, indicating that the magnetization dynamics are at play in the superconducting state. Moreover, the magnetoresistance hysteresis is strongly dependent on temperature, achieving a maximum near the superconducting temperature. This behavior is well described by the thermal activated phase slip model. Density function theory (DFT) calculations suggest that the magnetic moment is primarily contributed by the Ta 5dyz orbital, and the Stoner ferromagnetism is identified. Our findings provide new insights into the interaction of magnetism and superconductivity at KTaO3-based oxide heterointerfaces.