The parallel-magnetic-field H dependence of the superconducting transition temperature T c is studied for ultrathin films of In, Bi, and Al grown on GaAs(110). In the case of In films in the monolayer regime, T c exhibits a quadratic-like H dependence, which is one order of magnitude stronger than that previously observed in monolayer Pb films by the present authors [Phys. Rev. Lett. 111, 057005 (2013)]. The results are well reproduced by a model developed for an inhomogeneous two-dimensional superconducting state in the presence of a moderate Rashba spin-orbit interaction. The Rashba spin splitting is estimated to be 0.04 eV, which is much smaller than that expected for monolayer Pb films. In a few-monolayer Bi film, the suppression of T c with increasing H is comparable to that in monolayer Pb films. On the other hand, much stronger suppression, which is attributed to the Pauli paramagnetic effect, was observed for the Al film.